Filipe Manana [Tue, 4 Apr 2017 19:31:00 +0000 (20:31 +0100)]
Btrfs: send, fix file hole not being preserved due to inline extent
Normally we don't have inline extents followed by regular extents, but
there's currently at least one harmless case where this happens. For
example, when the page size is 4Kb and compression is enabled:
$ mkfs.btrfs -f /dev/sdb
$ mount -o compress /dev/sdb /mnt
$ xfs_io -f -c "pwrite -S 0xaa 0 4K" -c "fsync" /mnt/foobar
$ xfs_io -c "pwrite -S 0xbb 8K 4K" -c "fsync" /mnt/foobar
In this case we get a compressed inline extent, representing 4Kb of
data, followed by a hole extent and then a regular data extent. The
inline extent was not expanded/converted to a regular extent exactly
because it represents 4Kb of data. This does not cause any apparent
problem (such as the issue solved by commit
e1699d2d7bf6
("btrfs: add missing memset while reading compressed inline extents"))
except trigger an unexpected case in the incremental send code path
that makes us issue an operation to write a hole when it's not needed,
resulting in more writes at the receiver and wasting space at the
receiver.
So teach the incremental send code to deal with this particular case.
The issue can be currently triggered by running fstests btrfs/137 with
compression enabled (MOUNT_OPTIONS="-o compress" ./check btrfs/137).
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Filipe Manana [Mon, 3 Apr 2017 14:57:17 +0000 (15:57 +0100)]
Btrfs: fix extent map leak during fallocate error path
If the call to btrfs_qgroup_reserve_data() failed, we were leaking an
extent map structure. The failure can happen either due to an -ENOMEM
condition or, when quotas are enabled, due to -EDQUOT for example.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Filipe Manana [Wed, 8 Mar 2017 16:43:49 +0000 (16:43 +0000)]
Btrfs: fix incorrect space accounting after failure to insert inline extent
When using compression, if we fail to insert an inline extent we
incorrectly end up attempting to free the reserved data space twice,
once through extent_clear_unlock_delalloc(), because we pass it the
flag EXTENT_DO_ACCOUNTING, and once through a direct call to
btrfs_free_reserved_data_space_noquota(). This results in a trace
like the following:
[ 834.576240] ------------[ cut here ]------------
[ 834.576825] WARNING: CPU: 2 PID: 486 at fs/btrfs/extent-tree.c:4316 btrfs_free_reserved_data_space_noquota+0x60/0x9f [btrfs]
[ 834.579501] Modules linked in: btrfs crc32c_generic xor raid6_pq ppdev i2c_piix4 acpi_cpufreq psmouse tpm_tis parport_pc pcspkr serio_raw tpm_tis_core sg parport evdev i2c_core tpm button loop autofs4 ext4 crc16 jbd2 mbcache sr_mod cdrom sd_mod ata_generic virtio_scsi ata_piix virtio_pci libata virtio_ring virtio scsi_mod e1000 floppy [last unloaded: btrfs]
[ 834.592116] CPU: 2 PID: 486 Comm: kworker/u32:4 Not tainted 4.10.0-rc8-btrfs-next-37+ #2
[ 834.593316] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.9.1-0-gb3ef39f-prebuilt.qemu-project.org 04/01/2014
[ 834.595273] Workqueue: btrfs-delalloc btrfs_delalloc_helper [btrfs]
[ 834.596103] Call Trace:
[ 834.596103] dump_stack+0x67/0x90
[ 834.596103] __warn+0xc2/0xdd
[ 834.596103] warn_slowpath_null+0x1d/0x1f
[ 834.596103] btrfs_free_reserved_data_space_noquota+0x60/0x9f [btrfs]
[ 834.596103] compress_file_range.constprop.42+0x2fa/0x3fc [btrfs]
[ 834.596103] ? submit_compressed_extents+0x3a7/0x3a7 [btrfs]
[ 834.596103] async_cow_start+0x32/0x4d [btrfs]
[ 834.596103] btrfs_scrubparity_helper+0x187/0x3e7 [btrfs]
[ 834.596103] btrfs_delalloc_helper+0xe/0x10 [btrfs]
[ 834.596103] process_one_work+0x273/0x4e4
[ 834.596103] worker_thread+0x1eb/0x2ca
[ 834.596103] ? rescuer_thread+0x2b6/0x2b6
[ 834.596103] kthread+0x100/0x108
[ 834.596103] ? __list_del_entry+0x22/0x22
[ 834.596103] ret_from_fork+0x2e/0x40
[ 834.611656] ---[ end trace
719902fe6bdef08f ]---
So fix this by not calling directly btrfs_free_reserved_data_space_noquota()
if an error happened.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Filipe Manana [Mon, 6 Mar 2017 23:04:20 +0000 (23:04 +0000)]
Btrfs: fix invalid attempt to free reserved space on failure to cow range
When attempting to COW a file range (we are starting writeback and doing
COW), if we manage to reserve an extent for the range we will write into
but fail after reserving it and before creating the respective ordered
extent, we end up in an error path where we attempt to decrement the
data space's bytes_may_use counter after we already did it while
reserving the extent, leading to a warning/trace like the following:
[ 847.621524] ------------[ cut here ]------------
[ 847.625441] WARNING: CPU: 5 PID: 4905 at fs/btrfs/extent-tree.c:4316 btrfs_free_reserved_data_space_noquota+0x60/0x9f [btrfs]
[ 847.633704] Modules linked in: btrfs crc32c_generic xor raid6_pq acpi_cpufreq i2c_piix4 ppdev psmouse tpm_tis serio_raw pcspkr parport_pc tpm_tis_core i2c_core sg
[ 847.644616] CPU: 5 PID: 4905 Comm: xfs_io Not tainted 4.10.0-rc8-btrfs-next-37+ #2
[ 847.648601] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.9.1-0-gb3ef39f-prebuilt.qemu-project.org 04/01/2014
[ 847.648601] Call Trace:
[ 847.648601] dump_stack+0x67/0x90
[ 847.648601] __warn+0xc2/0xdd
[ 847.648601] warn_slowpath_null+0x1d/0x1f
[ 847.648601] btrfs_free_reserved_data_space_noquota+0x60/0x9f [btrfs]
[ 847.648601] btrfs_clear_bit_hook+0x140/0x258 [btrfs]
[ 847.648601] clear_state_bit+0x87/0x128 [btrfs]
[ 847.648601] __clear_extent_bit+0x222/0x2b7 [btrfs]
[ 847.648601] clear_extent_bit+0x17/0x19 [btrfs]
[ 847.648601] extent_clear_unlock_delalloc+0x3b/0x6b [btrfs]
[ 847.648601] cow_file_range.isra.39+0x387/0x39a [btrfs]
[ 847.648601] run_delalloc_nocow+0x4d7/0x70e [btrfs]
[ 847.648601] ? arch_local_irq_save+0x9/0xc
[ 847.648601] run_delalloc_range+0xa7/0x2b5 [btrfs]
[ 847.648601] writepage_delalloc.isra.31+0xb9/0x15c [btrfs]
[ 847.648601] __extent_writepage+0x249/0x2e8 [btrfs]
[ 847.648601] extent_write_cache_pages.constprop.33+0x28b/0x36c [btrfs]
[ 847.648601] ? arch_local_irq_save+0x9/0xc
[ 847.648601] ? mark_lock+0x24/0x201
[ 847.648601] extent_writepages+0x4b/0x5c [btrfs]
[ 847.648601] ? btrfs_writepage_start_hook+0xed/0xed [btrfs]
[ 847.648601] btrfs_writepages+0x28/0x2a [btrfs]
[ 847.648601] do_writepages+0x23/0x2c
[ 847.648601] __filemap_fdatawrite_range+0x5a/0x61
[ 847.648601] filemap_fdatawrite_range+0x13/0x15
[ 847.648601] btrfs_fdatawrite_range+0x20/0x46 [btrfs]
[ 847.648601] start_ordered_ops+0x19/0x23 [btrfs]
[ 847.648601] btrfs_sync_file+0x136/0x42c [btrfs]
[ 847.648601] vfs_fsync_range+0x8c/0x9e
[ 847.648601] vfs_fsync+0x1c/0x1e
[ 847.648601] do_fsync+0x31/0x4a
[ 847.648601] SyS_fsync+0x10/0x14
[ 847.648601] entry_SYSCALL_64_fastpath+0x18/0xad
[ 847.648601] RIP: 0033:0x7f5b05200800
[ 847.648601] RSP: 002b:
00007ffe204f71c8 EFLAGS:
00000246 ORIG_RAX:
000000000000004a
[ 847.648601] RAX:
ffffffffffffffda RBX:
ffffffff8109637b RCX:
00007f5b05200800
[ 847.648601] RDX:
00000000008bd0a0 RSI:
00000000008bd2e0 RDI:
0000000000000003
[ 847.648601] RBP:
ffffc90001d67f98 R08:
000000000000ffff R09:
000000000000001f
[ 847.648601] R10:
00000000000001f6 R11:
0000000000000246 R12:
0000000000000046
[ 847.648601] R13:
ffffc90001d67f78 R14:
00007f5b054be740 R15:
00007f5b054be740
[ 847.648601] ? trace_hardirqs_off_caller+0x3f/0xaa
[ 847.685787] ---[ end trace
2a4a3e15382508e8 ]---
So fix this by not attempting to decrement the data space info's
bytes_may_use counter if we already reserved the extent and an error
happened before creating the ordered extent. We are already correctly
freeing the reserved extent if an error happens, so there's no additional
measure needed.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Qu Wenruo [Wed, 8 Mar 2017 02:25:52 +0000 (10:25 +0800)]
btrfs: Handle delalloc error correctly to avoid ordered extent hang
[BUG]
If run_delalloc_range() returns error and there is already some ordered
extents created, btrfs will be hanged with the following backtrace:
Call Trace:
__schedule+0x2d4/0xae0
schedule+0x3d/0x90
btrfs_start_ordered_extent+0x160/0x200 [btrfs]
? wake_atomic_t_function+0x60/0x60
btrfs_run_ordered_extent_work+0x25/0x40 [btrfs]
btrfs_scrubparity_helper+0x1c1/0x620 [btrfs]
btrfs_flush_delalloc_helper+0xe/0x10 [btrfs]
process_one_work+0x2af/0x720
? process_one_work+0x22b/0x720
worker_thread+0x4b/0x4f0
kthread+0x10f/0x150
? process_one_work+0x720/0x720
? kthread_create_on_node+0x40/0x40
ret_from_fork+0x2e/0x40
[CAUSE]
|<------------------ delalloc range --------------------------->|
| OE 1 | OE 2 | ... | OE n |
|<>| |<---------- cleanup range --------->|
||
\_=> First page handled by end_extent_writepage() in __extent_writepage()
The problem is caused by error handler of run_delalloc_range(), which
doesn't handle any created ordered extents, leaving them waiting on
btrfs_finish_ordered_io() to finish.
However after run_delalloc_range() returns error, __extent_writepage()
won't submit bio, so btrfs_writepage_end_io_hook() won't be triggered
except the first page, and btrfs_finish_ordered_io() won't be triggered
for created ordered extents either.
So OE 2~n will hang forever, and if OE 1 is larger than one page, it
will also hang.
[FIX]
Introduce btrfs_cleanup_ordered_extents() function to cleanup created
ordered extents and finish them manually.
The function is based on existing
btrfs_endio_direct_write_update_ordered() function, and modify it to
act just like btrfs_writepage_endio_hook() but handles specified range
other than one page.
After fix, delalloc error will be handled like:
|<------------------ delalloc range --------------------------->|
| OE 1 | OE 2 | ... | OE n |
|<>|<-------- ----------->|<------ old error handler --------->|
|| ||
|| \_=> Cleaned up by cleanup_ordered_extents()
\_=> First page handled by end_extent_writepage() in __extent_writepage()
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Qu Wenruo [Wed, 8 Mar 2017 02:25:51 +0000 (10:25 +0800)]
btrfs: Fix metadata underflow caused by btrfs_reloc_clone_csum error
[BUG]
When btrfs_reloc_clone_csum() reports error, it can underflow metadata
and leads to kernel assertion on outstanding extents in
run_delalloc_nocow() and cow_file_range().
BTRFS info (device vdb5): relocating block group
12582912 flags data
BTRFS info (device vdb5): found 1 extents
assertion failed: inode->outstanding_extents >= num_extents, file: fs/btrfs//extent-tree.c, line: 5858
Currently, due to another bug blocking ordered extents, the bug is only
reproducible under certain block group layout and using error injection.
a) Create one data block group with one 4K extent in it.
To avoid the bug that hangs btrfs due to ordered extent which never
finishes
b) Make btrfs_reloc_clone_csum() always fail
c) Relocate that block group
[CAUSE]
run_delalloc_nocow() and cow_file_range() handles error from
btrfs_reloc_clone_csum() wrongly:
(The ascii chart shows a more generic case of this bug other than the
bug mentioned above)
|<------------------ delalloc range --------------------------->|
| OE 1 | OE 2 | ... | OE n |
|<----------- cleanup range --------------->|
|<----------- ----------->|
\/
btrfs_finish_ordered_io() range
So error handler, which calls extent_clear_unlock_delalloc() with
EXTENT_DELALLOC and EXTENT_DO_ACCOUNT bits, and btrfs_finish_ordered_io()
will both cover OE n, and free its metadata, causing metadata under flow.
[Fix]
The fix is to ensure after calling btrfs_add_ordered_extent(), we only
call error handler after increasing the iteration offset, so that
cleanup range won't cover any created ordered extent.
|<------------------ delalloc range --------------------------->|
| OE 1 | OE 2 | ... | OE n |
|<----------- ----------->|<---------- cleanup range --------->|
\/
btrfs_finish_ordered_io() range
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Liu Bo [Mon, 10 Apr 2017 19:36:26 +0000 (12:36 -0700)]
Btrfs: fix potential use-after-free for cloned bio
KASAN reports that there is a use-after-free case of bio in btrfs_map_bio.
If we need to submit IOs to several disks at a time, the original bio
would get cloned and mapped to the destination disk, but we really should
use the original bio instead of a cloned bio to do the sanity check
because cloned bios are likely to be freed by its endio.
Reported-by: Diego <diegocg@gmail.com>
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Liu Bo [Fri, 7 Apr 2017 20:11:10 +0000 (13:11 -0700)]
Btrfs: fix segmentation fault when doing dio read
Commit
2dabb3248453 ("Btrfs: Direct I/O read: Work on sectorsized blocks")
introduced this bug during iterating bio pages in dio read's endio hook,
and it could end up with segment fault of the dio reading task.
So the reason is 'if (nr_sectors--)', and it makes the code assume that
there is one more block in the same page, so page offset is increased and
the bio which is created to repair the bad block then has an incorrect
bvec.bv_offset, and a later access of the page content would throw a
segmentation fault.
This also adds ASSERT to check page offset against page size.
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Liu Bo [Wed, 5 Apr 2017 21:04:19 +0000 (14:04 -0700)]
Btrfs: fix invalid dereference in btrfs_retry_endio
When doing directIO repair, we have this oops:
[ 1458.532816] general protection fault: 0000 [#1] SMP
...
[ 1458.536291] Workqueue: btrfs-endio-repair btrfs_endio_repair_helper [btrfs]
[ 1458.536893] task:
ffff88082a42d100 task.stack:
ffffc90002b3c000
[ 1458.537499] RIP: 0010:btrfs_retry_endio+0x7e/0x1a0 [btrfs]
...
[ 1458.543261] Call Trace:
[ 1458.543958] ? rcu_read_lock_sched_held+0xc4/0xd0
[ 1458.544374] bio_endio+0xed/0x100
[ 1458.544750] end_workqueue_fn+0x3c/0x40 [btrfs]
[ 1458.545257] normal_work_helper+0x9f/0x900 [btrfs]
[ 1458.545762] btrfs_endio_repair_helper+0x12/0x20 [btrfs]
[ 1458.546224] process_one_work+0x34d/0xb70
[ 1458.546570] ? process_one_work+0x29e/0xb70
[ 1458.546938] worker_thread+0x1cf/0x960
[ 1458.547263] ? process_one_work+0xb70/0xb70
[ 1458.547624] kthread+0x17d/0x180
[ 1458.547909] ? kthread_create_on_node+0x70/0x70
[ 1458.548300] ret_from_fork+0x31/0x40
It turns out that btrfs_retry_endio is trying to get inode from a directIO
page.
This fixes the problem by using the saved inode pointer, done->inode.
btrfs_retry_endio_nocsum has the same problem, and it's fixed as well.
Also cleanup unused @start (which is too trivial for a separate patch).
Cc: David Sterba <dsterba@suse.cz>
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Adam Borowski [Fri, 31 Mar 2017 15:19:04 +0000 (17:19 +0200)]
btrfs: drop the nossd flag when remounting with -o ssd
The opposite case was already handled right in the very next switch entry.
And also when turning on nossd, drop ssd_spread.
Reported-by: Hans van Kranenburg <hans.van.kranenburg@mendix.com>
Signed-off-by: Adam Borowski <kilobyte@angband.pl>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Chris Mason [Wed, 29 Mar 2017 20:10:25 +0000 (13:10 -0700)]
Merge branch 'for-chris-4.11-rc5' of git://git./linux/kernel/git/kdave/linux into for-linus-4.11
Dan Carpenter [Fri, 17 Mar 2017 20:51:20 +0000 (23:51 +0300)]
Btrfs: fix an integer overflow check
This isn't super serious because you need CAP_ADMIN to run this code.
I added this integer overflow check last year but apparently I am
rubbish at writing integer overflow checks... There are two issues.
First, access_ok() works on unsigned long type and not u64 so on 32 bit
systems the access_ok() could be checking a truncated size. The other
issue is that we should be using a stricter limit so we don't overflow
the kzalloc() setting ctx->clone_roots later in the function after the
access_ok():
alloc_size = sizeof(struct clone_root) * (arg->clone_sources_count + 1);
sctx->clone_roots = kzalloc(alloc_size, GFP_KERNEL | __GFP_NOWARN);
Fixes:
f5ecec3ce21f ("btrfs: send: silence an integer overflow warning")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Reviewed-by: David Sterba <dsterba@suse.com>
[ added comment ]
Signed-off-by: David Sterba <dsterba@suse.com>
Goldwyn Rodrigues [Tue, 14 Mar 2017 10:25:09 +0000 (05:25 -0500)]
btrfs: Change qgroup_meta_rsv to 64bit
Using an int value is causing qg->reserved to become negative and
exclusive -EDQUOT to be reached prematurely.
This affects exclusive qgroups only.
TEST CASE:
DEVICE=/dev/vdb
MOUNTPOINT=/mnt
SUBVOL=$MOUNTPOINT/tmp
umount $SUBVOL
umount $MOUNTPOINT
mkfs.btrfs -f $DEVICE
mount /dev/vdb $MOUNTPOINT
btrfs quota enable $MOUNTPOINT
btrfs subvol create $SUBVOL
umount $MOUNTPOINT
mount /dev/vdb $MOUNTPOINT
mount -o subvol=tmp $DEVICE $SUBVOL
btrfs qgroup limit -e 3G $SUBVOL
btrfs quota rescan /mnt -w
for i in `seq 1 44000`; do
dd if=/dev/zero of=/mnt/tmp/test_$i bs=10k count=1
if [[ $? > 0 ]]; then
btrfs qgroup show -pcref $SUBVOL
exit 1
fi
done
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
[ add reproducer to changelog ]
Signed-off-by: David Sterba <dsterba@suse.com>
Liu Bo [Fri, 24 Mar 2017 22:04:50 +0000 (15:04 -0700)]
Btrfs: bring back repair during read
Commit
20a7db8ab3f2 ("btrfs: add dummy callback for readpage_io_failed
and drop checks") made a cleanup around readpage_io_failed_hook, and
it was supposed to keep the original sematics, but it also
unexpectedly disabled repair during read for dup, raid1 and raid10.
This fixes the problem by letting data's inode call the generic
readpage_io_failed callback by returning -EAGAIN from its
readpage_io_failed_hook in order to notify end_bio_extent_readpage to
do the rest. We don't call it directly because the generic one takes
an offset from end_bio_extent_readpage() to calculate the index in the
checksum array and inode's readpage_io_failed_hook doesn't offer that
offset.
Cc: David Sterba <dsterba@suse.cz>
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Reviewed-by: David Sterba <dsterba@suse.com>
[ keep the const function attribute ]
Signed-off-by: David Sterba <dsterba@suse.com>
Zygo Blaxell [Fri, 10 Mar 2017 21:45:44 +0000 (16:45 -0500)]
btrfs: add missing memset while reading compressed inline extents
This is a story about 4 distinct (and very old) btrfs bugs.
Commit
c8b978188c ("Btrfs: Add zlib compression support") added
three data corruption bugs for inline extents (bugs #1-3).
Commit
93c82d5750 ("Btrfs: zero page past end of inline file items")
fixed bug #1: uncompressed inline extents followed by a hole and more
extents could get non-zero data in the hole as they were read. The fix
was to add a memset in btrfs_get_extent to zero out the hole.
Commit
166ae5a418 ("btrfs: fix inline compressed read err corruption")
fixed bug #2: compressed inline extents which contained non-zero bytes
might be replaced with zero bytes in some cases. This patch removed an
unhelpful memset from uncompress_inline, but the case where memset is
required was missed.
There is also a memset in the decompression code, but this only covers
decompressed data that is shorter than the ram_bytes from the extent
ref record. This memset doesn't cover the region between the end of the
decompressed data and the end of the page. It has also moved around a
few times over the years, so there's no single patch to refer to.
This patch fixes bug #3: compressed inline extents followed by a hole
and more extents could get non-zero data in the hole as they were read
(i.e. bug #3 is the same as bug #1, but s/uncompressed/compressed/).
The fix is the same: zero out the hole in the compressed case too,
by putting a memset back in uncompress_inline, but this time with
correct parameters.
The last and oldest bug, bug #0, is the cause of the offending inline
extent/hole/extent pattern. Bug #0 is a subtle and mostly-harmless quirk
of behavior somewhere in the btrfs write code. In a few special cases,
an inline extent and hole are allowed to persist where they normally
would be combined with later extents in the file.
A fast reproducer for bug #0 is presented below. A few offending extents
are also created in the wild during large rsync transfers with the -S
flag. A Linux kernel build (git checkout; make allyesconfig; make -j8)
will produce a handful of offending files as well. Once an offending
file is created, it can present different content to userspace each
time it is read.
Bug #0 is at least 4 and possibly 8 years old. I verified every vX.Y
kernel back to v3.5 has this behavior. There are fossil records of this
bug's effects in commits all the way back to v2.6.32. I have no reason
to believe bug #0 wasn't present at the beginning of btrfs compression
support in v2.6.29, but I can't easily test kernels that old to be sure.
It is not clear whether bug #0 is worth fixing. A fix would likely
require injecting extra reads into currently write-only paths, and most
of the exceptional cases caused by bug #0 are already handled now.
Whether we like them or not, bug #0's inline extents followed by holes
are part of the btrfs de-facto disk format now, and we need to be able
to read them without data corruption or an infoleak. So enough about
bug #0, let's get back to bug #3 (this patch).
An example of on-disk structure leading to data corruption found in
the wild:
item 61 key (606890 INODE_ITEM 0) itemoff 9662 itemsize 160
inode generation 50 transid 50 size 47424 nbytes 49141
block group 0 mode 100644 links 1 uid 0 gid 0
rdev 0 flags 0x0(none)
item 62 key (606890 INODE_REF 603050) itemoff 9642 itemsize 20
inode ref index 3 namelen 10 name: DB_File.so
item 63 key (606890 EXTENT_DATA 0) itemoff 8280 itemsize 1362
inline extent data size 1341 ram 4085 compress(zlib)
item 64 key (606890 EXTENT_DATA 4096) itemoff 8227 itemsize 53
extent data disk byte
5367308288 nr 20480
extent data offset 0 nr 45056 ram 45056
extent compression(zlib)
Different data appears in userspace during each read of the 11 bytes
between 4085 and 4096. The extent in item 63 is not long enough to
fill the first page of the file, so a memset is required to fill the
space between item 63 (ending at 4085) and item 64 (beginning at 4096)
with zero.
Here is a reproducer from Liu Bo, which demonstrates another method
of creating the same inline extent and hole pattern:
Using 'page_poison=on' kernel command line (or enable
CONFIG_PAGE_POISONING) run the following:
# touch foo
# chattr +c foo
# xfs_io -f -c "pwrite -W 0 1000" foo
# xfs_io -f -c "falloc 4 8188" foo
# od -x foo
# echo 3 >/proc/sys/vm/drop_caches
# od -x foo
This produce the following on my box:
Correct output: file contains 1000 data bytes followed
by zeros:
0000000 cdcd cdcd cdcd cdcd cdcd cdcd cdcd cdcd
*
0001740 cdcd cdcd cdcd cdcd 0000 0000 0000 0000
0001760 0000 0000 0000 0000 0000 0000 0000 0000
*
0020000
Actual output: the data after the first 1000 bytes
will be different each run:
0000000 cdcd cdcd cdcd cdcd cdcd cdcd cdcd cdcd
*
0001740 cdcd cdcd cdcd cdcd 6c63 7400 635f 006d
0001760 5f74 6f43 7400 435f 0053 5f74 7363 7400
0002000 435f 0056 5f74 6164 7400 645f 0062 5f74
(...)
Signed-off-by: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Reviewed-by: Chris Mason <clm@fb.com>
Signed-off-by: Chris Mason <clm@fb.com>
Liu Bo [Tue, 7 Mar 2017 02:20:56 +0000 (18:20 -0800)]
Btrfs: fix regression in lock_delalloc_pages
The bug is a regression after commit
(
da2c7009f6ca "btrfs: teach __process_pages_contig about PAGE_LOCK operation")
and commit
(
76c0021db8fd "Btrfs: use helper to simplify lock/unlock pages").
So if the dirty pages which are under writeback got truncated partially
before we lock the dirty pages, we couldn't find all pages mapping to the
delalloc range, and the bug didn't return an error so it kept going on and
found that the delalloc range got truncated and got to unlock the dirty
pages, and then the ASSERT could caught the error, and showed
-----------------------------------------------------------------------------
assertion failed: page_ops & PAGE_LOCK, file: fs/btrfs/extent_io.c, line: 1716
-----------------------------------------------------------------------------
This fixes the bug by returning the proper -EAGAIN.
Cc: David Sterba <dsterba@suse.com>
Reported-by: Dave Jones <davej@codemonkey.org.uk>
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Dmitry V. Levin [Tue, 28 Feb 2017 23:12:50 +0000 (02:12 +0300)]
btrfs: remove btrfs_err_str function from uapi/linux/btrfs.h
btrfs_err_str function is not called from anywhere and is replicated
in the userspace headers for btrfs-progs.
It's removal also fixes the following linux/btrfs.h userspace
compilation error:
/usr/include/linux/btrfs.h: In function 'btrfs_err_str':
/usr/include/linux/btrfs.h:740:11: error: 'NULL' undeclared (first use in this function)
return NULL;
Suggested-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: Dmitry V. Levin <ldv@altlinux.org>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Chris Mason [Tue, 28 Feb 2017 22:35:09 +0000 (14:35 -0800)]
Merge branch 'for-chris-4.11-part2' of git://git./linux/kernel/git/kdave/linux into for-linus-4.11
David Sterba [Fri, 17 Feb 2017 15:24:29 +0000 (16:24 +0100)]
btrfs: add dummy callback for readpage_io_failed and drop checks
Make extent_io_ops::readpage_io_failed_hook callback mandatory and
define a dummy function for btrfs_extent_io_ops. As the failed IO
callback is not performance critical, the branch vs extra trade off does
not hurt.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 17 Feb 2017 14:59:35 +0000 (15:59 +0100)]
btrfs: drop checks for mandatory extent_io_ops callbacks
We know that eadpage_end_io_hook, submit_bio_hook and merge_bio_hook are
always defined so we can drop the checks before we call them.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 17 Feb 2017 14:27:44 +0000 (15:27 +0100)]
btrfs: document existence of extent_io ops callbacks
Some of the callbacks defined in btree_extent_io_ops and
btrfs_extent_io_ops do always exist so we don't need to check the
existence before each call. This patch just reorders the definition and
documents which are mandatory/optional.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 17 Feb 2017 14:18:32 +0000 (15:18 +0100)]
btrfs: let writepage_end_io_hook return void
There's no error path in any of the instances, always return 0.
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 17 Feb 2017 18:42:43 +0000 (19:42 +0100)]
btrfs: do proper error handling in btrfs_insert_xattr_item
The space check in btrfs_insert_xattr_item is duplicated in it's caller
(do_setxattr) so we won't hit the BUG_ON. Continuing without any check
could be disasterous so turn it to a proper error handling.
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Wed, 15 Feb 2017 08:35:01 +0000 (09:35 +0100)]
btrfs: handle allocation error in update_dev_stat_item
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Feb 2017 16:33:27 +0000 (17:33 +0100)]
btrfs: remove BUG_ON from __tree_mod_log_insert
All callers dereference the 'tm' parameter before it gets to this
function, the NULL check does not make much sense here.
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Feb 2017 18:45:05 +0000 (19:45 +0100)]
btrfs: derive maximum output size in the compression implementation
The value of max_out can be calculated from the parameters passed to the
compressors, which is number of pages and the page size, and we don't
have to needlessly pass it around.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Feb 2017 18:36:54 +0000 (19:36 +0100)]
btrfs: use predefined limits for calculating maximum number of pages for compression
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Feb 2017 18:30:39 +0000 (19:30 +0100)]
btrfs: export compression buffer limits in a header
Move the buffer limit definitions out of compress_file_range.
Reviewed-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Feb 2017 18:04:07 +0000 (19:04 +0100)]
btrfs: merge nr_pages input and output parameter in compress_pages
The parameter saying how many pages can be allocated at maximum can be
merged with the output page counter, to save some stack space. The
compression implementation will sink the parameter to a local variable
so everything works as before.
The nr_pages variables can also be simply merged in compress_file_range
into one.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Feb 2017 18:04:07 +0000 (19:04 +0100)]
btrfs: merge length input and output parameter in compress_pages
The length parameter is basically duplicated for input and output in the
top level caller of the compress_pages chain. We can simply use one
variable for that and reduce stack consumption. The compression
implementation will sink the parameter to a local variable so everything
works as before.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Feb 2017 17:33:53 +0000 (18:33 +0100)]
btrfs: constify name of subvolume in creation helpers
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Feb 2017 16:58:04 +0000 (17:58 +0100)]
btrfs: constify buffers used by compression helpers
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Feb 2017 17:03:49 +0000 (18:03 +0100)]
btrfs: constify input buffer of btrfs_csum_data
The function does not modify the input buffer, also update a typecast in
one caller.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Feb 2017 16:55:53 +0000 (17:55 +0100)]
btrfs: constify device path passed to relevant helpers
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:51:11 +0000 (13:51 +0200)]
btrfs: make btrfs_inode_resume_unlocked_dio take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:51:10 +0000 (13:51 +0200)]
btrfs: make btrfs_inode_block_unlocked_dio take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:51:09 +0000 (13:51 +0200)]
btrfs: Make btrfs_add_nondir take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:51:08 +0000 (13:51 +0200)]
btrfs: Make btrfs_add_link take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:51:07 +0000 (13:51 +0200)]
btrfs: Make btrfs_del_delalloc_inode take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:51:06 +0000 (13:51 +0200)]
btrfs: Make get_extent_t take btrfs_inode
In addition to changing the signature, this patch also switches
all the functions which are used as an argument to also take btrfs_inode.
Namely those are: btrfs_get_extent and btrfs_get_extent_filemap.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:51:05 +0000 (13:51 +0200)]
btrfs: Make check_extent_to_block take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:51:04 +0000 (13:51 +0200)]
btrfs: Make clone_update_extent_map take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:51:03 +0000 (13:51 +0200)]
btrfs: Make btrfs_clear_bit_hook take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:51:02 +0000 (13:51 +0200)]
btrfs: Make btrfs_extent_item_to_extent_map take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:51:01 +0000 (13:51 +0200)]
btrfs: make btrfs_log_inode_parent take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:51:00 +0000 (13:51 +0200)]
btrfs: Make check_parent_dirs_for_sync take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:59 +0000 (13:50 +0200)]
btrfs: Make btrfs_orphan_add take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:58 +0000 (13:50 +0200)]
btrfs: make btrfs_orphan_del take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:57 +0000 (13:50 +0200)]
btrfs: make btrfs_free_io_failure_record take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:56 +0000 (13:50 +0200)]
btrfs: make clean_io_failure take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:55 +0000 (13:50 +0200)]
btrfs: make repair_io_failure take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:54 +0000 (13:50 +0200)]
btrfs: make check_compressed_csum take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:53 +0000 (13:50 +0200)]
btrfs: make btrfs_print_data_csum_error take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:52 +0000 (13:50 +0200)]
btrfs: make free_io_failure take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:51 +0000 (13:50 +0200)]
btrfs: Make lock_and_cleanup_extent_if_need take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:50 +0000 (13:50 +0200)]
btrfs: Make check_can_nocow take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:49 +0000 (13:50 +0200)]
btrfs: Make btrfs_lookup_ordered_range take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:48 +0000 (13:50 +0200)]
btrfs: Make btrfs_mark_extent_written take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:47 +0000 (13:50 +0200)]
btrfs: Make fill_holes take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:46 +0000 (13:50 +0200)]
btrfs: Make hole_mergeable take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:45 +0000 (13:50 +0200)]
btrfs: Make btrfs_drop_extent_cache take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:44 +0000 (13:50 +0200)]
btrfs: Make btrfs_requeue_inode_defrag take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:43 +0000 (13:50 +0200)]
btrfs: Make (__)btrfs_add_inode_defrag take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:42 +0000 (13:50 +0200)]
btrfs: all btrfs_delalloc_release_metadata take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:41 +0000 (13:50 +0200)]
btrfs: Make btrfs_delalloc_reserve_metadata take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:40 +0000 (13:50 +0200)]
btrfs: Make btrfs_orphan_release_metadata take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:39 +0000 (13:50 +0200)]
btrfs: Make btrfs_orphan_reserve_metadata take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:38 +0000 (13:50 +0200)]
btrfs: Make calc_csum_metadata_size take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:37 +0000 (13:50 +0200)]
btrfs: Make drop_outstanding_extent take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:36 +0000 (13:50 +0200)]
btrfs: make btrfs_alloc_data_chunk_ondemand take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:35 +0000 (13:50 +0200)]
btrfs: make btrfs_is_free_space_inode take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:34 +0000 (13:50 +0200)]
btrfs: Make btrfs_i_size_write take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:33 +0000 (13:50 +0200)]
btrfs: Make btrfs_set_inode_index take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:32 +0000 (13:50 +0200)]
btrfs: make btrfs_set_inode_index_count take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:31 +0000 (13:50 +0200)]
btrfs: Make btrfs_insert_dir_item take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Mon, 20 Feb 2017 11:50:30 +0000 (13:50 +0200)]
btrfs: Make btrfs_log_all_parents take btrfs_inode
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Chris Mason [Mon, 27 Feb 2017 21:11:53 +0000 (13:11 -0800)]
Merge branch 'for-chris-4.11' of git://git./linux/kernel/git/fdmanana/linux into for-linus-4.11
Filipe Manana [Fri, 17 Feb 2017 18:43:57 +0000 (18:43 +0000)]
Btrfs: try harder to migrate items to left sibling before splitting a leaf
Before attempting to split a leaf we try to migrate items from the leaf to
its right and left siblings. We start by trying to move items into the
rigth sibling and, if the new item is meant to be inserted at the end of
our leaf, we try to free from our leaf an amount of bytes equal to the
number of bytes used by the new item, by setting the variable space_needed
to the byte size of that new item. However if we fail to move enough items
to the right sibling due to lack of space in that sibling, we then try
to move items into the left sibling, and in that case we try to free
an amount equal to the size of the new item from our leaf, when we need
only to free an amount corresponding to the size of the new item minus
the current free space of our leaf. So make sure that before we try to
move items to the left sibling we do set the variable space_needed with
a value corresponding to the new item's size minus the leaf's current
free space.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Filipe Manana [Tue, 14 Feb 2017 16:56:01 +0000 (16:56 +0000)]
Btrfs: fix data loss after truncate when using the no-holes feature
If we have a file with an implicit hole (NO_HOLES feature enabled) that
has an extent following the hole, delayed writes against regions of the
file behind the hole happened before but were not yet flushed and then
we truncate the file to a smaller size that lies inside the hole, we
end up persisting a wrong disk_i_size value for our inode that leads to
data loss after umounting and mounting again the filesystem or after
the inode is evicted and loaded again.
This happens because at inode.c:btrfs_truncate_inode_items() we end up
setting last_size to the offset of the extent that we deleted and that
followed the hole. We then pass that value to btrfs_ordered_update_i_size()
which updates the inode's disk_i_size to a value smaller then the offset
of the buffered (delayed) writes.
Example reproducer:
$ mkfs.btrfs -f /dev/sdb
$ mount /dev/sdb /mnt
$ xfs_io -f -c "pwrite -S 0x01 0K 32K" /mnt/foo
$ xfs_io -d -c "pwrite -S 0x02 -b 32K 64K 32K" /mnt/foo
$ xfs_io -c "truncate 60K" /mnt/foo
--> inode's disk_i_size updated to 0
$ md5sum /mnt/foo
3c5ca3c3ab42f4b04d7e7eb0b0d4d806 /mnt/foo
$ umount /dev/sdb
$ mount /dev/sdb /mnt
$ md5sum /mnt/foo
d41d8cd98f00b204e9800998ecf8427e /mnt/foo
--> Empty file, all data lost!
Cc: <stable@vger.kernel.org> # 3.14+
Fixes:
16e7549f045d ("Btrfs: incompatible format change to remove hole extents")
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Filipe Manana [Tue, 14 Feb 2017 17:56:32 +0000 (17:56 +0000)]
Btrfs: incremental send, fix unnecessary hole writes for sparse files
When using the NO_HOLES feature, during an incremental send we often issue
write operations for holes when we should not, because that range is already
a hole in the destination snapshot. While that does not change the contents
of the file at the receiver, it avoids preservation of file holes, leading
to wasted disk space and extra IO during send/receive.
A couple examples where the holes are not preserved follows.
$ mkfs.btrfs -O no-holes -f /dev/sdb
$ mount /dev/sdb /mnt
$ xfs_io -f -c "pwrite -S 0xaa 0 4K" /mnt/foo
$ xfs_io -f -c "pwrite -S 0xaa 0 4K" -c "pwrite -S 0xbb 1028K 4K" /mnt/bar
$ btrfs subvolume snapshot -r /mnt /mnt/snap1
# Now add one new extent to our first test file, increasing its size and
# leaving a 1Mb hole between the first extent and this new extent.
$ xfs_io -c "pwrite -S 0xbb 1028K 4K" /mnt/foo
# Now overwrite the last extent of our second test file.
$ xfs_io -c "pwrite -S 0xcc 1028K 4K" /mnt/bar
$ btrfs subvolume snapshot -r /mnt /mnt/snap2
$ xfs_io -r -c "fiemap -v" /mnt/snap2/foo
/mnt/snap2/foo:
EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS
0: [0..7]: 25088..25095 8 0x2000
1: [8..2055]: hole 2048
2: [2056..2063]: 24576..24583 8 0x2001
$ xfs_io -r -c "fiemap -v" /mnt/snap2/bar
/mnt/snap2/bar:
EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS
0: [0..7]: 25096..25103 8 0x2000
1: [8..2055]: hole 2048
2: [2056..2063]: 24584..24591 8 0x2001
$ btrfs send /mnt/snap1 -f /tmp/1.snap
$ btrfs send -p /mnt/snap1 /mnt/snap2 -f /tmp/2.snap
$ umount /mnt
# It's not relevant to enable no-holes in the new filesystem.
$ mkfs.btrfs -O no-holes -f /dev/sdc
$ mount /dev/sdc /mnt
$ btrfs receive /mnt -f /tmp/1.snap
$ btrfs receive /mnt -f /tmp/2.snap
$ xfs_io -r -c "fiemap -v" /mnt/snap2/foo
/mnt/snap2/foo:
EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS
0: [0..7]: 24576..24583 8 0x2000
1: [8..2063]: 25624..27679 2056 0x1
$ xfs_io -r -c "fiemap -v" /mnt/snap2/bar
/mnt/snap2/bar:
EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS
0: [0..7]: 24584..24591 8 0x2000
1: [8..2063]: 27680..29735 2056 0x1
The holes do not exist in the second filesystem and they were replaced
with extents filled with the byte 0x00, making each file take 1032Kb of
space instead of 8Kb.
So fix this by not issuing the write operations consisting of buffers
filled with the byte 0x00 when the destination snapshot already has a
hole for the respective range.
A test case for fstests will follow soon.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Filipe Manana [Sat, 4 Feb 2017 17:12:00 +0000 (17:12 +0000)]
Btrfs: fix use-after-free due to wrong order of destroying work queues
Before we destroy all work queues (and wait for their tasks to complete)
we were destroying the work queues used for metadata I/O operations, which
can result in a use-after-free problem because most tasks from all work
queues do metadata I/O operations. For example, the tasks from the caching
workers work queue (fs_info->caching_workers), which is destroyed only
after the work queue used for metadata reads (fs_info->endio_meta_workers)
is destroyed, do metadata reads, which result in attempts to queue tasks
into the later work queue, triggering a use-after-free with a trace like
the following:
[23114.613543] general protection fault: 0000 [#1] PREEMPT SMP
[23114.614442] Modules linked in: dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio libcrc32c btrfs xor raid6_pq dm_flakey dm_mod crc32c_generic
acpi_cpufreq tpm_tis tpm_tis_core tpm ppdev parport_pc parport i2c_piix4 processor sg evdev i2c_core psmouse pcspkr serio_raw button loop autofs4 ext4 crc16
jbd2 mbcache sr_mod cdrom sd_mod ata_generic virtio_scsi ata_piix virtio_pci libata virtio_ring virtio e1000 scsi_mod floppy [last unloaded: scsi_debug]
[23114.616932] CPU: 9 PID: 4537 Comm: kworker/u32:8 Not tainted 4.9.0-rc7-btrfs-next-36+ #1
[23114.616932] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.9.1-0-gb3ef39f-prebuilt.qemu-project.org 04/01/2014
[23114.616932] Workqueue: btrfs-cache btrfs_cache_helper [btrfs]
[23114.616932] task:
ffff880221d45780 task.stack:
ffffc9000bc50000
[23114.616932] RIP: 0010:[<
ffffffffa037c1bf>] [<
ffffffffa037c1bf>] btrfs_queue_work+0x2c/0x190 [btrfs]
[23114.616932] RSP: 0018:
ffff88023f443d60 EFLAGS:
00010246
[23114.616932] RAX:
0000000000000000 RBX:
6b6b6b6b6b6b6b6b RCX:
0000000000000102
[23114.616932] RDX:
ffffffffa0419000 RSI:
ffff88011df534f0 RDI:
ffff880101f01c00
[23114.616932] RBP:
ffff88023f443d80 R08:
00000000000f7000 R09:
000000000000ffff
[23114.616932] R10:
ffff88023f443d48 R11:
0000000000001000 R12:
ffff88011df534f0
[23114.616932] R13:
ffff880135963868 R14:
0000000000001000 R15:
0000000000001000
[23114.616932] FS:
0000000000000000(0000) GS:
ffff88023f440000(0000) knlGS:
0000000000000000
[23114.616932] CS: 0010 DS: 0000 ES: 0000 CR0:
0000000080050033
[23114.616932] CR2:
00007f0fb9f8e520 CR3:
0000000001a0b000 CR4:
00000000000006e0
[23114.616932] Stack:
[23114.616932]
ffff880101f01c00 ffff88011df534f0 ffff880135963868 0000000000001000
[23114.616932]
ffff88023f443da0 ffffffffa03470af ffff880149b37200 ffff880135963868
[23114.616932]
ffff88023f443db8 ffffffff8125293c ffff880149b37200 ffff88023f443de0
[23114.616932] Call Trace:
[23114.616932] <IRQ> [23114.616932] [<
ffffffffa03470af>] end_workqueue_bio+0xd5/0xda [btrfs]
[23114.616932] [<
ffffffff8125293c>] bio_endio+0x54/0x57
[23114.616932] [<
ffffffffa0377929>] btrfs_end_bio+0xf7/0x106 [btrfs]
[23114.616932] [<
ffffffff8125293c>] bio_endio+0x54/0x57
[23114.616932] [<
ffffffff8125955f>] blk_update_request+0x21a/0x30f
[23114.616932] [<
ffffffffa0022316>] scsi_end_request+0x31/0x182 [scsi_mod]
[23114.616932] [<
ffffffffa00235fc>] scsi_io_completion+0x1ce/0x4c8 [scsi_mod]
[23114.616932] [<
ffffffffa001ba9d>] scsi_finish_command+0x104/0x10d [scsi_mod]
[23114.616932] [<
ffffffffa002311f>] scsi_softirq_done+0x101/0x10a [scsi_mod]
[23114.616932] [<
ffffffff8125fbd9>] blk_done_softirq+0x82/0x8d
[23114.616932] [<
ffffffff814c8a4b>] __do_softirq+0x1ab/0x412
[23114.616932] [<
ffffffff8105b01d>] irq_exit+0x49/0x99
[23114.616932] [<
ffffffff81035135>] smp_call_function_single_interrupt+0x24/0x26
[23114.616932] [<
ffffffff814c7ec9>] call_function_single_interrupt+0x89/0x90
[23114.616932] <EOI> [23114.616932] [<
ffffffffa0023262>] ? scsi_request_fn+0x13a/0x2a1 [scsi_mod]
[23114.616932] [<
ffffffff814c5966>] ? _raw_spin_unlock_irq+0x2c/0x4a
[23114.616932] [<
ffffffff814c596c>] ? _raw_spin_unlock_irq+0x32/0x4a
[23114.616932] [<
ffffffff814c5966>] ? _raw_spin_unlock_irq+0x2c/0x4a
[23114.616932] [<
ffffffffa0023262>] scsi_request_fn+0x13a/0x2a1 [scsi_mod]
[23114.616932] [<
ffffffff8125590e>] __blk_run_queue_uncond+0x22/0x2b
[23114.616932] [<
ffffffff81255930>] __blk_run_queue+0x19/0x1b
[23114.616932] [<
ffffffff8125ab01>] blk_queue_bio+0x268/0x282
[23114.616932] [<
ffffffff81258f44>] generic_make_request+0xbd/0x160
[23114.616932] [<
ffffffff812590e7>] submit_bio+0x100/0x11d
[23114.616932] [<
ffffffff81298603>] ? __this_cpu_preempt_check+0x13/0x15
[23114.616932] [<
ffffffff812a1805>] ? __percpu_counter_add+0x8e/0xa7
[23114.616932] [<
ffffffffa03bfd47>] btrfsic_submit_bio+0x1a/0x1d [btrfs]
[23114.616932] [<
ffffffffa0377db2>] btrfs_map_bio+0x1f4/0x26d [btrfs]
[23114.616932] [<
ffffffffa0348a33>] btree_submit_bio_hook+0x74/0xbf [btrfs]
[23114.616932] [<
ffffffffa03489bf>] ? btrfs_wq_submit_bio+0x160/0x160 [btrfs]
[23114.616932] [<
ffffffffa03697a9>] submit_one_bio+0x6b/0x89 [btrfs]
[23114.616932] [<
ffffffffa036f5be>] read_extent_buffer_pages+0x170/0x1ec [btrfs]
[23114.616932] [<
ffffffffa03471fa>] ? free_root_pointers+0x64/0x64 [btrfs]
[23114.616932] [<
ffffffffa0348adf>] readahead_tree_block+0x3f/0x4c [btrfs]
[23114.616932] [<
ffffffffa032e115>] read_block_for_search.isra.20+0x1ce/0x23d [btrfs]
[23114.616932] [<
ffffffffa032fab8>] btrfs_search_slot+0x65f/0x774 [btrfs]
[23114.616932] [<
ffffffffa036eff1>] ? free_extent_buffer+0x73/0x7e [btrfs]
[23114.616932] [<
ffffffffa0331ba4>] btrfs_next_old_leaf+0xa1/0x33c [btrfs]
[23114.616932] [<
ffffffffa0331e4f>] btrfs_next_leaf+0x10/0x12 [btrfs]
[23114.616932] [<
ffffffffa0336aa6>] caching_thread+0x22d/0x416 [btrfs]
[23114.616932] [<
ffffffffa037bce9>] btrfs_scrubparity_helper+0x187/0x3b6 [btrfs]
[23114.616932] [<
ffffffffa037c036>] btrfs_cache_helper+0xe/0x10 [btrfs]
[23114.616932] [<
ffffffff8106cf96>] process_one_work+0x273/0x4e4
[23114.616932] [<
ffffffff8106d6db>] worker_thread+0x1eb/0x2ca
[23114.616932] [<
ffffffff8106d4f0>] ? rescuer_thread+0x2b6/0x2b6
[23114.616932] [<
ffffffff81072a81>] kthread+0xd5/0xdd
[23114.616932] [<
ffffffff810729ac>] ? __kthread_unpark+0x5a/0x5a
[23114.616932] [<
ffffffff814c6257>] ret_from_fork+0x27/0x40
[23114.616932] Code: 1f 44 00 00 55 48 89 e5 41 56 41 55 41 54 53 49 89 f4 48 8b 46 70 a8 04 74 09 48 8b 5f 08 48 85 db 75 03 48 8b 1f 49 89 5c 24 68 <83> 7b
64 ff 74 04 f0 ff 43 58 49 83 7c 24 08 00 74 2c 4c 8d 6b
[23114.616932] RIP [<
ffffffffa037c1bf>] btrfs_queue_work+0x2c/0x190 [btrfs]
[23114.616932] RSP <
ffff88023f443d60>
[23114.689493] ---[ end trace
6e48b6bc707ca34b ]---
[23114.690166] Kernel panic - not syncing: Fatal exception in interrupt
[23114.691283] Kernel Offset: disabled
[23114.691918] ---[ end Kernel panic - not syncing: Fatal exception in interrupt
The following diagram shows the sequence of operations that lead to the
use-after-free problem from the above trace:
CPU 1 CPU 2 CPU 3
caching_thread()
close_ctree()
btrfs_stop_all_workers()
btrfs_destroy_workqueue(
fs_info->endio_meta_workers)
btrfs_search_slot()
read_block_for_search()
readahead_tree_block()
read_extent_buffer_pages()
submit_one_bio()
btree_submit_bio_hook()
btrfs_bio_wq_end_io()
--> sets the bio's
bi_end_io callback
to end_workqueue_bio()
--> bio is submitted
bio completes
and its bi_end_io callback
is invoked
--> end_workqueue_bio()
--> attempts to queue
a task on fs_info->endio_meta_workers
btrfs_destroy_workqueue(
fs_info->caching_workers)
So fix this by destroying the queues used for metadata I/O tasks only
after destroying all the other queues.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Filipe Manana [Wed, 1 Feb 2017 22:39:50 +0000 (22:39 +0000)]
Btrfs: fix assertion failure when freeing block groups at close_ctree()
At close_ctree() we free the block groups and then only after we wait for
any running worker kthreads to finish and shutdown the workqueues. This
behaviour is racy and it triggers an assertion failure when freeing block
groups because while we are doing it we can have for example a block group
caching kthread running, and in that case the block group's reference
count can still be greater than 1 by the time we assert its reference count
is 1, leading to an assertion failure:
[19041.198004] assertion failed: atomic_read(&block_group->count) == 1, file: fs/btrfs/extent-tree.c, line: 9799
[19041.200584] ------------[ cut here ]------------
[19041.201692] kernel BUG at fs/btrfs/ctree.h:3418!
[19041.202830] invalid opcode: 0000 [#1] PREEMPT SMP
[19041.203929] Modules linked in: btrfs xor raid6_pq dm_flakey dm_mod crc32c_generic ppdev sg psmouse acpi_cpufreq pcspkr parport_pc evdev tpm_tis parport tpm_tis_core i2c_piix4 i2c_core tpm serio_raw processor button loop autofs4 ext4 crc16 jbd2 mbcache sr_mod cdrom sd_mod ata_generic virtio_scsi ata_piix virtio_pci libata virtio_ring virtio e1000 scsi_mod floppy [last unloaded: btrfs]
[19041.208082] CPU: 6 PID: 29051 Comm: umount Not tainted 4.9.0-rc7-btrfs-next-36+ #1
[19041.208082] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.9.1-0-gb3ef39f-prebuilt.qemu-project.org 04/01/2014
[19041.208082] task:
ffff88015f028980 task.stack:
ffffc9000ad34000
[19041.208082] RIP: 0010:[<
ffffffffa03e319e>] [<
ffffffffa03e319e>] assfail.constprop.41+0x1c/0x1e [btrfs]
[19041.208082] RSP: 0018:
ffffc9000ad37d60 EFLAGS:
00010286
[19041.208082] RAX:
0000000000000061 RBX:
ffff88015ecb4000 RCX:
0000000000000001
[19041.208082] RDX:
ffff88023f392fb8 RSI:
ffffffff817ef7ba RDI:
00000000ffffffff
[19041.208082] RBP:
ffffc9000ad37d60 R08:
0000000000000001 R09:
0000000000000000
[19041.208082] R10:
ffffc9000ad37cb0 R11:
ffffffff82f2b66d R12:
ffff88023431d170
[19041.208082] R13:
ffff88015ecb40c0 R14:
ffff88023431d000 R15:
ffff88015ecb4100
[19041.208082] FS:
00007f44f3d42840(0000) GS:
ffff88023f380000(0000) knlGS:
0000000000000000
[19041.208082] CS: 0010 DS: 0000 ES: 0000 CR0:
0000000080050033
[19041.208082] CR2:
00007f65d623b000 CR3:
00000002166f2000 CR4:
00000000000006e0
[19041.208082] Stack:
[19041.208082]
ffffc9000ad37d98 ffffffffa035989f ffff88015ecb4000 ffff88015ecb5630
[19041.208082]
ffff88014f6be000 0000000000000000 00007ffcf0ba6a10 ffffc9000ad37df8
[19041.208082]
ffffffffa0368cd4 ffff88014e9658e0 ffffc9000ad37e08 ffffffff811a634d
[19041.208082] Call Trace:
[19041.208082] [<
ffffffffa035989f>] btrfs_free_block_groups+0x17f/0x392 [btrfs]
[19041.208082] [<
ffffffffa0368cd4>] close_ctree+0x1c5/0x2e1 [btrfs]
[19041.208082] [<
ffffffff811a634d>] ? evict_inodes+0x132/0x141
[19041.208082] [<
ffffffffa034356d>] btrfs_put_super+0x15/0x17 [btrfs]
[19041.208082] [<
ffffffff8118fc32>] generic_shutdown_super+0x6a/0xeb
[19041.208082] [<
ffffffff8119004f>] kill_anon_super+0x12/0x1c
[19041.208082] [<
ffffffffa0343370>] btrfs_kill_super+0x16/0x21 [btrfs]
[19041.208082] [<
ffffffff8118fad1>] deactivate_locked_super+0x3b/0x68
[19041.208082] [<
ffffffff8118fb34>] deactivate_super+0x36/0x39
[19041.208082] [<
ffffffff811a9946>] cleanup_mnt+0x58/0x76
[19041.208082] [<
ffffffff811a99a2>] __cleanup_mnt+0x12/0x14
[19041.208082] [<
ffffffff81071573>] task_work_run+0x6f/0x95
[19041.208082] [<
ffffffff81001897>] prepare_exit_to_usermode+0xa3/0xc1
[19041.208082] [<
ffffffff81001a23>] syscall_return_slowpath+0x16e/0x1d2
[19041.208082] [<
ffffffff814c607d>] entry_SYSCALL_64_fastpath+0xab/0xad
[19041.208082] Code: c7 ae a0 3e a0 48 89 e5 e8 4e 74 d4 e0 0f 0b 55 89 f1 48 c7 c2 0b a4 3e a0 48 89 fe 48 c7 c7 a4 a6 3e a0 48 89 e5 e8 30 74 d4 e0 <0f> 0b 55 31 d2 48 89 e5 e8 d5 b9 f7 ff 5d c3 48 63 f6 55 31 c9
[19041.208082] RIP [<
ffffffffa03e319e>] assfail.constprop.41+0x1c/0x1e [btrfs]
[19041.208082] RSP <
ffffc9000ad37d60>
[19041.279264] ---[ end trace
23330586f16f064d ]---
This started happening as of kernel 4.8, since commit
f3bca8028bd9
("Btrfs: add ASSERT for block group's memory leak") introduced these
assertions.
So fix this by freeing the block groups only after waiting for all
worker kthreads to complete and shutdown the workqueues.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Filipe Manana [Wed, 1 Feb 2017 14:58:02 +0000 (14:58 +0000)]
Btrfs: do not create explicit holes when replaying log tree if NO_HOLES enabled
We log holes explicitly by using file extent items, however when replaying
a log tree, if a logged file extent item corresponds to a hole and the
NO_HOLES feature is enabled we do not need to copy the file extent item
into the fs/subvolume tree, as the absence of such file extent items is
the purpose of the NO_HOLES feature. So skip the copying of file extent
items representing holes when the NO_HOLES feature is enabled.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Robbie Ko [Fri, 7 Oct 2016 02:01:29 +0000 (10:01 +0800)]
Btrfs: fix leak of subvolume writers counter
When falling back from a nocow write to a regular cow write, we were
leaking the subvolume writers counter in 2 situations, preventing
snapshot creation from ever completing in the future, as it waits
for that counter to go down to zero before the snapshot creation
starts.
Signed-off-by: Robbie Ko <robbieko@synology.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
[Improved changelog and subject]
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Filipe Manana [Sat, 28 Jan 2017 01:47:56 +0000 (01:47 +0000)]
Btrfs: bulk delete checksum items in the same leaf
Very often we have the checksums for an extent spread in multiple items
in the checksums tree, and currently the algorithm to delete them starts
by looking for them one by one and then deleting them one by one, which
is not optimal since each deletion involves shifting all the other items
in the leaf and when the leaf reaches some low threshold, to move items
off the leaf into its left and right neighbor leafs. Also, after each
item deletion we release our search path and start a new search for other
checksums items.
So optimize this by deleting in bulk all the items in the same leaf that
contain checksums for the extent being freed.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Robbie Ko [Thu, 5 Jan 2017 08:24:58 +0000 (16:24 +0800)]
Btrfs: incremental send, do not issue invalid rmdir operations
When both the parent and send snapshots have a directory inode with the
same number but different generations (therefore they are different
inodes) and both have an entry with the same name, an incremental send
stream will contain an invalid rmdir operation that refers to the
orphanized name of the inode from the parent snapshot.
The following example scenario shows how this happens.
Parent snapshot:
.
|---- d259_old/ (ino 259, gen 9)
| |---- d1/ (ino 258, gen 9)
|
|---- f (ino 257, gen 9)
Send snapshot:
.
|---- d258/ (ino 258, gen 7)
|---- d259/ (ino 259, gen 7)
|---- d1/ (ino 257, gen 7)
When the kernel is processing inode 258 it notices that in both snapshots
there is an inode numbered 259 that is a parent of an inode 258. However
it ignores the fact that the inodes numbered 259 have different generations
in both snapshots, which means they are effectively different inodes.
Then it checks that both inodes 259 have a dentry named "d1" and because
of that it issues a rmdir operation with orphanized name of the inode 258
from the parent snapshot. This happens at send.c:process_record_refs(),
which calls send.c:did_overwrite_first_ref() that returns true and because
of that later on at process_recorded_refs() such rmdir operation is issued
because the inode being currently processed (258) is a directory and it
was deleted in the send snapshot (and replaced with another inode that has
the same number and is a directory too).
Fix this issue by comparing the generations of parent directory inodes
that have the same number and make send.c:did_overwrite_first_ref() when
the generations are different.
The following steps reproduce the problem.
$ mkfs.btrfs -f /dev/sdb
$ mount /dev/sdb /mnt
$ touch /mnt/f
$ mkdir /mnt/d1
$ mkdir /mnt/d259_old
$ mv /mnt/d1 /mnt/d259_old/d1
$ btrfs subvolume snapshot -r /mnt /mnt/snap1
$ btrfs send /mnt/snap1 -f /tmp/1.snap
$ umount /mnt
$ mkfs.btrfs -f /dev/sdc
$ mount /dev/sdc /mnt
$ mkdir /mnt/d1
$ mkdir /mnt/dir258
$ mkdir /mnt/dir259
$ mv /mnt/d1 /mnt/dir259/d1
$ btrfs subvolume snapshot -r /mnt /mnt/snap2
$ btrfs receive /mnt/ -f /tmp/1.snap
# Take note that once the filesystem is created, its current
# generation has value 7 so the inodes from the second snapshot all have
# a generation value of 7. And after receiving the first snapshot
# the filesystem is at a generation value of 10, because the call to
# create the second snapshot bumps the generation to 8 (the snapshot
# creation ioctl does a transaction commit), the receive command calls
# the snapshot creation ioctl to create the first snapshot, which bumps
# the filesystem's generation to 9, and finally when the receive
# operation finishes it calls an ioctl to transition the first snapshot
# (snap1) from RW mode to RO mode, which does another transaction commit
# and bumps the filesystem's generation to 10. This means all the inodes
# in the first snapshot (snap1) have a generation value of 9.
$ rm -f /tmp/1.snap
$ btrfs send /mnt/snap1 -f /tmp/1.snap
$ btrfs send -p /mnt/snap1 /mnt/snap2 -f /tmp/2.snap
$ umount /mnt
$ mkfs.btrfs -f /dev/sdd
$ mount /dev/sdd /mnt
$ btrfs receive /mnt -f /tmp/1.snap
$ btrfs receive -vv /mnt -f /tmp/2.snap
receiving snapshot mysnap2 uuid=
9c03962f-f620-0047-9f98-
32e5a87116d9, ctransid=7 parent_uuid=
d17a6e3f-14e5-df4f-be39-
a7951a5399aa, parent_ctransid=9
utimes
unlink f
mkdir o257-7-0
mkdir o259-7-0
rename o257-7-0 -> o259-7-0/d1
chown o259-7-0/d1 - uid=0, gid=0
chmod o259-7-0/d1 - mode=0755
utimes o259-7-0/d1
rmdir o258-9-0
ERROR: rmdir o258-9-0 failed: No such file or directory
Signed-off-by: Robbie Ko <robbieko@synology.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
[Rewrote changelog to be more precise and clear]
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Filipe Manana [Wed, 11 Jan 2017 02:15:39 +0000 (02:15 +0000)]
Btrfs: incremental send, do not delay rename when parent inode is new
When we are checking if we need to delay the rename operation for an
inode we not checking if a parent inode that exists in the send and
parent snapshots is really the same inode or not, that is, we are not
comparing the generation number of the parent inode in the send and
parent snapshots. Not only this results in unnecessarily delaying a
rename operation but also can later on make us generate an incorrect
name for a new inode in the send snapshot that has the same number
as another inode in the parent snapshot but a different generation.
Here follows an example where this happens.
Parent snapshot:
. (ino 256, gen 3)
|--- dir258/ (ino 258, gen 7)
| |--- dir257/ (ino 257, gen 7)
|
|--- dir259/ (ino 259, gen 7)
Send snapshot:
. (ino 256, gen 3)
|--- file258 (ino 258, gen 10)
|
|--- new_dir259/ (ino 259, gen 10)
|--- dir257/ (ino 257, gen 7)
The following steps happen when computing the incremental send stream:
1) When processing inode 257, its new parent is created using its orphan
name (o257-21-0), and the rename operation for inode 257 is delayed
because its new parent (inode 259) was not yet processed - this
decision to delay the rename operation does not make much sense
because the inode 259 in the send snapshot is a new inode, it's not
the same as inode 259 in the parent snapshot.
2) When processing inode 258 we end up delaying its rmdir operation,
because inode 257 was not yet renamed (moved away from the directory
inode 258 represents). We also create the new inode 258 using its
orphan name "o258-10-0", then rename it to its final name of "file258"
and then issue a truncate operation for it. However this truncate
operation contains an incorrect name, which corresponds to the orphan
name and not to the final name, which makes the receiver fail. This
happens because when we attempt to compute the inode's current name
we verify that there's another inode with the same number (258) that
has its rmdir operation pending and because of that we generate an
orphan name for the new inode 258 (we do this in the function
get_cur_path()).
Fix this by not delayed the rename operation of an inode if it has parents
with the same number but different generations in both snapshots.
The following steps reproduce this example scenario.
$ mkfs.btrfs -f /dev/sdb
$ mount /dev/sdb /mnt
$ mkdir /mnt/dir257
$ mkdir /mnt/dir258
$ mkdir /mnt/dir259
$ mv /mnt/dir257 /mnt/dir258/dir257
$ btrfs subvolume snapshot -r /mnt /mnt/snap1
$ mv /mnt/dir258/dir257 /mnt/dir257
$ rmdir /mnt/dir258
$ rmdir /mnt/dir259
# Remount the filesystem so that the next created inodes will have the
# numbers 258 and 259. This is because when a filesystem is mounted,
# btrfs sets the subvolume's inode counter to a value corresponding to
# the highest inode number in the subvolume plus 1. This inode counter
# is used to assign a unique number to each new inode and it's
# incremented by 1 after very inode creation.
# Note: we unmount and then mount instead of doing a mount with
# "-o remount" because otherwise the inode counter remains at value 260.
$ umount /mnt
$ mount /dev/sdb /mnt
$ touch /mnt/file258
$ mkdir /mnt/new_dir259
$ mv /mnt/dir257 /mnt/new_dir259/dir257
$ btrfs subvolume snapshot -r /mnt /mnt/snap2
$ btrfs send /mnt/snap1 -f /tmp/1.snap
$ btrfs send -p /mnt/snap1 /mnt/snap2 -f /tmp/2.snap
$ umount /mnt
$ mkfs.btrfs -f /dev/sdc
$ mount /dev/sdc /mnt
$ btrfs receive /mnt -f /tmo/1.snap
$ btrfs receive /mnt -f /tmo/2.snap -vv
receiving snapshot mysnap2 uuid=
e059b6d1-7f55-f140-8d7c-
9a3039d23c97, ctransid=10 parent_uuid=
77e98cb6-8762-814f-9e05-
e8ba877fc0b0, parent_ctransid=7
utimes
mkdir o259-10-0
rename dir258 -> o258-7-0
utimes
mkfile o258-10-0
rename o258-10-0 -> file258
utimes
truncate o258-10-0 size=0
ERROR: truncate o258-10-0 failed: No such file or directory
Reported-by: Robbie Ko <robbieko@synology.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Robbie Ko [Thu, 5 Jan 2017 08:24:55 +0000 (16:24 +0800)]
Btrfs: send, fix failure to rename top level inode due to name collision
Under certain situations, an incremental send operation can fail due to a
premature attempt to create a new top level inode (a direct child of the
subvolume/snapshot root) whose name collides with another inode that was
removed from the send snapshot.
Consider the following example scenario.
Parent snapshot:
. (ino 256, gen 8)
|---- a1/ (ino 257, gen 9)
|---- a2/ (ino 258, gen 9)
Send snapshot:
. (ino 256, gen 3)
|---- a2/ (ino 257, gen 7)
In this scenario, when receiving the incremental send stream, the btrfs
receive command fails like this (ran in verbose mode, -vv argument):
rmdir a1
mkfile o257-7-0
rename o257-7-0 -> a2
ERROR: rename o257-7-0 -> a2 failed: Is a directory
What happens when computing the incremental send stream is:
1) An operation to remove the directory with inode number 257 and
generation 9 is issued.
2) An operation to create the inode with number 257 and generation 7 is
issued. This creates the inode with an orphanized name of "o257-7-0".
3) An operation rename the new inode 257 to its final name, "a2", is
issued. This is incorrect because inode 258, which has the same name
and it's a child of the same parent (root inode 256), was not yet
processed and therefore no rmdir operation for it was yet issued.
The rename operation is issued because we fail to detect that the
name of the new inode 257 collides with inode 258, because their
parent, a subvolume/snapshot root (inode 256) has a different
generation in both snapshots.
So fix this by ignoring the generation value of a parent directory that
matches a root inode (number 256) when we are checking if the name of the
inode currently being processed collides with the name of some other
inode that was not yet processed.
We can achieve this scenario of different inodes with the same number but
different generation values either by mounting a filesystem with the inode
cache option (-o inode_cache) or by creating and sending snapshots across
different filesystems, like in the following example:
$ mkfs.btrfs -f /dev/sdb
$ mount /dev/sdb /mnt
$ mkdir /mnt/a1
$ mkdir /mnt/a2
$ btrfs subvolume snapshot -r /mnt /mnt/snap1
$ btrfs send /mnt/snap1 -f /tmp/1.snap
$ umount /mnt
$ mkfs.btrfs -f /dev/sdc
$ mount /dev/sdc /mnt
$ touch /mnt/a2
$ btrfs subvolume snapshot -r /mnt /mnt/snap2
$ btrfs receive /mnt -f /tmp/1.snap
# Take note that once the filesystem is created, its current
# generation has value 7 so the inode from the second snapshot has
# a generation value of 7. And after receiving the first snapshot
# the filesystem is at a generation value of 10, because the call to
# create the second snapshot bumps the generation to 8 (the snapshot
# creation ioctl does a transaction commit), the receive command calls
# the snapshot creation ioctl to create the first snapshot, which bumps
# the filesystem's generation to 9, and finally when the receive
# operation finishes it calls an ioctl to transition the first snapshot
# (snap1) from RW mode to RO mode, which does another transaction commit
# and bumps the filesystem's generation to 10.
$ rm -f /tmp/1.snap
$ btrfs send /mnt/snap1 -f /tmp/1.snap
$ btrfs send -p /mnt/snap1 /mnt/snap2 -f /tmp/2.snap
$ umount /mnt
$ mkfs.btrfs -f /dev/sdd
$ mount /dev/sdd /mnt
$ btrfs receive /mnt /tmp/1.snap
# Receive of snapshot snap2 used to fail.
$ btrfs receive /mnt /tmp/2.snap
Signed-off-by: Robbie Ko <robbieko@synology.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
[Rewrote changelog to be more precise and clear]
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Liu Bo [Tue, 21 Feb 2017 20:12:58 +0000 (12:12 -0800)]
Btrfs: use the correct type when creating cow dio extent
'BTRFS_ORDERED_REGULAR' was introduced for the cow case in patch
'Btrfs: specify a new ordered extent type for create_io_em',
but it missed the directIO cow case.
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: Chris Mason <clm@fb.com>
Filipe Manana [Tue, 21 Feb 2017 17:14:52 +0000 (17:14 +0000)]
Btrfs: fix deadlock between dedup on same file and starting writeback
If we are deduping two ranges of the same file we need to make sure that
we lock all pages in ascending order, that is, lock first the pages from
the range with lower offset and then the pages from the other range, as
otherwise we can deadlock with a concurrent task that is starting delalloc
(writeback). Example trace:
[74073.052218] INFO: task kworker/u32:10:17997 blocked for more than 120 seconds.
[74073.053889] Tainted: G W 4.9.0-rc7-btrfs-next-36+ #1
[74073.055071] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[74073.056696] kworker/u32:10 D 0 17997 2 0x00000000
[74073.058606] Workqueue: writeback wb_workfn (flush-btrfs-53176)
[74073.061370]
ffff880031e79858 ffff8802159d2580 ffff880237004580 ffff880031e79240
[74073.064784]
ffff88023f4978c0 ffffc9000817b638 ffffffff814c15e1 0000000000000000
[74073.068386]
ffff88023f4978d8 ffff88023f4978c0 000000000017b620 ffff880031e79240
[74073.071712] Call Trace:
[74073.072884] [<
ffffffff814c15e1>] ? __schedule+0x48f/0x6f4
[74073.075395] [<
ffffffff814c1c8b>] ? bit_wait+0x2f/0x2f
[74073.077511] [<
ffffffff814c18d2>] schedule+0x8c/0xa0
[74073.079440] [<
ffffffff814c4b36>] schedule_timeout+0x43/0xff
[74073.081637] [<
ffffffff8110953e>] ? time_hardirqs_on+0x9/0x14
[74073.083809] [<
ffffffff81095c67>] ? trace_hardirqs_on_caller+0x16/0x197
[74073.086314] [<
ffffffff810bde98>] ? timekeeping_get_ns+0x1e/0x32
[74073.100654] [<
ffffffff810be048>] ? ktime_get+0x41/0x52
[74073.102619] [<
ffffffff814c10f0>] io_schedule_timeout+0xa0/0x102
[74073.104771] [<
ffffffff814c10f0>] ? io_schedule_timeout+0xa0/0x102
[74073.106969] [<
ffffffff814c1ca6>] bit_wait_io+0x1b/0x39
[74073.108954] [<
ffffffff814c1fb8>] __wait_on_bit_lock+0x4f/0x99
[74073.110981] [<
ffffffff8112b692>] __lock_page+0x6b/0x6d
[74073.112833] [<
ffffffff8108ceb4>] ? autoremove_wake_function+0x3a/0x3a
[74073.115010] [<
ffffffffa031178b>] lock_page+0x2f/0x32 [btrfs]
[74073.116999] [<
ffffffffa0311d9f>] lock_delalloc_pages+0xc7/0x1a0 [btrfs]
[74073.119243] [<
ffffffffa0313d15>] find_lock_delalloc_range+0xc3/0x1a4 [btrfs]
[74073.121636] [<
ffffffffa0313e81>] writepage_delalloc.isra.31+0x8b/0x134 [btrfs]
[74073.124229] [<
ffffffffa0315d69>] __extent_writepage+0x1c1/0x2bf [btrfs]
[74073.126372] [<
ffffffffa03160f2>] extent_write_cache_pages.isra.30.constprop.49+0x28b/0x36c [btrfs]
[74073.129371] [<
ffffffffa03165b9>] extent_writepages+0x4b/0x5c [btrfs]
[74073.131440] [<
ffffffffa02fcb59>] ? insert_reserved_file_extent.constprop.42+0x261/0x261 [btrfs]
[74073.134303] [<
ffffffff811b4ce4>] ? writeback_sb_inodes+0xe0/0x4a1
[74073.136298] [<
ffffffffa02fab7f>] btrfs_writepages+0x28/0x2a [btrfs]
[74073.138248] [<
ffffffff81138200>] do_writepages+0x23/0x2c
[74073.139910] [<
ffffffff811b3cab>] __writeback_single_inode+0x105/0x6d2
[74073.142003] [<
ffffffff811b4e96>] writeback_sb_inodes+0x292/0x4a1
[74073.136298] [<
ffffffffa02fab7f>] btrfs_writepages+0x28/0x2a [btrfs]
[74073.138248] [<
ffffffff81138200>] do_writepages+0x23/0x2c
[74073.139910] [<
ffffffff811b3cab>] __writeback_single_inode+0x105/0x6d2
[74073.142003] [<
ffffffff811b4e96>] writeback_sb_inodes+0x292/0x4a1
[74073.143911] [<
ffffffff811b511b>] __writeback_inodes_wb+0x76/0xae
[74073.145787] [<
ffffffff811b53ca>] wb_writeback+0x1cc/0x4d7
[74073.147452] [<
ffffffff811b60cd>] wb_workfn+0x194/0x37d
[74073.149084] [<
ffffffff811b60cd>] ? wb_workfn+0x194/0x37d
[74073.150726] [<
ffffffff8106ce77>] ? process_one_work+0x154/0x4e4
[74073.152694] [<
ffffffff8106cf96>] process_one_work+0x273/0x4e4
[74073.154452] [<
ffffffff8106d6db>] worker_thread+0x1eb/0x2ca
[74073.156138] [<
ffffffff8106d4f0>] ? rescuer_thread+0x2b6/0x2b6
[74073.157837] [<
ffffffff81072a81>] kthread+0xd5/0xdd
[74073.159339] [<
ffffffff810729ac>] ? __kthread_unpark+0x5a/0x5a
[74073.161088] [<
ffffffff814c6257>] ret_from_fork+0x27/0x40
[74073.162680] INFO: lockdep is turned off.
[74073.163855] INFO: task do-dedup:30264 blocked for more than 120 seconds.
[74073.181180] Tainted: G W 4.9.0-rc7-btrfs-next-36+ #1
[74073.181180] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[74073.185296] fdm-stress D 0 30264 29974 0x00000000
[74073.186810]
ffff880089595118 ffff880211b8eac0 ffff880237030380 ffff880089594b00
[74073.188998]
ffff88023f2978c0 ffffc900063abb68 ffffffff814c15e1 0000000000000000
[74073.191070]
ffff88023f2978d8 ffff88023f2978c0 00000000003abb50 ffff880089594b00
[74073.193286] Call Trace:
[74073.193990] [<
ffffffff814c15e1>] ? __schedule+0x48f/0x6f4
[74073.195418] [<
ffffffff814c1c8b>] ? bit_wait+0x2f/0x2f
[74073.196796] [<
ffffffff814c18d2>] schedule+0x8c/0xa0
[74073.198163] [<
ffffffff814c4b36>] schedule_timeout+0x43/0xff
[74073.199621] [<
ffffffff81095df5>] ? trace_hardirqs_on+0xd/0xf
[74073.201100] [<
ffffffff810bde98>] ? timekeeping_get_ns+0x1e/0x32
[74073.202686] [<
ffffffff810be048>] ? ktime_get+0x41/0x52
[74073.204051] [<
ffffffff814c10f0>] io_schedule_timeout+0xa0/0x102
[74073.205585] [<
ffffffff814c10f0>] ? io_schedule_timeout+0xa0/0x102
[74073.207123] [<
ffffffff814c1ca6>] bit_wait_io+0x1b/0x39
[74073.208238] [<
ffffffff814c1fb8>] __wait_on_bit_lock+0x4f/0x99
[74073.208871] [<
ffffffff8112b692>] __lock_page+0x6b/0x6d
[74073.209430] [<
ffffffff8108ceb4>] ? autoremove_wake_function+0x3a/0x3a
[74073.210101] [<
ffffffff8112b800>] lock_page+0x2f/0x32
[74073.210636] [<
ffffffff8112c502>] pagecache_get_page+0x5e/0x153
[74073.211270] [<
ffffffffa03257eb>] gather_extent_pages+0x4e/0x109 [btrfs]
[74073.212166] [<
ffffffffa032a04c>] btrfs_dedupe_file_range+0x1e1/0x4dd [btrfs]
[74073.213257] [<
ffffffff8118d9b5>] vfs_dedupe_file_range+0x1c1/0x221
[74073.214086] [<
ffffffff8119e0c4>] do_vfs_ioctl+0x442/0x600
[74073.214767] [<
ffffffff811a7874>] ? rcu_read_unlock+0x5b/0x5d
[74073.215619] [<
ffffffff811a7953>] ? __fget+0x6b/0x77
[74073.216338] [<
ffffffff8119e2d9>] SyS_ioctl+0x57/0x79
[74073.217149] [<
ffffffff814c5fea>] entry_SYSCALL_64_fastpath+0x18/0xad
[74073.218102] [<
ffffffff81109552>] ? time_hardirqs_off+0x9/0x14
[74073.218968] [<
ffffffff810938ce>] ? trace_hardirqs_off_caller+0x1f/0xaa
[74073.219938] INFO: lockdep is turned off.
What happened was the following:
CPU 1 CPU 2
btrfs_dedupe_file_range()
--> using same inode as source
and target
--> src range is [768K, 1Mb[
--> dst range is [0, 256K[
btrfs_cmp_data_prepare()
--> calls gather_extent_pages()
for range [768K, 1Mb[ and
locks all pages in that range
do_writepages()
btrfs_writepages()
extent_writepages()
extent_write_cache_pages()
__extent_writepage()
writepage_delalloc()
find_lock_delalloc_range()
--> finds range [0, 1Mb[
lock_delalloc_pages()
--> locks all pages in the
range [0, 768K[
--> tries to lock page at
offset 768K
--> deadlock
--> calls gather_extent_pages()
to lock pages in the range
[0, 256K[
--> deadlock, task at CPU 1
already locked that
range and it's trying
to lock the range we
locked previously
So fix this by making sure that during a dedup we always lock first the
pages from the range with lower offset.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Chris Mason <clm@fb.com>
Jeff Mahoney [Wed, 15 Feb 2017 21:28:34 +0000 (16:28 -0500)]
btrfs: use btrfs_debug instead of pr_debug in transaction abort
Commit
e5d6b12fe14 (Btrfs: don't WARN() in btrfs_transaction_abort() for
IO errors) added a pr_debug call to be printed when a transaction is
aborted with -EIO instead of WARN. btrfs_debug prints which file system
the message is associated with so let's use that instead.
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Jeff Mahoney [Wed, 15 Feb 2017 21:28:32 +0000 (16:28 -0500)]
btrfs: btrfs_truncate_free_space_cache always allocates path
btrfs_truncate_free_space_cache always allocates a btrfs_path structure
but only uses it when the caller passes a block group. Let's move the
allocation and free into the conditional.
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Jeff Mahoney [Wed, 15 Feb 2017 21:28:30 +0000 (16:28 -0500)]
btrfs: free-space-cache, clean up unnecessary root arguments
The free space cache APIs accept a root but always use the tree root.
Also, btrfs_truncate_free_space_cache accepts a root AND an inode but
the inode always points to the root anyway, so let's just pass the inode.
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Jeff Mahoney [Wed, 15 Feb 2017 21:28:29 +0000 (16:28 -0500)]
btrfs: convert btrfs_inc_block_group_ro to accept fs_info
btrfs_inc_block_group_ro is either passed the extent root or the dev
root, but it doesn't do anything with the dev tree. Let's convert
to passing an fs_info and using the extent root.
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Jeff Mahoney [Wed, 15 Feb 2017 21:28:28 +0000 (16:28 -0500)]
btrfs: flush_space always takes fs_info->fs_root
We don't need to pass a root to flush_space since it always uses
the fs_root.
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Jeff Mahoney [Wed, 15 Feb 2017 21:28:27 +0000 (16:28 -0500)]
btrfs: pass fs_info to (more) routines that are only called with extent_root
Outside of interactions with qgroups, the roots passed in extent-tree.c
are usually passed to ensure that we don't do refcounts on log trees or
to get the allocation profile for an allocation request. Otherwise, it
operates on the extent root. This patch converts some more routines in
extent-tree.c that are always called with the extent root to accept
an fs_info instead.
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 15 Feb 2017 02:43:03 +0000 (10:43 +0800)]
btrfs: qgroup: Move half of the qgroup accounting time out of commit trans
Just as Filipe pointed out, the most time consuming parts of qgroup are
btrfs_qgroup_account_extents() and
btrfs_qgroup_prepare_account_extents().
Which both call btrfs_find_all_roots() to get old_roots and new_roots
ulist.
What makes things worse is, we're calling that expensive
btrfs_find_all_roots() at transaction committing time with
TRANS_STATE_COMMIT_DOING, which will blocks all incoming transaction.
Such behavior is necessary for @new_roots search as current
btrfs_find_all_roots() can't do it correctly so we do call it just
before switch commit roots.
However for @old_roots search, it's not necessary as such search is
based on commit_root, so it will always be correct and we can move it
out of transaction committing.
This patch moves the @old_roots search part out of
commit_transaction(), so in theory we can half the time qgroup time
consumption at commit_transaction().
But please note that, this won't speedup qgroup overall, the total time
consumption is still the same, just reduce the performance stall.
Cc: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 10 Feb 2017 19:30:23 +0000 (20:30 +0100)]
btrfs: remove unused parameter from adjust_slots_upwards
Never used.
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 10 Feb 2017 19:26:24 +0000 (20:26 +0100)]
btrfs: remove unused parameters from __btrfs_write_out_cache
Both unused after the call to update_cache_item has been moved to
__btrfs_wait_cache_io.
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 10 Feb 2017 19:23:00 +0000 (20:23 +0100)]
btrfs: remove unused parameter from cleanup_write_cache_enospc
bitmap_list is unused since the io_ctl framework.
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
projects / platform / kernel / linux-exynos.git / log