Btrfs-progs: make scrub IO priority configurable

Btrfs-progs: make scrub IO priority configurable

The btrfs tool is changed in order to support command line parameters
to configure the IO priority of the scrub tasks. Also the default is
changed. The default IO priority for scrub is the idle class now.

The behavior is the same as when one would type
'ionice ... btrfs scrub start ...' or 'ionice ... btrfs scrub resume ...'
(without this patch applied).
The only reason for adding this to the btrfs tool is that it was not
documented and not obvious that it worked like this, that all internal
scrub tasks inherited the IO priority values of the btrfs tool that is
starting or resuming the scrub operation.

Note that after applying the patch it is no longer possible to set
the IO priority using ionice since the btrfs tool always configures
the priority in order to run in the idle class by default.

Some basic performance measurements have been done with the goal to
measure which IO priority for scrub gives the best overall disk data
throughput. The kernel was configured to use the CFQ IO scheduler
with default configuration and without support for throttling. The
summary is, that the more the disk head movements are avoided, the
faster the overall disk transfer capacity is, which is not really a
big surprise. Therefore it makes sense that the best data throughput
was measured setting the scrub IO priority and the scrub readahead
IO priority to the idle class priority. Running with idle class IO
priority means that scrub and scrub readahead IO is paused while
other tasks access the disk. Doing the tasks one after the other
instead of concurrently avoids many disk head movements. The
overall data throughput of rotating disks is improved this way.

However, if it is desired to have the scrub task done within a
reasonable time, and if at the same time the filesystem is heavily
loaded, the idle IO priority should be avoided. Otherwise the scrub
operation will never take place and thus never terminate.

The best effort IO priority class with the subclass 7 (the lowest
one in the best effort class) is recommended in the case of always
heavily loaded hard disks. If the filesystem is not loaded all the
time and leaves some idle slots for scrub, the idle class IO priority
is recommended. The idle class now is the default if the scrub
operation is started with the btrfs-progs tools.

Note that the patch that sets the scrub readahead IO priority to the
idle class is a seperate patch, this needs to be done in the kernel.

Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de>