multipath: add followover failback mode

multipath: add followover failback mode

This patch adds a new failback mode, followover, to deal with multiple
computers accessing the same active/passive storage devices.  In these
cases, if only one node loses access to the primary paths, it will
force a trespass to the secondary paths.  If the nodes are configured
with immediate failback, the other nodes with trespass back to the
primary paths, and the machines will ping-pong the storage. If the
nodes are configured with manual failback, this won't happen. However
when the primary path is restored on the node that lost access to it,
the nodes won't automatically failback to it.  In followover mode, they
will.

Followover mode works by only failing back when a path comes back online
from a pathgroup that previously had no working paths.  For this to
work, the paths need an additional attribute, chkrstate. This is just like
the path state, except it is not updated when the paths state is changed
by the kernel, only when the path checker function sees that the path is
down.  This is necessary because when a trespass occurs, all the outstanding
IO to the previously active paths will fail, and the kernel will mark the
path as down.  But for failback to happen in followover mode, the paths must
actually be down, not just in a ghost state.

Signed-off-by: Benjamin Marzinski <bmarzins@redhat.com>