-.TH MULTIPATH.CONF 5 "30 November 2006"
+.\" ----------------------------------------------------------------------------
+.\" Update the date below if you make any significant change.
+.\" Make sure there are no errors with:
+.\" groff -z -wall -b -e -t multipath/multipath.conf.5
+.\" man --warnings -E UTF-8 -l -Tutf8 -Z multipath/multipath.conf.5 >/dev/null
+.\"
+.\" TODO: Look for XXX and ???
+.\"
+.\" ----------------------------------------------------------------------------
+.
+.TH MULTIPATH.CONF 5 2017-08-18 Linux
+.
+.
+.\" ----------------------------------------------------------------------------
.SH NAME
-multipath.conf \- multipath daemon configuration file
+.\" ----------------------------------------------------------------------------
+.
+multipath.conf \- multipath daemon configuration file.
+.
+.
+.\" ----------------------------------------------------------------------------
.SH DESCRIPTION
-.B "multipath.conf"
+.\" ----------------------------------------------------------------------------
+.
+.B "/etc/multipath.conf"
is the configuration file for the multipath daemon. It is used to
overwrite the built-in configuration table of \fBmultipathd\fP.
Any line whose first non-white-space character is a '#' is considered
a comment line. Empty lines are ignored.
+.PP
+Currently used multipathd configuration can be displayed with the \fBmultipath -t\fR
+or \fBmultipathd show config\fR command.
+.
+.
+.\" ----------------------------------------------------------------------------
.SH SYNTAX
+.\" ----------------------------------------------------------------------------
+.
The configuration file contains entries of the form:
.RS
.nf
<attribute> <value>
.I "..."
.RE
+.ft B
}
.RE
+.ft B
}
.ft R
.fi
.LP
Each \fIsection\fP contains one or more attributes or subsections. The
recognized keywords for attributes or subsections depend on the
-section in which they occor.
+section in which they occur.
.LP
+.
+.
The following \fIsection\fP keywords are recognized:
.TP 17
.B defaults
This section defines default values for attributes which are used
-whenever no specific setting is given.
+whenever no values are given in the appropriate device or multipath
+sections.
.TP
.B blacklist
This section defines which devices should be excluded from the
.B blacklist_exceptions
This section defines which devices should be included in the
multipath topology discovery, despite being listed in the
-.I blacklist
-section.
+\fIblacklist\fR section.
.TP
.B multipaths
This section defines the multipath topologies. They are indexed by a
-\fIWorld Wide Identifier\fR(wwid), which is the result of the
-\fIgetuid_callout\fR program.
+\fIWorld Wide Identifier\fR(WWID). For details on the WWID generation
+see section \fIWWID generation\fR below.
.TP
.B devices
This section defines the device-specific settings.
+.TP
+.B overrides
+This section defines values for attributes that should override the
+device-specific settings for all devices.
.RE
.LP
+.
+.
+.\" ----------------------------------------------------------------------------
.SH "defaults section"
-The
-.B defaults
-section recognizes the following keywords:
+.\" ----------------------------------------------------------------------------
+.
+The \fIdefaults\fR section recognizes the following keywords:
+.
+.
.TP 17
+.B verbosity
+Default verbosity. Higher values increase the verbosity level. Valid
+levels are between 0 and 6.
+.RS
+.TP
+The default is: \fB2\fR
+.RE
+.
+.
+.TP
.B polling_interval
-interval between two path checks in seconds; default is
-.I 5
+Interval between two path checks in seconds. For properly functioning paths,
+the interval between checks will gradually increase to \fImax_polling_interval\fR.
+This value will be overridden by the \fIWatchdogSec\fR
+setting in the multipathd.service definition if systemd is used.
+.RS
+.TP
+The default is: \fB5\fR
+.RE
+.
+.
+.TP
+.B max_polling_interval
+Maximal interval between two path checks in seconds.
+.RS
+.TP
+The default is: \fB4 * polling_interval\fR
+.RE
+.
+.
+.TP
+.B reassign_maps
+Enable reassigning of device-mapper maps. With this option multipathd
+will remap existing device-mapper maps to always point to multipath
+device, not the underlying block devices. Possible values are
+\fIyes\fR and \fIno\fR.
+.RS
+.TP
+The default is: \fBno\fR
+.RE
+.
+.
+.TP
+.B multipath_dir
+Directory where the dynamic shared objects are stored. Defined at compile time,
+commonly \fI/lib64/multipath/\fR or \fI/lib/multipath/\fR.
+.RS
+.TP
+The default is: \fB<system dependent>\fR
+.RE
+.
+.
+.TP
+.B path_selector
+The default path selector algorithm to use; they are offered by the
+kernel multipath target. There are three selector algorithms:
+.RS
+.TP 12
+.I "round-robin 0"
+Loop through every path in the path group, sending the same amount of I/O to
+each. Some aspects of behavior can be controlled with the attributes:
+\fIrr_min_io\fR, \fIrr_min_io_rq\fR and \fIrr_weight\fR.
+.TP
+.I "queue-length 0"
+(Since 2.6.31 kernel) Choose the path for the next bunch of I/O based on the amount
+of outstanding I/O to the path.
+.TP
+.I "service-time 0"
+(Since 2.6.31 kernel) Choose the path for the next bunch of I/O based on the amount
+of outstanding I/O to the path and its relative throughput.
+.TP
+The default is: \fBservice-time 0\fR
+.RE
+.
+.
+.TP
+.B path_grouping_policy
+The default path grouping policy to apply to unspecified
+multipaths. Possible values are:
+.RS
+.TP 12
+.I failover
+One path per priority group.
+.TP
+.I multibus
+All paths in one priority group.
+.TP
+.I group_by_serial
+One priority group per serial number.
+.TP
+.I group_by_prio
+One priority group per priority value. Priorities are determined by
+callout programs specified as a global, per-controller or
+per-multipath option in the configuration file.
+.TP
+.I group_by_node_name
+One priority group per target node name. Target node names are fetched
+in \fI/sys/class/fc_transport/target*/node_name\fR.
+.TP
+The default is: \fBfailover\fR
+.RE
+.
+.
+.TP
+.B uid_attrs
+The udev attribute providing a unique path identifier for corresponding
+type of path devices. If this field is configured and matched with type
+of device, it would override any other methods providing for device
+unique identifier in config file, and it would activate merging uevents
+according to the identifier to promote effiecncy in processing uevents.
+It has no default value, if you want to identify path by udev attribute
+and to activate merging uevents for SCSI and DASD devices, you can set
+its value as: \fIuid_attrs "sd:ID_SERIAL dasd:ID_UID"\fR.
+.RS
+.TP
+The default is: \fB<unset>\fR
+.RE
+.
+.
+.TP
+.B uid_attribute
+The udev attribute providing a unique path identifier.
+.RS
+.TP
+The default is: for SCSI devices \fBID_SERIAL\fR
+.TP
+The default is: for DASD devices \fBID_UID\fR
+.TP
+The default is: for NVME devices \fBID_WWN\fR
+.RE
+.
+.
+.TP
+.B getuid_callout
+(Superseded by \fIuid_attribute\fR) The default program and args to callout
+to obtain a unique path identifier. Should be specified with an absolute path.
+.RS
+.TP
+The default is: \fB<unset>\fR
+.RE
+.
+.
+.TP
+.B prio
+The name of the path priority routine. The specified routine
+should return a numeric value specifying the relative priority
+of this path. Higher number have a higher priority.
+\fI"none"\fR is a valid value. Currently the following path priority routines
+are implemented:
+.RS
+.TP 12
+.I const
+Return a constant priority of \fI1\fR.
+.TP
+.I sysfs
+Use the sysfs attributes \fIaccess_state\fR and \fIpreferred_path\fR to
+generate the path priority. This prioritizer accepts the optional prio_arg
+\fIexclusive_pref_bit\fR.
+.TP
+.I emc
+(Hardware-dependent)
+Generate the path priority for DGC class arrays as CLARiiON CX/AX and
+EMC VNX and Unity families.
+.TP
+.I alua
+(Hardware-dependent)
+Generate the path priority based on the SCSI-3 ALUA settings. This prioritizer
+accepts the optional prio_arg \fIexclusive_pref_bit\fR.
+.TP
+.I ontap
+(Hardware-dependent)
+Generate the path priority for NetApp ONTAP class and OEM arrays as IBM NSeries.
+.TP
+.I rdac
+(Hardware-dependent)
+Generate the path priority for LSI/Engenio/NetApp RDAC class as NetApp SANtricity
+E/EF Series, and OEM arrays from IBM DELL SGI STK and SUN.
+.TP
+.I hp_sw
+(Hardware-dependent)
+Generate the path priority for HP/COMPAQ/DEC HSG80 and MSA/HSV arrays with
+Active/Standby mode exclusively.
+.TP
+.I hds
+(Hardware-dependent)
+Generate the path priority for Hitachi AMS 2000 and HUS 100 families of arrays.
+.TP
+.I random
+Generate a random priority between 1 and 10.
+.TP
+.I weightedpath
+Generate the path priority based on the regular expression and the
+priority provided as argument. Requires prio_args keyword.
+.TP
+.I path_latency
+Generate the path priority based on a latency algorithm.
+Requires prio_args keyword.
+.TP
+.I datacore
+(Hardware-dependent)
+Generate the path priority for some DataCore storage arrays. Requires prio_args
+keyword.
+.TP
+.I iet
+(iSCSI only)
+Generate path priority for iSCSI targets based on IP address. Requires
+prio_args keyword.
+.PP
+The default depends on the \fBdetect_prio\fR setting: If \fBdetect_prio\fR is
+\fByes\fR (default), the default priority algorithm is \fBsysfs\fR (except for
+NetAPP E-Series, where it is \fBalua\fR). If \fBdetect_prio\fR is
+\fBno\fR, the default priority algorithm is \fBconst\fR.
+.RE
+.
+.
+.TP
+.B prio_args
+Arguments to pass to to the prio function. This only applies to certain
+prioritizers:
+.RS
+.TP 12
+.I weighted
+Needs a value of the form
+\fI"<hbtl|devname|serial|wwn> <regex1> <prio1> <regex2> <prio2> ..."\fR
+.RS
+.TP 8
+.I hbtl
+Regex can be of SCSI H:B:T:L format. For example: 1:0:.:. , *:0:0:.
+.TP
+.I devname
+Regex can be of device name format. For example: sda , sd.e
+.TP
+.I serial
+Regex can be of serial number format. For example: .*J1FR.*324 . The serial can
+be looked up through sysfs or by running multipathd show paths format "%z". For
+example: 0395J1FR904324
+.TP
+.I wwn
+Regex can be of the form \fI"host_wwnn:host_wwpn:target_wwnn:target_wwpn"\fR
+these values can be looked up through sysfs or by running \fImultipathd show paths format
+"%N:%R:%n:%r"\fR. For example: 0x200100e08ba0aea0:0x210100e08ba0aea0:.*:.* , .*:.*:iqn.2009-10.com.redhat.msp.lab.ask-06:.*
+.RE
+.TP 12
+.I path_latency
+Needs a value of the form "io_num=\fI<20>\fR base_num=\fI<10>\fR"
+.RS
+.TP 8
+.I io_num
+The number of read IOs sent to the current path continuously, used to calculate the average path latency.
+Valid Values: Integer, [2, 200].
+.TP
+.I base_num
+The base number value of logarithmic scale, used to partition different priority ranks. Valid Values: Integer,
+[2, 10]. And Max average latency value is 100s, min average latency value is 1us.
+For example: If base_num=10, the paths will be grouped in priority groups with path latency <=1us, (1us, 10us],
+(10us, 100us], (100us, 1ms], (1ms, 10ms], (10ms, 100ms], (100ms, 1s], (1s, 10s], (10s, 100s], >100s.
+.RE
+.TP 12
+.I alua
+If \fIexclusive_pref_bit\fR is set, paths with the \fIpreferred path\fR bit
+set will always be in their own path group.
+.TP
+.I sysfs
+If \fIexclusive_pref_bit\fR is set, paths with the \fIpreferred path\fR bit
+set will always be in their own path group.
+.TP
+.I datacore
+.RS
+.TP 8
+.I preferredsds
+(Mandatory) The preferred "SDS name".
+.TP
+.I timeout
+(Optional) The timeout for the INQUIRY, in ms.
+.RE
+.TP 12
+.I iet
+.RS
+.TP 8
+.I preferredip=...
+(Mandatory) Th preferred IP address, in dotted decimal notation, for iSCSI targets.
+.RE
+.TP
+The default is: \fB<unset>\fR
+.RE
+.
+.
+.TP
+.B features
+Specify any device-mapper features to be used. Syntax is \fInum list\fR
+where \fInum\fR is the number, between 0 and 8, of features in \fIlist\fR.
+Possible values for the feature list are:
+.RS
+.TP 12
+.I queue_if_no_path
+(Deprecated, superseded by \fIno_path_retry\fR) Queue I/O if no path is active.
+Identical to the \fIno_path_retry\fR with \fIqueue\fR value. If both this
+feature and \fIno_path_retry\fR are set, the latter value takes
+precedence. See KNOWN ISSUES.
+.TP
+.I pg_init_retries <times>
+(Since kernel 2.6.24) Number of times to retry pg_init, it must be between 1 and 50.
+.TP
+.I pg_init_delay_msecs <msecs>
+(Since kernel 2.6.38) Number of msecs before pg_init retry, it must be between 0 and 60000.
+.TP
+.I queue_mode <mode>
+(Since kernel 4.8) Select the the queueing mode per multipath device.
+<mode> can be \fIbio\fR, \fIrq\fR or \fImq\fR, which corresponds to
+bio-based, request-based, and block-multiqueue (blk-mq) request-based,
+respectively.
+The default depends on the kernel parameter \fBdm_mod.use_blk_mq\fR. It is
+\fImq\fR if the latter is set, and \fIrq\fR otherwise.
+.TP
+The default is: \fB<unset>\fR
+.RE
+.
+.
+.TP
+.B path_checker
+The default method used to determine the paths state. Possible values
+are:
+.RS
+.TP 12
+.I readsector0
+(Deprecated) Read the first sector of the device. This checker is being
+deprecated, please use \fItur\fR instead.
+.TP
+.I tur
+Issue a \fITEST UNIT READY\fR command to the device.
+.TP
+.I emc_clariion
+(Hardware-dependent)
+Query the DGC/EMC specific EVPD page 0xC0 to determine the path state
+for CLARiiON CX/AX and EMC VNX and Unity arrays families.
+.TP
+.I hp_sw
+(Hardware-dependent)
+Check the path state for HP/COMPAQ/DEC HSG80 and MSA/HSV arrays with
+Active/Standby mode exclusively.
+.TP
+.I rdac
+(Hardware-dependent)
+Check the path state for LSI/Engenio/NetApp RDAC class as NetApp SANtricity E/EF
+Series, and OEM arrays from IBM DELL SGI STK and SUN.
+.TP
+.I directio
+(Deprecated) Read the first sector with direct I/O. This checker is being
+deprecated, it could cause spurious path failures under high load.
+Please use \fItur\fR instead.
+.TP
+.I cciss_tur
+(Hardware-dependent)
+Check the path state for HP/COMPAQ Smart Array(CCISS) controllers.
+.TP
+.I none
+Do not check the device, fallback to use the values retrieved from sysfs
+.TP
+.I rbd
+Check if the path is in the Ceph blacklist and remap the path if it is.
+.TP
+The default is: \fBtur\fR
+.RE
+.
+.
+.TP
+.B alias_prefix
+The \fIuser_friendly_names\fR prefix.
+.RS
+.TP
+The default is: \fBmpath\fR
+.RE
+.
+.
+.TP
+.B failback
+Tell multipathd how to manage path group failback.
+To select \fIimmediate\fR or a \fIvalue\fR, it's mandatory that the device
+has support for a working prioritizer.
+.RS
+.TP 12
+.I immediate
+Immediately failback to the highest priority pathgroup that contains
+active paths.
+.TP
+.I manual
+Do not perform automatic failback.
+.TP
+.I followover
+Used to deal with multiple computers accessing the same Active/Passive storage
+devices. Only perform automatic failback when the first path of a pathgroup
+becomes active. This keeps a cluster node from automatically failing back when
+another node requested the failover.
+.TP
+.I values > 0
+Deferred failback (time to defer in seconds).
+.TP
+The default is: \fBmanual\fR
+.RE
+.
+.
+.TP
+.B rr_min_io
+Number of I/O requests to route to a path before switching to the next in the
+same path group. This is only for \fIBlock I/O\fR(BIO) based multipath and
+only apply to \fIround-robin\fR path_selector.
+.RS
+.TP
+The default is: \fB1000\fR
+.RE
+.
+.
+.TP
+.B rr_min_io_rq
+Number of I/O requests to route to a path before switching to the next in the
+same path group. This is only for \fIRequest\fR based multipath and
+only apply to \fIround-robin\fR path_selector.
+.RS
+.TP
+The default is: \fB1\fR
+.RE
+.
+.
+.TP
+.B max_fds
+Specify the maximum number of file descriptors that can be opened by multipath
+and multipathd. This is equivalent to ulimit \-n. A value of \fImax\fR will set
+this to the system limit from \fI/proc/sys/fs/nr_open\fR. If this is not set, the
+maximum number of open fds is taken from the calling process. It is usually
+1024. To be safe, this should be set to the maximum number of paths plus 32,
+if that number is greated than 1024.
+.RS
+.TP
+The default is: \fBmax\fR
+.RE
+.
+.
+.TP
+.B rr_weight
+If set to \fIpriorities\fR the multipath configurator will assign path weights
+as "path prio * rr_min_io". Possible values are
+.I priorities
+or
+.I uniform .
+Only apply to \fIround-robin\fR path_selector.
+.RS
+.TP
+The default is: \fBuniform\fR
+.RE
+.
+.
+.TP
+.B no_path_retry
+Specify what to do when all paths are down. Possible values are:
+.RS
+.TP 12
+.I value > 0
+Number of retries until disable I/O queueing.
+.TP
+.I fail
+For immediate failure (no I/O queueing).
+.TP
+.I queue
+For never stop I/O queueing, similar to \fIqueue_if_no_path\fR. See KNOWN ISSUES.
+.TP
+The default is: \fBfail\fR
+.RE
+.
+.
+.TP
+.B queue_without_daemon
+If set to
+.I no
+, when multipathd stops, queueing will be turned off for all devices.
+This is useful for devices that set no_path_retry. If a machine is
+shut down while all paths to a device are down, it is possible to hang waiting
+for I/O to return from the device after multipathd has been stopped. Without
+multipathd running, access to the paths cannot be restored, and the kernel
+cannot be told to stop queueing I/O. Setting queue_without_daemon to
+.I no
+, avoids this problem.
+.RS
+.TP
+The default is: \fBno\fR
+.RE
+.
+.
+.TP
+.B checker_timeout
+Specify the timeout to use for path checkers and prioritizers that issue SCSI
+commands with an explicit timeout, in seconds.
+.RS
+.TP
+The default is: in \fB/sys/block/sd<x>/device/timeout\fR
+.RE
+.
+.
+.TP
+.B flush_on_last_del
+If set to
+.I yes
+, multipathd will disable queueing when the last path to a device has been
+deleted.
+.RS
+.TP
+The default is: \fBno\fR
+.RE
+.
+.
+.TP
+.B user_friendly_names
+If set to
+.I yes
+, using the bindings file \fI/etc/multipath/bindings\fR to assign a persistent
+and unique alias to the multipath, in the form of mpath<n>. If set to
+.I no
+use the WWID as the alias. In either case this be will
+be overridden by any specific aliases in the \fImultipaths\fR section.
+.RS
+.TP
+The default is: \fBno\fR
+.RE
+.
+.
+.TP
+.B fast_io_fail_tmo
+Specify the number of seconds the SCSI layer will wait after a problem has been
+detected on a FC remote port before failing I/O to devices on that remote port.
+This should be smaller than dev_loss_tmo. Setting this to
+.I off
+will disable the timeout.
+.RS
+.TP
+The default is: in \fB5\fR
+.RE
+.
+.
+.TP
+.B dev_loss_tmo
+Specify the number of seconds the SCSI layer will wait after a problem has
+been detected on a FC remote port before removing it from the system. This
+can be set to "infinity" which sets it to the max value of 2147483647
+seconds, or 68 years. It will be automatically adjusted to the overall
+retry interval \fIno_path_retry\fR * \fIpolling_interval\fR
+if a number of retries is given with \fIno_path_retry\fR and the
+overall retry interval is longer than the specified \fIdev_loss_tmo\fR value.
+The Linux kernel will cap this value to \fI600\fR if \fIfast_io_fail_tmo\fR
+is not set. See KNOWN ISSUES.
+.RS
+.TP
+The default is: \fB600\fR
+.RE
+.
+.
+.TP
+.B bindings_file
+The full pathname of the binding file to be used when the user_friendly_names
+option is set.
+.RS
+.TP
+The default is: \fB/etc/multipath/bindings\fR
+.RE
+.
+.
+.TP
+.B wwids_file
+The full pathname of the WWIDs file, which is used by multipath to keep track
+of the WWIDs for LUNs it has created multipath devices on in the past.
+.RS
+.TP
+The default is: \fB/etc/multipath/wwids\fR
+.RE
+.
+.
+.TP
+.B prkeys_file
+The full pathname of the prkeys file, which is used by multipathd to keep
+track of the persistent reservation key used for a specific WWID, when
+\fIreservation_key\fR is set to \fBfile\fR.
+.RS
+.TP
+The default is \fB/etc/multipath/prkeys\fR
+.RE
+.
+.
+.TP
+.B log_checker_err
+If set to
+.I once
+, multipathd logs the first path checker error at logging level 2. Any later
+errors are logged at level 3 until the device is restored. If set to
+.I always
+, multipathd always logs the path checker error at logging level 2.
+.RS
+.TP
+The default is: \fBalways\fR
+.RE
+.
+.
+.TP
+.B reservation_key
+This is the service action reservation key used by mpathpersist. It must be
+set for all multipath devices using persistent reservations, and it must be
+the same as the RESERVATION KEY field of the PERSISTENT RESERVE OUT parameter
+list which contains an 8-byte value provided by the application client to the
+device server to identify the I_T nexus.
+.RS
+.PP
+Alternatively, this can be set to \fBfile\fR, which will store the RESERVATION
+KEY registered by mpathpersist in the \fIprkeys_file\fR. multipathd will then
+use this key to register additional paths as they appear. When the
+registration is removed, the RESERVATION KEY is removed from the
+\fIprkeys_file\fR.
+.TP
+The default is: \fB<unset>\fR
+.RE
+.
+.
+.TP
+.B retain_attached_hw_handler
+(Obsolete for kernels >= 4.3) If set to
+.I yes
+and the SCSI layer has already attached a hardware_handler to the device,
+multipath will not force the device to use the hardware_handler specified by
+mutipath.conf. If the SCSI layer has not attached a hardware handler,
+multipath will continue to use its configured hardware handler.
+.RS
+.PP
+The default is: \fByes\fR
+.PP
+\fBImportant Note:\fR Linux kernel 4.3 or newer always behaves as if
+\fB"retain_attached_hw_handler yes"\fR was set.
+.RE
+.
+.
+.TP
+.B detect_prio
+If set to
+.I yes
+, multipath will try to detect if the device supports SCSI-3 ALUA. If so, the
+device will automatically use the \fIsysfs\fR prioritizer if the required sysf
+attributes \fIaccess_state\fR and \fIpreferred_path\fR are supported, or the
+\fIalua\fR prioritizer if not. If set to
+.I no
+, the prioritizer will be selected as usual.
+.RS
+.TP
+The default is: \fByes\fR
+.RE
+.
+.
+.TP
+.B detect_checker
+if set to
+.I yes
+, multipath will try to detect if the device supports SCSI-3 ALUA. If so, the
+device will automatically use the \fItur\fR checker. If set to
+.I no
+, the checker will be selected as usual.
+.RS
+.TP
+The default is: \fByes\fR
+.RE
+.
+.
+.TP
+.B force_sync
+If set to
+.I yes
+, multipathd will call the path checkers in sync mode only. This means that
+only one checker will run at a time. This is useful in the case where many
+multipathd checkers running in parallel causes significant CPU pressure.
+.RS
+.TP
+The default is: \fBno\fR
+.RE
+.
+.
+.TP
+.B strict_timing
+If set to
+.I yes
+, multipathd will start a new path checker loop after exactly one second,
+so that each path check will occur at exactly \fIpolling_interval\fR
+seconds. On busy systems path checks might take longer than one second;
+here the missing ticks will be accounted for on the next round.
+A warning will be printed if path checks take longer than \fIpolling_interval\fR
+seconds.
+.RS
+.TP
+The default is: \fBno\fR
+.RE
+.
+.
+.TP
+.B deferred_remove
+If set to
+.I yes
+, multipathd will do a deferred remove instead of a regular remove when the
+last path device has been deleted. This means that if the multipath device is
+still in use, it will be freed when the last user closes it. If path is added
+to the multipath device before the last user closes it, the deferred remove
+will be canceled.
+.RS
+.TP
+The default is: \fBno\fR
+.RE
+.
+.
+.TP
+.B partition_delimiter
+If this value is not set, when multipath renames a device, it will act just
+like the kpartx default does, only adding a \fI"p"\fR to names ending in a
+number. If this parameter is set, multipath will act like kpartx does with
+the \fI-p\fR option is used, and always add delimiter.
+.RS
+.TP
+The default is: \fB<unset>\fR
+.RE
+.
+.
+.TP
+.B config_dir
+If set to anything other than "", multipath will search this directory
+alphabetically for file ending in ".conf" and it will read configuration
+information from them, just as if it was in \fI/etc/multipath.conf\fR.
+config_dir must either be "" or a fully qualified directory name.
+.RS
.TP
-.B udev_dir
-directory where udev creates its device nodes; default is
-.I /dev
+The default is: \fB/etc/multipath/conf.d/\fR
+.RE
+.
+.
.TP
-.B verbosity
-default verbosity. Higher values increase the verbosity level. Valid
-levels are between 0 and 6; default is 2.
-.I /dev
+.B marginal_path_double_failed_time
+One of the four parameters of supporting path check based on accounting IO
+error such as intermittent error. When a path failed event occurs twice in
+\fImarginal_path_double_failed_time\fR seconds due to an IO error and all the
+other three parameters are set, multipathd will fail the path and enqueue
+this path into a queue of which members are sent a couple of continuous
+direct reading asynchronous IOs at a fixed sample rate of 10HZ to start IO
+error accounting process.
+.RS
.TP
-.B selector
-The default path selector algorithm to use; they are offered by the
-kernel multipath target. The only currently implemented is
-.I "round-robin 0"
+The default is: \fBno\fR
+.RE
+.
+.
.TP
-.B path_grouping_policy
-The default path grouping policy to apply to unspecified
-multipaths. Possible values are
+.B marginal_path_err_sample_time
+One of the four parameters of supporting path check based on accounting IO
+error such as intermittent error. If it is set to a value no less than 120,
+when a path fail event occurs twice in \fImarginal_path_double_failed_time\fR
+second due to an IO error, multipathd will fail the path and enqueue this
+path into a queue of which members are sent a couple of continuous direct
+reading asynchronous IOs at a fixed sample rate of 10HZ to start the IO
+accounting process for the path will last for
+\fImarginal_path_err_sample_time\fR.
+If the rate of IO error on a particular path is greater than the
+\fImarginal_path_err_rate_threshold\fR, then the path will not reinstate for
+\fImarginal_path_err_recheck_gap_time\fR seconds unless there is only one
+active path. After \fImarginal_path_err_recheck_gap_time\fR expires, the path
+will be requeueed for rechecking. If checking result is good enough, the
+path will be reinstated.
.RS
-.TP 12
-.B failover
-1 path per priority group
.TP
-.B multibus
-all paths in 1 priority group
+The default is: \fBno\fR
+.RE
+.
+.
.TP
-.B group_by_serial
-1 priority group per serial number
+.B marginal_path_err_rate_threshold
+The error rate threshold as a permillage (1/1000). One of the four parameters
+of supporting path check based on accounting IO error such as intermittent
+error. Refer to \fImarginal_path_err_sample_time\fR. If the rate of IO errors
+on a particular path is greater than this parameter, then the path will not
+reinstate for \fImarginal_path_err_rate_threshold\fR seconds unless there is
+only one active path.
+.RS
.TP
-.B group_by_prio
-1 priority group per priority value. Priorities are determined by
-callout programs specified as a global, per-controller or
-per-multipath option in the configuration file.
+The default is: \fBno\fR
+.RE
+.
+.
.TP
-.B group_by_node_name
-1 priority group per target node name. Target node names are fetched
-in /sys/class/fc_transport/target*/node_name.
+.B marginal_path_err_recheck_gap_time
+One of the four parameters of supporting path check based on accounting IO
+error such as intermittent error. Refer to
+\fImarginal_path_err_sample_time\fR. If this parameter is set to a positive
+value, the failed path of which the IO error rate is larger than
+\fImarginal_path_err_rate_threshold\fR will be kept in failed state for
+\fImarginal_path_err_recheck_gap_time\fR seconds. When
+\fImarginal_path_err_recheck_gap_time\fR seconds expires, the path will be
+requeueed for checking. If checking result is good enough, the path will be
+reinstated, or else it will keep failed.
+.RS
.TP
-Default value is \fImultibus\fR.
+The default is: \fBno\fR
.RE
+.
+.
.TP
-.B getuid_callout
-The default program and args to callout to obtain a unique path
-identifier. Should be specified with an absolute path. Default value
-is
-.I /lib/udev/scsi_id --whitelisted --device=/dev/%n
-.TP
-.B prio_callout
-The default program and args to callout to obtain a path priority
-value. The specified program will be executed and should return a
-numeric value specifying the relative priority of this path. Higher
-number have a higher priority. A '%n' in the command line will be expanded
-to the device name, a '%b' will be expanded to the device number in
-.I major:minor
-format.
-.I "none"
-is a valid value. Currently the following path priority programs are
-implemented:
+.B delay_watch_checks
+If set to a value greater than 0, multipathd will watch paths that have
+recently become valid for this many checks. If they fail again while they are
+being watched, when they next become valid, they will not be used until they
+have stayed up for \fIdelay_wait_checks\fR checks.
.RS
-.TP 12
-.B mpath_prio_emc /dev/%n
-Generate the path priority for EMC arrays
.TP
-.B mpath_prio_alua /dev/%n
-Generate the path priority based on the SCSI-3 ALUA settings.
+The default is: \fBno\fR
+.RE
+.
+.
+.TP
+.B delay_wait_checks
+If set to a value greater than 0, when a device that has recently come back
+online fails again within \fIdelay_watch_checks\fR checks, the next time it
+comes back online, it will marked and delayed, and not used until it has passed
+\fIdelay_wait_checks\fR checks.
+.RS
.TP
-.B mpath_prio_netapp /dev/%n
-Generate the path priority for NetApp arrays.
+The default is: \fBno\fR
+.RE
+.
+.
.TP
-.B mpath_prio_rdac /dev/%n
-Generate the path priority for LSI/Engenio RDAC controller.
+.B find_multipaths
+If set to
+.I yes
+, instead of trying to create a multipath device for every non-blacklisted
+path, multipath will only create a device if one of three condidions are
+met.
+.I 1
+There are at least two non-blacklisted paths with the same WWID,
+.I 2
+the user manually forces the creation, by specifying a device with the multipath
+command, or
+.I 3
+a path has the same WWID as a multipath device that was previously created
+while find_multipaths was set (even if that multipath device doesn't currently
+exist).
+Whenever a multipath device is created with find_multipaths set, multipath will
+remember the WWID of the device, so that it will automatically create the
+device again, as soon as it sees a path with that WWID. This should allow most
+users to have multipath automatically choose the correct paths to make into
+multipath devices, without having to edit the blacklist.
+.RS
.TP
-.B mpath_prio_hp_sw /dev/%n
-Generate the path priority for Compaq/HP controller in
-active/standby mode.
+The default is: \fBno\fR
+.RE
+.
+.
.TP
-.B mpath_prio_hds_modular %b
-Generate the path priority for Hitachi HDS Modular storage arrays.
+.B uxsock_timeout
+CLI receive timeout in milliseconds. For larger systems CLI commands
+might timeout before the multipathd lock is released and the CLI command
+can be processed. This will result in errors like
+"timeout receiving packet" to be returned from CLI commands.
+In these cases it is recommended to increase the CLI timeout to avoid
+those issues.
+.RS
.TP
-Default value is \fBnone\fR.
+The default is: \fB1000\fR
.RE
+.
+.
.TP
-.B features
-Specify any device-mapper features to be used. The most common of
-these features is
-.I "1 queue_if_no_path"
-Note that this can also be set via the
-.I no_path_retry
-keyword.
+.B retrigger_tries
+Sets the number of times multipathd will try to retrigger a uevent to get the
+WWID.
+.RS
.TP
-.B path_checker
-The default method used to determine the paths' state. Possible values
-are
+The default is: \fB3\fR
+.RE
+.
+.
+.TP
+.B retrigger_delay
+Sets the amount of time, in seconds, to wait between retriggers.
.RS
-.TP 12
-.B readsector0
-Read the first sector of the device
.TP
-.B tur
-Issue a
-.I TEST UNIT READY
-command to the device.
+The default is: \fB10\fR
+.RE
+.
+.
+.TP
+.B missing_uev_wait_timeout
+Controls how many seconds multipathd will wait, after a new multipath device
+is created, to receive a change event from udev for the device, before
+automatically enabling device reloads. Usually multipathd will delay reloads
+on a device until it receives a change uevent from the initial table load.
+.RS
.TP
-.B emc_clariion
-Query the EMC Clariion specific EVPD page 0xC0 to determine the path
-state.
+The default is: \fB30\fR
+.RE
+.
+.
.TP
-.B hp_sw
-Check the path state for HP storage arrays with Active/Standby firmware.
+.B skip_kpartx
+If set to
+.I yes
+, kpartx will not automatically create partitions on the device.
+.RS
.TP
-.B rdac
-Check the path state for LSI/Engenio RDAC storage controller.
+The default is: \fBno\fR
+.RE
+.
+.
.TP
-.B directio
-Read the first sector with direct I/O.
+.B disable_changed_wwids
+If set to \fIyes\fR, multipathd will check the path wwid on change events, and
+if it has changed from the wwid of the multipath device, multipathd will
+disable access to the path until the wwid changes back.
+.RS
.TP
-Default value is \fIreadsector0\fR.
+The default is: \fBno\fR
.RE
+.
+.
.TP
-.B failback
-Tell the daemon to manage path group failback, or not to. 0 or
-.I immediate
-means immediate failback, values >0 means deferred failback (in
-seconds).
-.I manual
-means no failback. Default value is
-.I manual
+.B remove_retries
+This sets how may times multipath will retry removing a device that is in-use.
+Between each attempt, multipath will sleep 1 second.
+.RS
.TP
-.B rr_min_io
-The number of IO to route to a path before switching to the next in
-the same path group. Default is
-.I 1000
+The default is: \fB0\fR
+.RE
+.
+.
.TP
-.B rr_weight
-If set to \fIpriorities\fR the multipath configurator will assign
-path weights as "path prio * rr_min_io". Possible values are
-.I priorities
-or
-.I uniform
-. Default is
-.I uniform
+.B max_sectors_kb
+Sets the max_sectors_kb device parameter on all path devices and the multipath
+device to the specified value.
+.RS
.TP
-.B no_path_retry
-Specify the number of retries until disable queueing, or
-.I fail
-for immediate failure (no queueing),
-.I queue
-for never stop queueing. Default is 0.
+The default is: \fB<device dependent>\fR
+.RE
+.
+.
.TP
-.B user_friendly_names
-If set to
-.I yes
-, using the bindings file
-.I /var/lib/multipath/bindings
-to assign a persistent and unique alias to the multipath, in the form of mpath<n>.
-If set to
-.I no
-use the WWID as the alias. In either case this be will
-be overriden by any specific aliases in the \fImultipaths\fR section.
-Default is
-.I no
+.B ghost_delay
+Sets the number of seconds that multipath will wait after creating a device
+with only ghost paths before marking it ready for use in systemd. This gives
+the active paths time to appear before the multipath runs the hardware handler
+to switch the ghost paths to active ones. Setting this to \fI0\fR or \fIon\fR
+makes multipath immediately mark a device with only ghost paths as ready.
+.RS
+.TP
+The default is \fBno\fR
+.RE
.
+.
+.\" ----------------------------------------------------------------------------
.SH "blacklist section"
-The
-.I blacklist
-section is used to exclude specific device from inclusion in the
-multipath topology. It is most commonly used to exclude local disks or
-LUNs for the array controller.
+.\" ----------------------------------------------------------------------------
+.
+The \fIblacklist\fR section is used to exclude specific device from inclusion in
+the multipath topology. It is most commonly used to exclude local disks or LUNs
+for the array controller.
.LP
+.
+.
The following keywords are recognized:
.TP 17
+.B devnode
+Regular expression of the device nodes to be excluded.
+.RS
+.TP
+The default is: \fB^(ram|raw|loop|fd|md|dm-|sr|scd|st|dcssblk)[0-9]\fR and \fB^(td|hd|vd)[a-z]\fR
+.RE
+.TP
.B wwid
The \fIWorld Wide Identification\fR of a device.
.TP
-.B devnode
-Regular expression of the device nodes to be excluded.
+.B property
+Regular expression of the udev property to be excluded.
.TP
.B device
Subsection for the device description. This subsection recognizes the
-.I vendor
+.B vendor
and
-.I product
+.B product
keywords. For a full description of these keywords please see the
-.I devices
-section description.
+\fIdevices\fR section description.
+.
+.
+.\" ----------------------------------------------------------------------------
.SH "blacklist_exceptions section"
-The
-.I blacklist_exceptions
-section is used to revert the actions of the
-.I blacklist
-section, ie to include specific device in the
-multipath topology. This allows to selectively include devices which
-would normally be excluded via the
-.I blacklist
-section.
+.\" ----------------------------------------------------------------------------
+.
+The \fIblacklist_exceptions\fR section is used to revert the actions of the
+\fIblacklist\fR section. For example to include specific device in the
+multipath topology. This allows one to selectively include devices which
+would normally be excluded via the \fIblacklist\fR section.
.LP
+.
+.
The following keywords are recognized:
.TP 17
+.B devnode
+Regular expression of the device nodes to be whitelisted.
+.TP
.B wwid
The \fIWorld Wide Identification\fR of a device.
.TP
-.B devnode
-Regular expression of the device nodes to be excluded.
+.B property
+Regular expression of the udev property to be whitelisted.
+.RS
+.TP
+The default is: \fB(SCSI_IDENT_|ID_WWN)\fR
+.RE
.TP
.B device
Subsection for the device description. This subsection recognizes the
-.I vendor
+.B vendor
and
-.I product
-keywords. For a full description of these keywords please see the
-.I devices
+.B product
+keywords. For a full description of these keywords please see the \fIdevices\fR
section description.
+.LP
+The \fIproperty\fR blacklist and whitelist handling is different from the usual
+handling in the sense that the whitelist \fIhas\fR to be set, otherwise the
+device will be blacklisted. In these cases the message \fIblacklisted, udev
+property missing\fR will be displayed.
+.
+.
+.\" ----------------------------------------------------------------------------
.SH "multipaths section"
-The only recognized attribute for the
-.B multipaths
-section is the
-.I multipath
-subsection.
+.\" ----------------------------------------------------------------------------
+.
+The only recognized attribute for the \fImultipaths\fR section is the
+\fImultipath\fR subsection.
.LP
-The
-.B multipath
-subsection recognizes the following attributes:
+.
+.
+The \fImultipath\fR subsection recognizes the following attributes:
.TP 17
.B wwid
-Index of the container. Mandatory for this subsection.
+(Mandatory) Index of the container.
.TP
.B alias
-(Optional) symbolic name for the multipath map.
+Symbolic name for the multipath map.
.LP
+.
+.
The following attributes are optional; if not set the default values
-are taken from the
-.I defaults
-section:
+are taken from the \fIdefaults\fR or \fIdevices\fR section:
.sp 1
.PD .1v
.RS
.TP
.B path_selector
.TP
+.B prio
+.TP
+.B prio_args
+.TP
.B failback
.TP
+.B rr_weight
+.TP
.B no_path_retry
.TP
.B rr_min_io
+.TP
+.B rr_min_io_rq
+.TP
+.B flush_on_last_del
+.TP
+.B features
+.TP
+.B reservation_key
+.TP
+.B user_friendly_names
+.TP
+.B deferred_remove
+.TP
+.B marginal_path_err_sample_time
+.TP
+.B marginal_path_err_rate_threshold
+.TP
+.B marginal_path_err_recheck_gap_time
+.TP
+.B marginal_path_double_failed_time
+.TP
+.B delay_watch_checks
+.TP
+.B delay_wait_checks
+.TP
+.B skip_kpartx
+.TP
+.B max_sectors_kb
+.TP
+.B ghost_delay
.RE
.PD
.LP
+.
+.
+.\" ----------------------------------------------------------------------------
.SH "devices section"
-The only recognized attribute for the
-.B devices
-section is the
-.I device
+.\" ----------------------------------------------------------------------------
+.
+The only recognized attribute for the \fIdevices\fR section is the \fIdevice\fR
subsection.
.LP
-The
-.I device
-subsection recognizes the following attributes:
+.
+.
+The \fIdevice\fR subsection recognizes the following attributes:
+.TP
+vendor, product, revision and product_blacklist are POSIX Extended regex.
.TP 17
.B vendor
-(Mandatory) Vendor identifier
+(Mandatory) Vendor identifier.
.TP
.B product
-(Mandatory) Product identifier
+(Mandatory) Product identifier.
+.TP
+.B revision
+Revision identfier.
.TP
.B product_blacklist
-Product strings to blacklist for this vendor
+Product strings to blacklist for this vendor.
+.TP
+.B alias_prefix
+The user_friendly_names prefix to use for this
+device type, instead of the default "mpath".
.TP
.B hardware_handler
-(Optional) The hardware handler to use for this device type.
+The hardware handler to use for this device type.
The following hardware handler are implemented:
.RS
.TP 12
-.B 1 emc
-Hardware handler for EMC storage arrays.
+.I 1 emc
+(Hardware-dependent)
+Hardware handler for DGC class arrays as CLARiiON CX/AX and EMC VNX and Unity
+families.
+.TP
+.I 1 rdac
+(Hardware-dependent)
+Hardware handler for LSI/Engenio/NetApp RDAC class as NetApp SANtricity E/EF
+Series, and OEM arrays from IBM DELL SGI STK and SUN.
+.TP
+.I 1 hp_sw
+(Hardware-dependent)
+Hardware handler for HP/COMPAQ/DEC HSG80 and MSA/HSV arrays with
+Active/Standby mode exclusively.
+.TP
+.I 1 alua
+(Hardware-dependent)
+Hardware handler for SCSI-3 ALUA compatible arrays.
+.PP
+The default is: \fB<unset>\fR
+.PP
+\fBImportant Note:\fR Linux kernels 4.3 and newer automatically attach a device
+handler to known devices (which includes all devices supporting SCSI-3 ALUA)
+and disallow changing the handler
+afterwards. Setting \fBhardware_handler\fR for such devices on these kernels
+has no effect.
.RE
+.
+.
.LP
The following attributes are optional; if not set the default values
-are taken from the
-.I defaults
+are taken from the \fIdefaults\fR
section:
.sp 1
.PD .1v
.TP 18
.B path_grouping_policy
.TP
+.B uid_attribute
+.TP
+.B path_selector
+.TP
+.B path_checker
+.TP
+.B prio
+.TP
+.B prio_args
+.TP
+.B features
+.TP
+.B failback
+.TP
+.B rr_weight
+.TP
+.B no_path_retry
+.TP
+.B rr_min_io
+.TP
+.B rr_min_io_rq
+.TP
+.B fast_io_fail_tmo
+.TP
+.B dev_loss_tmo
+.TP
+.B flush_on_last_del
+.TP
+.B retain_attached_hw_handler
+.TP
+.B detect_prio
+.TP
+.B detect_checker
+.TP
+.B deferred_remove
+.TP
+.B marginal_path_err_sample_time
+.TP
+.B marginal_path_err_rate_threshold
+.TP
+.B marginal_path_err_recheck_gap_time
+.TP
+.B marginal_path_double_failed_time
+.TP
+.B delay_watch_checks
+.TP
+.B delay_wait_checks
+.TP
+.B skip_kpartx
+.TP
+.B max_sectors_kb
+.TP
+.B ghost_delay
+.RE
+.PD
+.LP
+.
+.
+.\" ----------------------------------------------------------------------------
+.SH "overrides section"
+.\" ----------------------------------------------------------------------------
+.
+The overrides section recognizes the following optional attributes; if not set
+the values are taken from the \fIdevices\fR or \fIdefaults\fR sections:
+.sp 1
+.PD .1v
+.RS
+.TP 18
+.B path_grouping_policy
+.TP
+.B uid_attribute
+.TP
.B getuid_callout
.TP
.B path_selector
.TP
.B path_checker
.TP
+.B alias_prefix
+.TP
.B features
.TP
-.B prio_callout
+.B prio
+.TP
+.B prio_args
.TP
.B failback
.TP
.B no_path_retry
.TP
.B rr_min_io
+.TP
+.B rr_min_io_rq
+.TP
+.B flush_on_last_del
+.TP
+.B fast_io_fail_tmo
+.TP
+.B dev_loss_tmo
+.TP
+.B user_friendly_names
+.TP
+.B retain_attached_hw_handler
+.TP
+.B detect_prio
+.TP
+.B detect_checker
+.TP
+.B deferred_remove
+.TP
+.B marginal_path_err_sample_time
+.TP
+.B marginal_path_err_rate_threshold
+.TP
+.B marginal_path_err_recheck_gap_time
+.TP
+.B marginal_path_double_failed_time
+.TP
+.B delay_watch_checks
+.TP
+.B delay_wait_checks
+.TP
+.B skip_kpartx
+.TP
+.B ghost_delay
.RE
.PD
.LP
+.
+.
+.\" ----------------------------------------------------------------------------
+.SH "WWID generation"
+.\" ----------------------------------------------------------------------------
+.
+Multipath uses a \fIWorld Wide Identification\fR (WWID) to determine
+which paths belong to the same device. Each path presenting the same
+WWID is assumed to point to the same device.
+.LP
+The WWID is generated by three methods (in the order of preference):
+.TP 17
+.B getuid_callout
+Use the specified external program; cf \fIgetuid_callout\fR above.
+Care should be taken when using this method; the external program
+needs to be loaded from disk for execution, which might lead to
+deadlock situations in an all-paths-down scenario.
+.TP
+.B uid_attribute
+Use the value of the specified udev attribute; cf \fIuid_attribute\fR
+above. This method is preferred to \fIgetuid_callout\fR as multipath
+does not need to call any external programs here. However, under
+certain circumstances udev might not be able to generate the requested
+variable.
+.TP
+.B vpd_pg83
+If none of the \fIgetuid_callout\fR or \fIuid_attribute\fR parameters
+are present multipath will try to use the sysfs attribute
+\fIvpd_pg83\fR to generate the WWID.
+.
+.
+.\" ----------------------------------------------------------------------------
.SH "KNOWN ISSUES"
-The usage of
-.B queue_if_no_path
-option can lead to
-.B D state
-processes being hung and not killable in situations where all the paths to the LUN go offline.
-It is advisable to use the
-.B no_path_retry
-option instead.
+.\" ----------------------------------------------------------------------------
+.
+The usage of \fIqueue_if_no_path\fR option can lead to \fID state\fR
+processes being hung and not killable in situations where all the paths to the
+LUN go offline. It is advisable to use the \fIno_path_retry\fR option instead.
+.P
+The use of \fIqueue_if_no_path\fR or \fIno_path_retry\fR might lead to a
+deadlock if the \fIdev_loss_tmo\fR setting results in a device being removed
+while I/O is still queued. The multipath daemon will update the \fIdev_loss_tmo\fR
+setting accordingly to avoid this deadlock. Hence if both values are
+specified the order of precedence is \fIno_path_retry, queue_if_no_path, dev_loss_tmo\fR.
+.
+.
+.\" ----------------------------------------------------------------------------
.SH "SEE ALSO"
+.\" ----------------------------------------------------------------------------
+.
.BR udev (8),
-.BR dmsetup (8)
-.BR multipath (8)
-.BR multipathd (8)
+.BR dmsetup (8),
+.BR multipath (8),
+.BR multipathd (8).
+.
+.
+.\" ----------------------------------------------------------------------------
.SH AUTHORS
-.B multipath
-was developed by Christophe Varoqui, <christophe.varoqui@free.fr> and others.
+.\" ----------------------------------------------------------------------------
+.
+\fImultipath-tools\fR was developed by Christophe Varoqui, <christophe.varoqui@opensvc.com>
+and others.
+.\" EOF
projects / platform / upstream / multipath-tools.git / blobdiff