scsi: mpt3sas: Reduce the performance drop

scsi: mpt3sas: Reduce the performance drop

This patch is to reduce the performance drop depth observed on SATA HDD
when ATA PT command is outstanding.

Driver returns IO commands with status "SAM_STAT_BUSY" whenever ATA PT
command is outstanding. With this, IO commands will be retried until this
outstanding ATA PT to complete and hence we will observe drop in
performance.

As the driver is completing the subsequent IOs commands with SAM_STAT_BUSY
status, these IOs has to go though the block layer.  Hence it adds latency
to the IOs and large performance drop is observed.

So to reduce this performance dropp, added improvement in driver to return
the subsequent IOs with SCSI_MLQUEUE_DEVICE_BUSY status instead of
completing the IOs with SAM_STAT_BUSY status when ATA PT command is
outstanding. Sending command back with SCSI_MLQUEUE_DEVICE_BUSY does not go
through complete block layer stack (as scsi_done won't be called) SML will
immediately retry the command and this method will avoid latency of block
layer stack and the performance impact will be reduced.

On Local setup, ran 512k sequential read IO operation on HGST SATA drive
with existing driver & with this improvement drivers and here is the
result,

1. With existing driver: IOs are running at bandwidth of ~230 rMB/s and
whenever any ATA PT command is outstanding (e.g issued from systemd-udevd
daemon) then this bandwidth drops to ~150 rMB/s.

2. With this improvement driver: IOs are running at bandwidth of ~230 rMB/s
and whenever any ATA PT command is outstanding then this bandwidth drops to
just ~190 rMB/s.

Signed-off-by: Suganath Prabu <suganath-prabu.subramani@broadcom.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>