PCI: hv: Do not set PCI_COMMAND_MEMORY to reduce VM boot time
authorDexuan Cui <decui@microsoft.com>
Mon, 2 May 2022 07:42:55 +0000 (00:42 -0700)
committerWei Liu <wei.liu@kernel.org>
Tue, 3 May 2022 10:59:10 +0000 (10:59 +0000)
commit23e118a48acf7be223e57d98e98da8ac5a4071ac
tree004688c4e29c9fa31a58bd7aeb1c0efd21502226
parent455880dfe292a2bdd3b4ad6a107299fce610e64b
PCI: hv: Do not set PCI_COMMAND_MEMORY to reduce VM boot time

Currently when the pci-hyperv driver finishes probing and initializing the
PCI device, it sets the PCI_COMMAND_MEMORY bit; later when the PCI device
is registered to the core PCI subsystem, the core PCI driver's BAR detection
and initialization code toggles the bit multiple times, and each toggling of
the bit causes the hypervisor to unmap/map the virtual BARs from/to the
physical BARs, which can be slow if the BAR sizes are huge, e.g., a Linux VM
with 14 GPU devices has to spend more than 3 minutes on BAR detection and
initialization, causing a long boot time.

Reduce the boot time by not setting the PCI_COMMAND_MEMORY bit when we
register the PCI device (there is no need to have it set in the first place).
The bit stays off till the PCI device driver calls pci_enable_device().
With this change, the boot time of such a 14-GPU VM is reduced by almost
3 minutes.

Link: https://lore.kernel.org/lkml/20220419220007.26550-1-decui@microsoft.com/
Tested-by: Boqun Feng (Microsoft) <boqun.feng@gmail.com>
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Reviewed-by: Michael Kelley <mikelley@microsoft.com>
Acked-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Jake Oshins <jakeo@microsoft.com>
Link: https://lore.kernel.org/r/20220502074255.16901-1-decui@microsoft.com
Signed-off-by: Wei Liu <wei.liu@kernel.org>
drivers/pci/controller/pci-hyperv.c