ASoC: SOF: topology: do not power down primary core during topology removal

ASoC: SOF: topology: do not power down primary core during topology removal

[ Upstream commit ec626334eaffe101df9ed79e161eba95124e64ad ]

When removing the topology components, do not power down
the primary core. Doing so will result in an IPC timeout
when the SOF PCI device runtime suspends.

Fixes: 0dcdf84289fb ("ASoC: SOF: add a "core" parameter to widget loading functions")

Signed-off-by: Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
Reviewed-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
Reviewed-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
Link: https://lore.kernel.org/r/20211006104041.27183-1-peter.ujfalusi@linux.intel.com
Signed-off-by: Mark Brown <broonie@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>