Felix Kuehling [Tue, 1 Nov 2022 19:02:48 +0000 (15:02 -0400)]
drm/amdkfd: Fix error handling in criu_checkpoint
Checkpoint BOs last. That way we don't need to close dmabuf FDs if
something else fails later. This avoids problematic access to user mode
memory in the error handling code path.
criu_checkpoint_bos has its own error handling and cleanup that does not
depend on access to user memory.
In the private data, keep BOs before the remaining objects. This is
necessary to restore things in the correct order as restoring events
depends on the events-page BO being restored first.
Fixes:
be072b06c739 ("drm/amdkfd: CRIU export BOs as prime dmabuf objects")
Reported-by: Jann Horn <jannh@google.com>
CC: Rajneesh Bhardwaj <Rajneesh.Bhardwaj@amd.com>
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-and-tested-by: Rajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Felix Kuehling [Thu, 3 Nov 2022 21:01:46 +0000 (17:01 -0400)]
drm/amdkfd: Fix error handling in kfd_criu_restore_events
mutex_unlock before the exit label because all the error code paths that
jump there didn't take that lock. This fixes unbalanced locking errors
in case of restore errors.
Fixes:
40e8a766a761 ("drm/amdkfd: CRIU checkpoint and restore events")
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Rajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Tim Huang [Thu, 3 Nov 2022 03:05:19 +0000 (11:05 +0800)]
drm/amd/pm: update SMU IP v13.0.4 msg interface header
Some of the unused messages that were used earlier in development have
been freed up as spare messages, no intended functional changes.
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Tim Huang <tim.huang@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Dave Airlie [Wed, 9 Nov 2022 00:48:44 +0000 (10:48 +1000)]
Merge branch '00.06-gr-ampere' of https://gitlab.freedesktop.org/skeggsb/nouveau into drm-next
This is the pull request for a whole bunch of fixes and prep-work that
was done to support Ampere acceleration prior to GSP-RM being
available. It uses the ACR firmware released by NVIDIA in
linux-firmware, as we do on earlier GPUs. The work to support running
on top of GSP-RM also heavily depends on various pieces of this
series.
In addition to the new HW support, general stability of the driver
should be improved, especially around recovering HW from bugs that can
be generated by userspace driver components.
Signed-off-by: Dave Airlie <airlied@redhat.com>
From: Ben Skeggs <bskeggs@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/CABDvA==s+nZD0n7CuRWLPE=Pj+02CN13r+ZQJxoHQ_EmR+o=XQ@mail.gmail.com
Ben Skeggs [Wed, 1 Jun 2022 10:48:35 +0000 (20:48 +1000)]
drm/nouveau/gr/ga102: initial support
v2:
- whitespace
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Gourav Samaiya <gsamaiya@nvidia.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:34 +0000 (20:48 +1000)]
drm/nouveau/ltc/ga102: initial support
v2. fixup for ga103 early merge
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:33 +0000 (20:48 +1000)]
drm/nouveau/acr/ga102: initial support
v2. fixup for ga103 early merge
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Gourav Samaiya <gsamaiya@nvidia.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:32 +0000 (20:48 +1000)]
drm/nouveau/fb/ga102: load and boot VPR scrubber FW
v2. fixup for ga103 early merge
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Gourav Samaiya <gsamaiya@nvidia.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:22 +0000 (20:48 +1000)]
drm/nouveau/gr/tu102: remove gv100_grctx_unkn88c
Match RM.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:21 +0000 (20:48 +1000)]
drm/nouveau/gr/tu102: add gv100_gr_init_4188a4
Match RM.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:21 +0000 (20:48 +1000)]
drm/nouveau/gr/tu102-: fix support for sw_bundle64_init
We weren't sending the high bits, though they're zero currently anyway.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:20 +0000 (20:48 +1000)]
drm/nouveau/gr/tu102-: use sw_veid_bundle_init from firmware
NVIDIA provided this on Turing, but we kept using the hardcoded version
from Volta (where they didn't).
Switch to the firmware version prior to Ampere.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:19 +0000 (20:48 +1000)]
drm/nouveau/gr/gv100-: drop a write from init_shader_exceptions()
Match RM.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:19 +0000 (20:48 +1000)]
drm/nouveau/gr/gv100-: move init_419bd8() after sw_ctx load
Match RM.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:18 +0000 (20:48 +1000)]
drm/nouveau/gr/gv100-: add NV_PGRAPH_PRI_PD_AB_DIST_CONFIG_1 to patch list
Match RM.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:17 +0000 (20:48 +1000)]
drm/nouveau/gr/gv100-: fix number of tile map registers
Match RM.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:17 +0000 (20:48 +1000)]
drm/nouveau/gr/gv100-: port smid mapping code from nvgpu
Essentially ripped verbatim from NVGPU, comments and all, and adapted to
nvkm's structs and style.
- maybe fixes an nvgpu bug though, a small tweak was needed to match RM
v2:
- remove unnecessary WARN_ON
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:16 +0000 (20:48 +1000)]
drm/nouveau/gr/gp100-: modify init_fecs_exceptions
Match RM.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:15 +0000 (20:48 +1000)]
drm/nouveau/gr/gk20a,gm20b,gp10b: split out netlist parsing from fw loading
We'll want to reuse the former for loading from proper netlist images.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:14 +0000 (20:48 +1000)]
drm/nouveau/gr/gp100-: fix number of zcull tile regs
Match RM.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:14 +0000 (20:48 +1000)]
drm/nouveau/gr/gf117-: make ppc_nr[gpc] accurate
We're going to be pulling in a chunk of code from NVGPU to fixup our
SMID mappings on Volta and above, which depends on ppc_nr[gpc]
reflecting the actual number of PPCs present, not the maximum number.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:13 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: switch to newer style interrupt handler
Ampere.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:12 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: move some init to init_exception2()
Ampere.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:12 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: move some init to init_rop_exceptions()
Ampere.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:11 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: move reset during golden ctx init to fecs_reset()
Ampere.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:10 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: wfi after register-bashing golden init
Match RM.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:10 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: gpfifo_ctl zero before init
Match RM.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:09 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: wait for FE_PWR_MODE_AUTO
This doesn't fix any known issue, but RM started doing it at some point,
so presumably it's needed for something.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:08 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: call FECS HALT_PIPE method before RC reset
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:08 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: call FECS WFI_GOLDEN_SAVE method
This won't work on Ampere, and, it's questionable whether we should have
been using our FW's method of storing the golden context image with NV's
firmware to begin with.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:07 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: stop using NV_PGRAPH_FECS_CTXSW_MAILBOX_CLEAR
This doesn't work on Ampere for some reason, switch to directly modifying
NV_PGRAPH_FECS_CTXSW_MAILBOX instead.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:07 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: make global attrib_cb actually global
This was thought to be per-channel initially - it's not. The backing
pages for the VMM mappings are shared for all channels.
- switches to more straight-forward patch interfaces
- prepares for sub-context support
- this is saving a *sizeable* amount of vram
v2:
- whitespace
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:06 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: move misc context patching out of attrib_cb funcs
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:06 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: make global bundle_cb actually global
This was thought to be per-channel initially - it's not. The backing
pages for the VMM mappings are shared for all channels.
- switches to more straight-forward patch interfaces
- prepares for sub-context support
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:05 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: make global pagepool actually global
This was thought to be per-channel initially - it's not. The backing
pages for the VMM mappings are shared for all channels.
- switches to more straight-forward patch interfaces
- prepares for sub-context support
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:05 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: generate golden context during first object alloc
Needed for GV100 (and only GV100 for some reason) for WFI_GOLDEN_SAVE.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:04 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: move some code around to make next commits nicer
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:04 +0000 (20:48 +1000)]
drm/nouveau/fifo: expose function to read engine ctxsw status
Needed to support Ampere differences in gr/gf100-:
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:03 +0000 (20:48 +1000)]
drm/nouveau/ltc: split color vs depth/stencil zbc counts
These differ on Ampere.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:48:02 +0000 (20:48 +1000)]
drm/nouveau/engine: add HAL for engine-specific rc reset procedure
Will be used to improve gr reset on GF100 and newer.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:53 +0000 (20:47 +1000)]
drm/nouveau/sec2: dump tracepc info on halt
- useful to distinguish between different issues.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:52 +0000 (20:47 +1000)]
drm/nouveau/acr: use common falcon HS FW code for ACR FWs
Adds context binding and support for FWs with a bootloader to the code
that was added to load VPR scrubber HS binaries, and ports ACR over to
using all of it.
- gv100 split from gp108 to handle FW exit status differences
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:52 +0000 (20:47 +1000)]
drm/nouveau/fb/gp102-: unlock VPR right after devinit
Under memory load, instmem allocations could end up in the regions of
VRAM that are inaccessible right after boot, and be corrupted after a
suspend/resume cycle as a result of being restored before booting the
mem unlock firmware.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:51 +0000 (20:47 +1000)]
drm/nouveau/fb: handle sysmem flush page from common code
- also executes pre-DEVINIT, so early boot is able to DMA sysmem
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:51 +0000 (20:47 +1000)]
drm/nouveau/flcn: new code to load+boot simple HS FWs (VPR scrubber)
Adds the start of common interfaces to load and boot the HS binaries
provided by NVIDIA that enable the usage of GR.
ACR already handles most of this, but it's very much tied into ACR's
init process, and there's other code that could benefit from reusing
a lot of this stuff too (ie. VBIOS DEVINIT/PreOS, VPR scrubber).
The VPR scrubber code is fairly independent, and a good first target.
- adds better debug output to fw loading process, to ease bring-up/debug
v2:
- whitespace, 0->false
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:51 +0000 (20:47 +1000)]
drm/nouveau/flcn: rework falcon reset
Mostly preparation to fit in Ampere changes, but should result in reset
sequences a lot closer to RM's, and perhaps help out with the issues we
sometimes see reported in this area.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:50 +0000 (20:47 +1000)]
drm/nouveau/sec2: switch to newer style interrupt handler
Ampere.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:49 +0000 (20:47 +1000)]
drm/nouveau/sec2: unload RTOS before tearing down WPR
Reset regs won't be available on Ampere while SEC2 RTOS is running, and
we're apparently supposed to be doing this on earlier GPUs too.
v2:
- fixed some excessive indentation
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:49 +0000 (20:47 +1000)]
drm/nouveau/pmu/gm20b,gp10b: boot RTOS from PMU init
Cleanup before falcon changes.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:48 +0000 (20:47 +1000)]
drm/nouveau/pmu: move init() falcon reset to non-nvfw code
Cleanup before falcon changes.
- fixes (attempt at?) reset of pmu while rtos is running, on gm20b
v2:
- remove extra whitespace
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:48 +0000 (20:47 +1000)]
drm/nouveau/pmu: move preinit() falcon reset to devinit
Cleanup before falcon changes.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:47 +0000 (20:47 +1000)]
drm/nouveau/gsp: add funcs
Ampere.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:39 +0000 (20:47 +1000)]
drm/nouveau/fifo/ga100-: initial support
- replaces the hacked-up version that existed solely to support TTM
v2. remove earlier hack preventing use of non-stall intr for fences
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:38 +0000 (20:47 +1000)]
drm/nouveau/ce/ga100-: initial support
- replaces the hacked-up version that existed solely to support TTM
- noop until the next commit, adding proper support for ampere host
v2. fixup for ga103 early merge
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:38 +0000 (20:47 +1000)]
drm/nouveau/fifo: add new channel classes
Exposes a bunch of the new features that became possible as a result
of the earlier commits. DRM will build on this in the future to add
support for features such as SCG ("async compute") and multi-device
rendering, as part of the work necessary to be able to write a half-
decent vulkan driver - finally.
For the moment, this just crudely ports DRM to the API changes.
- channel class interfaces now the same for all HW classes
- channel group class exposed (SCG)
- channel runqueue selector exposed (SCG)
- channel sub-device id control exposed (multi-device rendering)
- channel names in logging will reflect creating process, not fd owner
- explicit USERD allocation required by VOLTA_CHANNEL_GPFIFO_A and newer
- drm is smarter about determining the appropriate channel class to use
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:37 +0000 (20:47 +1000)]
drm/nouveau/fifo: add new engine object handling
Simplifies the GPU-specific code, completing the switch to newer HALs.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:37 +0000 (20:47 +1000)]
drm/nouveau/fifo: add new engine context handling
Builds on the context tracking that was added earlier.
- marks engine context PTEs as 'priv' where possible
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:36 +0000 (20:47 +1000)]
drm/nouveau/fifo: add RAMFC info to nvkm_chan_func
- adds support for specifying SUBDEVICE_ID for channel
- rounds non-power-of-two GPFIFO sizes down, rather than up
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:36 +0000 (20:47 +1000)]
drm/nouveau/fifo: add USERD info to nvkm_chan_func
And use it to cleanup multiple implementations of almost the same thing.
- prepares for non-polled / client-provided USERD
- only zeroes relevant "registers", rather than entire USERD
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:35 +0000 (20:47 +1000)]
drm/nouveau/fifo: add RAMIN info to nvkm_chan_func
Currently provided by {chan,dma,gpfifo}*.c, and those are going away.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:35 +0000 (20:47 +1000)]
drm/nouveau/fifo: add common runlist control
- less dependence on waiting for runlist updates, on GPUs that allow it
- supports runqueue selector in RAMRL entries
- completes switch to common runl/cgrp/chan topology info
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:34 +0000 (20:47 +1000)]
drm/nouveau/fifo: add common channel recovery
That sure was fun to untangle.
- handled per-runlist, rather than globally
- more straight-forward process in general
- various potential SW/HW races have been fixed
- fixes lockdep issues that were present in >=gk104's prior implementation
- volta recovery now actually stands a chance of working
- volta/turing waiting for PBDMA idle before engine reset
- turing using hw-provided TSG info for CTXSW_TIMEOUT
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:33 +0000 (20:47 +1000)]
drm/nouveau/fifo: kill channel on NV_PPBDMA_INTR_1_CTXNOTVALID
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:33 +0000 (20:47 +1000)]
drm/nouveau/fifo: kill channel on a selection of PBDMA errors
A bunch of these can be handled in such a way that the channel can
continue, however, any of these are a pretty decent sign something
has gone horribly wrong, and the safest option is to disable the
channel.
This is a bit of a hack, we will want to handle these individually
and dump relevant debug info for each at some point.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:33 +0000 (20:47 +1000)]
drm/nouveau/fifo: add chan/cgrp preempt()
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:32 +0000 (20:47 +1000)]
drm/nouveau/fifo: add chan start()/stop()
- nvkm_chan_error() built on top, stops channel and sends 'killed' event
- removes an odd double-bashing of channel enable regs on kepler and up
- pokes doorbell on turing and up, after enabling channel
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:31 +0000 (20:47 +1000)]
drm/nouveau/fifo: add chan bind()/unbind()
- stops programming (non-existent) runl id field on bind(), from maxwell
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:31 +0000 (20:47 +1000)]
drm/nouveau/fifo: add runlist block()/allow()
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:30 +0000 (20:47 +1000)]
drm/nouveau/fifo: add runlist wait()
- adds g8x/turing registers, which were missing before
- switches fermi to polled wait, like later hw (see:
4f2fc25c0f8bc...)
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:30 +0000 (20:47 +1000)]
drm/nouveau/fifo: add new engine context tracking
Channel groups have somewhat more complicated requirements than what we
currently support. An engine context is shared between all channels in
a channel group, VEID/subctx support (later) brings per-VEID components,
and we need to track an individual channel's engine context pointers.
This commit adds the structures and refcounting to support the above,
wrapping the prior implementation for the moment.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:29 +0000 (20:47 +1000)]
drm/nouveau/fifo: add new channel lookup interfaces
- supports per-runlist CHIDs
- channel group lock held across reference, rather than global lock
v2:
- remove unnecessary parenthesis
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:29 +0000 (20:47 +1000)]
drm/nouveau/fifo: merge mmu fault handlers together
After updating GF100 implementation from the GK104/TU102 ones, and using
the new runlist/engine topology info, all three handlers become (almost)
identical.
- there's a temporary kludge to call through to the HW-specific recovery
- engine fault mapping info determined at load time, not on every fault
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:29 +0000 (20:47 +1000)]
drm/nouveau/fifo: move PBDMA intr to runq
- merges gf100/gk104- NV_PFIFO_INTR_0_PBDMA and NV_PPBDMA_INTR_0 code
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:28 +0000 (20:47 +1000)]
drm/nouveau/fifo: move PBDMA init to runq
- bumps pbdma timeout to value RM uses on newer HW
- bumps fb timeout to max from boot default
- one/both of these greatly improves stability on // piglit runs
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Mon, 3 Oct 2022 03:20:07 +0000 (13:20 +1000)]
drm/nouveau/fifo: program NV_PFIFO_FB_TIMEOUT on init
NVGPU and RM both program this value.
Fixes a bunch of random hangs running parallel piglit.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Ben Skeggs [Mon, 3 Oct 2022 03:19:08 +0000 (13:19 +1000)]
drm/nouveau/fifo: tidy global PBDMA init
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:27 +0000 (20:47 +1000)]
drm/nouveau/fifo: tidy up non-stall intr handling
- removes a layer of indirection in the intr handling
- prevents non-stall ctrl racing with unknown intrs
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:26 +0000 (20:47 +1000)]
drm/nouveau/fifo: use explicit intr interfaces
More control, and shallower call-chain to get to the point.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:26 +0000 (20:47 +1000)]
drm/nouveau/fifo: use runlist engine info to lookup engine classes
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:26 +0000 (20:47 +1000)]
drm/nouveau/fifo: add cgrp, have all channels be part of one
Engine context tracking will move to nvkm_cgrp in later commits, so we
create SW-only channel groups on HW without support for them.
- switches to nvkm_chid for TSG/channel ID allocation
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:25 +0000 (20:47 +1000)]
drm/nouveau/fifo: expose per-runlist CHID information
DRM uses this to setup fence-related items.
- nouveau_chan.runlist will always be "0" for the moment, not an issue
as GPUs prior to ampere have system-wide channel IDs,
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:25 +0000 (20:47 +1000)]
drm/nouveau/fifo: expose runlist topology info on all chipsets
Previously only available from Kepler onwards.
- also fixes the info() queries causing fifo init()/fini() unnecessarily
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:24 +0000 (20:47 +1000)]
drm/nouveau/fifo: add common runlist/engine topology
Creates an nvkm_runl for each runlist on the GPU, and an nvkm_engn for
each engine that is reachable from a runlist.
- basically what gk104- already does, but extended to all chips
- adds per-runlist CHID allocators (Ampere)
- splits g98/gt2xx out from g84 (different target engines)
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:23 +0000 (20:47 +1000)]
drm/nouveau/fifo: add runq
Creates an nvkm_runq for each PBDMA, these will be associated with the
relevant runlist(s) later.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:23 +0000 (20:47 +1000)]
drm/nouveau/fifo: add chid allocator
We need to be able to allocate TSG IDs as well as channel IDs, also,
Ampere has per-runlist channel IDs.
- holds per-ID private data, which will be used for/to protect lookup
- holds an nvkm_event which will be used for events tied to IDs
- not used yet beyond setup, and switching use of "fifo->nr - 1" for
channel ID mask to "chid->mask"
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:22 +0000 (20:47 +1000)]
drm/nouveau/fifo: merge gk104_fifo_func into nvkm_host_func
This makes it easier to transition everything.
- a couple of function renames for collisions
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:22 +0000 (20:47 +1000)]
drm/nouveau/fifo: pre-move some blocks of code around
- will make subsequent patches more obvious
- no code changes
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:21 +0000 (20:47 +1000)]
drm/nouveau/fifo: unify handling of channel classes
Adds the basic skeleton for common channel (group) interfaces.
- common behaviour between <gk104 and >=gk104 impl's
- separates priv/user channel objects
- passthrough to existing object for now, kludges removed later
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:21 +0000 (20:47 +1000)]
drm/nouveau/fifo: add chid_nr()
- reads channel count from GPU from gm200 onwards
- removes gm20b/gp10b (they become identical to gm200/gp100)
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:21 +0000 (20:47 +1000)]
drm/nouveau/imem: allow bar2 mapping of user allocations
Will be used to init client-allocated USERD to default values.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:20 +0000 (20:47 +1000)]
drm/nouveau/flcn: show falcon user in debug output
Displays both owner/user of the falcon (when they differ), and takes
both subdevs' debug levels into account when deciding whether to log
the message.
- runlist debugging will use one of the alternate macros added here
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:47:19 +0000 (20:47 +1000)]
drm/nouveau/nvkm: add locking to subdev/engine init paths
This wasn't really needed before; the main place this could race is with
channel recovery, but (through potentially fragile means) shouldn't have
been possible.
However, a number of upcoming patches benefit from having better control
over subdev init, necessitating some improvements here.
- allows subdev/engine oneinit() without init() (host/fifo patches)
- merges engine use locking/tracking into subdev, and extends it to fix
some issues that will arise with future usage patterns (acr patches)
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:46:55 +0000 (20:46 +1000)]
drm/nouveau/mc/ga100: switch to using NV_PMC_DEVICE_ENABLE
- NV_PMC_ENABLE still exists, but we don't touch anything in it yet
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:46:55 +0000 (20:46 +1000)]
drm/nouveau/mc: move NV_PMC_ENABLE bashing to chipset-specific code
Ampere needs different handling here, most of what we touch has moved.
We probably want to refactor these interfaces in general, but I'm not
yet sure how they should look, this will get the job done for now.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:46:55 +0000 (20:46 +1000)]
drm/nouveau/mc: implement intr handling on top of nvkm_intr
- new-style handlers can now be used here too
- decent clean-up
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:46:54 +0000 (20:46 +1000)]
drm/nouveau/fault/ga100: initial support
TU102 implementation should be OK for Ampere now.
v2. fixup for ga103 early merge
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:46:54 +0000 (20:46 +1000)]
drm/nouveau/fault/tu102: switch to explicit intr handlers
- reads vectors from HW, rather than being hardcoded
- removes hacks to support routing via old interfaces
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:46:54 +0000 (20:46 +1000)]
drm/nouveau/vfn/tu102-: support new-style interrupt tree
- switches ampere over now, and removes its hack mc implementation
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:46:53 +0000 (20:46 +1000)]
drm/nouveau/vfn: move NV_USERMODE class from host
- uses proper class IDs for Turing/Ampere
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Ben Skeggs [Wed, 1 Jun 2022 10:46:53 +0000 (20:46 +1000)]
drm/nouveau/vfn: add stub subdev for dev_func
Initially for NV_USERMODE class, and Turing/Ampere's new interrupt tree.
v2. fixup for ga103 early merge
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>