platform/kernel/linux-rpi.git
20 months agodrm/nouveau/gr/gk20a,gm20b,gp10b: split out netlist parsing from fw loading
Ben Skeggs [Wed, 1 Jun 2022 10:48:15 +0000 (20:48 +1000)]
drm/nouveau/gr/gk20a,gm20b,gp10b: split out netlist parsing from fw loading

We'll want to reuse the former for loading from proper netlist images.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/gr/gp100-: fix number of zcull tile regs
Ben Skeggs [Wed, 1 Jun 2022 10:48:14 +0000 (20:48 +1000)]
drm/nouveau/gr/gp100-: fix number of zcull tile regs

Match RM.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/gr/gf117-: make ppc_nr[gpc] accurate
Ben Skeggs [Wed, 1 Jun 2022 10:48:14 +0000 (20:48 +1000)]
drm/nouveau/gr/gf117-: make ppc_nr[gpc] accurate

We're going to be pulling in a chunk of code from NVGPU to fixup our
SMID mappings on Volta and above, which depends on ppc_nr[gpc]
reflecting the actual number of PPCs present, not the maximum number.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/gr/gf100-: switch to newer style interrupt handler
Ben Skeggs [Wed, 1 Jun 2022 10:48:13 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: switch to newer style interrupt handler

Ampere.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/gr/gf100-: move some init to init_exception2()
Ben Skeggs [Wed, 1 Jun 2022 10:48:12 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: move some init to init_exception2()

Ampere.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/gr/gf100-: move some init to init_rop_exceptions()
Ben Skeggs [Wed, 1 Jun 2022 10:48:12 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: move some init to init_rop_exceptions()

Ampere.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/gr/gf100-: move reset during golden ctx init to fecs_reset()
Ben Skeggs [Wed, 1 Jun 2022 10:48:11 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: move reset during golden ctx init to fecs_reset()

Ampere.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/gr/gf100-: wfi after register-bashing golden init
Ben Skeggs [Wed, 1 Jun 2022 10:48:10 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: wfi after register-bashing golden init

Match RM.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/gr/gf100-: gpfifo_ctl zero before init
Ben Skeggs [Wed, 1 Jun 2022 10:48:10 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: gpfifo_ctl zero before init

Match RM.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/gr/gf100-: wait for FE_PWR_MODE_AUTO
Ben Skeggs [Wed, 1 Jun 2022 10:48:09 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: wait for FE_PWR_MODE_AUTO

This doesn't fix any known issue, but RM started doing it at some point,
so presumably it's needed for something.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/gr/gf100-: call FECS HALT_PIPE method before RC reset
Ben Skeggs [Wed, 1 Jun 2022 10:48:08 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: call FECS HALT_PIPE method before RC reset

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/gr/gf100-: call FECS WFI_GOLDEN_SAVE method
Ben Skeggs [Wed, 1 Jun 2022 10:48:08 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: call FECS WFI_GOLDEN_SAVE method

This won't work on Ampere, and, it's questionable whether we should have
been using our FW's method of storing the golden context image with NV's
firmware to begin with.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/gr/gf100-: stop using NV_PGRAPH_FECS_CTXSW_MAILBOX_CLEAR
Ben Skeggs [Wed, 1 Jun 2022 10:48:07 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: stop using NV_PGRAPH_FECS_CTXSW_MAILBOX_CLEAR

This doesn't work on Ampere for some reason, switch to directly modifying
NV_PGRAPH_FECS_CTXSW_MAILBOX instead.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/gr/gf100-: make global attrib_cb actually global
Ben Skeggs [Wed, 1 Jun 2022 10:48:07 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: make global attrib_cb actually global

This was thought to be per-channel initially - it's not.  The backing
pages for the VMM mappings are shared for all channels.

- switches to more straight-forward patch interfaces
- prepares for sub-context support
- this is saving a *sizeable* amount of vram

v2:
- whitespace

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/gr/gf100-: move misc context patching out of attrib_cb funcs
Ben Skeggs [Wed, 1 Jun 2022 10:48:06 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: move misc context patching out of attrib_cb funcs

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/gr/gf100-: make global bundle_cb actually global
Ben Skeggs [Wed, 1 Jun 2022 10:48:06 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: make global bundle_cb actually global

This was thought to be per-channel initially - it's not.  The backing
pages for the VMM mappings are shared for all channels.

- switches to more straight-forward patch interfaces
- prepares for sub-context support

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/gr/gf100-: make global pagepool actually global
Ben Skeggs [Wed, 1 Jun 2022 10:48:05 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: make global pagepool actually global

This was thought to be per-channel initially - it's not.  The backing
pages for the VMM mappings are shared for all channels.

- switches to more straight-forward patch interfaces
- prepares for sub-context support

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/gr/gf100-: generate golden context during first object alloc
Ben Skeggs [Wed, 1 Jun 2022 10:48:05 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: generate golden context during first object alloc

Needed for GV100 (and only GV100 for some reason) for WFI_GOLDEN_SAVE.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/gr/gf100-: move some code around to make next commits nicer
Ben Skeggs [Wed, 1 Jun 2022 10:48:04 +0000 (20:48 +1000)]
drm/nouveau/gr/gf100-: move some code around to make next commits nicer

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: expose function to read engine ctxsw status
Ben Skeggs [Wed, 1 Jun 2022 10:48:04 +0000 (20:48 +1000)]
drm/nouveau/fifo: expose function to read engine ctxsw status

Needed to support Ampere differences in gr/gf100-:

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/ltc: split color vs depth/stencil zbc counts
Ben Skeggs [Wed, 1 Jun 2022 10:48:03 +0000 (20:48 +1000)]
drm/nouveau/ltc: split color vs depth/stencil zbc counts

These differ on Ampere.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/engine: add HAL for engine-specific rc reset procedure
Ben Skeggs [Wed, 1 Jun 2022 10:48:02 +0000 (20:48 +1000)]
drm/nouveau/engine: add HAL for engine-specific rc reset procedure

Will be used to improve gr reset on GF100 and newer.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/sec2: dump tracepc info on halt
Ben Skeggs [Wed, 1 Jun 2022 10:47:53 +0000 (20:47 +1000)]
drm/nouveau/sec2: dump tracepc info on halt

- useful to distinguish between different issues.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/acr: use common falcon HS FW code for ACR FWs
Ben Skeggs [Wed, 1 Jun 2022 10:47:52 +0000 (20:47 +1000)]
drm/nouveau/acr: use common falcon HS FW code for ACR FWs

Adds context binding and support for FWs with a bootloader to the code
that was added to load VPR scrubber HS binaries, and ports ACR over to
using all of it.

- gv100 split from gp108 to handle FW exit status differences

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fb/gp102-: unlock VPR right after devinit
Ben Skeggs [Wed, 1 Jun 2022 10:47:52 +0000 (20:47 +1000)]
drm/nouveau/fb/gp102-: unlock VPR right after devinit

Under memory load, instmem allocations could end up in the regions of
VRAM that are inaccessible right after boot, and be corrupted after a
suspend/resume cycle as a result of being restored before booting the
mem unlock firmware.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fb: handle sysmem flush page from common code
Ben Skeggs [Wed, 1 Jun 2022 10:47:51 +0000 (20:47 +1000)]
drm/nouveau/fb: handle sysmem flush page from common code

- also executes pre-DEVINIT, so early boot is able to DMA sysmem

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/flcn: new code to load+boot simple HS FWs (VPR scrubber)
Ben Skeggs [Wed, 1 Jun 2022 10:47:51 +0000 (20:47 +1000)]
drm/nouveau/flcn: new code to load+boot simple HS FWs (VPR scrubber)

Adds the start of common interfaces to load and boot the HS binaries
provided by NVIDIA that enable the usage of GR.

ACR already handles most of this, but it's very much tied into ACR's
init process, and there's other code that could benefit from reusing
a lot of this stuff too (ie. VBIOS DEVINIT/PreOS, VPR scrubber).

The VPR scrubber code is fairly independent, and a good first target.

- adds better debug output to fw loading process, to ease bring-up/debug

v2:
- whitespace, 0->false

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/flcn: rework falcon reset
Ben Skeggs [Wed, 1 Jun 2022 10:47:51 +0000 (20:47 +1000)]
drm/nouveau/flcn: rework falcon reset

Mostly preparation to fit in Ampere changes, but should result in reset
sequences a lot closer to RM's, and perhaps help out with the issues we
sometimes see reported in this area.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/sec2: switch to newer style interrupt handler
Ben Skeggs [Wed, 1 Jun 2022 10:47:50 +0000 (20:47 +1000)]
drm/nouveau/sec2: switch to newer style interrupt handler

Ampere.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/sec2: unload RTOS before tearing down WPR
Ben Skeggs [Wed, 1 Jun 2022 10:47:49 +0000 (20:47 +1000)]
drm/nouveau/sec2: unload RTOS before tearing down WPR

Reset regs won't be available on Ampere while SEC2 RTOS is running, and
we're apparently supposed to be doing this on earlier GPUs too.

v2:
- fixed some excessive indentation

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/pmu/gm20b,gp10b: boot RTOS from PMU init
Ben Skeggs [Wed, 1 Jun 2022 10:47:49 +0000 (20:47 +1000)]
drm/nouveau/pmu/gm20b,gp10b: boot RTOS from PMU init

Cleanup before falcon changes.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/pmu: move init() falcon reset to non-nvfw code
Ben Skeggs [Wed, 1 Jun 2022 10:47:48 +0000 (20:47 +1000)]
drm/nouveau/pmu: move init() falcon reset to non-nvfw code

Cleanup before falcon changes.

- fixes (attempt at?) reset of pmu while rtos is running, on gm20b

v2:
- remove extra whitespace

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/pmu: move preinit() falcon reset to devinit
Ben Skeggs [Wed, 1 Jun 2022 10:47:48 +0000 (20:47 +1000)]
drm/nouveau/pmu: move preinit() falcon reset to devinit

Cleanup before falcon changes.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/gsp: add funcs
Ben Skeggs [Wed, 1 Jun 2022 10:47:47 +0000 (20:47 +1000)]
drm/nouveau/gsp: add funcs

Ampere.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo/ga100-: initial support
Ben Skeggs [Wed, 1 Jun 2022 10:47:39 +0000 (20:47 +1000)]
drm/nouveau/fifo/ga100-: initial support

- replaces the hacked-up version that existed solely to support TTM

v2. remove earlier hack preventing use of non-stall intr for fences

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
20 months agodrm/nouveau/ce/ga100-: initial support
Ben Skeggs [Wed, 1 Jun 2022 10:47:38 +0000 (20:47 +1000)]
drm/nouveau/ce/ga100-: initial support

- replaces the hacked-up version that existed solely to support TTM
- noop until the next commit, adding proper support for ampere host

v2. fixup for ga103 early merge

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
20 months agodrm/nouveau/fifo: add new channel classes
Ben Skeggs [Wed, 1 Jun 2022 10:47:38 +0000 (20:47 +1000)]
drm/nouveau/fifo: add new channel classes

Exposes a bunch of the new features that became possible as a result
of the earlier commits.  DRM will build on this in the future to add
support for features such as SCG ("async compute") and multi-device
rendering, as part of the work necessary to be able to write a half-
decent vulkan driver - finally.

For the moment, this just crudely ports DRM to the API changes.

- channel class interfaces now the same for all HW classes
- channel group class exposed (SCG)
- channel runqueue selector exposed (SCG)
- channel sub-device id control exposed (multi-device rendering)
- channel names in logging will reflect creating process, not fd owner
- explicit USERD allocation required by VOLTA_CHANNEL_GPFIFO_A and newer
- drm is smarter about determining the appropriate channel class to use

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: add new engine object handling
Ben Skeggs [Wed, 1 Jun 2022 10:47:37 +0000 (20:47 +1000)]
drm/nouveau/fifo: add new engine object handling

Simplifies the GPU-specific code, completing the switch to newer HALs.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: add new engine context handling
Ben Skeggs [Wed, 1 Jun 2022 10:47:37 +0000 (20:47 +1000)]
drm/nouveau/fifo: add new engine context handling

Builds on the context tracking that was added earlier.

- marks engine context PTEs as 'priv' where possible

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: add RAMFC info to nvkm_chan_func
Ben Skeggs [Wed, 1 Jun 2022 10:47:36 +0000 (20:47 +1000)]
drm/nouveau/fifo: add RAMFC info to nvkm_chan_func

- adds support for specifying SUBDEVICE_ID for channel
- rounds non-power-of-two GPFIFO sizes down, rather than up

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: add USERD info to nvkm_chan_func
Ben Skeggs [Wed, 1 Jun 2022 10:47:36 +0000 (20:47 +1000)]
drm/nouveau/fifo: add USERD info to nvkm_chan_func

And use it to cleanup multiple implementations of almost the same thing.

- prepares for non-polled / client-provided USERD
- only zeroes relevant "registers", rather than entire USERD

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: add RAMIN info to nvkm_chan_func
Ben Skeggs [Wed, 1 Jun 2022 10:47:35 +0000 (20:47 +1000)]
drm/nouveau/fifo: add RAMIN info to nvkm_chan_func

Currently provided by {chan,dma,gpfifo}*.c, and those are going away.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: add common runlist control
Ben Skeggs [Wed, 1 Jun 2022 10:47:35 +0000 (20:47 +1000)]
drm/nouveau/fifo: add common runlist control

- less dependence on waiting for runlist updates, on GPUs that allow it
- supports runqueue selector in RAMRL entries
- completes switch to common runl/cgrp/chan topology info

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: add common channel recovery
Ben Skeggs [Wed, 1 Jun 2022 10:47:34 +0000 (20:47 +1000)]
drm/nouveau/fifo: add common channel recovery

That sure was fun to untangle.

- handled per-runlist, rather than globally
- more straight-forward process in general
- various potential SW/HW races have been fixed
- fixes lockdep issues that were present in >=gk104's prior implementation
- volta recovery now actually stands a chance of working
- volta/turing waiting for PBDMA idle before engine reset
- turing using hw-provided TSG info for CTXSW_TIMEOUT

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: kill channel on NV_PPBDMA_INTR_1_CTXNOTVALID
Ben Skeggs [Wed, 1 Jun 2022 10:47:33 +0000 (20:47 +1000)]
drm/nouveau/fifo: kill channel on NV_PPBDMA_INTR_1_CTXNOTVALID

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: kill channel on a selection of PBDMA errors
Ben Skeggs [Wed, 1 Jun 2022 10:47:33 +0000 (20:47 +1000)]
drm/nouveau/fifo: kill channel on a selection of PBDMA errors

A bunch of these can be handled in such a way that the channel can
continue, however, any of these are a pretty decent sign something
has gone horribly wrong, and the safest option is to disable the
channel.

This is a bit of a hack, we will want to handle these individually
and dump relevant debug info for each at some point.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: add chan/cgrp preempt()
Ben Skeggs [Wed, 1 Jun 2022 10:47:33 +0000 (20:47 +1000)]
drm/nouveau/fifo: add chan/cgrp preempt()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: add chan start()/stop()
Ben Skeggs [Wed, 1 Jun 2022 10:47:32 +0000 (20:47 +1000)]
drm/nouveau/fifo: add chan start()/stop()

- nvkm_chan_error() built on top, stops channel and sends 'killed' event
- removes an odd double-bashing of channel enable regs on kepler and up
- pokes doorbell on turing and up, after enabling channel

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: add chan bind()/unbind()
Ben Skeggs [Wed, 1 Jun 2022 10:47:31 +0000 (20:47 +1000)]
drm/nouveau/fifo: add chan bind()/unbind()

- stops programming (non-existent) runl id field on bind(), from maxwell

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: add runlist block()/allow()
Ben Skeggs [Wed, 1 Jun 2022 10:47:31 +0000 (20:47 +1000)]
drm/nouveau/fifo: add runlist block()/allow()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: add runlist wait()
Ben Skeggs [Wed, 1 Jun 2022 10:47:30 +0000 (20:47 +1000)]
drm/nouveau/fifo: add runlist wait()

- adds g8x/turing registers, which were missing before
- switches fermi to polled wait, like later hw (see: 4f2fc25c0f8bc...)

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: add new engine context tracking
Ben Skeggs [Wed, 1 Jun 2022 10:47:30 +0000 (20:47 +1000)]
drm/nouveau/fifo: add new engine context tracking

Channel groups have somewhat more complicated requirements than what we
currently support.  An engine context is shared between all channels in
a channel group, VEID/subctx support (later) brings per-VEID components,
and we need to track an individual channel's engine context pointers.

This commit adds the structures and refcounting to support the above,
wrapping the prior implementation for the moment.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: add new channel lookup interfaces
Ben Skeggs [Wed, 1 Jun 2022 10:47:29 +0000 (20:47 +1000)]
drm/nouveau/fifo: add new channel lookup interfaces

- supports per-runlist CHIDs
- channel group lock held across reference, rather than global lock

v2:
- remove unnecessary parenthesis

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: merge mmu fault handlers together
Ben Skeggs [Wed, 1 Jun 2022 10:47:29 +0000 (20:47 +1000)]
drm/nouveau/fifo: merge mmu fault handlers together

After updating GF100 implementation from the GK104/TU102 ones, and using
the new runlist/engine topology info, all three handlers become (almost)
identical.

- there's a temporary kludge to call through to the HW-specific recovery
- engine fault mapping info determined at load time, not on every fault

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: move PBDMA intr to runq
Ben Skeggs [Wed, 1 Jun 2022 10:47:29 +0000 (20:47 +1000)]
drm/nouveau/fifo: move PBDMA intr to runq

- merges gf100/gk104- NV_PFIFO_INTR_0_PBDMA and NV_PPBDMA_INTR_0 code

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: move PBDMA init to runq
Ben Skeggs [Wed, 1 Jun 2022 10:47:28 +0000 (20:47 +1000)]
drm/nouveau/fifo: move PBDMA init to runq

- bumps pbdma timeout to value RM uses on newer HW
- bumps fb timeout to max from boot default
- one/both of these greatly improves stability on // piglit runs

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: program NV_PFIFO_FB_TIMEOUT on init
Ben Skeggs [Mon, 3 Oct 2022 03:20:07 +0000 (13:20 +1000)]
drm/nouveau/fifo: program NV_PFIFO_FB_TIMEOUT on init

NVGPU and RM both program this value.

Fixes a bunch of random hangs running parallel piglit.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
20 months agodrm/nouveau/fifo: tidy global PBDMA init
Ben Skeggs [Mon, 3 Oct 2022 03:19:08 +0000 (13:19 +1000)]
drm/nouveau/fifo: tidy global PBDMA init

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
20 months agodrm/nouveau/fifo: tidy up non-stall intr handling
Ben Skeggs [Wed, 1 Jun 2022 10:47:27 +0000 (20:47 +1000)]
drm/nouveau/fifo: tidy up non-stall intr handling

- removes a layer of indirection in the intr handling
- prevents non-stall ctrl racing with unknown intrs

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: use explicit intr interfaces
Ben Skeggs [Wed, 1 Jun 2022 10:47:26 +0000 (20:47 +1000)]
drm/nouveau/fifo: use explicit intr interfaces

More control, and shallower call-chain to get to the point.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: use runlist engine info to lookup engine classes
Ben Skeggs [Wed, 1 Jun 2022 10:47:26 +0000 (20:47 +1000)]
drm/nouveau/fifo: use runlist engine info to lookup engine classes

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: add cgrp, have all channels be part of one
Ben Skeggs [Wed, 1 Jun 2022 10:47:26 +0000 (20:47 +1000)]
drm/nouveau/fifo: add cgrp, have all channels be part of one

Engine context tracking will move to nvkm_cgrp in later commits, so we
create SW-only channel groups on HW without support for them.

- switches to nvkm_chid for TSG/channel ID allocation

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: expose per-runlist CHID information
Ben Skeggs [Wed, 1 Jun 2022 10:47:25 +0000 (20:47 +1000)]
drm/nouveau/fifo: expose per-runlist CHID information

DRM uses this to setup fence-related items.

- nouveau_chan.runlist will always be "0" for the moment, not an issue
  as GPUs prior to ampere have system-wide channel IDs,

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: expose runlist topology info on all chipsets
Ben Skeggs [Wed, 1 Jun 2022 10:47:25 +0000 (20:47 +1000)]
drm/nouveau/fifo: expose runlist topology info on all chipsets

Previously only available from Kepler onwards.

- also fixes the info() queries causing fifo init()/fini() unnecessarily

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: add common runlist/engine topology
Ben Skeggs [Wed, 1 Jun 2022 10:47:24 +0000 (20:47 +1000)]
drm/nouveau/fifo: add common runlist/engine topology

Creates an nvkm_runl for each runlist on the GPU, and an nvkm_engn for
each engine that is reachable from a runlist.

- basically what gk104- already does, but extended to all chips
- adds per-runlist CHID allocators (Ampere)
- splits g98/gt2xx out from g84 (different target engines)

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: add runq
Ben Skeggs [Wed, 1 Jun 2022 10:47:23 +0000 (20:47 +1000)]
drm/nouveau/fifo: add runq

Creates an nvkm_runq for each PBDMA, these will be associated with the
relevant runlist(s) later.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: add chid allocator
Ben Skeggs [Wed, 1 Jun 2022 10:47:23 +0000 (20:47 +1000)]
drm/nouveau/fifo: add chid allocator

We need to be able to allocate TSG IDs as well as channel IDs, also,
Ampere has per-runlist channel IDs.

- holds per-ID private data, which will be used for/to protect lookup
- holds an nvkm_event which will be used for events tied to IDs
- not used yet beyond setup, and switching use of "fifo->nr - 1" for
  channel ID mask to "chid->mask"

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: merge gk104_fifo_func into nvkm_host_func
Ben Skeggs [Wed, 1 Jun 2022 10:47:22 +0000 (20:47 +1000)]
drm/nouveau/fifo: merge gk104_fifo_func into nvkm_host_func

This makes it easier to transition everything.

- a couple of function renames for collisions

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: pre-move some blocks of code around
Ben Skeggs [Wed, 1 Jun 2022 10:47:22 +0000 (20:47 +1000)]
drm/nouveau/fifo: pre-move some blocks of code around

- will make subsequent patches more obvious
- no code changes

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: unify handling of channel classes
Ben Skeggs [Wed, 1 Jun 2022 10:47:21 +0000 (20:47 +1000)]
drm/nouveau/fifo: unify handling of channel classes

Adds the basic skeleton for common channel (group) interfaces.

- common behaviour between <gk104 and >=gk104 impl's
- separates priv/user channel objects
- passthrough to existing object for now, kludges removed later

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: add chid_nr()
Ben Skeggs [Wed, 1 Jun 2022 10:47:21 +0000 (20:47 +1000)]
drm/nouveau/fifo: add chid_nr()

- reads channel count from GPU from gm200 onwards
- removes gm20b/gp10b (they become identical to gm200/gp100)

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/imem: allow bar2 mapping of user allocations
Ben Skeggs [Wed, 1 Jun 2022 10:47:21 +0000 (20:47 +1000)]
drm/nouveau/imem: allow bar2 mapping of user allocations

Will be used to init client-allocated USERD to default values.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/flcn: show falcon user in debug output
Ben Skeggs [Wed, 1 Jun 2022 10:47:20 +0000 (20:47 +1000)]
drm/nouveau/flcn: show falcon user in debug output

Displays both owner/user of the falcon (when they differ), and takes
both subdevs' debug levels into account when deciding whether to log
the message.

- runlist debugging will use one of the alternate macros added here

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/nvkm: add locking to subdev/engine init paths
Ben Skeggs [Wed, 1 Jun 2022 10:47:19 +0000 (20:47 +1000)]
drm/nouveau/nvkm: add locking to subdev/engine init paths

This wasn't really needed before; the main place this could race is with
channel recovery, but (through potentially fragile means) shouldn't have
been possible.

However, a number of upcoming patches benefit from having better control
over subdev init, necessitating some improvements here.

- allows subdev/engine oneinit() without init() (host/fifo patches)
- merges engine use locking/tracking into subdev, and extends it to fix
  some issues that will arise with future usage patterns (acr patches)

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/mc/ga100: switch to using NV_PMC_DEVICE_ENABLE
Ben Skeggs [Wed, 1 Jun 2022 10:46:55 +0000 (20:46 +1000)]
drm/nouveau/mc/ga100: switch to using NV_PMC_DEVICE_ENABLE

- NV_PMC_ENABLE still exists, but we don't touch anything in it yet

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/mc: move NV_PMC_ENABLE bashing to chipset-specific code
Ben Skeggs [Wed, 1 Jun 2022 10:46:55 +0000 (20:46 +1000)]
drm/nouveau/mc: move NV_PMC_ENABLE bashing to chipset-specific code

Ampere needs different handling here, most of what we touch has moved.

We probably want to refactor these interfaces in general, but I'm not
yet sure how they should look, this will get the job done for now.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/mc: implement intr handling on top of nvkm_intr
Ben Skeggs [Wed, 1 Jun 2022 10:46:55 +0000 (20:46 +1000)]
drm/nouveau/mc: implement intr handling on top of nvkm_intr

- new-style handlers can now be used here too
- decent clean-up

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fault/ga100: initial support
Ben Skeggs [Wed, 1 Jun 2022 10:46:54 +0000 (20:46 +1000)]
drm/nouveau/fault/ga100: initial support

TU102 implementation should be OK for Ampere now.

v2. fixup for ga103 early merge

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
20 months agodrm/nouveau/fault/tu102: switch to explicit intr handlers
Ben Skeggs [Wed, 1 Jun 2022 10:46:54 +0000 (20:46 +1000)]
drm/nouveau/fault/tu102: switch to explicit intr handlers

- reads vectors from HW, rather than being hardcoded
- removes hacks to support routing via old interfaces

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/vfn/tu102-: support new-style interrupt tree
Ben Skeggs [Wed, 1 Jun 2022 10:46:54 +0000 (20:46 +1000)]
drm/nouveau/vfn/tu102-: support new-style interrupt tree

- switches ampere over now, and removes its hack mc implementation

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/vfn: move NV_USERMODE class from host
Ben Skeggs [Wed, 1 Jun 2022 10:46:53 +0000 (20:46 +1000)]
drm/nouveau/vfn: move NV_USERMODE class from host

- uses proper class IDs for Turing/Ampere

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/vfn: add stub subdev for dev_func
Ben Skeggs [Wed, 1 Jun 2022 10:46:53 +0000 (20:46 +1000)]
drm/nouveau/vfn: add stub subdev for dev_func

Initially for NV_USERMODE class, and Turing/Ampere's new interrupt tree.

v2. fixup for ga103 early merge

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
20 months agodrm/nouveau/intr: add nvkm_subdev_intr() compatibility
Ben Skeggs [Wed, 1 Jun 2022 10:46:53 +0000 (20:46 +1000)]
drm/nouveau/intr: add nvkm_subdev_intr() compatibility

It's quite a lot of tedious and error-prone work to switch over all the
subdevs at once, so allow an nvkm_intr to request new-style handlers to
be created that wrap the existing interfaces.

This will allow a more gradual transition.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/intr: support multiple trees, and explicit interfaces
Ben Skeggs [Wed, 1 Jun 2022 10:46:52 +0000 (20:46 +1000)]
drm/nouveau/intr: support multiple trees, and explicit interfaces

Turing adds a second top-level interrupt tree in HW, in addition to the
trees available via NV_PMC.  Most of the interrupts we care about are
exposed in both trees, but not all of them, and we have some rather
nasty hacks to route the fault buffer interrupts.

Ampere removes the NV_PMC trees entirely.

Here we add some infrastructure to be able to handle all of this more
cleanly, as well as providing more explicit control over handlers.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/intr: add shared interrupt plumbing between pci/tegra
Ben Skeggs [Wed, 1 Jun 2022 10:46:52 +0000 (20:46 +1000)]
drm/nouveau/intr: add shared interrupt plumbing between pci/tegra

Unifies the handling between PCI-based and Tegra GPUs, and makes more
explicit/obvious where device interrupts can be expected.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/top: parse device topology right after devinit
Ben Skeggs [Wed, 1 Jun 2022 10:46:51 +0000 (20:46 +1000)]
drm/nouveau/top: parse device topology right after devinit

We're going to want this information available earlier than it is now.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/nvkm: give each nvkm_event its own lockdep class
Ben Skeggs [Wed, 1 Jun 2022 10:46:51 +0000 (20:46 +1000)]
drm/nouveau/nvkm: give each nvkm_event its own lockdep class

The vblank and nonstall events have some annoying interactions with DRM
locking, and aren't able to do certain things as a result.

However, other uses of event notifications don't have such requirements,
and upcoming patches take advantage of this for various improvements.

Having separate classes for each nvkm_event's spinlocks allows lockdep
to distinguish between them and avoid false-positives.

v2: __always_inline + comment

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/kms: switch to drm fbdev helpers
Ben Skeggs [Wed, 1 Jun 2022 10:46:40 +0000 (20:46 +1000)]
drm/nouveau/kms: switch to drm fbdev helpers

This removes support for accelerated fbcon rendering, and fixes a number
of races/crashes/issues around suspend/resume/module unload etc.

Losing HW accelerated rendering isn't ideal, but it's been significantly
reduced in performance since the removal of accelerated scrolling in the
kernel anyway - not to mention, can be racey (skips cpu<->gpu sync) from
certain contexts.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
20 months agodrm/nouveau/nvkm: rip out old notify
Ben Skeggs [Wed, 1 Jun 2022 10:46:40 +0000 (20:46 +1000)]
drm/nouveau/nvkm: rip out old notify

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: expose channel killed in host channel event class
Ben Skeggs [Wed, 1 Jun 2022 10:46:39 +0000 (20:46 +1000)]
drm/nouveau/fifo: expose channel killed in host channel event class

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fifo: expose non-stall intr in host channel event class
Ben Skeggs [Wed, 1 Jun 2022 10:46:39 +0000 (20:46 +1000)]
drm/nouveau/fifo: expose non-stall intr in host channel event class

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/disp: expose page flip event class
Ben Skeggs [Wed, 1 Jun 2022 10:46:39 +0000 (20:46 +1000)]
drm/nouveau/disp: expose page flip event class

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/disp: expose conn event class
Ben Skeggs [Wed, 1 Jun 2022 10:46:38 +0000 (20:46 +1000)]
drm/nouveau/disp: expose conn event class

This removes some now-unnecessary nesting of workqueues.

v2:
- use ?: (lyude)

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/disp: expose head event class
Ben Skeggs [Wed, 1 Jun 2022 10:46:38 +0000 (20:46 +1000)]
drm/nouveau/disp: expose head event class

Also fixes vblank interrupts being left enabled when they're not meant
to be as a result of races/bugs in previous event handling code.

v2:
- use ?: (lyude)

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/disp: switch vblank semaphore release to nvkm_event_ntfy
Ben Skeggs [Wed, 1 Jun 2022 10:46:37 +0000 (20:46 +1000)]
drm/nouveau/disp: switch vblank semaphore release to nvkm_event_ntfy

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fault: expose replayable fault buffer event class
Ben Skeggs [Wed, 1 Jun 2022 10:46:37 +0000 (20:46 +1000)]
drm/nouveau/fault: expose replayable fault buffer event class

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/fault: switch non-replayable faults to nvkm_event_ntfy
Ben Skeggs [Wed, 1 Jun 2022 10:46:37 +0000 (20:46 +1000)]
drm/nouveau/fault: switch non-replayable faults to nvkm_event_ntfy

v2: fix flush_work() being called uninitialised during init

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/nvkm: add a replacement for nvkm_notify
Ben Skeggs [Wed, 1 Jun 2022 10:46:36 +0000 (20:46 +1000)]
drm/nouveau/nvkm: add a replacement for nvkm_notify

This replaces the twisty, confusing, relationship between nvkm_event and
nvkm_notify with something much simpler, and less racey.  It also places
events in the object tree hierarchy, which will allow a heap of the code
tracking events across allocation/teardown/suspend to be removed.

This commit just adds the new interfaces, and passes the owning subdev to
the event constructor to enable debug-tracing in the new code.

v2:
- use ?: (lyude)

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/disp: move head scanoutpos method
Ben Skeggs [Wed, 1 Jun 2022 10:46:36 +0000 (20:46 +1000)]
drm/nouveau/disp: move head scanoutpos method

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
20 months agodrm/nouveau/disp: add head class
Ben Skeggs [Wed, 1 Jun 2022 10:46:36 +0000 (20:46 +1000)]
drm/nouveau/disp: add head class

v2: remove extra whitespace

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>