drm/amdgpu: fix amdgpu_bo_va_mapping flags

They are 64bit not 32 for a while now.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: access stolen VRAM directly on CZ (v2)

We don't need to use the PCI BAR on APUs. This allows us to access
the full VRAM directly without being limited by the BAR size.

v2: squash in 64bit shift fix

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: access stolen VRAM directly on KV/KB (v2)

We don't need to use the PCI BAR on APUs. This allows us to access
the full VRAM directly without being limited by the BAR size.

v2: squash in 64bit shift fix

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: fix kernel panic when dpm disabled on Kv.

Return early if it's disabled.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: fix dpm bug on Kv.

1. current_ps/request_ps not update.
2. compare crrent_ps and request_ps, if same,
   don't re-set power state.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: fix regresstion issue can't set manual dpm mode.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: handle vfct with multiple vbios images

The vfct table can contain multiple vbios images if the
platform contains multiple GPUs. Noticed by netkas on
phoronix forums.  This patch fixes those platforms.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: handle vfct with multiple vbios images

The vfct table can contain multiple vbios images if the
platform contains multiple GPUs. Noticed by netkas on
phoronix forums.  This patch fixes those platforms.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm/amdgpu: move misc si headers into amdgpu

Move these to the amdgpu directory to match what we
do for other asics.

Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove unused header si_reg.h

All of these are available elsewhere.

Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: drop pitcairn dpm quirks

No longer necessary with the new 58 mc ucode.

bug:
https://bugs.freedesktop.org/show_bug.cgi?id=76490

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: drop pitcairn dpm quirks

No longer necessary with the new 58 mc ucode.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm: radeon: radeon_ttm: Handle return NULL error from ioremap_nocache

Here, If ioremap_nocache will fail. It will return NULL.
Kernel can run into a NULL-pointer dereference.
This error check will avoid NULL pointer dereference.

Signed-off-by: Arvind Yadav <arvind.yadav.cs@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/amdgpu/amdgpu_ttm: Handle return NULL error from ioremap_nocache

Here, If ioremap_nocache will fail. It will return NULL.
Kernel can run into a NULL-pointer dereference.
This error check will avoid NULL pointer dereference.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Arvind Yadav <arvind.yadav.cs@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add new virtual display ID

In the case of pass-through, amdgpu.ko may be included into a image
with the hard code ID therefore loading driver with specified virtual
display ID will lose efficacy when the BDF of GPU modifies.So add the
new ID string "all" for it as same as vf case what does.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Yintian Tao <yttao@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/amdgpu: remove the uncessary parameter for ib scheduler

Signed-off-by: Junwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Bring bo creation in line with radeon driver (v2)

Add the bo creation changes that have been done to the radeon driver in
recent times, e.g. disable GTT WC on 32 bit because it is broken there,
and also disable it generally (and print a warning message) when
CONFIG_X86_PAT is not set.

v2: agd: fix warning in defined(CONFIG_X86) && !defined(CONFIG_X86_PAT)
    case

Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Nils Holland <nholland@tisys.org>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: fix misspelling in header guard

In smu7_clockpowergating.h, the #ifndef statement which prevents
multiple inclusions of the header file uses _SMU7_CLOCK_POWER_GATING_H_
but the following #define statement uses _SMU7_CLOCK__POWER_GATING_H_.

Signed-off-by: Nicolas Iooss <nicolas.iooss_linux@m4x.org>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: revert "add optional LRU removal callback v2"

Without the custom LRU management the callback is not used any more.

agd: fix trivial warning

Reviewed-by: Sinclair Yeh <syeh@vmware.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-and-Tested-by: Roger.He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: revert "implement LRU add callbacks v2"

The additional housekeeping had too much CPU overhead,
let's use the BO priorities instead.

agd: also revert hibmc changes

Reviewed-by: Sinclair Yeh <syeh@vmware.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-and-Tested-by: Roger.He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: double the priority of kernel allocations

Give kernel allocations a higher priority cause it is often
more work to swap them back in.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger.He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: user BO priority instead of self coding it (v2)

Keeping groups of BOs on the LRU is to time consuming on command submission.

Instead use the newly added BO priority to give a certain eviction order.

v2: agd: trivial warning fix

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger.He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: add BO priorities for the LRUs

This way the driver can specify a priority for a BO which has the effect that
a BO is only evicted when all other BOs with a lower priority are evicted
first.

Reviewed-by: Sinclair Yeh <syeh@vmware.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger.He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: remove allow_errors parameter from ttm_bo_force_list_clean

Not allowing errors here is completely pointless and actually dangerous
cause trying to continue on an error can cause an endless loop.

Reviewed-by: Sinclair Yeh <syeh@vmware.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger.He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/vi: fix mailbox irq mistake

For virt, freed mailbox irq should be handled in hw fini, not hw
init. Correct it.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Reviewed-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: fix reboot failure issue for virtualization

Reboot process will call HW fini functions of IP blocks. For virt,
need to send event three before hw fini and send event four after
hw fini.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Reviewed-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/amdgpu: Add PCI info to gca_config debugfs

So we can determine which device the entry is before connecting
a display.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:insert switch buffer only for VM submit

for non-VM submit which is from kernel side, no need
to switch buffer at all.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:Preamble is forbid to be ignored in SRIOV

SR-IOV requires the preamble.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:set cond_exec polling value to 1 in ring_init

no need to set it per ib_schedule(), hw won't override
this polling address.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:in cntx_ctrl we need insert meta-init for CE/DE(V2)

to support SRIOV preemption.

v2:
fix emit_frame_size

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:introduce new flag to identify VM domain for cntx_ctrl

To determine whether the context uses GPUVM or not.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:implement CE/DE meta-init routines

those package need to insert into ring buffer for
SRIOV case. they are used to let CP do preemption.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:implement ring_write_multiple

Write multiple dwords to the ring.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:add META_DATA struct for CSA/SRIOV v2

META-DATA is used in GFX cmd submit, we have two
format suit for META-DATA-init, one is legacy and another
is for chained-ib preempt, which is used in vulkan
UMD.

v2: drop use CP version number to judge if chain-ib
supports or not, we wait for it mature

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: refine ci uvd dpm code.

Fix up the powergating logic.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Ack-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: simplify allocation of scratch regs

The scratch regs are sequential so there's no need to keep
them in an array, we can just return the index of the first
free register + the base register. Also change the array
of bools for keeping track of the free regs to a bitfield.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Nils Wallménius <nils.wallmenius@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Refactor flip into prepare submit and submit. (v3)

Make pflip atomic friendly. Split the fuinction into
whatever can fail part and the actual flip submit part.
Call the pre-submit function before atomic states
are swapped so in case of error we can fail the
IOCTL.

v2:
Update due to  target_vblank code change.
Fix identetation.
Change return type for amdgpu_crtc_submit_flip to void
v3: agd: fix formatting

Signed-off-by: Andrey Grodzovsky <Andrey.Grodzovsky@amd.com>
Reviewed-by Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Switch error code when bo pin fails. (v3)

Switching to -ENOMEM resolves a hang in case the driver
uses this function in atomic code path.

v2: Update commit message.
v3:
Remove forcing -EINVAL in case of amdgpu_bo_pin fail
since there is no apparent reason for not just passing
through the the return value.

Signed-off-by: Andrey Grodzovsky <Andrey.Grodzovsky@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by Harry Wentland <harry.wentland@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/gfx8: fix bugon.cocci warnings

 Use BUG_ON instead of a if condition followed by BUG.

Generated by: scripts/coccinelle/misc/bugon.cocci

Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewd-by: Xiangliang.Yu <Xiangliang.Yu@amd.com>
CC: Xiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: Julia Lawall <julia.lawall@lip6.fr>
Signed-off-by: Fengguang Wu <fengguang.wu@intel.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: use the num_rings variable for checking vce rings

Difference families may have different numbers of rings. Use
the variable rather than a hardcoded number.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/amdgpu: fix locking in bo creation error path

Unlock the resv lock only if we were the ones to lock it in the first
place.

Signed-off-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/amdgpu: lock reservation object while creating shadow bo

ttm_bo_init checks that the reservation object is locked. This is
the caller's responsibility when resv != NULL. Otherwise, the inline
reservation object of the newly allocated buffer is used and must
explicitly be locked.

Uninterruptible w/w locks without an acquire context are always
successful.

v2: use ww_mutex_lock

Signed-off-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net> (v1)
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: fix typo of CGTS

Fixes: 9e8590861e9 ('drm/amdgpu: add parse clock gating state')
Reported-by: Tom StDenis <Tom.StDenis@amd.com>
Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: do not reset gpu for virtualization

Current amdgpu reset process only works on bare-metal and for
SRIOV many inside it need re-work to adapt to vf device.

This is a temporary workaround to skip gpu reset.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/virt: enable virtual display

Virtual display is default setting for virtualization, enable it.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Monk Liu <Monk.Liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/vi: add support virtualization

Call VI virtualization functions if device is Vf.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: request/release full gpu access if device is vf

For gpu vf device, first need to request full gpu access before
accessing gpu registers, and release full gpu access after the
access is done.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/virt: implement VI virt operation interfaces

VI has asic specific virt support, which including mailbox and
golden registers init.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Signed-off-by: shaoyunl <Shaoyun.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/virt: add high level interfaces for virt

Add high level interfaces that is not relate to specific asic. So
asic files just need to implement the interfaces to support
virtualization.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Monk Liu <Monk.Liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/virt: use kiq to access registers (v2)

For virtualization, it is must for driver to use KIQ to access
registers when it is out of GPU full access mode.

v2: agd: rebase

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Monk Liu <Monk.Liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/virt: add runtime flag

Add new flag to define gpu runtime that is out of full gpu access.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/gfx8: implement emit_rreg/wreg function

Implement emit_rreg/wreg function for kiq ring.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/ring: add two interfaces to support r/w registers with kiq

During virtual runtime, need to send command to kiq ring to
read/write GPU registers. Add two interface to support the two
actions.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: Monk Linu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:invoke CSA functions (v2)

Make sure the CSA is mapped.

v2: agd: rebase.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:add new file for SRIOV

for SRIOV usage, CSA is only used per device and each
VM will map on it.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:new field members for SRIOV

and implement CSA functions in this file

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/amdgpu: add check that shadow page tables are GPU-accessible

Skip amdgpu_gem_va_update_vm otherwise. Also clean up the check for the
non-shadow page tables using the new helper function.

This fixes a crash with the stack trace:

amdgpu_gem_va_update_vm
-> amdgpu_vm_update_page_directory
 -> amdgpu_ttm_bind
  -> amdgpu_gtt_mgr_alloc

v2: actually check bo->shadow instead of just checking bo twice

Signed-off-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/amdgpu: add check that shadow page directory is GPU-accessible

Skip amdgpu_gem_va_update_vm when shadow the page directory is swapped out.
Clean up the check for non-shadow BOs as well using the new helper function.

This fixes a crash with the stack trace:

amdgpu_gem_va_update_vm
-> amdgpu_vm_update_page_directory
 -> amdgpu_ttm_bind
  -> amdgpu_gtt_mgr_alloc

Signed-off-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/amdgpu: add amdgpu_bo_gpu_accessible helper function

Signed-off-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/amdgpu: move eviction counting to amdgpu_bo_move_notify

This catches evictions of shadow page tables from the GART. Since shadow
page tables are always stored in system memory, amdgpu_bo_move is never
called for them.

This fixes a crash during command submission that occurs when only a shadow
page table and no other BOs were evicted since the last submission.

Fixes: 1baa439fb2f4e586 ("drm/amdgpu: allocate shadow for pd/pt bo V2")
Signed-off-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: add evict parameter to ttm_bo_driver::move_notify

Ensure that the driver can listen to evictions even when they don't take the
path through ttm_bo_driver::move.

This is crucial for amdgpu, which relies on an eviction counter to skip
re-binding page tables when possible.

Signed-off-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: refine vce3.0 initialize.

1. disable vce cg when vce hw initialize.
2. initizlize vce clock to 10KHz fo dgpu,
   so no need to set bypass clock to vce.

Change-Id: I934c2c4820cc95c1bfa2fa41ff0f40a0d3cd1c40
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: extend profiling mode.

in profiling mode, powerplay will fix power state
as stable as possible.and disable gfx cg and LBPW feature.

profile_standard: as a prerequisite, ensure power and thermal
sustainable, set clocks ratio as close to the highest clock
ratio as possible.
profile_min_sclk: fix mclk as profile_normal, set lowest sclk
profile_min_mclk: fix sclk as profile_normal, set lowest mclk
profile_peak: set highest sclk and mclk, power and thermal not
sustainable
profile_exit: exit profile mode. enable gfx cg/lbpw feature.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: refine DIDT feature in Powerplay.

Updating SQ DIDT settings and block mask
so SQ uses PCC on Polaris11.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: add new smu message.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: Configuring DIDT blocks only SQ enabled on Polaris11.

following firmware's request.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: fix bug return invalid value to sysfs.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add get clockgating_state method for vce v3

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add get clockgating_state method for uvd v5&v6

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add get clockgating_state method for vi common

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add get clockgating_state method for sdma v3

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add get clockgating_state method for gmc v8

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add get clockgating_state method for gfx v8

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add parse clock gating state

Suggested-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Cc: William Lewis <minutemaidpark@hotmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: introduce an interface to get clock gating status dynamically

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove detect_hw_virtualization interface

Call detection function driectly, so remove the interface.

V2: ci and si also need to call the detect function.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/vi: move virtualization detection forward

Move the detection forward into vi_set_ip_blocks function, then
add ip blocks virtualization need if device is VF.

V2: add ip blocks according to asic type.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/virt: rename fieldes of virtualization structure

Use acronym to rename fields to make easy to spell out.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/gfx8: add support kernel interface queue(KIQ)

KIQ is queue-memory based initialization method: setup KIQ queue
firstly, then send command to KIQ to setup other queues, without
accessing registers.

For virtualization, need KIQ to access virtual function registers
when running on guest mode.

V2: use amdgpu_bo_create/free_kernel to allocate BO.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: PCI I/O bar can be disabled

PCI I/O bar can be disabled in VBIOS to save the resource. It is often
disabled in large aperture VBIOS. Don't call it an error.

Signed-off-by: Amber Lin <Amber.Lin@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: change function name to make code more readable

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: refine smumgr code

1. delete asic_smum_init functions, export asic private functions
   to smumgr directly, make code more readable.
2. create asic private data in asic_init_func.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: fix memory leak in smu7_hwmgr

add pp_smu7_thermal_fini function to free related
data when smu7_hwmgr_fini.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: fix memory leak in cz_hwmgr.c

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: not free hwmgr/smumgr in asic private functions.

struct smumgr/hwmgr will be freed on amd_powerplay_destory

and if we free them in one of asic private functions, other private
date may not be freed. for example: power state and power table
in hwmgr.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: not use module parameter in powerplay.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: refine powerplay interface.

v2: add pp_check function to check pp_instance
   valid.

1. powerplay export two new interface to amdgpu,
   amd_powerplay_create/amd_powerplay_destroy.
2. create pp_instance/smumgr/hwmgr/eventmgr in
   early init, destroy them when lata_fini.
3. in sw_init, create and init asic private smumgr
   data, and free them when sw_fini.
4. in hw_init, create and init asic private hwmgr
   data, and free them when hw_fini.
5. export powerplay state: PP_DPM_DISABLED.
   when user disabled powerplay or hwmgr/eventmgr
   init failed, powerplay return this state to amdgpu.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: use same enter/exit safe mode for gfx_8.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: add profiling mode in dpm level

In some case, App need to run under max stable clock.
so export profiling mode: GFX CG was disabled.
and user can select the max stable clock of the device.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: delete dead definitions of dpm_ip_funcs

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: delete dead module parameter:amdgpu_powerplay.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: fix issue can't load another smu firmware.

need to release failed smu firmware before load another
firmware.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: refine code in cz_smumgr.c

cz_smu_init will be called in sw_init.
so it should not touch other blocks's
firmware as they were not ready.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: fix bug dpm level unexpectly be reset to auto.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: Unify dpm level defines

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: delete dpm code for Cz/St.

The powerplay implementation has been the default for a
while now.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: update all printk to pr_* on smumgr

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: update all printk to pr_* on hwmgr

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: update all printk to pr_* on eventmgr

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>