drm/i915/gt: Enable only one CCS for compute workload

Enable only one CCS engine by default with all the compute sices
allocated to it.

While generating the list of UABI engines to be exposed to the
user, exclude any additional CCS engines beyond the first
instance.

This change can be tested with igt i915_query.

Fixes: d2eae8e98d59 ("drm/i915/dg2: Drop force_probe requirement")
Signed-off-by: Andi Shyti <andi.shyti@linux.intel.com>
Cc: Chris Wilson <chris.p.wilson@linux.intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Cc: <stable@vger.kernel.org> # v6.2+
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Acked-by: Michal Mrozek <michal.mrozek@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20240328073409.674098-4-andi.shyti@linux.intel.com
(cherry picked from commit 2bebae0112b117de7e8a7289277a4bd2403b9e17)
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/i915/gt: Disable HW load balancing for CCS

The hardware should not dynamically balance the load between CCS
engines. Wa_14019159160 recommends disabling it across all
platforms.

Fixes: d2eae8e98d59 ("drm/i915/dg2: Drop force_probe requirement")
Signed-off-by: Andi Shyti <andi.shyti@linux.intel.com>
Cc: Chris Wilson <chris.p.wilson@linux.intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Cc: <stable@vger.kernel.org> # v6.2+
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Acked-by: Michal Mrozek <michal.mrozek@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20240328073409.674098-2-andi.shyti@linux.intel.com
(cherry picked from commit f5d2904cf814f20b79e3e4c1b24a4ccc2411b7e0)
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/i915/mtl: Update workaround 14018575942

Applying WA 14018575942 only on Compute engine has impact on
some apps like chrome. Updating this WA to apply on Render
engine as well as it is helping with performance on Chrome.

Note: There is no concern from media team thus not applying
WA on media engines. We will revisit if any issues reported
from media team.

V2(Matt):
- Use correct WA number

Fixes: 668f37e1ee11 ("drm/i915/mtl: Update workaround 14018778641")
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Signed-off-by: Andi Shyti <andi.shyti@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20240228103738.2018458-1-tejas.upadhyay@intel.com
(cherry picked from commit 71271280175aa0ed6673e40cce7c01296bcd05f6)
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/i915: switch from drm_debug_printer() to device specific drm_dbg_printer()

Prefer the device specific debug printer.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Reviewed-by: Luca Coelho <luciano.coelho@intel.com>
Acked-by: Maxime Ripard <mripard@kernel.org>
Link: https://patchwork.freedesktop.org/patch/msgid/f2614dfcba295be20c650cdab24c3979d265f422.1705410327.git.jani.nikula@intel.com

drm/i915/xelpg: Extend some workarounds/tuning to gfx version 12.74

Some of our existing Xe_LPG workarounds and tuning are also applicable
to the version 12.74 variant. Extend the condition bounds accordingly.
Also fix the comment on Wa_14018575942 while we're at it.

v2: Extend some more workarounds (Harish)

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Harish Chegondi <harish.chegondi@intel.com>
Signed-off-by: Haridhar Kalvala <haridhar.kalvala@intel.com>
Reviewed-by: Matt Atwood <matthew.s.atwood@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20240108122738.14399-4-haridhar.kalvala@intel.com

drm/i915/xelpg: Add workaround 14019877138

WA 14019877138 needed for Graphics 12.70/71 both

V2(Jani):
- Use drm/i915

Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20240103053111.763172-1-tejas.upadhyay@intel.com

drm/i915: Add Wa_14019877138

Enable Force Dispatch Ends Collection for DG2.

BSpec: 46001

Signed-off-by: Haridhar Kalvala <haridhar.kalvala@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20231213064612.480032-1-haridhar.kalvala@intel.com

drm/i915/dg2: Drop Wa_22014600077

This workaround has been dropped from all DG2 variants in the latest
workaround database update.

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20231127190043.4099109-2-matthew.d.roper@intel.com

drm/i915/dg2: Wa_18028616096 now applies to all DG2

The workaround database was just updated to extend this workaround to
DG2-G11 (whereas previously it applied only to G10 and G12).

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20231115182117.2551522-2-matthew.d.roper@intel.com

drm/i915/xelpmp: Add Wa_16021867713

This workaround applies to all steppings of Xe_LPM+. Implement the KMD
part.

v2:
- Put the definition of VDBOX_CGCTL3F1C() in the correct sort order.
(Matt)

Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Gustavo Sousa <gustavo.sousa@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20231106201959.156943-1-gustavo.sousa@intel.com

drm/i915: Set copy engine arbitration for Wa_16018031267 / Wa_16018063123

Set copy engine arbitration into round robin mode
for part of Wa_16018031267 / Wa_16018063123 mitigation.

Signed-off-by: Nirmoy Das <nirmoy.das@intel.com>
Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
Signed-off-by: Andrzej Hajda <andrzej.hajda@intel.com>
Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20231026-wabb-v6-4-4aa7d55d0a8a@intel.com

drm/i915/mtl: Add Wa_22016670082

Implemented workaround for XeLPM+
BSpec: 51762

Signed-off-by: Dnyaneshwar Bhadane <dnyaneshwar.bhadane@intel.com>
Reviewed-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20231025131709.3368517-1-dnyaneshwar.bhadane@intel.com

drm/i915/gt: More use of GT specific print helpers

A bunch of print messages got missed in the update to using sub-system
specific helpers. So update those.

Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20231009183802.673882-2-John.C.Harrison@Intel.com

drm/i915: Add Wa_18028616096

Drop UGM per set fragment threshold to 3

BSpec: 54833
Signed-off-by: Shekhar Chauhan <shekhar.chauhan@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
[mattrope: moved above xehpsdv block for consistency]
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230922155356.583595-1-shekhar.chauhan@intel.com

drm/i915/gt: skip WA verification for GEN7_MISCCPCTL on DG2

Some DG2 firmware locks this register for modification. Using wa_add
with read_mask 0 allows to skip checks of such registers.

Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/8945
Signed-off-by: Andrzej Hajda <andrzej.hajda@intel.com>
Reviewed-by: Nirmoy Das <nirmoy.das@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230912073521.2106162-1-andrzej.hajda@intel.com

drm/i915: Add Wa_14015150844

Disables Atomic-chaining of Typed Writes.

BSpec: 54040
Signed-off-by: Shekhar Chauhan <shekhar.chauhan@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230901045700.2553994-1-shekhar.chauhan@intel.com

drm/i915: Eliminate IS_MTL_GRAPHICS_STEP

Several workarounds are guarded by IS_MTL_GRAPHICS_STEP. However none
of these workarounds are actually tied to MTL as a platform; they only
relate to the Xe_LPG graphics IP, regardless of what platform it appears
in. At the moment MTL is the only platform that uses Xe_LPG with IP
versions 12.70 and 12.71, but we can't count on this being true in the
future. Switch these to use a new IS_GFX_GT_IP_STEP() macro instead
that is purely based on IP version. IS_GFX_GT_IP_STEP() is also
GT-based rather than device-based, which will help prevent mistakes
where we accidentally try to apply Xe_LPG graphics workarounds to the
Xe_LPM+ media GT and vice-versa.

v2:
- Switch to a more generic and shorter IS_GT_IP_STEP macro that can be
used for both graphics and media IP (and any other kind of GTs that
show up in the future).
v3:
- Switch back to long-form IS_GFX_GT_IP_STEP macro. (Jani)
- Move macro to intel_gt.h. (Andi)
v4:
- Build IS_GFX_GT_IP_STEP on top of IS_GFX_GT_IP_RANGE and
IS_GRAPHICS_STEP building blocks and name the parameters from/until
rather than begin/fixed. (Jani)
- Fix usage examples in comment.
v5:
- Tweak comment on macro. (Gustavo)

Cc: Gustavo Sousa <gustavo.sousa@intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
Cc: Andi Shyti <andi.shyti@linux.intel.com>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230821180619.650007-15-matthew.d.roper@intel.com

drm/i915/xelpg: Call Xe_LPG workaround functions based on IP version

Although some of our Xe_LPG workarounds were already being applied based
on IP version correctly, others were matching on MTL as a base platform,
which is incorrect. Although MTL is the only platform right now that
uses Xe_LPG IP, this may not always be the case. If a future platform
re-uses this graphics IP, the same workarounds should be applied, even
if it isn't a "MTL" platform.

We were also incorrectly applying Xe_LPG workarounds/tuning to the
Xe_LPM+ media IP in one or two places; we should make sure that we don't
try to apply graphics workarounds to the media GT and vice versa where
they don't belong. A new helper macro IS_GT_IP_RANGE() is added to help
ensure this is handled properly -- it checks that the GT matches the IP
type being tested as well as the IP version falling in the proper range.

Note that many of the stepping-based workarounds are still incorrectly
checking for a MTL base platform; that will be remedied in a later
patch.

v2:
- Rework macro into a slightly more generic IS_GT_IP_RANGE() that can
be used for either GFX or MEDIA checks.

v3:
- Switch back to separate macros for gfx and media. (Jani)
- Move macro to intel_gt.h. (Andi)

Cc: Gustavo Sousa <gustavo.sousa@intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Andi Shyti <andi.shyti@linux.intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230821180619.650007-14-matthew.d.roper@intel.com

drm/i915/xelpmp: Don't assume workarounds extend to future platforms

The currently implemented Xe_LPM+ workarounds are specific to media
version 13.00. When new IP versions show up in the future, they'll need
their own workaround lists.

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230821180619.650007-13-matthew.d.roper@intel.com

drm/i915: Tidy workaround definitions

Removal of the DG2 pre-production workarounds has left duplicate
condition blocks in a couple places, as well as some inconsistent
platform ordering. Reshuffle and consolidate some of the workarounds to
reduce the number of condition blocks and to more consistently follow
the "newest platform first" convention. Code movement only; no
functional change.

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Acked-by: Jani Nikula <jani.nikula@intel.com>
Reviewed-by: Matt Atwood <matthew.s.atwood@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230816214201.534095-11-matthew.d.roper@intel.com

drm/i915/dg2: Drop pre-production GT workarounds

DG2 first production steppings were C0 (for DG2-G10), B1 (for DG2-G11),
and A1 (for DG2-G12). Several workarounds that apply onto to
pre-production hardware can be dropped. Furthermore, several
workarounds that apply to all production steppings can have their
conditions simplified to no longer check the GT stepping.

v2:
- Keep Wa_16011777198 in place for now; it will be removed separately
in a follow-up patch to keep review easier.

Bspec: 44477
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Acked-by: Jani Nikula <jani.nikula@intel.com>
Reviewed-by: Matt Atwood <matthew.s.atwood@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230816214201.534095-10-matthew.d.roper@intel.com

drm/i915/dg2: Remove Wa_15010599737

Since this Wa is specific to DirectX, this is not required on Linux.

Signed-off-by: Shekhar Chauhan <shekhar.chauhan@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230814150215.873941-1-shekhar.chauhan@intel.com

drm/i915/jsl: s/JSL/JASPERLAKE for platform/subplatform defines

Follow consistent naming convention. Replace JSL with
JASPERLAKE. Unroll IS_JSL_EHL() define with IS_JASPERLAKE() ||
IS_ELKHARTLAKE() condition. Change in the display step define for
Jasperlake.

v2:
- Change subject prefix skl instead of SKL(Anusha)

v3:
- Remove the use of define IS_JSL_EHL.
- Replace with IS_JASPERLAKE() || IS_ELKHARTLAKE()
- Unrolled wrapper IS_JSL_ELK_DISPLAY_STEP (Jani/Tvrtko)

v4:
- Removed unused macro

v5:
- Resolved valid checkpatch warning(Jani)

Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Anusha Srivatsa <anusha.srivatsa@intel.com>
Signed-off-by: Dnyaneshwar Bhadane <dnyaneshwar.bhadane@intel.com>
Acked-by: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230801135344.3797924-9-dnyaneshwar.bhadane@intel.com

drm/i915/kbl: s/KBL/KABYLAKE for platform/subplatform defines

Follow consistent naming convention. Replace KBL with
KABYLAKE.Replace IS_KBL_GRAPHICS_STEP with
IS_KABYLAKE () && IS_GRAPHICS_STEP().

v2:
- s/KBL/kbl in the subject prefix(Anusha)

v3:
- Unrolled wrapper IS_KBL_GRAPHICS_STEP.
- Replace with IS_PLATFORM && DISPLAY_STEP(tvrtko/jani)

v4:
- Removed unused macro.

Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Anusha Srivatsa <Anusha.Srivatsa@intel.com>
Signed-off-by: Dnyaneshwar Bhadane <dnyaneshwar.bhadane@intel.com>
Reviewed-by: Anusha Srivatsa <anusha.srivatsa@intel.com>
Acked-by: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230801135344.3797924-5-dnyaneshwar.bhadane@intel.com

drm/i915/skl: s/SKL/SKYLAKE for platform/subplatform defines

Follow consistent naming convention. Replace SKL with
SKYLAKE and Replace IS_SKL_GRAPHICS_STEP with
IS_SKYLAKE() && IS_GRAPHICS_STEP().

v2:
- Change subject skl instead of SKL(Anusha)

v3:
- Unrolled wrapper IS_SKL_GRAPHICS_STEP.
- Replace with IS_PLATFORM && DISPLAY_STEP(tvrtko/jani)

v4:
- Removed the unused macro.

Cc: Anusha Srivatsa <anusha.srivatsa@intel.com>
Signed-off-by: Dnyaneshwar Bhadane <dnyaneshwar.bhadane@intel.com>
Reviewed-by: Anusha Srivatsa <anusha.srivatsa@intel.com>
Acked-by: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230801135344.3797924-4-dnyaneshwar.bhadane@intel.com

drm/i915/bdw: s/BDW/BROADWELL for platform/subplatform defines

Follow consistent naming convention. Replace BDW with
BROADWELL.

Signed-off-by: Dnyaneshwar Bhadane <dnyaneshwar.bhadane@intel.com>
Reviewed-by: Anusha Srivatsa <anusha.srivatsa@intel.com>
Acked-by: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230801135344.3797924-3-dnyaneshwar.bhadane@intel.com

drm/i915: use direct alias for i915 in requests

i915_request contains direct alias to i915, there is no point to go via
rq->engine->i915.

v2: added missing rq.i915 initialization in measure_breadcrumb_dw.

Signed-off-by: Andrzej Hajda <andrzej.hajda@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Acked-by: Nirmoy Das <nirmoy.das@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230720113002.1541572-1-andrzej.hajda@intel.com

drm/i915/gt: Also check set bits in clr_set()

When checking if the workarounds were applied successfully, the
read-back mask should also contain the bits being set: it's possible
that in a call to wa_write_clr_set(), the cleared bits are not a
superset of the set bits.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Link: https://patchwork.freedesktop.org/patch/msgid/20230630203509.1635216-8-lucas.demarchi@intel.com

drm/i915/gt: Remove bogus comment on IVB_FBC_RT_BASE_UPPER

The comment on the parameter being 0 to avoid the read back doesn't
apply as this is not a call to wa_add(), but rather to
wa_write_clr_set(). So, this register is actually checked and it's
according to the Bspec that the register is RW, not RO.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Link: https://patchwork.freedesktop.org/patch/msgid/20230630203509.1635216-7-lucas.demarchi@intel.com

drm/i915/gt: Enable read back on XEHP_FF_MODE2

Contrary to GEN12_FF_MODE2, platforms using XEHP_FF_MODE2 are not
affected by Wa_1608008084, hence read back can be enabled.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Link: https://patchwork.freedesktop.org/patch/msgid/20230630203509.1635216-6-lucas.demarchi@intel.com

drm/i915/gt: Drop read from GEN8_L3CNTLREG in ICL workaround

Now that non-masked registers are already read before programming the
context reads, the additional read became redudant, so remove it.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Link: https://patchwork.freedesktop.org/patch/msgid/20230630203509.1635216-5-lucas.demarchi@intel.com

drm/i915/gt: Fix context workarounds with non-masked regs

Most of the context workarounds tweak masked registers, but not all. For
masked registers, when writing the value it's sufficient to just write
the wa->set_bits since that will take care of both the clr and set bits
as well as not overwriting other bits.

However there are some workarounds, the registers are non-masked. Up
until now the driver was simply emitting a MI_LOAD_REGISTER_IMM with the
set_bits to program the register via the GPU in the WA bb. This has the
side effect of overwriting the content of the register outside of bits
that should be set and also doesn't handle the bits that should be
cleared.

Kenneth reported that on DG2, mesa was seeing a weird behavior due to
the kernel programming of L3SQCREG5 in dg2_ctx_gt_tuning_init(). With
the GPU idle, that register could be read via intel_reg as 0x00e001ff,
but during a 3D workload it would change to 0x0000007f. So the
programming of that tuning was affecting more than the bits in
L3_PWM_TIMER_INIT_VAL_MASK. Matt Roper noticed the lack of rmw for the
context workarounds due to the use of MI_LOAD_REGISTER_IMM.

So, for registers that are not masked, read its value via mmio, modify
and then set it in the buffer to be written by the GPU. This should take
care in a simple way of programming just the bits required by the
tuning/workaround. If in future there are registers that involved that
can't be read by the CPU, a more complex approach may be required like
a) issuing additional instructions to read and modify; or b) scan the
golden context and patch it in place before saving it; or something
else. But for now this should suffice.

Scanning the context workarounds for all platforms, these are the
impacted ones with the respective registers

mtl: DRAW_WATERMARK
mtl/dg2: XEHP_L3SQCREG5, XEHP_FF_MODE2

ICL has some non-masked registers in the context workarounds:
GEN8_L3CNTLREG, IVB_FBC_RT_BASE and VB_FBC_RT_BASE_UPPER, but there
shouldn't be an impact. The first is already being manually read and the
other 2 are intentionally overwriting the entire register. Same
reasoning applies to GEN12_FF_MODE2: the WA is intentionally
overwriting all the bits to avoid a read-modify-write.

v2: Reword commit message wrt GEN12_FF_MODE2 and the changed behavior
on preparatory patches.
v3: Also skip reading if clear|set bits covers everything

Cc: Kenneth Graunke <kenneth@whitecape.org>
Cc: Matt Roper <matthew.d.roper@intel.com>
Link: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23783#note_1968971
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Link: https://patchwork.freedesktop.org/patch/msgid/20230630203509.1635216-4-lucas.demarchi@intel.com

drm/i915/gt: Clear all bits from GEN12_FF_MODE2

Right now context workarounds don't do a rmw and instead only write to
the register. Since 2 separate programmings to the same register are
coalesced into a single write, this is not problematic for
GEN12_FF_MODE2 since both TDS and GS timer are going to be written
together and the other remaining bits be zeroed.

However in order to fix other workarounds that may want to preserve the
unrelated bits in the same register, context workarounds need to
be changed to a rmw. To prepare for that, move the programming of
GEN12_FF_MODE2 to a single place so the value passed for "clear" can
be all the bits. Otherwise the second workaround would be dropped as
it'd be detected as overwriting a previously programmed workaround.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230630203509.1635216-3-lucas.demarchi@intel.com

drm/i915/gt: Move wal_get_fw_for_rmw()

Move helper function to get all the forcewakes required by the wa list
to the top, so it can be re-used by other functions.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Link: https://patchwork.freedesktop.org/patch/msgid/20230630203509.1635216-2-lucas.demarchi@intel.com

drm/i915: Extend Wa_14015795083 platforms

This workaround was already implemented for DG2, PVC, and some steppings
of MTL, but the workaround database has now been updated to extend this
workaround to TGL, RKL, DG1, and ADL.

v2:
- Skip readback verification for these extra gen12lp platforms. On
some of the platforms, the firmware locks this register, preventing
the driver from making any modifications. We should still try to
apply the workaround, but if the register is locked and the value
doesn't stick, that's semi-expected and not something we want to flag
as a driver error on debug builds.

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Haridhar Kalvala <haridhar.kalvala@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230616225041.3922719-1-matthew.d.roper@intel.com

drm/i915/mtl: Update workaround 14018778641

WA 14018778641 needs an update after recent
performance data on MTL, aligning driver here with
HW WA update.

Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Signed-off-by: Andi Shyti <andi.shyti@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230619090326.3039040-1-tejas.upadhyay@intel.com

drm/i915/mtl: Add MTL performance tuning changes

MTL reuses the tuning parameters for DG2. Extend the dg2
performance tuning parameters to MTL.

v2: Add DRAW_WATERMARK tuning parameter.
v3: Limit DRAW_WATERMARK tuning to non A0 step.
v4: Reorder platform checks.
Restrict Blend fill caching optimization to Render GT.
v5: Move mtl tuning params to its own function

Bspec: 68331
Cc: Haridhar Kalvala <haridhar.kalvala@intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Signed-off-by: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230517233111.297542-1-radhakrishna.sripada@intel.com

drm/i915/mtl: Fix the wa number for Wa_22016670082

Fixes the right lineage number for the workaround.

Fixes: a7fa1537b791 ("drm/i915/mtl: Implement Wa_14019141245")
Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230505234544.4029535-1-radhakrishna.sripada@intel.com

drm/i915/mtl: Implement Wa_14019141245

Enable strict RAR to prevent spurious GPU hangs.

v1.1: Rebase
Bspec: 51762
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
Reviewed-by:Haridhar Kalvala <haridhar.kalvala@intel.com>
Reviewed-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230425183011.865085-1-radhakrishna.sripada@intel.com

drm/i915/mtl: Add workaround 14018778641

WA 18018781329 is applicable now across all MTL
steppings.

V2:
- Remove IS_MTL check, code already running for MTL - Matt

Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Reviewed-by: Andrzej Hajda <andrzej.hajda@intel.com>
Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Signed-off-by: Andi Shyti <andi.shyti@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230424101749.3719600-1-tejas.upadhyay@intel.com

drm/i915/mtl: WA to clear RDOP clock gating

Workaround implementation to clear RDOP clock gating.

Bspec: 66622
Signed-off-by: Haridhar Kalvala <haridhar.kalvala@intel.com>
Signed-off-by: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
Reviewed-by: Matt Atwood <matthew.s.atwood@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230418220446.2205509-5-radhakrishna.sripada@intel.com

drm/i915: disable sampler indirect state in bindless heap

By default the indirect state sampler data (border colors) are stored
in the same heap as the SAMPLER_STATE structure. For userspace drivers
that can be 2 different heaps (dynamic state heap & bindless sampler
state heap). This means that border colors have to copied in 2
different places so that the same SAMPLER_STATE structure find the
right data.

This change is forcing the indirect state sampler data to only be in
the dynamic state pool (more convenient for userspace drivers, they
only have to have one copy of the border colors). This is reproducing
the behavior of the Windows drivers.

BSpec: 46052

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: stable@vger.kernel.org
Reviewed-by: Haridhar Kalvala <haridhar.kalvala@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230407093237.3296286-1-lionel.g.landwerlin@intel.com

drm/i915: disable sampler indirect state in bindless heap

By default the indirect state sampler data (border colors) are stored
in the same heap as the SAMPLER_STATE structure. For userspace drivers
that can be 2 different heaps (dynamic state heap & bindless sampler
state heap). This means that border colors have to copied in 2
different places so that the same SAMPLER_STATE structure find the
right data.

This change is forcing the indirect state sampler data to only be in
the dynamic state pool (more convenient for userspace drivers, they
only have to have one copy of the border colors). This is reproducing
the behavior of the Windows drivers.

BSpec: 46052

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: stable@vger.kernel.org
Reviewed-by: Haridhar Kalvala <haridhar.kalvala@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230407093237.3296286-1-lionel.g.landwerlin@intel.com
(cherry picked from commit 16fc9c08f0ec7b1c95f1ea4a16097acdb3fc943d)
Signed-off-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>

drm/i915/mtl: Add Wa_14017856879

Wa_14017856879 implementation for mtl.

Bspec: 46046

Signed-off-by: Haridhar Kalvala <haridhar.kalvala@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230404173220.3175577-1-haridhar.kalvala@intel.com

drm/i915/mtl: Add Wa_22015279794

Wa_22015279794 applies to MTL P from stepping A0 to B0 (exclusive).

Signed-off-by: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
Signed-off-by: Gustavo Sousa <gustavo.sousa@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230329212336.106161-3-gustavo.sousa@intel.com

drm/i915/mtl: Add workarounds Wa_14017066071 and Wa_14017654203

Both workarounds require the same implementation and apply to MTL P and
M from stepping A0 to B0 (exclusive).

v2:
- Remove unrelated brace removal. (Matt)

Signed-off-by: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
Signed-off-by: Gustavo Sousa <gustavo.sousa@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230329212336.106161-2-gustavo.sousa@intel.com

drm/i915: Move DG2 tuning to the right function

Use gt_tuning_settings() for the recommended tunings rather than the one
for workarounds.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230306204954.753739-2-lucas.demarchi@intel.com

drm/i915: Remove redundant check for DG1

dg1_gt_workarounds_init() is only ever called for DG1, so there is no
point checking it again.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230306204954.753739-1-lucas.demarchi@intel.com

drm/i915/xelp: Implement Wa_1606376872

Wa_1606376872 applies to all Xe_LP IPs except DG1.

Signed-off-by: Gustavo Sousa <gustavo.sousa@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230307032238.300674-1-gustavo.sousa@intel.com

drm/i915: Stop whitelisting CS_CTX_TIMESTAMP on Xe_HP platforms

Xe_HP architecture already makes the CS_CTX_TIMESTAMP readable by
userspace on all engines; there's no longer a need to add it to the
software-managed whitelist for the non-RCS engines.

Bspec: 45545
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Acked-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230224002300.3578985-2-matthew.d.roper@intel.com

drm/i915: Whitelist COMMON_SLICE_CHICKEN3 for UMD access

A recommended tuning setting for both gen12 and Xe_HP platforms requires
that we grant userspace r/w access to the COMMON_SLICE_CHICKEN3
register.

Bspec: 73993, 73994, 31870, 68331
Cc: Dongwon Kim <dongwon.kim@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230224002300.3578985-1-matthew.d.roper@intel.com

drm/i915/xehp: LNCF/LBCF workarounds should be on the GT list

Although registers in the L3 bank/node configuration ranges are marked
as having "DEV" reset characteristics in the bspec, this appears to be a
hold-over from pre-Xe_HP platforms. In reality, these registers
maintain their values across engine resets, meaning that workarounds
and tuning settings targeting them should be placed on the GT
workaround list rather than an engine workaround list.

Note that an extra clue here is that these registers moved from the
RENDER forcewake domain to the GT forcewake domain in Xe_HP; generally
RCS/CCS engine resets should not lead to the reset of a register that
lives outside the RENDER domain.

Re-applying these registers on engine resets wouldn't actually hurt
anything, but is unnecessary and just makes it more confusing to anyone
trying to decipher how these registers really work.

v2:
- Also move DG2's Wa_14010648519 to the GT list. (Gustavo)

Cc: Gustavo Sousa <gustavo.sousa@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230209232228.859317-1-matthew.d.roper@intel.com

drm/i915: Fix GEN8_MISCCPCTL

Register 0x9424 is not replicated on any platform, so it shouldn't be
declared with REG_MCR(). Declaring it with _MMIO() is basically
duplicate of the GEN7 version, so just remove the GEN8 and change all
the callers to use the right functions.

Old versions of the gen8 bspec page used to contain a table with MCR
registers, apparently implying 0x9400 - 0x94ff registers were
replicated. However that table went away and there is no information
related to the ranges for gen8 anymore. Moreover the current behavior of
the driver wouldn't do anything special for 0x9424 since there is no
equivalent table in intel_gt_mcr.c: the driver would just fallback to
intel_uncore_{read,write}(). Therefore, do not care about the possible
special case for gen8 and just use the register as non-MCR for all the
platforms.

One place doing read + write is also converted to intel_uncore_rmw().

v2: Reword commit message adding the justification wrt gen8

Fixes: a9e69428b1b4 ("drm/i915: Define MCR registers explicitly")
Cc: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Gustavo Sousa <gustavo.sousa@intel.com>
Cc: Matt Atwood <matthew.s.atwood@intel.com>
Cc: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230206165410.3056073-1-lucas.demarchi@intel.com

drm/i915/gen11: Wa_1408615072/Wa_1407596294 should be on GT list

The UNSLICE_UNIT_LEVEL_CLKGATE register programmed by this workaround
has 'BUS' style reset, indicating that it does not lose its value on
engine resets. Furthermore, this register is part of the GT forcewake
domain rather than the RENDER domain, so it should not be impacted by
RCS engine resets. As such, we should implement this on the GT
workaround list rather than an engine list.

Bspec: 19219
Fixes: 3551ff928744 ("drm/i915/gen11: Moving WAs to rcs_engine_wa_init()")
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230201222831.608281-2-matthew.d.roper@intel.com

drm/i915/pvc: Annotate two more workaround/tuning registers as MCR

XEHPC_LNCFMISCCFGREG0 and XEHPC_L3SCRUB are both in MCR register ranges
on PVC (with HALFBSLICE and L3BANK replication respectively), so they
should be explicitly declared as MCR registers and use MCR-aware
workaround handlers.

The workarounds/tuning settings should still be applied properly on PVC
even without the MCR annotation, but readback verification on
CONFIG_DRM_I915_DEBUG_GEM builds could potentitally give false positive
"workaround lost on load" warnings on parts fused such that a unicast
read targets a terminated register instance.

Fixes: a9e69428b1b4 ("drm/i915: Define MCR registers explicitly")
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230201222831.608281-1-matthew.d.roper@intel.com

drm/i915/doc: Escape wildcard in method names

Stephen Rothwell reported htmldocs warnings:

Documentation/gpu/i915:64: drivers/gpu/drm/i915/gt/intel_workarounds.c:32: WARNING: Inline emphasis start-string without end-string.
Documentation/gpu/i915:64: drivers/gpu/drm/i915/gt/intel_workarounds.c:57: WARNING: Inline emphasis start-string without end-string.
Documentation/gpu/i915:64: drivers/gpu/drm/i915/gt/intel_workarounds.c:66: WARNING: Inline emphasis start-string without end-string.

Escape wildcards in *_ctx_workarounds_init(), *_gt_workarounds_init(), and
*_whitelist_build() to fix above warnings.

Link: https://lore.kernel.org/linux-next/20230203134622.0b6315b9@canb.auug.org.au/
Fixes: 0c3064cf33fbfa ("drm/i915/doc: Document where to implement register workarounds")
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230203100215.31852-2-bagasdotme@gmail.com

drm/i915/dg1: Drop support for pre-production steppings

Several post-DG1 platforms have been brought up now, so we're well past
the point where we usually drop the workarounds that are only applicable
to internal/pre-production hardware.

Production DG1 hardware always has a B0 stepping for both display and
GT.

Bspec: 44463
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230127224313.4042331-3-matthew.d.roper@intel.com

drm/i915/tgl: Drop support for pre-production steppings

Several post-TGL platforms have been brought up now, so we're well past
the point where we usually drop the workarounds that are only applicable
to internal/pre-production hardware.

Production TGL hardware always has display stepping C0 or later and GT
stepping B0 or later (this is true for both the original TGL and the U/Y
subplatform).

Bspec 44455
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230127224313.4042331-2-matthew.d.roper@intel.com

drm/i915: Fix GEN8_MISCCPCTL

Register 0x9424 is not replicated on any platform, so it shouldn't be
declared with REG_MCR(). Declaring it with _MMIO() is basically
duplicate of the GEN7 version, so just remove the GEN8 and change all
the callers to use the right functions.

Old versions of the gen8 bspec page used to contain a table with MCR
registers, apparently implying 0x9400 - 0x94ff registers were
replicated. However that table went away and there is no information
related to the ranges for gen8 anymore. Moreover the current behavior of
the driver wouldn't do anything special for 0x9424 since there is no
equivalent table in intel_gt_mcr.c: the driver would just fallback to
intel_uncore_{read,write}(). Therefore, do not care about the possible
special case for gen8 and just use the register as non-MCR for all the
platforms.

One place doing read + write is also converted to intel_uncore_rmw().

v2: Reword commit message adding the justification wrt gen8

Fixes: a9e69428b1b4 ("drm/i915: Define MCR registers explicitly")
Cc: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Gustavo Sousa <gustavo.sousa@intel.com>
Cc: Matt Atwood <matthew.s.atwood@intel.com>
Cc: Ashutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230206165410.3056073-1-lucas.demarchi@intel.com
(cherry picked from commit 869bace73ae2b4227e57ee3fd994bfa7d4808938)
Signed-off-by: Jani Nikula <jani.nikula@intel.com>

drm/i915/pvc: Annotate two more workaround/tuning registers as MCR

XEHPC_LNCFMISCCFGREG0 and XEHPC_L3SCRUB are both in MCR register ranges
on PVC (with HALFBSLICE and L3BANK replication respectively), so they
should be explicitly declared as MCR registers and use MCR-aware
workaround handlers.

The workarounds/tuning settings should still be applied properly on PVC
even without the MCR annotation, but readback verification on
CONFIG_DRM_I915_DEBUG_GEM builds could potentitally give false positive
"workaround lost on load" warnings on parts fused such that a unicast
read targets a terminated register instance.

Fixes: a9e69428b1b4 ("drm/i915: Define MCR registers explicitly")
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230201222831.608281-1-matthew.d.roper@intel.com
(cherry picked from commit 4039e44237e8ebb06f0e4af549fbedf7c41df9db)
Signed-off-by: Jani Nikula <jani.nikula@intel.com>

drm/i915/doc: Escape wildcard in method names

Stephen Rothwell reported htmldocs warnings:

Documentation/gpu/i915:64: drivers/gpu/drm/i915/gt/intel_workarounds.c:32: WARNING: Inline emphasis start-string without end-string.
Documentation/gpu/i915:64: drivers/gpu/drm/i915/gt/intel_workarounds.c:57: WARNING: Inline emphasis start-string without end-string.
Documentation/gpu/i915:64: drivers/gpu/drm/i915/gt/intel_workarounds.c:66: WARNING: Inline emphasis start-string without end-string.

Escape wildcards in *_ctx_workarounds_init(), *_gt_workarounds_init(), and
*_whitelist_build() to fix above warnings.

Link: https://lore.kernel.org/linux-next/20230203134622.0b6315b9@canb.auug.org.au/
Fixes: 0c3064cf33fbfa ("drm/i915/doc: Document where to implement register workarounds")
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230203100215.31852-2-bagasdotme@gmail.com
(cherry picked from commit ec852e3c88d5caa457557406c0c787b56c36dffb)
Signed-off-by: Jani Nikula <jani.nikula@intel.com>

drm/i915/xehp: Annotate a couple more workaround registers as MCR

GAMSTLB_CTRL and GAMCNTRL_CTRL became multicast/replicated registers on
Xe_HP. They should be defined accordingly and use MCR-aware operations.

These registers have only been used for some dg2/xehpsdv workarounds, so
this fix is mostly just for consistency/future-proofing; even lacking
the MCR annotation, workarounds will always be properly applied in a
multicast manner on these platforms.

Cc: Gustavo Sousa <gustavo.sousa@intel.com>
Fixes: 58bc2453ab8a ("drm/i915: Define multicast registers as a new type")
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230125234159.3015385-3-matthew.d.roper@intel.com

drm/i915/mtl: Correct implementation of Wa_18018781329

Workaround Wa_18018781329 has applied to several recent Xe_HP-based
platforms. However there are some extra gotchas to implementing this
properly for MTL that we need to take into account:

* Due to the separation of media and render/compute into separate GTs,
this workaround needs to be implemented on each GT, not just the
primary GT. Since each class of register only exists on one of the
two GTs, we should program the appropriate registers on each GT.

* As with past Xe_HP platforms, the registers on the primary GT (Xe_LPG
IP) are multicast/replicated registers and should be handled with the
MCR-aware functions. However the registers on the media GT (Xe_LPM+
IP) are regular singleton registers and should _not_ use MCR
handling. We need to create separate register definitions for the
Xe_HP multicast form and the Xe_LPM+ singleton form and use each in
the appropriate place.

* Starting with MTL, workarounds documented by the hardware teams are
technically associated with IP versions/steppings rather than
top-level platforms. That means we should take care to check the
media IP version rather than the graphics IP version when deciding
whether the workaround is needed on the Xe_LPM+ media GT (in this
case the workaround applies to both IPs and the stepping bounds are
identical, but we should still write the code appropriately to set a
proper precedent for future workaround implementations).

* It's worth noting that the GSC register and the CCS register are
defined with the same MMIO offset (0xCF30). Since the CCS is only
relevant to the primary GT and the GSC is only relevant to the media
GT there isn't actually a clash here (the media GT automatically adds
the additional 0x380000 GSI offset). However there's currently a
glitch in the bspec where the CCS register doesn't show up at all and
the GSC register is listed as existing on both GTs. That's a known
documentation problem for several registers with shared GSC/CCS
offsets; rest assured that the CCS register really does still exist.

Cc: Gustavo Sousa <gustavo.sousa@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Fixes: 41bb543f5598 ("drm/i915/mtl: Add initial gt workarounds")
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230125234159.3015385-2-matthew.d.roper@intel.com

drm/i915/xehp: GAM registers don't need to be re-applied on engine resets

Register reset characteristics (i.e., whether the register maintains or
loses its value on engine reset) is an important factor that determines
which wa_list we want to add workarounds to. We recently found out that
the bspec documentation for the Xe_HP's "GAM" registers in the 0xC800 -
0xCFFF range was misleading; these registers do not actually lose their
value on engine resets as the documentation implied. This means there's
no need to re-apply workarounds touching these registers after a reset,
and the corresponding workarounds should be moved from the 'engine'
lists back to the 'gt' list.

v2:
- Don't add Wa_18018781329 to xehpsdv; the original condition didn't
include that platform. (Gustavo)
- Move the MTL code to the GT function as-is for now; we'll take care
of the additional fixes needed in a follow-up patch.

Cc: Gustavo Sousa <gustavo.sousa@intel.com>
Fixes: edf176f48d87 ("drm/i915/dg2: Move misplaced 'ctx' & 'gt' wa's to engine wa list")
Fixes: b2006061ae28 ("drm/i915/xehpsdv: Move render/compute engine reset domains related workarounds")
Fixes: 41bb543f5598 ("drm/i915/mtl: Add initial gt workarounds")
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230125234159.3015385-1-matthew.d.roper@intel.com

drm/i915/gt: Convert PSS_MODE2 to multicast register

That register became a multicast register as of Xe_HP and it is
currently used only for DG2. Use a proper prefix since there could be
usage of the same register for previous platforms in the future, which
would require a different definition (i.e. using _MMIO).

Note that, in its current state, the code does not cause functional
problems, since the actual application of the workaround would
implicitly use multicast mode. This fix is more toward consistency and
being future-proof uses of this register outside of workarounds.

v2:
- Add paragraph noting that this change is for consistency and
making the code future-proof. (Matt)

Signed-off-by: Gustavo Sousa <gustavo.sousa@intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Cc: Matthew Atwood <matthew.s.atwood@intel.com>
Fixes: 468a4e630c7d ("drm/i915/dg2: Introduce Wa_18018764978")
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230120181423.90507-1-gustavo.sousa@intel.com

drm/i915/gt: Move LSC_CHICKEN_BIT* workarounds to correct function

That register doesn't belong to a specific engine, so the proper
placement for workarounds programming it should be
general_render_compute_wa_init().

Signed-off-by: Gustavo Sousa <gustavo.sousa@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230118155249.41551-3-gustavo.sousa@intel.com

drm/i915/doc: Document where to implement register workarounds

Extend the existing documentation in gt/intel_workarounds.c to make it
clear which functions register workarounds should be implemented in
according to their types.

Signed-off-by: Gustavo Sousa <gustavo.sousa@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230118155249.41551-2-gustavo.sousa@intel.com

drm/i915/gt: Cover rest of SVG unit MCR registers

CHICKEN_RASTER_{1,2} got overlooked with the move done in commit
a9e69428b1b4 ("drm/i915: Define MCR registers explicitly"). Registers
from the SVG unit became multicast as of Xe_HP graphics.

BSpec: 66534
Fixes: a9e69428b1b4 ("drm/i915: Define MCR registers explicitly")
Signed-off-by: Gustavo Sousa <gustavo.sousa@intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230105133701.19556-1-gustavo.sousa@intel.com

drm/i915/mtl: Add initial gt workarounds

This patch introduces initial gt workarounds for the MTL platform.

v2: drop redundant/stale comments specifying wa platforms affected
(Lucas).
v3: drop additional redundant stale comments (MattR)

Bspec: 66622

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Atwood <matthew.s.atwood@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230105234408.277750-1-matthew.s.atwood@intel.com

drm/i915/dg2: Return Wa_22012654132 to just specific steppings

Programming of the ENABLE_PREFETCH_INTO_IC bit originally showed up in
both the general DG2 tuning guide (applicable to all DG2
variants/steppings) and under Wa_22012654132 (applicable only to
specific steppings). It has now been removed from the tuning guide, and
the guidance is to only program it in the specific steppings associated
with the workaround.

Bspec: 68331
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Matt Atwood <matthew.s.atwood@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221213234119.2963317-1-matthew.d.roper@intel.com

drm/i915/gen12: Apply recommended L3 hashing mask

The TGL/RKL/DG1/ADL performance tuning guide suggests programming a
literal value of 0x2FC0100F for this register. The register's hardware
default value is 0x2FC0108F, so this translates to just clearing one
bit.

Take this opportunity to also clean up the register definition and
re-write its existing bits/fields in the preferred notation.

Bspec: 31870
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221201222210.344152-1-matthew.d.roper@intel.com

drm/i915/pvc: Implement recommended caching policy

As per the performance tuning guide, set the HOSTCACHEEN bit to
implement the recommended caching policy on PVC.

Signed-off-by: Wayne Boyer <wayne.boyer@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221130170723.2460014-1-wayne.boyer@intel.com

drm/i915/gt: Add dedicated MCR lock

We've been overloading uncore->lock to protect access to the MCR
steering register. That's not really what uncore->lock is intended for,
and it would be better if we didn't need to hold such a high-traffic
spinlock for the whole sequence of (apply steering, access MCR register,
restore steering). Let's create a dedicated MCR lock to protect the
steering control register over this critical section and stop relying on
the high-traffic uncore->lock.

For now the new lock is a software lock. However some platforms (MTL
and beyond) have a hardware-provided locking mechanism that can be used
to serialize not only software accesses, but also hardware/firmware
accesses as well; support for that hardware level lock will be added in
a future patch.

v2:
- Use irqsave/irqrestore spinlock calls; platforms using execlist
submission rather than GuC submission can perform MCR accesses in
interrupt context because reset -> errordump happens in a tasklet.

Cc: Chris Wilson <chris.p.wilson@linux.intel.com>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221128233014.4000136-4-matthew.d.roper@intel.com

drm/i915/dg2: Introduce Wa_18019271663

Wa_18019271663 applies to all DG2 steppings and skus.

Bspec: 66622

Signed-off-by: Matt Atwood <matthew.s.atwood@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221123183648.407058-2-matthew.s.atwood@intel.com

drm/i915/dg2: Introduce Wa_18018764978

Wa_18018764978 applies to specific steppings of DG2 (G10 C0+,
G11 and G12 A0+). Clean up style in function at the same time.

Bspec: 66622

Signed-off-by: Matt Atwood <matthew.s.atwood@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221123183648.407058-1-matthew.s.atwood@intel.com

drm/i915: Fix workarounds on Gen2-3

In 3653727560d0 ("drm/i915: Simplify internal helper function signature")
I broke the old platforms by not noticing engine workaround init does not
initialize the list on old platforms. Fix it by always initializing which
already does the right thing by mostly not doing anything if there aren't
any workarounds on the list.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Fixes: 3653727560d0 ("drm/i915: Simplify internal helper function signature")
Reported-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221118115249.2683946-1-tvrtko.ursulin@linux.intel.com

drm/i915/gen11: Wa_1408615072/Wa_1407596294 should be on GT list

The UNSLICE_UNIT_LEVEL_CLKGATE register programmed by this workaround
has 'BUS' style reset, indicating that it does not lose its value on
engine resets. Furthermore, this register is part of the GT forcewake
domain rather than the RENDER domain, so it should not be impacted by
RCS engine resets. As such, we should implement this on the GT
workaround list rather than an engine list.

Bspec: 19219
Fixes: 3551ff928744 ("drm/i915/gen11: Moving WAs to rcs_engine_wa_init()")
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230201222831.608281-2-matthew.d.roper@intel.com
(cherry picked from commit 5f21dc07b52eb54a908e66f5d6e05a87bcb5b049)
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/i915/dg2: Introduce Wa_18019271663

Wa_18019271663 applies to all DG2 steppings and skus.

Bspec: 66622

Signed-off-by: Matt Atwood <matthew.s.atwood@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221123183648.407058-2-matthew.s.atwood@intel.com
(cherry picked from commit 900a80c5836587d95db32742f66e1f34f7b40fcb)
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/i915/dg2: Introduce Wa_18018764978

Wa_18018764978 applies to specific steppings of DG2 (G10 C0+,
G11 and G12 A0+). Clean up style in function at the same time.

Bspec: 66622

Signed-off-by: Matt Atwood <matthew.s.atwood@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221123183648.407058-1-matthew.s.atwood@intel.com
(cherry picked from commit 468a4e630c7da8cf586f85cc498d6097aed1ab4b)
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/i915/gt: Cover rest of SVG unit MCR registers

CHICKEN_RASTER_{1,2} got overlooked with the move done in commit
a9e69428b1b4 ("drm/i915: Define MCR registers explicitly"). Registers
from the SVG unit became multicast as of Xe_HP graphics.

BSpec: 66534
Fixes: a9e69428b1b4 ("drm/i915: Define MCR registers explicitly")
Signed-off-by: Gustavo Sousa <gustavo.sousa@intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230105133701.19556-1-gustavo.sousa@intel.com
(cherry picked from commit 10903b0a0f4d4964b352fa3df12d3d2ef5fb7a3b)
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/i915: Fix workarounds on Gen2-3

In 3653727560d0 ("drm/i915: Simplify internal helper function signature")
I broke the old platforms by not noticing engine workaround init does not
initialize the list on old platforms. Fix it by always initializing which
already does the right thing by mostly not doing anything if there aren't
any workarounds on the list.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Fixes: 3653727560d0 ("drm/i915: Simplify internal helper function signature")
Reported-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221118115249.2683946-1-tvrtko.ursulin@linux.intel.com
(cherry picked from commit 71feb6f901ecba962177a0a029dc545c91a4b396)
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/i915: call i915_request_await_object from _i915_vma_move_to_active

Since almost all calls to i915_vma_move_to_active are prepended with
i915_request_await_object, let's call the latter from
_i915_vma_move_to_active by default and add flag allowing bypassing it.
Adjust all callers accordingly.
The patch should not introduce functional changes.

Signed-off-by: Andrzej Hajda <andrzej.hajda@intel.com>
Acked-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221019215906.295296-2-andrzej.hajda@intel.com

drm/i915: Update workaround documentation

There were several updates in the driver on how the workarounds are
handled since its documentation was written. Update the documentation to
reflect the current reality.

v2:
- Remove footnote that was wrongly referenced, adding back the
reference in the correct paragraph.
- Remove "Display workarounds" and just mention "display IP" under
"Other" category since all of them are peppered around the driver.

Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Acked-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com> # v1
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221115192611.179981-1-lucas.demarchi@intel.com

drm/i915: Simplify internal helper function signature

Since we are now storing the GT backpointer in the wa list we can drop the
explicit struct intel_gt * argument to wa_list_apply.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Andrzej Hajda <andrzej.hajda@intel.com>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221110124633.3135026-1-tvrtko.ursulin@linux.intel.com

drm/i915: stop including i915_irq.h from i915_trace.h

Turns out many of the files that need i915_reg.h get it implicitly via
{display/intel_de.h, gt/intel_context.h} -> i915_trace.h -> i915_irq.h
-> i915_reg.h. Since i915_trace.h doesn't actually need i915_irq.h,
makes sense to drop it, but that requires adding quite a few new
includes all over the place.

Prefer including i915_reg.h where needed instead of adding another
implicit include, because eventually we'll want to split up i915_reg.h
and only include the specific registers at each place.

Also some places actually needed i915_irq.h too.

Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/6e78a2e0ac1bffaf5af3b5ccc21dff05e6518cef.1668008071.git.jani.nikula@intel.com

drm/i915: Partial abandonment of legacy DRM logging macros

Convert some usages of legacy DRM logging macros into versions which tell
us on which device have the events occurred.

v2:
* Don't have struct drm_device as local. (Jani, Ville)

v3:
* Store gt, not i915, in workaround list. (John)

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Acked-by: Jani Nikula <jani.nikula@intel.com>
Cc: Jani Nikula <jani.nikula@intel.com>
Cc: John Harrison <John.C.Harrison@Intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: Andrzej Hajda <andrzej.hajda@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221109104633.2579245-1-tvrtko.ursulin@linux.intel.com

drm/i915/dg2: Introduce Wa_18017747507

WA 18017747507 applies to all DG2 skus.

BSpec: 56035, 46121, 68173

Signed-off-by: Wayne Boyer <wayne.boyer@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221031131509.3411195-1-wayne.boyer@intel.com

drm/i915/xelp: Add Wa_1806527549

Workaround to be applied to platforms using XE_LP graphics.

BSpec: 52890
Signed-off-by: Gustavo Sousa <gustavo.sousa@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221019161334.119885-1-gustavo.sousa@intel.com

drm/i915: Extend Wa_1607297627 to Alderlake-P

Workaround 1607297627 was missed for Alderlake-P, so here extending it
to it and adding the fixes tag so this WA is backported to all
stable kernels.

v2:
- fixed subject
- added Fixes tag

BSpec: 54369
Cc: <stable@vger.kernel.org> # v5.17+
Fixes: dfb924e33927 ("drm/i915/adlp: Remove require_force_probe protection")
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221017132432.112850-1-jose.souza@intel.com

drm/i915/xelpmp: Add multicast steering for media GT

MTL's media IP (Xe_LPM+) only has a single type of steering ("OAADDRM")
which selects between media slice 0 and media slice 1. We'll always
steer to media slice 0 unless it is fused off (which is the case when
VD0, VE0, and SFC0 are all reported as unavailable).

Bspec: 67789
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221014230239.1023689-15-matthew.d.roper@intel.com

drm/i915/xelpg: Add multicast steering

MTL's graphics IP (Xe_LPG) once again changes the multicast register
types and steering details. Key changes from past platforms:
* The number of instances of some MCR types (NODE, OAAL2, and GAM) vary
according to the MTL subplatform and cannot be read from fuse
registers. However steering to instance #0 will always provided a
non-terminated value, so we can lump these all into a single
"instance0" table.
* The MCR steering register (and its bitfields) has changed.

Unlike past platforms, we will be explicitly steering all types of MCR
accesses, including those for "SLICE" and "DSS" ranges; we no longer
rely on implicit steering. On previous platforms, various
hardware/firmware agents that needed to access registers typically had
their own steering control registers, allowing them to perform multicast
steering without clobbering the CPU/kernel steering. Starting with MTL,
more of these agents now share a single steering register (0xFD4) and it
is no longer safe for us to assume that the value will remain unchanged
from how we initialized it during startup. There is also a slight
chance of race conditions between the driver and a hardware/firmware
agent, so the hardware provides a semaphore register that can be used to
coordinate access to the steering register. Support for the semaphore
register will be introduced in a future patch.

v2:
- Use Xe_LPG terminology instead of "MTL 3D" since it's the IP version
we're matching on now rather than the platform.
- Don't combine l3bank and mslice masks into a union. It's not related
to the other changes here and we might still need both of them on
some future platform.
- Separate debug dumping of steering settings to a separate helper
function. (Tvrtko)
- Update debug dumping to include DSS ranges (and future-proof it so
that any new ranges added on future platforms will also be dumped).
- Restore MULTICAST bit at the end of rw_with_mcr_steering_fw() if we
cleared it. Also force the MULTICAST bit to true at the beginning of
multicast writes just to be safe. (Bala)

Bspec: 67788, 67112
Cc: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
Cc: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221014230239.1023689-14-matthew.d.roper@intel.com

drm/i915: Define multicast registers as a new type

Rather than treating multicast registers as 'i915_reg_t' let's define
them as a completely new type. This will allow the compiler to help us
make sure we're using multicast-aware functions to operate on multicast
registers.

This plan does break down a bit in places where we're just maintaining
heterogeneous lists of registers (e.g., various MMIO whitelists used by
perf, GVT, etc.) rather than performing reads/writes. We only really
care about the offset in those cases, so for now we can "cast" the
registers as non-MCR, leaving us with a list of i915_reg_t's, but we may
want to look for better ways to store mixed collections of i915_reg_t
and i915_mcr_reg_t in the future.

v2:
- Add TLB invalidation registers
v3:
- Make type checking of i915_mmio_reg_offset() stricter. It will
accept either i915_reg_t or i915_mcr_reg_t, but will now raise a
compile error if any other type is passed, even if that type contains
a 'reg' field. (Jani)
- Drop a ton of GVT changes; allowing i915_mmio_reg_offset() to take
either an i915_reg_t or an i915_mcr_reg_t means that the huge lists
of MMIO_D*() macros used in GVT will continue to work without
modification. We need only make changes to structures that have an
explicit i915_reg_t in them now.

Cc: Jani Nikula <jani.nikula@linux.intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221014230239.1023689-13-matthew.d.roper@intel.com

drm/i915/gt: Add MCR-specific workaround initializers

Let's be more explicit about which of our workarounds are updating MCR
registers.

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221014230239.1023689-12-matthew.d.roper@intel.com

drm/i915/gt: Correct prefix on a few registers

We have a few registers that have existed for several hardware
generations, but are only used by the driver on Xe_HP and beyond. In
cases where the Xe_HP version of the register is now replicated and uses
multicast behavior, but earlier generations were singleton, let's change
the register prefix to "XEHP_" to help clarify that we're using the
newer multicast form of the register.

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221014230239.1023689-5-matthew.d.roper@intel.com

drm/i915/xehp: Create separate reg definitions for new MCR registers

Starting in Xe_HP, several registers our driver works with have been
converted from singleton registers into replicated registers with
multicast behavior. Although the registers are still located at the
same MMIO offsets as on previous platforms, let's duplicate the register
definitions in preparation for upcoming patches that will handle
multicast registers in a special manner.

The registers that are now replicated on Xe_HP are:
* PAT_INDEX (mslice replication)
* FF_MODE2 (gslice replication)
* COMMON_SLICE_CHICKEN3 (gslice replication)
* SLICE_COMMON_ECO_CHICKEN1 (gslice replication)
* SLICE_UNIT_LEVEL_CLKGATE (gslice replication)
* LNCFCMOCS (lncf replication)

Note that there are a couple places in selftest_mocs.c where the
gen9 version of LNCFCMOCS is still used without regards for which
platform we're on. Those cases are just doing an offset lookup and not
issuing any CPU reads/writes of the register, so the potentially
multicast nature of the register doesn't come into play.

v2:
- Add commit message note about the unconditional GEN9_LNCFCMOCS usage
in selftest_mocs. (Bala)
- Include some additional TLB registers.

Bspec: 66534
Cc: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221014230239.1023689-3-matthew.d.roper@intel.com

drm/i915/gen8: Create separate reg definitions for new MCR registers

Gen8 was the first time our hardware had multicast registers (or at
least the first time the multicast nature was exposed and MMIO accesses
could be steered). There are some registers that transitioned from
singleton behavior to multicast during the gen7 -> gen8 transition;
let's duplicate the register definitions for those registers in
preparation for upcoming patches that will handle MCR registers in a
special manner.

The registers adjusted are:
* MISCCPCTL
* SAMPLER_INSTDONE
* ROW_INSTDONE
* ROW_CHICKEN2
* HALF_SLICE_CHICKEN1
* HALF_SLICE_CHICKEN3

v2:
- Use the gen8 version of HALF_SLICE_CHICKEN3 in GVT's gen9 engine MMIO
list. (Bala)
- Update to the gen8 version of MISCCPCTL in a couple new workarounds
that were recently added for DG2/PVC. (Bala)

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Reviewed-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221014230239.1023689-2-matthew.d.roper@intel.com

drm/i915: Document and future-proof preemption control policy

Intel hardware allows some preemption settings to be controlled either
by the kernel-mode driver exclusively, or placed under control of the
user-mode drivers; on Linux we always select the userspace control
option. The various registers involved in this are not documented very
clearly; let's add some clarifying comments to help explain how this all
works and provide some history on why our Linux drivers take the
approach they do (which I believe differs from the path taken by certain
other operating systems' drivers).

While we're at it, let's also remove the graphics version 12 upper bound
on this programming. As described, we don't have any plans to move away
from UMD control of preemption settings on future platforms, and there's
currently no reason to believe that the hardware will fundamentally
change how these registers and settings work after version 12.

Bspec: 45921, 45858, 45863
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Suggested-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Wayne Boyer <wayne.boyer@intel.com>
Acked-by: Tapani Pälli <tapani.palli@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220907212410.22623-1-matthew.d.roper@intel.com

drm/i915/dg2: introduce Wa_22015475538

Wa_22015475538 applies to all DG2 (and ATSM) skus. The workaround
implementation is identical to Wa_16011620976. LSC_CHICKEN_BIT_0_UDW is
a general render register instead of rcs so adding this move to the
proper wa init function.

bspec:54077

Signed-off-by: Matt Atwood <matthew.s.atwood@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220920204359.103370-1-matthew.s.atwood@intel.com

drm/i915: Split GAM and MSLICE steering

Although the bspec lists several MMIO ranges as "MSLICE," it turns out
that a subset of these are of a "GAM" subclass that has unique rules and
doesn't followed regular mslice steering behavior.

* Xe_HP SDV: GAM ranges must always be steered to 0,0. These
registers share the regular steering control register (0xFDC) with
other steering types

* DG2: GAM ranges must always be steered to 1,0. GAM registers have a
dedicated steering control register (0xFE0) so we can set the value
once at startup and rely on implicit steering. Technically the
hardware default should already be set to 1,0 properly, but it never
hurts to ensure that in the driver.

Bspec: 66534
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Prathap Kumar Valsan <prathap.kumar.valsan@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220916014345.3317739-1-matthew.d.roper@intel.com

drm/i915: Extend Wa_1607297627 to Alderlake-P

Workaround 1607297627 was missed for Alderlake-P, so here extending it
to it and adding the fixes tag so this WA is backported to all
stable kernels.

v2:
- fixed subject
- added Fixes tag

BSpec: 54369
Cc: <stable@vger.kernel.org> # v5.17+
Fixes: dfb924e33927 ("drm/i915/adlp: Remove require_force_probe protection")
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221017132432.112850-1-jose.souza@intel.com
(cherry picked from commit 847eec69f01a28ca44f5ac7e1d71d3a60263d680)
Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

drm/i915/ats-m: Add thread execution tuning setting

On client DG2 platforms, optimal performance is achieved with the
hardware's default "age based" thread execution setting. However on
ATS-M, switching this to "round robin after dependencies" provides
better performance. We'll add a new "tuning" feature flag to the ATS-M
device info to enable/disable this setting.

Bspec: 68331
Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Matt Atwood <matthew.s.atwood@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220826212718.409948-1-matthew.d.roper@intel.com

Revert "drm/i915/dg2: Add preemption changes for Wa_14015141709"

This reverts commit ca6920811aa5428270dd78af0a7a36b10119065a.

The intent of Wa_14015141709 was to inform us that userspace can no
longer control object-level preemption as it has on past platforms
(i.e., by twiddling register bit CS_CHICKEN1[0]). The description of
the workaround in the spec wasn't terribly well-written, and when we
requested clarification from the hardware teams we were told that on the
kernel side we should also probably stop setting
FF_SLICE_CS_CHICKEN1[14], which is the register bit that directs the
hardware to honor the settings in per-context register CS_CHICKEN1. It
turns out that this guidance about FF_SLICE_CS_CHICKEN1[14] was a
mistake; even though CS_CHICKEN1[0] is non-operational and useless to
userspace, there are other bits in the register that do still work and
might need to be adjusted by userspace in the future (e.g., to implement
other workarounds that show up). If we don't set
FF_SLICE_CS_CHICKEN1[14] in i915, then those future workarounds would
not take effect.

This miscommunication came to light because another workaround
(Wa_16013994831) has now shown up that requires userspace to adjust the
value of CS_CHICKEN[10] in certain circumstances. To ensure userspace's
updates to this chicken bit are handled properly by the hardware, we
need to make sure that FF_SLICE_CS_CHICKEN1[14] is once again set by the
kernel.

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Matt Atwood <matthew.s.atwood@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220826210233.406482-1-matthew.d.roper@intel.com

drm/i915/dg2: Incorporate Wa_16014892111 into DRAW_WATERMARK tuning

Although register tuning settings are generally implemented via the
workaround infrastructure, it turns out that the DRAW_WATERMARK register
is not properly saved/restored by hardware around power events (i.e.,
RC6 entry) so updates to the value cannot be applied in the usual
manner. New workaround Wa_16014892111 informs us that any tuning
updates to this register must instead be applied via an INDIRECT_CTX
batch buffer. This will ensure that the necessary value is re-applied
when a context begins running, even if an RC6 entry had wiped the
register back to hardware defaults since the last context ran.

Fixes: 6dc85721df74 ("drm/i915/dg2: Add additional tuning settings")
Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/6642
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220823202449.83727-1-matthew.d.roper@intel.com

drm/i915/dg2: Add additional tuning settings

Some additional MMIO tuning settings have appeared in the bspec's
performance tuning guide section.

One of the tuning settings here is also documented as formal workaround
Wa_22012654132 for some steppings of DG2. However the tuning setting
applies to all DG2 variants and steppings, making it a superset of the
workaround.

v2:
- Move DRAW_WATERMARK to engine workaround section. It only moves into
the engine context on future platforms. (Lucas)
- CHICKEN_RASTER_2 needs to be handled as a masked register. (Lucas)

Bspec: 68331
Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Cc: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220816210601.2041572-2-matthew.d.roper@intel.com

drm/i915/gt: Add dedicated function for non-ctx register tuning settings

The bspec performance tuning section gives recommended settings that the
driver should program for various MMIO registers. Although these
settings aren't "workarounds" we use the workaround infrastructure to do
this programming to make sure it is handled at the appropriate places
and doesn't conflict with any real workarounds.

Since more of these are starting to show up on recent platforms, it's a
good time to create a dedicated function to hold them so that there's
less ambiguity about how/where to implement new ones.

Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220816210601.2041572-1-matthew.d.roper@intel.com

drm/i915/dg2: Add Wa_1509727124

Bspec: 46052
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Harish Chegondi <harish.chegondi@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220801213839.8549-1-harish.chegondi@intel.com

drm/i915/dg2: Add Wa_15010599737

This workaround may need to be extended to other platforms soon, but for
now it's marked as DG2-specific.

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Arun R Murthy <arun.r.murthy@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220708215804.2889246-1-matthew.d.roper@intel.com

drm/i915/pvc: Implement w/a 16016694945

A new PVC-specific workaround has just been added to the BSpec.

BSpec: 64027

Signed-off-by: Gustavo Sousa <gustavo.sousa@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220630201407.16770-1-gustavo.sousa@intel.com

drm/i915/gt: Cleanup interface for MCR operations

Let's replace the assortment of intel_gt_* and intel_uncore_* functions
that operate on MCR registers with a cleaner set of interfaces:

* intel_gt_mcr_read -- unicast read from specific instance
* intel_gt_mcr_read_any[_fw] -- unicast read from any non-terminated
instance
* intel_gt_mcr_unicast_write -- unicast write to specific instance
* intel_gt_mcr_multicast_write[_fw] -- multicast write to all instances

We'll also replace the historic "slice" and "subslice" terminology with
"group" and "instance" to match the documentation for more recent
platforms; these days MCR steering applies to more types of replication
than just slice/subslice.

v2:
- Reference the new kerneldoc from i915.rst. (Jani)
- Tweak the wording of the documentation for a couple functions to
clarify the difference between "_fw" and non-"_fw" forms.

v3:
- s/read/write/ to fix copy-paste mistake in a couple comments.
(Harish)

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Acked-by: Jani Nikula <jani.nikula@linux.intel.com>
Reviewed-by: Harish Chegondi <harish.chegondi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220615001019.1821989-3-matthew.d.roper@intel.com

drm/i915/gt: Move multicast register handling to a dedicated file

Handling of multicast/replicated registers is spread across intel_gt.c
and intel_uncore.c today. As multicast handling and the related
steering logic gets more complicated with the addition of new platforms
and new rules it makes sense to centralize it all in one place.

For now the existing functions have been moved to the new .c/.h as-is.
Function renames and updates to operate in a more consistent manner will
be done in subsequent patches.

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Acked-by: Jani Nikula <jani.nikula@linux.intel.com>
Reviewed-by: Harish Chegondi <harish.chegondi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220615001019.1821989-2-matthew.d.roper@intel.com

drm/i915/pvc: Add recommended MMIO setting

As with past platforms, the bspec's performance tuning guide provides
recommended MMIO settings. Although not technically "workarounds" we
apply these through the workaround framework to ensure that they're
re-applied at the proper times (e.g., on engine resets) and that any
conflicts with real workarounds are flagged.

Bspec: 72161
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220613165314.862029-1-matthew.d.roper@intel.com

drm/i915/pvc: Add register steering

Ponte Vecchio no longer has MSLICE or LNCF steering, but the bspec does
document several new types of multicast register ranges. Fortunately,
most of the different MCR types all provide valid values at instance
(0,0) so there's no need to read fuse registers and calculate a
non-terminated instance. We'll lump all of those range types (BSLICE,
HALFBSLICE, TILEPSMI, CC, and L3BANK) into a single category called
"INSTANCE0" to keep things simple. We'll also perform explicit steering
for each of these multicast register types, even if the implicit
steering setup for COMPUTE/DSS ranges would have worked too; this is
based on guidance from our hardware architects who suggested that we
move away from implicit steering and start explicitly steer all MCR
register accesses on modern platforms (we'll work on transitioning
COMPUTE/DSS to explicit steering in the future).

Note that there's one additional MCR range type defined in the bspec
(SQIDI) that we don't handle here. Those ranges use a different
steering control register that we never touch; since instance 0 is also
always a valid setting there, we can just ignore those ranges.

Finally, we'll rename the HAS_MSLICES() macro to HAS_MSLICE_STEERING().
PVC hardware still has units referred to as mslices, but there's no
register steering based on mslice for this platform.

v2:
- Rebase on other recent changes
- Swap two table rows to keep table sorted & easy to read. (Harish)

Bspec: 67609
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Harish Chegondi <harish.chegondi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220608170700.4026648-1-matthew.d.roper@intel.com

drm/i915/xehp: Correct steering initialization

Another mistake during the conversion to DSS bitmaps: after retrieving
the DSS ID intel_sseu_find_first_xehp_dss() we forgot to modulo it down
to obtain which ID within the current gslice it is.

Fixes: b87d39019651 ("drm/i915/sseu: Disassociate internal subslice mask representation from uapi")
Cc: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Acked-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220607175716.3338661-1-matthew.d.roper@intel.com

drm/i915: More PVC+DG2 workarounds

A new PVC+DG2 workaround has appeared recently:
- Wa_16015675438

And a couple existing DG2 workarounds have been extended to PVC:
- Wa_14015795083
- Wa_18018781329

Note that Wa_16015675438 asks us to program a register that is in the
0x2xxx range typically associated with the RCS engine, even though PVC
does not have an RCS. By default the GuC will think we've made a
mistake and throw an exception when it sees this register on a CCS
engine's save/restore list, so we need to pass an extra GuC control flag
to tell it that this is expected and not a problem.

Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
Signed-off-by: Badal Nilawar <badal.nilawar@intel.com>
Signed-off-by: Prathap Kumar Valsan <prathap.kumar.valsan@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Anshuman Gupta <anshuman.gupta@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220608005108.3717895-1-matthew.d.roper@intel.com

drm/i915/dg2: Correct DSS check for Wa_1308578152

When converting our DSS masks to bitmaps, we fumbled the condition used
to check whether any DSS are present in the first gslice. Since
intel_sseu_find_first_xehp_dss() returns a 0-based number, we need a >=
condition rather than >.

Fixes: b87d39019651 ("drm/i915/sseu: Disassociate internal subslice mask representation from uapi")
Reported-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Acked-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220607154724.3155521-1-matthew.d.roper@intel.com

drm/i915/dg2: Add Wa_14015795083

i915 must disable Render DOP clock gating globally.

v2:
- Addressed cosmetic review comments.

Bspec: 52621
Cc: Matt Roper <matthew.d.roper@intel.com>
Cc: Badal Nilawar <badal.nilawar@intel.com>
Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220607104542.8559-1-anshuman.gupta@intel.com

drm/i915/sseu: Disassociate internal subslice mask representation from uapi

As with EU masks, it's easier to store subslice/DSS masks internally in
a format that's more natural for the driver to work with, and then only
covert into the u8[] uapi form when the query ioctl is invoked. Since
the hardware design changed significantly with Xe_HP, we'll use a union
to choose between the old "hsw-style" subslice masks or the newer xehp
mask. HSW-style masks will be stored in an array of u8's, indexed by
slice (there's never more than 6 subslices per slice on older
platforms). For Xe_HP and beyond where slices no longer exist, we only
need a single bitmask. However we already know that this mask is
eventually going to grow too large for a simple u64 to hold, so we'll
represent it in a manner that can be operated on by the utilities in
linux/bitmap.h.

v2:
- Fix typo: BIT(s) -> BIT(ss) in gen9_sseu_device_status()

v3:
- Eliminate sseu->ss_stride and just calculate the stride while
specifically handling uapi. (Tvrtko)
- Use BITMAP_BITS() macro to refer to size of masks rather than
passing I915_MAX_SS_FUSE_BITS directly. (Tvrtko)
- Report compute/geometry DSS masks separately when dumping Xe_HP SSEU
info. (Tvrtko)
- Restore dropped range checks to intel_sseu_has_subslice(). (Tvrtko)

v4:
- Make the bitmap size macro check the size of the .xehp field rather
than the containing union. (Tvrtko)
- Don't add GEM_BUG_ON() intel_sseu_has_subslice()'s check for whether
slice or subslice ID exceed sseu->max_[sub]slices; various loops
in the driver are expected to exceed these, so we should just
silently return 'false.'

v5:
- Move XEHP_BITMAP_BITS() to the header so that we can also replace a
usage of I915_MAX_SS_FUSE_BITS in one of the inline functions.
(Bala)
- Change the local variable in intel_slicemask_from_xehp_dssmask() from
u16 to 'unsigned long' to make it a bit more future-proof.

Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
Cc: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Acked-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220601150725.521468-6-matthew.d.roper@intel.com

drm/i915/pvc: Add initial PVC workarounds

Bspec: 64027
Signed-off-by: Stuart Summers <stuart.summers@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Matt Atwood <matthew.s.atwood@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220527163348.1936146-3-matthew.d.roper@intel.com

drm/i915/dg2: Add workaround 22014600077

Signed-off-by: Swathi Dhanavanthri <swathi.dhanavanthri@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220517212905.24212-1-swathi.dhanavanthri@intel.com

drm/i915/pvc: Define MOCS table for PVC

v2 (MattR):
- Clarify comment above RING_CMD_CCTL programming.
- Remove bspec reference from field definition. (Lucas)
- Add WARN if we try to use a (presumably uninitialized) wb_index of 0.
On most platforms 0 is an invalid MOCS entry and even on the ones
where it isn't, it isn't the right setting for wb_index. (Lucas)

Bspec: 45101, 72161
Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Ayaz A Siddiqui <ayaz.siddiqui@intel.com>
Signed-off-by: Fei Yang <fei.yang@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220505213812.3979301-4-matthew.d.roper@intel.com

drm/i915/dg2: Add workaround 18019627453

A new DG2 workaround added to fix some corner cases hangs.

v2:
- implementing the second and preferred option for this workaround

BSpec: 54077
BSpec: 68173
BSpec: 71488
Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220419182753.364237-2-jose.souza@intel.com

drm/i915/dg2: Add Wa_22014226127

New DG2 workaround added to specification.

BSpec: 54077
BSpec: 66622
BSpec: 54833
Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220325142249.81443-1-jose.souza@intel.com

drm/i915/dg2: Add preemption changes for Wa_14015141709

Starting with DG2, preemption can no longer be controlled using userspace
on a per-context basis. Instead, the hardware only allows us to enable or
disable preemption in a global, system-wide basis. Also, we lose the
ability to specify the preemption granularity (such as batch-level vs
command-level vs object-level).

v2 (MattR):
- Move debugfs interface to a separate patch. (Jani)
v3 (MattR):
- Drop the debugfs support completely for now.

Cc: Matt Roper <matthew.d.roper@intel.com>
Cc: Prathap Kumar Valsan <prathap.kumar.valsan@intel.com>
Cc: John Harrison <john.c.harrison@intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
Signed-off-by: Akeem G Abodunrin <akeem.g.abodunrin@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Anusha Srivatsa <anusha.srivatsa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220318021051.2073847-1-matthew.d.roper@intel.com

drm/i915: Report steering details in debugfs

Add a new 'steering' node in each gt's debugfs directory that tells
whether we're using explicit steering for various types of MCR ranges
and, if so, what MMIO ranges it applies to.

We're going to be transitioning away from implicit steering, even for
slice/dss steering soon, so the information reported here will become
increasingly valuable once that happens.

v2:
- Adding missing 'static' on intel_steering_types[] (Jose, sparse)
v3:
- "static const char *" -> "static const char * const" (sparse)

Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220315170250.954380-1-matthew.d.roper@intel.com

drm/i915/xehp: Support platforms with CCS engines but no RCS

In the past we've always assumed that an RCS engine is present on every
platform. However now that we have compute engines there may be
platforms that have CCS engines but no RCS, or platforms that are
designed to have both, but have the RCS engine fused off.

Various engine-centric initialization that only needs to be done a
single time for the group of RCS+CCS engines can't rely on being setup
with the RCS now; instead we add a I915_ENGINE_FIRST_RENDER_COMPUTE flag
that will be assigned to a single engine in the group; whichever engine
has this flag will be responsible for some of the general setup
(RCU_MODE programming, initialization of certain workarounds, etc.).

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220303223435.2793124-1-matthew.d.roper@intel.com

drm/i915/xehpsdv: Move render/compute engine reset domains related workarounds

Registers that exist in the shared render/compute reset domain need to
be placed on an engine workaround list to ensure that they are properly
re-applied whenever an RCS or CCS engine is reset. We have a number of
workarounds (updating registers MLTICTXCTL, L3SQCREG1_CCS0,
GEN12_MERT_MOD_CTRL, and GEN12_GAMCNTRL_CTRL) that are incorrectly
implemented on the 'gt' workaround list and need to be moved
accordingly.

Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Srinivasan Shanmugam <srinivasan.s@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220301231549.1817978-14-matthew.d.roper@intel.com

drm/i915/xehp: Add compute workarounds

Additional workarounds are required once we start exposing CCS engines.

Note that we have a number of workarounds that update registers in the
shared render/compute reset domain. Historically we've just added such
registers to the RCS engine's workaround list. But going forward we
should be more careful to place such workarounds on a wa_list for an
engine that definitely exists and is not fused off (e.g., a platform
with no RCS would never apply the RCS wa_list). We'll keep
rcs_engine_wa_init() focused on RCS-specific workarounds that only need
to be applied if the RCS engine is present. A separate
general_render_compute_wa_init() function will be used to define
workarounds that touch registers in the shared render/compute reset
domain and that we need to apply regardless of what render and/or
compute engines actually exist. Any workarounds defined in this new
function will internally be added to the first present RCS or CCS
engine's workaround list to ensure they get applied (and only get
applied once rather than being needlessly re-applied several times).

Co-author: Srinivasan Shanmugam
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220301231549.1817978-13-matthew.d.roper@intel.com

drm/i915/dg2: add Wa_14014947963

BSPEC: 46123
v2: Address review feedback [MattR]
v3: move register definition to gt_regs [MattR]
Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Clint Taylor <clinton.a.taylor@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220211052333.12306-1-clinton.a.taylor@intel.com

drm/i915/dg2: Move misplaced 'ctx' & 'gt' wa's to engine wa list

Registers that belong to the shared render/compute reset domain need to
be placed on an engine workaround list to ensure that they are properly
re-applied whenever any RCS or CCS engine is reset, even if the
registers do not belong to a specific engine's MMIO range. We have a
number of workarounds today that are incorrectly implemented on the 'gt'
workaround list and need to be moved accordingly. We also have one
workaround (Wa_22012532006) that is incorrectly implemented on the
context workaround list, even though the register it is adjusting is not
part of the RCS engine's context image; it must also be moved.

We'll have some workaround refactoring coming in the near future that
deals with registers in the reset domain in a more clear way. But in
the meantime, we should just move these workarounds to
rcs_engine_wa_init() to place them on the RCS engine's workaround list.
All production DG2 platforms will have an RCS engine (it's never fused
off) so these registers will be properly restored after a domain reset
triggered via an RCS engine _or_ a CCS engine.

Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Srinivasan Shanmugam <srinivasan.s@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220215235531.2236399-1-matthew.d.roper@intel.com

drm/i915/gt: Use parameterized RING_MI_MODE

We have both a parameterized RING_MI_MODE() macro and an RCS-specific
MI_MODE; drop the latter and use the former everywhere.

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Matt Atwood <matthew.s.atwood@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220209051140.1599643-4-matthew.d.roper@intel.com

drm/i915: Move GT registers to their own header file

This is a huge, chaotic mass of registers copied over as-is without any
real cleanup. We'll come back and organize these better, align on
consistent coding style, remove dead code, etc. in separate patches
later that will be easier to review.

v2:
- Add missing include in intel_pxp_irq.c
v3:
- Correct a few indentation errors (Lucas)
- Minor conflict resolution

Cc: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220127234334.4016964-6-matthew.d.roper@intel.com

drm/i915: Introduce G12 subplatform of DG2

Another fork of the DG2 design has appeared, known as "DG2-G12;" let's
add it as a new subplatform. As with G11, the GT stepping resets back
to A0 (so a DG2-G12 A0 is similar, but not identical, to a DG2-G10 C0)
but the display steppings continue to use the same numbering scheme as
G10 and G11.

Some existing DG2 workarounds are starting to be extended to the DG2-G12
subplatform. So far only workarounds that were "permanent" for both
DG2-G10 and DG2-G11 have been tagged for DG2-G12, but more
stepping-specific workarounds are likely to show up in the future.

Bspec: 44477
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220120235016.1209326-1-matthew.d.roper@intel.com

drm/i915/dg2: s/engine->i915/i915/ for engine workarounds

rcs_engine_wa_init() has a local 'i915' variable; we should use that
rather than 'engine->i915' for consistency with how we handle other
platforms.

Suggested-by: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220128170125.4121819-1-matthew.d.roper@intel.com

drm/i915/dg2: Add Wa_14015227452

Note that the bspec doesn't list the bit we're programming here (bit 11)
as being present on DG2, but we've confirmed with the hardware team that
this is a documentation mistake and the bit does indeed exist on all
Xe_HP-based platforms.

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220127194855.3963296-1-matthew.d.roper@intel.com
Reviewed-by: Swathi Dhanavanthri <swathi.dhanavanthri@intel.com>

drm/i915/dg2: Add Wa_18018781329

A few more MOD registers need to be programmed on DG2.

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220120234147.1200574-1-matthew.d.roper@intel.com
Reviewed-by: Swathi Dhanavanthri <swathi.dhanavanthri@intel.com>

drm/i915/gt: Move engine registers to their own header

Let's continue breaking up and cleaning up the massive i915_reg.h file
by moving all registers that are defined in relation to an engine base
to their own header.

There are probably a bunch of other "engine registers" that we haven't
moved yet (especially those that belong to the render engine in the
0x2??? range), but this is a relatively straightforward first step.

Cc: Jani Nikula <jani.nikula@linux.intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220111051600.3429104-8-matthew.d.roper@intel.com

drm/i915: Replace GFX_MODE_GEN7 with RING_MODE_GEN7

It's preferable to use parameterized register macros where possible.

Cc: Jani Nikula <jani.nikula@linux.intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Jani Nikula <jani.nikula@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220111051600.3429104-6-matthew.d.roper@intel.com

drm/i915: Use RING_PSMI_CTL rather than per-engine macros

We have a parameterized macro for RING_PSMI_CTL; let's use that instead
of the per-engine definitions where possible.

Cc: Jani Nikula <jani.nikula@linux.intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Jani Nikula <jani.nikula@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220111051600.3429104-5-matthew.d.roper@intel.com

drm/i915: Parameterize ECOSKPD

Combine the separate render and blitter register definitions into a
single definition. We already know we have some workarounds on an
upcoming platform that will need to update the ECOSKPD register for
other engines too, so this helps pave the way for that.

Cc: Jani Nikula <jani.nikula@linux.intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Jani Nikula <jani.nikula@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220111051600.3429104-4-matthew.d.roper@intel.com

drm/i915/gt: Use to_gt() helper

Use to_gt() helper consistently throughout the codebase.
Pure mechanical s/i915->gt/to_gt(i915). No functional changes.

Signed-off-by: Michał Winiarski <michal.winiarski@intel.com>
Signed-off-by: Andi Shyti <andi.shyti@linux.intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20211214193346.21231-5-andi.shyti@linux.intel.com

drm/i915/gen11: Moving WAs to icl_gt_workarounds_init()

Bspec page says "Reset: BUS", Accordingly moving w/a's:
Wa_1407352427,Wa_1406680159 to proper function icl_gt_workarounds_init()
Which will resolve guc enabling error

v2:
- Previous patch rev2 was created by email client which caused the
Build failure, This v2 is to resolve the previous broken series

Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
Signed-off-by: Raviteja Goud Talla <ravitejax.goud.talla@intel.com>
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20211203145603.4006937-1-ravitejax.goud.talla@intel.com

drm/i915: Add workaround numbers to GEN7_COMMON_SLICE_CHICKEN1 whitelisting

Those two workarounds needs to be implemented in UMD, KMD only needs
to whitelist the registers, so here only adding the workaround number
to facilitate future workaroud table checks.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Matt Atwood <matthew.s.atwood@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20211119140931.32791-2-jose.souza@intel.com

Revert "drm/i915: Implement Wa_1508744258"

This workarounds are causing hangs, because I missed the fact that it
needs to be enabled for all cases and disabled when doing a resolve
pass.

So KMD only needs to whitelist it and UMD will be the one setting it
on per case.

This reverts commit 28ec02c9cbebf3feeaf21a59df9dfbc02bda3362.

Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/4145
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Fixes: 28ec02c9cbeb ("drm/i915: Implement Wa_1508744258")
Reviewed-by: Matt Atwood <matthew.s.atwood@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20211119140931.32791-1-jose.souza@intel.com

drm/i915: Don't read query SSEU for non-existent slice 0 on old platforms

Pre-HSW platforms don't use the gt SSEU structures; this means that
calling intel_sseu_get_subslices() on slice 0 for these platforms will
trip a GEM_BUG_ON(slice >= sseu->max_slices) warning.

Let's move the DSS lookup for a DG2 workaround into a helper function
that will only get called after we've already decided that we're on a
DG2 platform.

Fixes: 645cc0b9d972 ("drm/i915/dg2: Add initial gt/ctx/engine workarounds")
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20211112160107.1593906-1-matthew.d.roper@intel.com

drm/i915/dg2: Program recommended HW settings

The bspec's performance guide suggests programming specific values into
a few registers for optimal performance. Although these aren't
workarounds, it's easiest to handle them inside the GT workaround
functions (which will also ensure that the values set here are properly
melded with other bits in the same registers that _are_ set by
workarounds).

Bspec: 68331, 45395

Cc: Matt Atwood <matthew.s.atwood@intel.com>
Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Cc: Siddiqui Ayaz A <ayaz.siddiqui@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Clint Taylor <Clinton.A.Taylor@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20211102222511.534310-4-matthew.d.roper@intel.com

drm/i915/dg2: Add initial gt/ctx/engine workarounds

Bspec: 54077,68173,54833
Cc: Anusha Srivatsa <anusha.srivatsa@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Anusha Srivatsa <anusha.srivatsa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20211102222511.534310-3-matthew.d.roper@intel.com

drm/i915/xehpsdv: Add initial workarounds

Add the initial set of workarounds for Xe_HP SDV.

There are some additional workarounds specific to the compute engines
that we're holding back for now. Those will be added later, after
general compute engine support lands.

Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Stuart Summers <stuart.summers@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Clint Taylor <Clinton.A.Taylor@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20211102222511.534310-2-matthew.d.roper@intel.com

drm/i915: Rename GT_STEP to GRAPHICS_STEP

As now graphics and media can have different steppings this patch is
renaming all _GT_STEP macros to _GRAPHICS_STEP.

Future platforms will properly choose between _MEDIA_STEP and
_GRAPHICS_STEP for each new workaround.

Cc: Matt Atwood <matthew.s.atwood@intel.com>
Cc: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Caz Yokoyama <caz.yokoyama@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20211020002353.193893-3-jose.souza@intel.com

drm/i915: Enable WaProgramMgsrForCorrectSliceSpecificMmioReads for Gen9

This implements WaProgramMgsrForCorrectSliceSpecificMmioReads which
was omitted by mistake from Gen9 documentation, while it is actually
applicable to fused off parts.

Workaround consists of making sure MCR packet control register is
programmed to point to enabled slice/subslice pair before doing any
MMIO reads from the affected registers.

Failure do to this can result in complete system hangs when running
certain workloads. Two known cases which can cause system hangs are:

1. "test_basic progvar_prog_scope_uninit" test which is part of
Khronos OpenCL conformance suite
(https://github.com/KhronosGroup/OpenCL-CTS) with the Intel
OpenCL driver (https://github.com/intel/compute-runtime).

2. VP8 media hardware encoding using the full-feature build of the
Intel media-driver (https://github.com/intel/media-driver) and
ffmpeg.

For the former case patch was verified to fix the hard system hang
when executing the OCL test on Intel Pentium CPU 6405U which contains
fused off GT1 graphics.

Reference: HSD#1508045018,1405586840, BSID#0575

Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Jani Nikula <jani.nikula@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
Cc: William Tseng <william.tseng@intel.com>
Cc: Shawn C Lee <shawn.c.lee@intel.com>
Cc: Pawel Wilma <pawel.wilma@intel.com>
Signed-off-by: Cooper Chiou <cooper.chiou@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Acked-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20211025042623.3876-1-cooper.chiou@intel.com

drm/i915/gen11: Moving WAs to icl_gt_workarounds_init()

Bspec page says "Reset: BUS", Accordingly moving w/a's:
Wa_1407352427,Wa_1406680159 to proper function icl_gt_workarounds_init()
Which will resolve guc enabling error

v2:
- Previous patch rev2 was created by email client which caused the
Build failure, This v2 is to resolve the previous broken series

Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
Signed-off-by: Raviteja Goud Talla <ravitejax.goud.talla@intel.com>
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20211203145603.4006937-1-ravitejax.goud.talla@intel.com
(cherry picked from commit 67b858dd89932086ae0ee2d0ce4dd070a2c88bb3)
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Revert "drm/i915: Implement Wa_1508744258"

This workarounds are causing hangs, because I missed the fact that it
needs to be enabled for all cases and disabled when doing a resolve
pass.

So KMD only needs to whitelist it and UMD will be the one setting it
on per case.

This reverts commit 28ec02c9cbebf3feeaf21a59df9dfbc02bda3362.

Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/4145
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Fixes: 28ec02c9cbeb ("drm/i915: Implement Wa_1508744258")
Reviewed-by: Matt Atwood <matthew.s.atwood@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20211119140931.32791-1-jose.souza@intel.com
(cherry picked from commit f3799ff16fcfacd44aee55db162830df461b631f)
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/i915: Make wa list per-gt

Support for multiple GT's within a single i915 device will be arriving
soon. Since each GT may have its own fusing and require different
workarounds, we need to make the GT workaround functions and multicast
steering setup per-gt.

Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Signed-off-by: Venkata Sandeep Dhanalakota <venkata.s.dhanalakota@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210917170845.836358-1-matthew.d.roper@intel.com

drm/i915/gt: Set BLIT_CCTL reg to un-cached

Blitter commands which do not have MOCS fields rely on
cacheability of BlitterCacheControlRegister which was mapped
to index 0 by default.Once we changed the MOCS value of
index 0 to L3 WB, tests like gem_linear_blits started failing
due to a change in cacheability from UC to WB.

Program and place the BlitterCacheControlRegister in
build_aux_regs().

Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Ayaz A Siddiqui <ayaz.siddiqui@intel.com>
Signed-off-by: Ramalingam C <ramalingam.c@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210903092153.535736-4-ayaz.siddiqui@intel.com

drm/i915/gt: Set CMD_CCTL to UC for Gen12 Onward

Cache-control registers for Command Stream(CMD_CCTL) are used
to set catchability for memory writes and reads outputted by
Command Streamers on Gen12 onward platforms.

These registers need to point un-cached(UC) MOCS index.

Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Ayaz A Siddiqui <ayaz.siddiqui@intel.com>
Signed-off-by: Ramalingam C <ramalingam.c@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210903092153.535736-3-ayaz.siddiqui@intel.com

drm/i915: Ensure wa_init_finish() is called for ctx workaround list

A recent restructuring of our context workaround list initialization
added an early return for non-render engines; this caused us to
potentially miss the wa_init_finish() call at the end of the function.
The mistake is pretty harmless --- the only impact is that non-render
engines on graphics version 12.50+ platforms we don't trim down the
workaround list to reclaim some memory, and we don't print the usual
"Initialized 1 context workaround" message in dmesg. Let's change the
early return to a jump down to the wa_init_finish() call at the bottom
of the function.

Reported-by: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
Fixes: 9e9dfd080201 ("drm/i915/dg2: Maintain backward-compatible nested batch behavior")
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210826033559.1209020-1-matthew.d.roper@intel.com

drm/i915/dg2: Maintain backward-compatible nested batch behavior

For tgl+, the per-context setting of MI_MODE[12] determines whether
the bits of a nested MI_BATCH_BUFFER_START instruction should be
interpreted in the traditional manner or whether they should
instead use a new tgl+ meaning that breaks backward compatibility, but
allows nesting into 3rd-level batchbuffers. For previous platforms,
the hardware default for this register bit is to maintain
backward-compatible behavior unless a context intentionally opts into
the new behavior; however Xe_HPG flips the hardware default behavior.

From a SW perspective, we want to maintain the backward-compatible
behavior for userspace, so we'll apply a fake workaround to set it back
to the legacy behavior on platforms where the hardware default is to
break compatibility. At the moment there is no Linux userspace that
utilizes third-level batchbuffers, so this will avoid userspace from
needing to make any changes. using the legacy meaning is the correct
thing to do. If/when we have userspace consumers that want to utilize
third-level batch nesting, we can provide a context parameter to allow
them to opt-in.

Bspec: 45974, 45718
Cc: John Harrison <John.C.Harrison@Intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210805163647.801064-9-matthew.d.roper@intel.com
Reviewed-by: Anusha Srivatsa <anusha.srivatsa@intel.com>

drm/i915: Make shadow tables range-based

Rather than defining our shadow tables as a list of individual
registers, provide them as a list of register ranges; we'll have some
ranges of multiple registers being added soon (and we already have a
couple adjacent registers that we can squash into a single range now).

This change also defines the table with hex literal values rather than
symbolic register names; since that's how the tables are defined in the
bspec, this change will make it easier to review the tables overall.

v2:
- Force signed comparison on range overlap sanity check

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Caz Yokoyama <caz.yokoyama@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210729152158.2646246-1-matthew.d.roper@intel.com

drm/i915/dg2: Add SQIDI steering

Although DG2_G10 platforms will always have all SQIDI's present and
don't need steering for registers in a SQIDI MMIO range, this isn't true
for DG2_G11 platforms; only SQIDI's 2 and 3 can be used on those.

We handle SQIDI ranges a bit differently from other types of explicit
steering. The SQIDI ranges belong to either the MCFG unit or the SF
unit, both of which have their own dedicated steering registers and do
not use the typical 0xFDC steering control that all other types of
ranges use. Thus we only need to worry about picking a valid initial
value for the MCFG and SF steering registers (0xFD0 and 0xFD8
respectively) at driver init; they won't change after we set them up so
we don't need to worry about re-steering them explicitly at runtime.

Given that any SQIDI value should work fine for DG2-G10 and XeHP SDV,
while only values of 2 and 3 are valid for DG2-G11, we'll just
initialize the MCFG and SF steering registers to a constant value of "2"
for all XeHP-based platforms for simplicity --- that will work in all
cases.

Bspec: 66534
Cc: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210729170008.2836648-6-matthew.d.roper@intel.com

drm/i915/xehpsdv: Define steering tables

Define and initialize the MMIO ranges for which XeHP SDV requires MSLICE
and LNCF steering.

Bspec: 66534
Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210729170008.2836648-3-matthew.d.roper@intel.com

drm/i915/xehp: handle new steering options

Xe_HP is more modular than its predecessors and as a consequence it has
more types of replicated registers. As with l3bank regions on previous
platforms, we may need to explicitly re-steer accesses to these new
types of ranges at runtime if we can't find a single default steering
value that satisfies the fusing of all types.

v2:
- Add a local 'i915' variable to reduce gt->i915 usage. (Caz)
- Drop unused 'intel_gt_read_register' prototype. (Caz)

v3:
- Drop unnecessary comment text. (Lucas)
- Drop unused register bit definition. (Lucas)

Bspec: 66534
Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
Cc: Caz Yokoyama <caz.yokoyama@intel.com>
Signed-off-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210729170008.2836648-2-matthew.d.roper@intel.com

drm/i915/xehp: Fix missing sentinel on mcr_ranges_xehp

There's a missing sentinel since we are not using ARRAY_SIZE(), but rather
checking that the .start is 0 to stop the iteration in mcr_range().

BUG: KASAN: global-out-of-bounds in mcr_range.isra.0+0x69/0xa0 [i915]
Read of size 4 at addr ffffffffa0889928 by task modprobe/3881

Fixes: d8905ba705ab ("drm/i915/xehp: Define multicast register ranges")
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210730191115.2514581-1-lucas.demarchi@intel.com

drm/i915/guc: Provide mmio list to be saved/restored on engine reset

The driver must provide GuC with a list of mmio registers
that should be saved/restored during a GuC-based engine reset.
Unfortunately, the list must be dynamically allocated as its size is
variable. That means the driver must generate the list twice - once to
work out the size and a second time to actually save it.

v2:
(Alan / CI)
- GEN7_GT_MODE -> GEN6_GT_MODE to fix WA selftest failure

Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Signed-off-by: Fernando Pacheco <fernando.pacheco@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210727002348.97202-16-matthew.brost@intel.com

drm/i915/xehp: Define multicast register ranges

Since we can't steer multicast register reads during ring-based
workaround verification, we need to define the multicast ranges where
failure to steer could potentially cause us to read back from a
fused-off register instance.

As with gen12, we can ignore the multicast ranges that the bspec
describes as 'SQIDI' since all instances of those registers will always
be present and we'll always be able to read back a workaround value that
was written with multicast.

Bspec: 66534
Cc: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210714031540.3539704-11-matthew.d.roper@intel.com

drm/i915: Extend Wa_1406941453 to adl-p

Workaround also needed for alderlake-P.

HSDES: 14010801662
Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210722192041.92346-1-jose.souza@intel.com

drm/i915: Make GT workaround upper bounds exclusive

Workarounds are documented in the bspec with an exclusive upper bound
(i.e., a "fixed" stepping that no longer needs the workaround). This
makes our driver's use of an inclusive upper bound for stepping ranges
confusing; the differing notation between code and bspec makes it very
easy for mistakes to creep in.

Let's switch the upper bound of our IS_{GT,DISP}_STEP macros over to use
an exclusive upper bound like the bspec does. This also has the benefit
of helping make sure workarounds are properly handled for new minor
steppings that show up (e.g., an A1 between the A0 and B0 we already
knew about) --- if the new intermediate stepping pulls in hardware fixes
early, there will be an update to the workaround definition which lets
us know we need to change our code. If the new stepping does not pull a
hardware fix earlier, then the new stepping will already be captured
properly by the "[begin, fix)" range in the code.

We'll probably need to be extra vigilant in code review of new
workarounds for the near future to make sure developers notice the new
semantics of workaround bounds. But we just migrated a bunch of our
platforms from the IS_REVID bounds over to IS_{GT,DISP}_STEP, so people
are already adjusting to the new macros and now is a good time to make
this change too.

[mattrope: Split out GT changes to apply through gt-next tree]
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210717051426.4120328-8-matthew.d.roper@intel.com

drm/i915: Program DFR enable/disable as a GT workaround

DFR programming (which we enable as an optimization on gen11, but must
ensure is disabled on gen12) should be handled as a GT workaround rather
than clock gating initialization. This will ensure that the programming
of these registers is verified with our typical workaround checks.

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210717051426.4120328-4-matthew.d.roper@intel.com

drm/i915/icl: Drop a couple unnecessary workarounds

While doing a quick sanity check of the ICL workarounds in the driver I
noticed a few things that should be updated:

* There's no mention in the bspec that WaPipelineFlushCoherentLines
is needed on gen11 (both the current WA database and the old,
deprecated page 20196 were checked); it appears this might have just
been copied from the gen9 list? Even if this were needed, it doesn't
seem like this was the correct implementation anyway since the gen9
workaround is supposed to be implemented in the indirect context bb
(as we do in gen8_emit_flush_coherentl3_wa() on gen8/gen9).

* WaForwardProgressSoftReset does not appear in the current workaround
database. The old deprecated workaround list has a note indicating
the workaround was dropped in 2017, so we should be safe to drop it
from the code too.

While we're at it, add the formal workaround ID number to
WaDisableBankHangMode (our hardware team made a transition from
text-based workaround names to ID numbers partway through the
development of ICL, which is why some workarounds only have names, some
only have numbers, and some have both).

Bspec: 33450
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210717051426.4120328-3-matthew.d.roper@intel.com

drm/i915: Fix application of WaInPlaceDecompressionHang

On SKL we've been applying this workaround on H0+ steppings, which is
actually backwards; H0 is supposed to be the first stepping where the
workaround is no longer needed. Flip the bounds so that the workaround
applies to all steppings _before_ H0.

On BXT we've been applying this workaround to all steppings, but the
bspec tells us it's only needed until C0. Pre-C0 GT steppings only
appeared in pre-production hardware, which we no longer support in the
driver, so we can drop the workaround completely for this platform.

On ICL we've been applying this workaround to all steppings, but there
doesn't seem to be any indication that this workaround was ever needed
for this platform (even now-deprecated page 20196 of the bspec doesn't
mention it). We can go ahead and drop it.

I also don't see any mention of this workaround being needed for KBL,
although this may be an oversight since the workaround is needed for all
steppings of CFL. I'll leave the workaround in place for KBL to be
safe.

Bspec: 14091, 33450
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210717051426.4120328-2-matthew.d.roper@intel.com

drm/i915/icl: Drop workarounds that only apply to pre-production steppings

We're past the point at which we usually drop workarounds that were
never needed on production hardware. The driver will already print an
error and apply taint if loaded on pre-production hardware.

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Anusha Srivatsa <anusha.srivatsa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210713193635.3390052-13-matthew.d.roper@intel.com

drm/i915/cnl: Drop all workarounds

All of the Cannon Lake hardware that came out had graphics fused off,
and our userspace drivers have already dropped their support for the
platform; CNL-specific code in i915 that isn't inherited by subsequent
platforms is effectively dead code. Let's remove all of the
CNL-specific workarounds as a quick and easy first step.

References: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6899
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Anusha Srivatsa <anusha.srivatsa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210713193635.3390052-12-matthew.d.roper@intel.com

drm/i915/dg1: Use revid->stepping tables

Switch DG1 to use a revid->stepping table as we're trying to do on all
platforms going forward.

This removes the last use of IS_REVID() and REVID_FOREVER, so remove
those now-unused macros as well to prevent their accidental use on
future platforms.

v2:
- Use COMMON_STEP() macro in table. (Anusha)

Bspec: 44463
Cc: Anusha Srivatsa <anusha.srivatsa@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Anusha Srivatsa <anusha.srivatsa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210713193635.3390052-11-matthew.d.roper@intel.com

drm/i915/jsl_ehl: Use revid->stepping tables

Switch JSL/EHL to use a revid->stepping table as we're trying to do on
all platforms going forward.

v2:
- Use COMMON_STEP(). (Anusha)

Bspec: 29153
Cc: Anusha Srivatsa <anusha.srivatsa@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Anusha Srivatsa <anusha.srivatsa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210713193635.3390052-9-matthew.d.roper@intel.com

drm/i915/icl: Use revid->stepping tables

Switch ICL to use a revid->stepping table as we're trying to do on all
platforms going forward. While we're at it, let's include some
additional steppings that have popped up, even if we don't yet have any
workarounds tied to those steppings (we probably need to audit our
workaround list soon to see if any of the bounds have moved or if new
workarounds have appeared).

Note that the current bspec table is missing information about how to
map PCI revision ID to GT/display steppings; it only provides an SoC
stepping. The mapping to GT/display steppings (which aren't always the
same as the SoC stepping) used to be in the bspec, but was apparently
dropped during an update in Nov 2019; I've made my changes here based on
an older bspec snapshot that still had the necessary information. We've
requested that the missing information be restored.

I'm only including the production revids in the table here since we're
past the point at which we usually stop trying to support pre-production
hardware. An appropriate check is added to
intel_detect_preproduction_hw() to print an error and taint the kernel
just in case someone still tries to load the driver on old
pre-production hardware.

v2:
- Drop pre-production steppings and add error/taint at startup when
loading on pre-production hardware.

Bspec: 21141 # pre-Nov 2019 snapshot
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210713193635.3390052-8-matthew.d.roper@intel.com

drm/i915/skl: Use revid->stepping tables

Switch SKL to use a revid->stepping table as we're trying to do on all
platforms going forward. Also drop the preproduction revisions and add
the newer steppings we hadn't already handled.

Note that SKL has a case where a newer revision ID corresponds to an
older GT/disp stepping (0x9 -> STEP_J0, 0xA -> STEP_I1). Also, the lack
of a revision ID 0x8 in the table is intentional and not an oversight.
We'll re-write the KBL-specific comment to make it clear that these kind
of quirks are expected.

v2:
- Since GT and display steppings are always identical on SKL use a
macro to set both values at once in a more readable manner. (Anusha)
- Drop preproduction steppings.

Bspec: 13626
Cc: Anusha Srivatsa <anusha.srivatsa@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Anusha Srivatsa <anusha.srivatsa@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210713193635.3390052-4-matthew.d.roper@intel.com

drm/i915/adl_s: Extend Wa_1406941453

BSpec: 54370
Cc: Gwan-gyeong Mun <gwan-gyeong.mun@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210713003854.143197-4-jose.souza@intel.com

drm/i915: Implement Wa_1508744258

Same bit was required for Wa_14012131227 in DG1 now it is also
required as Wa_1508744258 to TGL, RKL, DG1, ADL-S and ADL-P.

Cc: Gwan-gyeong Mun <gwan-gyeong.mun@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210713003854.143197-3-jose.souza@intel.com

drm/i915: Settle on "adl-x" in WA comments

Most of the places are using this format so lets consolidate it.

v2:
- split patch in two: display and non-display because of conflicts
between drm-intel-gt-next x drm-intel-next

Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210713003854.143197-2-jose.souza@intel.com

drm/i915: Add support for explicit L3BANK steering

Because Render Power Gating restricts us to just a single subslice as a
valid steering target for reads of multicast registers in a SUBSLICE
range, the default steering we setup at init may not lead to a suitable
target for L3BANK multicast register. In cases where it does not, use
explicit runtime steering whenever an L3BANK multicast register is read.

While we're at it, let's simplify the function a little bit and drop its
support for gen10/CNL since no such platforms ever materialized for real
use. Multicast register steering is already an area that causes enough
confusion; no need to complicate it with what's effectively dead code.

v2:
- Use gt->uncore instead of gt->i915->uncore. (Tvrtko)
- Use {} as table terminator. (Rodrigo)

v3:
- L3bank fuse register is a disable mask rather than an enable mask.
We need to invert it before use. (CI)

v4:
- L3bank ID goes in the subslice field, not the slice field. (CI)

Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210617211425.1943662-4-matthew.d.roper@intel.com

drm/i915: Add GT support for multiple types of multicast steering

Although most of our multicast registers are replicated per-subslice, we
also have a small number of multicast registers that are replicated
per-l3 bank instead. For both types of multicast registers we need to
make sure we steer reads of these registers to a valid instance.
Ideally we'd like to find a specific instance ID that would steer reads
of either type of multicast register to a valid instance (i.e., not
fused off and not powered down), but sometimes the combination of
part-specific fusing and the additional restrictions imposed by Render
Power Gating make it impossible to find any overlap between the set of
valid subslices and valid l3 banks. This problem will become even more
noticeable on our upcoming platforms since they will be adding
additional types of multicast registers with new types of replication
and rules for finding valid instances for reads.

To handle this we'll continue to pick a suitable subslice instance at
driver startup and program this as the default (sliceid,subsliceid)
setting in the steering control register (0xFDC). In cases where we
need to read another type of multicast GT register, but the default
subslice steering would not correspond to a valid instance, we'll
explicitly re-steer the single read to a valid value, perform the read,
and then reset the steering to it's "subslice" default.

This patch adds the general functionality to prepare for this explicit
steering of other multicast register types. We'll plug L3 bank steering
into this in the next patch, and then add additional types of multicast
registers when the support for our next upcoming platform arrives.

v2:
- Use entry->end==0 as table terminator. (Rodrigo)
- Grab forcewake in wa_list_verify() now that we're using accessors
that assume forcewake is already held.

v3:
- Fix loop condition when iterating over steering range tables.
(Rodrigo)

Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210617211425.1943662-3-matthew.d.roper@intel.com

drm/i915/adl_p: Add initial ADL_P Workarounds

Most of the context WA are already implemented.
Adding adl_p platform tag to reflect so.

v2: adjust comments for clarity (MattR)

BSpec: 54369
Cc: Matt Roper <matthew.d.roper@intel.com>
Cc: Aditya Swarup <aditya.swarup@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
Signed-off-by: Anusha Srivatsa <anusha.srivatsa@intel.com>
Signed-off-by: Madhumitha Tolakanahalli Pradeep <madhumitha.tolakanahalli.pradeep@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Swathi Dhanavanthri <swathi.dhanavanthri@intel.com>
Signed-off-by: Clint Taylor <clinton.a.taylor@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210608174721.17593-1-clinton.a.taylor@intel.com

drm/i915/gt: replace IS_GEN and friends with GRAPHICS_VER

This was done by the following semantic patch:

@@ expression i915; @@
- INTEL_GEN(i915)
+ GRAPHICS_VER(i915)

@@ expression i915; expression E; @@
- INTEL_GEN(i915) >= E
+ GRAPHICS_VER(i915) >= E

@@ expression dev_priv; expression E; @@
- !IS_GEN(dev_priv, E)
+ GRAPHICS_VER(dev_priv) != E

@@ expression dev_priv; expression E; @@
- IS_GEN(dev_priv, E)
+ GRAPHICS_VER(dev_priv) == E

@@
expression dev_priv;
expression from, until;
@@
- IS_GEN_RANGE(dev_priv, from, until)
+ IS_GRAPHICS_VER(dev_priv, from, until)

@def@
expression E;
identifier id =~ "^gen$";
@@
- id = GRAPHICS_VER(E)
+ ver = GRAPHICS_VER(E)

@@
identifier def.id;
@@
- id
+ ver

It also takes care of renaming the variable we assign to GRAPHICS_VER()
so to use "ver" rather than "gen".

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210605155356.4183026-2-lucas.demarchi@intel.com

drm/i915: Drop duplicate WaDisable4x2SubspanOptimization:hsw

Same workaround was listed two times - once under the Gen7 block and once
under the Haswell section.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210507084926.2423003-1-tvrtko.ursulin@linux.intel.com

drm/i915: Add Wa_14011060649

This is a permanent workaround for TGL,RKL,DG1 and ADLS.

Signed-off-by: Swathi Dhanavanthri <swathi.dhanavanthri@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210324200502.1731265-5-lucas.demarchi@intel.com

drm/i915: Move Wa_16011163337 to gen12_ctx_workarounds_init()

This WA is needed in all gen12 platforms, moving it to
gen12_ctx_workarounds_init() allow us to remove the duplicated
implementation.
Also allow us to remove the tgl_ctx_workarounds_init() that after the
WA move above was empty.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210324200502.1731265-4-lucas.demarchi@intel.com

drm/i915/icl: add Wa_22010271021 for all gen11

Wa_22010271021 does not apply only to EHL, but to all gen11 and other
gen12 platforms. Gen12 is already covered in another code path, but we
need to stop checking for EHL when handling gen11.

Bspec: 33450, 52887

v2: Remove "gen11" suffix as it also applies to gen12 platforms

Cc: Clinton Taylor <clinton.a.taylor@intel.com>
Signed-off-by: Caz Yokoyama <caz.yokoyama@intel.com>
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210324200502.1731265-3-lucas.demarchi@intel.com

drm/i915/gen12: Add recommended hardware tuning value

Follow Bspec 31870 to set recommended tuning values for certain GT
register. These values aren't workarounds per-se, but it's best to
handle them in the same general area of the driver, especially since
there may be real workarounds that update other bits of the same
registers.

At the moment the only value we need to worry about is the
TDS_TIMER setting in FF_MODE2. This setting was previously
described as "Wa_1604555607" on some platforms, but the spec
tells us that we should continue to program this on all current
gen12 platforms, even those that do not have that WA.

Bspec: 31870

v2: Rephrase some comments to make them clearer (Matt)

Cc: Clinton Taylor <clinton.a.taylor@intel.com>
Signed-off-by: Caz Yokoyama <caz.yokoyama@intel.com>
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210324200502.1731265-2-lucas.demarchi@intel.com

drm/i915: rename DISP_STEPPING->DISPLAY_STEP and GT_STEPPING->GT_STEP

Matter of taste. STEP matches the enums.

Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/cf2dccd1c9c7fdcf5de08ea10a9265292b45d8c7.1616764798.git.jani.nikula@intel.com

drm/i915: switch KBL to the new stepping scheme

Add new symbolic names for revision ids, and convert KBL revids to use
them via the new stepping check macros.

This also fixes theoretical out of bounds access to kbl_revids array.

v3: upgrade dbg to warn on unknown revid (José)

v2: Rename stepping->step

Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/79b6c48211c6b214165391d350d556bad748f747.1616764798.git.jani.nikula@intel.com

drm/i915: split out stepping info to a new file

gt/intel_workarounds.c is decidedly the wrong place for handling
stepping info. Add new intel_step.[ch] for the data, and move the
stepping arrays there. No functional changes.

v2: Rename stepping->step

Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/f69baf82819a8a35815fca25a520de5c38a7e1b5.1616764798.git.jani.nikula@intel.com

drm/i915/gt: Replace 'return' with a fall-through

Checkpatch worries that the 'return' before an else clause might be
redundant. In this case, it is avoiding hitting the MISSING_CASE()
warning. Let us appease checkpatch by falling through to the end of the
function, which typically means that we then clean up the unused
wa_list.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210122192913.4518-10-chris@chris-wilson.co.uk
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>

drm/i915/gt: Fixup misaligned function parameters

Remember to align parameters to the '(', thanks checkpatch

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210122192913.4518-4-chris@chris-wilson.co.uk
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>

drm/i915/gt: Remove repeated words from comments

Checkpatch spotted a few repeated words in the comment, genuine
mistakes.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210122192913.4518-3-chris@chris-wilson.co.uk
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>

drm/i915/gt: SPDX cleanup

Clean up the SPDX licence declarations to comply with checkpatch.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210122192913.4518-1-chris@chris-wilson.co.uk
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>

drm/i915: Move pinning to inside engine_wa_list_verify()

This should be done as part of the ww loop, in order to remove a
i915_vma_pin that needs ww held.

Now only i915_ggtt_pin() callers remaining.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210323155059.628690-25-maarten.lankhorst@linux.intel.com

drm/i915/adl_s: Add GT and CTX WAs for ADL-S

- Extend Wa_1606931601 and Wa_1409804808 to ADL-S.
- Extend Wa_14010919138 and Wa_14010229206 to ADL-S (Madhumitha)
- Extend Wa_22010271021 to ADLS (cyokoyam)

v2:
- Extend Wa_1409804808 and remove unnecessary branching/redundant
adls workaround placeholder functions.
- Split WAs properly based on previous platforms and applicable ADLS
WA.

Cc: Jani Nikula <jani.nikula@intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Imre Deak <imre.deak@intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Madhumitha Tolakanahalli Pradeep <madhumitha.tolakanahalli.pradeep@intel.com>
Signed-off-by: Aditya Swarup <aditya.swarup@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210129182945.217078-9-aditya.swarup@intel.com

drm/i915/adl_s: Add ADL-S platform info and PCI ids

- Add the initial platform information for Alderlake-S.
- Specify ppgtt_size value
- Add dma_mask_size
- Add ADLS REVIDs
- HW tracking(Selective Update Tracking Enable) has been
removed from ADLS. Disable PSR2 till we enable software/
manual tracking.

v2:
- Add support for different ADLS SOC steppings to select
correct GT/DISP stepping based on Bspec 53655 based on
feedback from Matt Roper.(aswarup)

v3:
- Make display/gt steppings info generic for reuse with TGL and ADLS.
- Modify the macros to reuse tgl_revids_get()
- Add HTI support to adls device info.(mdroper)

v4:
- Rebase on TGL patch for applying WAs based on stepping info from
Matt Roper's feedback.(aswarup)

v5:
- Replace macros with PCI IDs in revid to stepping table.

v6: remove stray adls_revids (Lucas)

Bspec: 53597
Bspec: 53648
Bspec: 53655
Bspec: 48028
Bspec: 53650
BSpec: 50422

Cc: José Roberto de Souza <jose.souza@intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Cc: Anusha Srivatsa <anusha.srivatsa@intel.com>
Cc: Jani Nikula <jani.nikula@intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Imre Deak <imre.deak@intel.com>
Signed-off-by: Caz Yokoyama <caz.yokoyama@intel.com>
Signed-off-by: Aditya Swarup <aditya.swarup@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210119192931.1116500-2-lucas.demarchi@intel.com

drm/i915/tgl: Use TGL stepping info for applying WAs

TGL adds another level of indirection for applying WA based on stepping
information rather than PCI REVID. So change TGL_REVID enum into
stepping enum and use PCI REVID as index into revid to stepping table to
fetch correct display and GT stepping for application of WAs as
suggested by Matt Roper.

Cc: Matt Roper <matthew.d.roper@intel.com>
Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Cc: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Aditya Swarup <aditya.swarup@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210119192931.1116500-1-lucas.demarchi@intel.com

drm/i915: Disable atomics in L3 for gen9

Enabling atomic operations in L3 leads to unrecoverable GPU hangs, as
the machine stops responding milliseconds after receipt of the reset
request [GDRT]. By disabling the cached atomics, the hang do not occur
and we presume the GPU would reset normally for similar hangs.

Sadly this is a shotgun approach, but since the impact is critical it is
better to err on the safe side and work back from there.

Reported-by: Jason Ekstrand <jason@jlekstrand.net>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=110998
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Jason Ekstrand <jason@jlekstrand.net>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: Jason Ekstrand <jason@jlesktrand.net>
Link: https://patchwork.freedesktop.org/patch/msgid/20210125220152.24070-1-chris@chris-wilson.co.uk
Cc: stable@vger.kernel.org
(cherry picked from commit b267c7ae0ad5b437b068f46919b17f85000154b4)
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/i915/gt: Prune inlines

Remove all the manual inlines from non-critical sections in gt/

add/remove: 2/0 grow/shrink: 0/3 up/down: 762/-1473 (-711)
Function old new delta
mi_set_context.isra - 602 +602
write_dma_entry - 160 +160
__set_pd_entry 214 69 -145
clear_pd_entry 190 42 -148
ring_request_alloc 2021 841 -1180
Total: Before=1605086, After=1604375, chg -0.04%

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Jani Nikula <jani.nikula@intel.com>
Reviewed-by: Jani Nikula <jani.nikula@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210113152224.29794-1-chris@chris-wilson.co.uk

drm/i915/gt: Rearrange ivb workarounds

Some rcs0 workarounds were being incorrectly applied to the GT, and so
we failed to restore the expected register settings after a reset.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210113225144.30810-2-chris@chris-wilson.co.uk

drm/i915/gt: Rearrange vlv workarounds

Some rcs0 workarounds were being incorrectly applied to the GT, and so
we failed to restore the expected register settings after a reset.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210113225144.30810-1-chris@chris-wilson.co.uk

drm/i915/gt: Mark up a debug-only function

drivers/gpu/drm/i915//gt/intel_workarounds.c:1394:20: error: function 'is_nonpriv_flags_valid' is not needed and will not be emitted [-Werror,-Wunneeded-internal-declaration]
static inline bool is_nonpriv_flags_valid(u32 flags)

This is only used by debug build, so mark it as maybe-unused to keep the
compiler from complaining.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Andi Shyti <andi.shyti@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210109163455.28466-2-chris@chris-wilson.co.uk

drm/i915/gt: Rearrange hsw workarounds

Some rcs0 workarounds were being incorrectly applied to the GT, and so
we failed to restore the expected register settings after a reset.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210104114914.30165-2-chris@chris-wilson.co.uk

drm/i915/gt: Rearrange snb workarounds

Some rcs0 workarounds were being incorrectly applied to the GT, and so
we failed to restore the expected register settings after a reset.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210104114914.30165-1-chris@chris-wilson.co.uk

drm/i915: Clarify error message on failed workaround

Let's modify the "workaround lost" error message slightly to make it
more clear what the various numbers represent. Also, the 'expected'
value needs to be &'d with wa->read so that it doesn't include the mask
bits for masked registers (those bits are write-only in the hardware and
will usually always read out as 0's).

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20201231191103.854519-1-matthew.d.roper@intel.com

drm/i915/gt: Provide a utility to create a scratch buffer

Primarily used by selftests, but also by runtime debugging of engine
w/a, is a routine to create a temporarily bound buffer for readback.
Almagamate the duplicated routines into one.

Suggested-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201219020343.22681-2-chris@chris-wilson.co.uk

drm/i915/gt: Move gen8 CS emitters into gen8_engine_cs.h

Reduce the pollution of intel_engine.h by moving gen8_emit_pipe_control
and friends to gen8_engine_cs.h

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201216135452.6063-1-chris@chris-wilson.co.uk

drm/i915: Correct location of Wa_1408615072

The above workaround was added as an engine workaround not a GT
workaround. Moved it to the correct location.

Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20201210170615.3107266-1-lucas.demarchi@intel.com

drm/i915/gt: document masked registers

Document what a masked register is according to bspec so we avoid
developers using the wrong functions to implement WAs.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20201209045246.2905675-3-lucas.demarchi@intel.com

drm/i915/gt: rename wa_write_masked_or()

The use of "masked" in this function is due to its history. Once upon a
time it received a mask and a value as parameter. Since
commit eeec73f8a4a4 ("drm/i915/gt: Skip rmw for masked registers")
that is not true anymore and now there is a clear and a set parameter.
Depending on the case, that can still be thought as a mask and value,
but there are some subtle differences: what we clear doesn't need to be
the same bits we are setting, particularly when we are using masked
registers.

The fact that we also have "masked registers", i.e. registers whose mask
is stored in the upper 16 bits of the register, makes it even more
confusing, because "masked" in wa_write_masked_or() has little to do
with masked registers, but rather refers to the old mask parameter the
function received (that can also, but not exclusively, be used to write
to masked register).

Avoid the ambiguity and misnomer by renaming it to something else,
hopefully less confusing: wa_write_clr_set(), to designate that we are
doing both clr and set operations in the register.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20201209045246.2905675-2-lucas.demarchi@intel.com

drm/i915/gt: stop ignoring read with wa_masked_field_set

When using masked registers, there is nothing to clear since a masked
register has the mask in the upper 16b: we can just write to the
location we want and use the mask to control what bits we are writing
to.

However that doesn't mean we don't want to read back the register and
check the value actually matched what we wanted to write, i.e. that
the WA stick. That should be an explicit opt-out for registers that are
either write-only or that are affected by hardware misbehavior.

Moreover both wa_masked_en() and wa_masked_dis() check the WA stick, so
skipping the check just because the field is more than 1 bit is
surprising and error-prone.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20201209045246.2905675-1-lucas.demarchi@intel.com

drm/i915: remove WA_SET_FIELD_MASKED()

Remove the last macro and implement it as a function like the rest of
the operations that don't assume there is a `wal` list, but rather
receive it as argument.

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20201205092542.2325477-4-lucas.demarchi@intel.com

drm/i915: remove WA_CLR_BIT_MASKED()

Just ommitting the list it's operating on doesn't save much typing
and adds another way to do the same thing. Just replace it with
wa_masked_dis().

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20201205092542.2325477-3-lucas.demarchi@intel.com

drm/i915: remove WA_SET_BIT_MASKED()

Just ommitting the list it's operating on doesn't save much typing and
adds another way to do the same thing. Just replace it with
wa_masked_en().

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20201205092542.2325477-2-lucas.demarchi@intel.com

drm/i915/dg1: Implement WA_16011163337

Set GS Timer to 224 to prevent a HS/DS hang.

Bspec: 53508

v2: reword commit message and add comment explaining why read
verification is ignored (Chris)

Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Swathi Dhanavanthri <swathi.dhanavanthri@intel.com>
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20201205092542.2325477-1-lucas.demarchi@intel.com

drm/i915/tgl, rkl, dg1: Apply WA_1406941453 to TGL, RKL and DG1

This workaround is applicable only for tgl,rkl and dg1.

Bspec: 52890, 53273, 53508.

Signed-off-by: Swathi Dhanavanthri <swathi.dhanavanthri@intel.com>
Reviewed-by: Clint Taylor <Clinton.A.Taylor@intel.com>
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201201175735.1377372-1-lucas.demarchi@intel.com

drm/i915/dg1: make Wa_22010271021 permanent

Just like for rkl and tgl, this should be permanent as well for dg1
instead just for A0. The commit making it permanent for those platforms
ended up "racing" with the commit adding the DG1 WAs, so now fix that up.

v2: Add "tgl,dg1" to WA comment (Matt)

Cc: Swathi Dhanavanthri <swathi.dhanavanthri@intel.com>
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201027043228.696518-3-lucas.demarchi@intel.com

drm/i915: Avoid memory leak with more than 16 workarounds on a list

I forgot to free the old list when growing past 16 entries.

Luckily, as much as I checked, none of the current platforms has more than
16 workarounds on a single list.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Fixes: 452420d22d5b ("drm/i915: Fuse per-context workaround handling with the common framework")
Reported-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20201113132510.2298483-1-tvrtko.ursulin@linux.intel.com

drm/i915: Avoid memory leak with more than 16 workarounds on a list

I forgot to free the old list when growing past 16 entries.

Luckily, as much as I checked, none of the current platforms has more than
16 workarounds on a single list.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Fixes: 452420d22d5b ("drm/i915: Fuse per-context workaround handling with the common framework")
Reported-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20201113132510.2298483-1-tvrtko.ursulin@linux.intel.com
(cherry picked from commit 77c296966e866a795742a46fc52a218771894867)
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/i915/dg1: Add initial DG1 workarounds

DG1 shares some workarounds with TGL and RKL and also has some
additional workarounds of its own.

v2: Correct location of Wa_1408615072 (JohnH).
v3: Apply WAs 1606700617, 18011464164 and 22010931296 to DG1 (José)
v4 (Anusha)
- Add Wa_22010271021
- s/Wa_14010096844/Wa_1409836686
v5:
- Extend Wa_14010919138 to all revs (Matt Atwood)
- Power gate media is global gen12 design. (Rodrigo)
- Rebase (Lucas)
v6: use REG_BIT() to fix checkpatch warning (Lucas)

BSpec: 53508

Cc: Matt Atwood <matthew.s.atwood@intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Cc: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
Cc: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Stuart Summers <stuart.summers@intel.com>
Signed-off-by: Anusha Srivatsa <anusha.srivatsa@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201014191937.1266226-8-lucas.demarchi@intel.com

drm/i915/jsl: Split EHL/JSL platform info and PCI ids

Recently we came across requirement to identify EHL and JSL
platform to program them differently. Thus Split the basic
platform definition, macros, and PCI IDs to differentiate
between EHL and JSL platforms. Also, IS_ELKHARTLAKE is replaced
with IS_JSL_EHL everywhere.

Changes since V1 :
- Rebased to avoid merge conflicts
- Added missed check for jasperlake in intel_uc_fw.c

Cc : Matt Roper <matthew.d.roper@intel.com>
Cc : Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Tejas Upadhyay <tejaskumarx.surendrakumar.upadhyay@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201013192948.63470-1-tejaskumarx.surendrakumar.upadhyay@intel.com

drm/i915: Update gen12 multicast register ranges

The updated bspec forcewake table also provides us with new multicast
ranges that should be reflected in our workaround code.

Note that there are different types of multicast registers with
different styles of replication and different steering registers. The
i915 MCR range lists we're updating here are only used to ensure we can
verify workarounds properly (i.e., if we can't steer register reads we
don't want to verify workarounds where an unsteered read might hit a
fused-off instance of the unit). Because of this, we don't need to
include any of the multicast ranges where all instances of the register
will always present and fusing doesn't play a role. Specifically, that
means that we are not including the MCR ranges designated as "SQIDI" in
the bspec.

Bspec: 66696
Cc: Caz Yokoyama <caz.yokoyama@intel.com>
Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Cc: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201009194442.3668677-4-matthew.d.roper@intel.com
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>

drm/i915/tgl, rkl: Make Wa_1606700617/22010271021 permanent

This workaround applies to all TGL and RKL steppings.

Signed-off-by: Swathi Dhanavanthri <swathi.dhanavanthri@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200911221158.4700-1-swathi.dhanavanthri@intel.com

drm/i915: Kill last user of intel_context_create_request outside of selftests

Instead of using intel_context_create_request(), use intel_context_pin()
and i915_create_request directly.

Now all those calls are gone outside of selftests. :)

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200819140904.1708856-17-maarten.lankhorst@linux.intel.com
Signed-off-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>

drm/i915/tgl: Fix stepping WA matching

TGL made stepping a litte mess, workarounds refer to the stepping of
the IP(GT or Display) not of the GPU stepping so it would already
require the same solution as used in commit 96c5a15f9f39
("drm/i915/kbl: Fix revision ID checks").
But to make things even more messy it have a different IP stepping
mapping between SKUs and the same stepping revision of GT do not match
the same HW between TGL U/Y and regular TGL.

So it was required to have 2 different macros to check GT WAs while
for Display we are able to use just one macro that uses the right
revids table.

All TGL workarounds checked and updated accordingly.

v2:
- removed TODO to check if WA 14010919138 applies to regular TGL.
- fixed display stepping in regular TGL (Anusha)

BSpec: 52890
BSpec: 55378
BSpec: 44455
Reviewed-by: Anusha Srivatsa <anusha.srivtsa@intel.com>
Cc: Anusha Srivatsa <anusha.srivatsa@intel.com>
Cc: Penne Lee <penne.y.lee@intel.com>
Cc: Guangyao Bai <guangyao.bai@intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200827233943.400946-1-jose.souza@intel.com

drm/i915/gt: Implement WA_1406941453

Enable HW Default flip for small PL.

bspec: 52890
bspec: 53508
bspec: 53273

v2: rebase to drm-tip
v3: move from ctx to gt workarounds. Remove whitelist.
v4: move to rcs WA init

Cc: Matt Atwood <matthew.s.atwood@intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Cc: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Clint Taylor <clinton.a.taylor@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200826025724.20944-1-clinton.a.taylor@intel.com

drm/i915/kbl: Fix revision ID checks

We usually assume that increasing PCI device revision ID's translates to
newer steppings; macros like IS_KBL_REVID() that we use rely on this
behavior. Unfortunately this turns out to not be true on KBL; the
newer device 2 revision ID's sometimes go backward to older steppings.
The situation is further complicated by different GT and display
steppings associated with each revision ID.

Let's work around this by providing a table to map the revision ID to
specific GT and display steppings, and then perform our comparisons on
the mapped values.

v2:
- Move the kbl_revids[] array to intel_workarounds.c to avoid compiler
warnings about an unused variable in files that don't call the
macros (kernel test robot).

Bspec: 18329
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200811032105.2819370-1-matthew.d.roper@intel.com
Reviewed-by: Swathi Dhanavanthri <swathi.dhanavanthri@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/i915/rkl: Add initial workarounds

RKL and TGL share some general gen12 workarounds, but each platform also
has its own platform-specific workarounds.

v2:
- Add Wa_1604555607 for RKL. This makes RKL's ctx WA list identical to
TGL's, so we'll have both functions call the tgl_ function for now;
this workaround isn't listed for DG1 so we don't want to add it to
the general gen12_ function.

Cc: Matt Atwood <matthew.s.atwood@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200716220551.2730644-3-matthew.d.roper@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

drm/i915/tgl: Implement WAs 18011464164 and 22010931296

As today those 2 WAs have different implementation between TGL and DG1
WA pages but checking the HSD it is clear that DG1 implementation
should be used for both, also to do so is easier as we just need to
extend WA 1407928979 to B* stepping.

Both WAs are need to fix some possible render corruptions.

DG1 initial patches were not merged yet, as soon it is this WAs should
be applied to DG1 as well.

BSpec: 53508
BSpec: 52890
Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200708212947.40178-1-jose.souza@intel.com

drm/i915/sseu: Move sseu_info under gt_info

SSEUs are a GT capability, so track them under gt_info.

Signed-off-by: Venkata Sandeep Dhanalakota <venkata.s.dhanalakota@intel.com>
Signed-off-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Andi Shyti <andi.shyti@intel.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20200708003952.21831-8-daniele.ceraolospurio@intel.com

drm/i915/gt: Move gen4 GT workarounds from init_clock_gating to workarounds

Rescue the GT workarounds from being buried inside init_clock_gating so
that we remember to apply them after a GT reset, and that they are
included in our verification that the workarounds are applied.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: stable@vger.kernel.org
Link: https://patchwork.freedesktop.org/patch/msgid/20200611080140.30228-6-chris@chris-wilson.co.uk
(cherry picked from commit 2bcefd0d263ab4a72f0d61921ae6b0dc81606551)
Signed-off-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>

drm/i915/gt: Move ilk GT workarounds from init_clock_gating to workarounds

Rescue the GT workarounds from being buried inside init_clock_gating so
that we remember to apply them after a GT reset, and that they are
included in our verification that the workarounds are applied.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: stable@vger.kernel.org
Link: https://patchwork.freedesktop.org/patch/msgid/20200611080140.30228-5-chris@chris-wilson.co.uk
(cherry picked from commit 806a45c0838d253e306a6384057e851b65d11099)
Signed-off-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>

drm/i915/gt: Move snb GT workarounds from init_clock_gating to workarounds

Rescue the GT workarounds from being buried inside init_clock_gating so
that we remember to apply them after a GT reset, and that they are
included in our verification that the workarounds are applied.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: stable@vger.kernel.org
Link: https://patchwork.freedesktop.org/patch/msgid/20200611080140.30228-4-chris@chris-wilson.co.uk
(cherry picked from commit c3b93a943f2c9ee4a106db100a2fc3b2f126bfc5)
Signed-off-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>

drm/i915/gt: Move vlv GT workarounds from init_clock_gating to workarounds

Rescue the GT workarounds from being buried inside init_clock_gating so
that we remember to apply them after a GT reset, and that they are
included in our verification that the workarounds are applied.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: stable@vger.kernel.org
Link: https://patchwork.freedesktop.org/patch/msgid/20200611080140.30228-3-chris@chris-wilson.co.uk
(cherry picked from commit 7331c356b6d2d8a01422cacab27478a1dba9fa2a)
Signed-off-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>

drm/i915/gt: Move ivb GT workarounds from init_clock_gating to workarounds

Rescue the GT workarounds from being buried inside init_clock_gating so
that we remember to apply them after a GT reset, and that they are
included in our verification that the workarounds are applied.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: stable@vger.kernel.org
Link: https://patchwork.freedesktop.org/patch/msgid/20200611080140.30228-2-chris@chris-wilson.co.uk
(cherry picked from commit 19f1f627b33385a2f0855cbc7d33d86d7f4a1e78)
Signed-off-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>

drm/i915/gt: Move hsw GT workarounds from init_clock_gating to workarounds

Rescue the GT workarounds from being buried inside init_clock_gating so
that we remember to apply them after a GT reset, and that they are
included in our verification that the workarounds are applied.

v2: Leave HSW_SCRATCH to set an explicit value, not or in our disable
bit.

Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/2011
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: stable@vger.kernel.org
Link: https://patchwork.freedesktop.org/patch/msgid/20200611093015.11370-1-chris@chris-wilson.co.uk
(cherry picked from commit f93ec5fb563779bda4501890b1854526de58e0f1)
Signed-off-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>

drm/i915/gt: Move gen4 GT workarounds from init_clock_gating to workarounds

Rescue the GT workarounds from being buried inside init_clock_gating so
that we remember to apply them after a GT reset, and that they are
included in our verification that the workarounds are applied.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: stable@vger.kernel.org
Link: https://patchwork.freedesktop.org/patch/msgid/20200611080140.30228-6-chris@chris-wilson.co.uk

drm/i915/gt: Move ilk GT workarounds from init_clock_gating to workarounds

Rescue the GT workarounds from being buried inside init_clock_gating so
that we remember to apply them after a GT reset, and that they are
included in our verification that the workarounds are applied.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: stable@vger.kernel.org
Link: https://patchwork.freedesktop.org/patch/msgid/20200611080140.30228-5-chris@chris-wilson.co.uk

drm/i915/gt: Move snb GT workarounds from init_clock_gating to workarounds

Rescue the GT workarounds from being buried inside init_clock_gating so
that we remember to apply them after a GT reset, and that they are
included in our verification that the workarounds are applied.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: stable@vger.kernel.org
Link: https://patchwork.freedesktop.org/patch/msgid/20200611080140.30228-4-chris@chris-wilson.co.uk

drm/i915/gt: Move vlv GT workarounds from init_clock_gating to workarounds

Rescue the GT workarounds from being buried inside init_clock_gating so
that we remember to apply them after a GT reset, and that they are
included in our verification that the workarounds are applied.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: stable@vger.kernel.org
Link: https://patchwork.freedesktop.org/patch/msgid/20200611080140.30228-3-chris@chris-wilson.co.uk

drm/i915/gt: Move ivb GT workarounds from init_clock_gating to workarounds

Rescue the GT workarounds from being buried inside init_clock_gating so
that we remember to apply them after a GT reset, and that they are
included in our verification that the workarounds are applied.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: stable@vger.kernel.org
Link: https://patchwork.freedesktop.org/patch/msgid/20200611080140.30228-2-chris@chris-wilson.co.uk

drm/i915/gt: Move hsw GT workarounds from init_clock_gating to workarounds

Rescue the GT workarounds from being buried inside init_clock_gating so
that we remember to apply them after a GT reset, and that they are
included in our verification that the workarounds are applied.

v2: Leave HSW_SCRATCH to set an explicit value, not or in our disable
bit.

Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/2011
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: stable@vger.kernel.org
Link: https://patchwork.freedesktop.org/patch/msgid/20200611093015.11370-1-chris@chris-wilson.co.uk

drm/i915/tgl: Implement WA_16011163337

Set GS Timer to 224. Combine with Wa_1604555607 due to register FF_MODE2
not being able to be read.

V2: Math issue fixed

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Caz Yokoyama <caz.yokoyama@intel.com>
Cc: Matt Atwood <matthew.s.atwood@intel.com>
Signed-off-by: Clint Taylor <clinton.a.taylor@intel.com>
Acked-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20200603221150.14745-1-clinton.a.taylor@intel.com

drm/i915: Drop i915_request.i915 backpointer

We infrequently use the direct i915 backpointer from the i915_request,
so do we really need to waste the space in the struct for it? 8 bytes
from the most frequently allocated struct vs an 3 bytes and pointer
chasing in using rq->engine->i915?

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Akeem G Abodunrin <akeem.g.abodunrin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200602220953.21178-1-chris@chris-wilson.co.uk

drm/i915/gt: Make the CTX_TIMESTAMP readable on !rcs

For reasons that be, the HW only allows usersace to read its own
CTX_TIMESTAMP [context local HW runtime] on rcs. Make it available for
all by adding it to the whitelists.

v2: The change took effect from Cometlake.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200602154839.6902-1-chris@chris-wilson.co.uk

drm/i915: Identify Cometlake platform

Cometlake is a small refresh of Coffeelake, but since we have found out a
difference in the plaforms, we need to identify them as separate platforms.

Since we previously took Coffeelake/Cometlake as identical, update all
IS_COFFEELAKE() to also include IS_COMETLAKE().

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200602140541.5481-1-chris@chris-wilson.co.uk

drm/i915/gt: Move legacy context wa to intel_workarounds

Use the central mechanism for recording and verifying that we restore
the w/a for the older devices as well.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200601072446.19548-3-chris@chris-wilson.co.uk

drm/i915/ehl: Wa_22010271021

Reflect recent Bspec changes.

Bspec: 33451

Signed-off-by: Matt Atwood <matthew.s.atwood@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Swathi Dhanavanthri <swathi.dhanavanthri@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200519162534.10035-1-matthew.s.atwood@intel.com

drm/i915/ehl: Restrict w/a 1607087056 for EHL/JSL

This w/a is fixed in B0 stepping and needs to be restricted for
A0 stepping only.
Bspec: 33451

Signed-off-by: Swathi Dhanavanthri <swathi.dhanavanthri@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200512180050.6785-1-swathi.dhanavanthri@intel.com

drm/i915: Remove cnl pre-prod workarounds

Remove all the stepping dependent cnl workarounds. Bspec lists
more steppings than this so presumably these are classed as
pre-production. And this is cnl after all so no one should
really care anyway.

Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200430125822.21985-2-ville.syrjala@linux.intel.com
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>

drm/i915/tgl: Initialize multicast register steering for workarounds

Even though the bspec is missing gen12 register details for the MCR
selector register (0xFDC), this is confirmed by hardware folks to be a
mistake; the register does exist and we do indeed need to steer
multicast register reads to an appropriate instance the same as we did
on gen11.

Note that despite the lack of documentation we were still using the MCR
selector to read INSTDONE and such in read_subslice_reg() too.

Cc: Matt Atwood <matthew.s.atwood@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200414211118.2787489-4-matthew.d.roper@intel.com
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>

drm/i915/tgl: Make Wa_14010229206 permanent

This workaround now applies to all steppings, not just A0.
Wa_1409085225 is a temporary A0-only W/A however it is
identical to Wa_14010229206 and hence the combined workaround
is made permanent.
Bspec: 52890

Signed-off-by: Swathi Dhanavanthri <swathi.dhanavanthri@intel.com>
Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
[mattrope: added missing blank line]
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200326234955.16155-1-swathi.dhanavanthri@intel.com

drm/i915/workarounds: convert to drm_device based logging macros.

Replace the use of printk based drm logging macros with the struct
drm_device based logging macros.

Note that this converts DRM_DEBUG_DRIVER() to drm_dbg().

References: https://lists.freedesktop.org/archives/dri-devel/2020-January/253381.html
Signed-off-by: Wambui Karuga <wambui.karugax@gmail.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200314183344.17603-8-wambui.karugax@gmail.com

drm/i915: Handle all MCR ranges

The bspec documents multiple MCR ranges; make sure they're all captured
by the driver.

Bspec: 13991, 52079
Fixes: 592a7c5e082e ("drm/i915: Extend non readable mcr range")
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200311162300.1838847-2-matthew.d.roper@intel.com
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
(cherry picked from commit 415d1269975d3fc21c13a6ae8de7b5fe0e6febb1)
Signed-off-by: Jani Nikula <jani.nikula@intel.com>

drm/i915: Add Wa_1605460711 / Wa_1408767742 to ICL and EHL

This workaround appears under two different numbers (and with somewhat
confused stepping applicability on ICL). Ultimately it appears we
should just implement this for all stepping of ICL and EHL.

Note that this is identical to Wa_1407928979:tgl that already exists in
our driver too...yet another number referencing the same actual
workaround.

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200311162300.1838847-7-matthew.d.roper@intel.com
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>

drm/i915: Apply Wa_1406680159:icl,ehl as an engine workaround

The register this workaround updates is a render engine register in the
MCR range, so we should initialize this in rcs_engine_wa_init() rather
than gt_wa_init().

Closes: https://gitlab.freedesktop.org/drm/intel/issues/1222
Fixes: 36204d80bacb ("drm/i915/icl: Wa_1406680159")
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200311162300.1838847-6-matthew.d.roper@intel.com
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>

drm/i915: Add Wa_1406306137:icl,ehl

v2:
- Move to context workarounds. ROW_CHICKEN4 is part of the context
image on gen11 (although it isn't on gen12).

Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200311162300.1838847-5-matthew.d.roper@intel.com
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>

drm/i915: Add Wa_1604278689:icl,ehl

The bspec description for this workaround tells us to program
0xFFFF_FFFF into both FBC_RT_BASE_ADDR_REGISTER_* registers, but we've
previously found that this leads to failures in CI. Our suspicion is
that the failures are caused by this valid turning on the "address valid
bit" even though we're intentionally supplying an invalid address.
Experimentation has shown that setting all bits _except_ for the
RT_VALID bit seems to avoid these failures.

v2:
- Mask off the RT_VALID bit. Experimentation with CI trybot indicates
that this is necessary to avoid reset failures on BCS.

v3:
- Program RT_BASE before RT_BASE_UPPER so that the valid bit is turned
off by the first write. (Chris)

Bspec: 11388
Bspec: 33451
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20200311162300.1838847-4-matthew.d.roper@intel.com
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>

drm/i915: Handle all MCR ranges

The bspec documents multiple MCR ranges; make sure they're all captured
by the driver.

Bspec: 13991, 52079
Fixes: 592a7c5e082e ("drm/i915: Extend non readable mcr range")
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200311162300.1838847-2-matthew.d.roper@intel.com
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>

drm/i915/tgl: Don't treat unslice registers as masked

The UNSLICE_UNIT_LEVEL_CLKGATE and UNSLICE_UNIT_LEVEL_CLKGATE2 registers
that we update in a few engine workarounds are not masked registers
(i.e., we don't have to write a mask bit in the top 16 bits when
updating one of the lower 16 bits). As such, these workarounds should
be applied via wa_write_or() rather than wa_masked_en()

v2:
- Rebase

Reported-by: Nick Desaulniers <ndesaulniers@google.com>
Reported-by: kernelci.org bot <bot@kernelci.org>
References: https://github.com/ClangBuiltLinux/linux/issues/918
Fixes: 50148a25f841 ("drm/i915/tgl: Move and restrict Wa_1408615072")
Fixes: 3551ff928744 ("drm/i915/gen11: Moving WAs to rcs_engine_wa_init()")
Cc: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Tested-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200306171139.1414649-1-matthew.d.roper@intel.com

drm/i915/tgl: Make Wa_1606700617 permanent

This workaround is to disable FF DOP Clock gating. The fix
in B0 was backed out due to timing reasons and decided to
be made permanent.

Bspec: 52890
Signed-off-by: Swathi Dhanavanthri <swathi.dhanavanthri@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200305181204.28856-1-swathi.dhanavanthri@intel.com

drm/i915/tgl: WaDisableGPGPUMidThreadPreemption

Enable FtrPerCtxtPreemptionGranularityControl bit and select thread-
group as the default preemption level.

v2:
* Remove register whitelisting (Rafael, Tony).

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Michał Winiarski <michal.winiarski@intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: piotr.zdunowski@intel.com
Cc: michal.mrozek@intel.com
Cc: Tony Ye <tony.ye@intel.com>
Cc: Rafael Antognolli <rafael.antognolli@intel.com>
Acked-by: Rafael Antognolli <rafael.antognolli@intel.com>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Acked-by: Chris Wilson <chris@chris-wilson.co.uk>
Acked-by: Tony Ye <tony.ye@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200304153144.10675-1-tvrtko.ursulin@linux.intel.com

drm/i915/tgl: Move and restrict Wa_1408615072

Following the changes in the previous patch
"drm/i915/gen11: Moving WAs to rcs_engine_wa_init()" also moving TGL
Wa_1408615072 to rcs_engine_wa_init() this way after a engine
reset it will be reapplied also restricting it to A0 as it is fixed in
B0 stepping.

BSpec: 52890
Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200302231421.224322-2-jose.souza@intel.com

drm/i915/gen11: Moving WAs to rcs_engine_wa_init()

This are register of render engine, so after a render reset those
would return to the default value and init_clock_gating() is not
called for single engine reset.
So here moving it rcs_engine_wa_init() that will guarantee that this
WAs will not be lost.

Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200302231421.224322-1-jose.souza@intel.com

drm/i915/tgl: Add Wa number to WaAllowPMDepthAndInvocationCountAccessFromUMD

Just to make easier to check that the Wa was implemetend when
comparing to the number in BSpec.

BSpec: 52890
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200227220101.321671-10-jose.souza@intel.com