sfc: support offloading ct(nat) action in RHS rules

If an IP address and/or L4 port for NAPT is available from a CT match,
the MAE will perform the edits; if no CT lookup has been performed for
this packet, the CT lookup did not return a match, or the matched CT
entry did not include NAPT, the action will have no effect.

Reviewed-by: Pieter Jansen van Vuuren <pieter.jansen-van-vuuren@amd.com>
Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sfc: support TC rules which require OR-AR-CT-AR flow

When a foreign LHS rule (TC rule from a tunnel netdev which requests
conntrack lookup) matches on inner headers or enc_key_id, these matches
cannot be performed by the Outer Rule table, as the keys are only
available after the tunnel type has been identified (by the OR lookup)
and the rest of the headers parsed accordingly.
Offload such rules with an Action Rule, using the LOOKUP_CONTROL section
of the AR response to specify the conntrack and/or recirculation actions,
combined with an Outer Rule which performs only the usual Encap Match
duties.
This processing flow, as it requires two AR lookups per packet, is less
performant than OR-CT-AR, so only use it where necessary.

Reviewed-by: Pieter Jansen van Vuuren <pieter.jansen-van-vuuren@amd.com>
Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sfc: support TC left-hand-side rules on foreign netdevs

Allow a tunnel netdevice (such as a vxlan) to offload conntrack lookups,
in much the same way as efx netdevs.
To ensure this rule does not overlap with other tunnel rules on the same
sip,dip,dport tuple, register a pseudo encap match of a new type
(EFX_TC_EM_PSEUDO_OR), which unlike PSEUDO_MASK may only be referenced
once (because an actual Outer Rule in hardware exists, although its
fw_id is not recorded in the encap match entry).

Reviewed-by: Pieter Jansen van Vuuren <pieter.jansen-van-vuuren@amd.com>
Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sfc: add decrement ttl by offloading set ipv4 ttl actions

Offload pedit set ipv4 ttl field, where the ttl field has already been
matched and the new value is one less, by translating it to a decrement.

Co-developed-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: Pieter Jansen van Vuuren <pieter.jansen-van-vuuren@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sfc: introduce ethernet pedit set action infrastructure

Introduce the initial ethernet pedit set action infrastructure in
preparation for adding mac src and dst pedit action offloads.

Co-developed-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: Pieter Jansen van Vuuren <pieter.jansen-van-vuuren@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sfc: offload left-hand side rules for conntrack

Handle the (comparatively) simple case of a -trk rule on an efx netdev
(i.e. not a tunnel decap rule) with ct and goto chain actions.

Reviewed-by: Pieter Jansen van Vuuren <pieter.jansen-van-vuuren@amd.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sfc: conntrack state matches in TC rules

Parse ct_state trk/est, mark and zone out of flower keys, and plumb
them through to the hardware, performing some minor translations.
Nothing can actually hit them yet as we're not offloading any DO_CT
actions.

Reviewed-by: Pieter Jansen van Vuuren <pieter.jansen-van-vuuren@amd.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sfc: functions to insert/remove conntrack entries to MAE hardware

Translate from software struct efx_tc_ct_entry objects to the key
and response bitstrings, and implement insertion and removal of
these entries from the hardware table.
Callers of these functions will be added in subsequent patches.

Reviewed-by: Pieter Jansen van Vuuren <pieter.jansen-van-vuuren@amd.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sfc: add MAE table machinery for conntrack table

Access to the connection tracking table in EF100 hardware is through
a "generic" table mechanism, whereby a firmware call at probe time
gives the driver a description of the field widths and offsets, so
that the driver can then construct key and response bitstrings at
runtime.
Probe the NIC for this information and populate the needed metadata
into a new meta_ct field of struct efx_tc_state.

Reviewed-by: Pieter Jansen van Vuuren <pieter.jansen-van-vuuren@amd.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sfc: MAE functions to create/update/delete encap headers

Besides the raw header data, also pass the tunnel type, so that the
hardware knows it needs to update the IP Total Length and UDP Length
fields (and corresponding checksums) for each packet.
Also, populate the ENCAP_HEADER_ID field in efx_mae_alloc_action_set()
with the fw_id returned from efx_mae_allocate_encap_md().

Reviewed-by: Pieter Jansen van Vuuren <pieter.jansen-van-vuuren@amd.com>
Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

sfc: add function to atomically update a rule in the MAE

efx_mae_update_rule() changes the action-set-list attached to an MAE
flow rule in the Action Rule Table.
We will use this when neighbouring updates change encap actions.

Reviewed-by: Pieter Jansen van Vuuren <pieter.jansen-van-vuuren@amd.com>
Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

sfc: support TC decap rules matching on enc_src_port

Allow efx_tc_encap_match entries to include a udp_sport and a
udp_sport_mask. As with enc_ip_tos, use pseudos to enforce that all
encap matches within a given <src_ip,dst_ip,udp_dport> tuple have
the same udp_sport_mask.
Note that since we use a single layer of pseudos for both fields, two
matches that differ in (say) udp_sport value aren't permitted to have
different ip_tos_mask, even though this would technically be safe.
Current userland TC does not support setting enc_src_port; this patch
was tested with an iproute2 patched to support it.

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sfc: populate enc_ip_tos matches in MAE outer rules

Currently tc.c will block them before they get here, but following
patch will change that.
Use the extack message from efx_mae_check_encap_match_caps() instead
of writing a new one, since there's now more being fed in than just
an IP version.

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sfc: add offloading of 'foreign' TC (decap) rules

A 'foreign' rule is one for which the net_dev is not the sfc netdevice
or any of its representors. The driver registers indirect flow blocks
for tunnel netdevs so that it can offload decap rules. For example:

tc filter add dev vxlan0 parent ffff: protocol ipv4 flower \
enc_src_ip 10.1.0.2 enc_dst_ip 10.1.0.1 \
enc_key_id 1000 enc_dst_port 4789 \
action tunnel_key unset \
action mirred egress redirect dev $REPRESENTOR

When notified of a rule like this, register an encap match on the IP
and dport tuple (creating an Outer Rule table entry) and insert an MAE
action rule to perform the decapsulation and deliver to the representee.

Moved efx_tc_delete_rule() below efx_tc_flower_release_encap_match() to
avoid the need for a forward declaration.

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sfc: add functions to insert encap matches into the MAE

An encap match corresponds to an entry in the exact-match Outer Rule
table; the lookup response includes the encap type (protocol) allowing
the hardware to continue parsing into the inner headers.

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sfc: add notion of match on enc keys to MAE machinery

Extend the MAE caps check to validate that the hardware supports these
outer-header matches where used by the driver.
Extend efx_mae_populate_match_criteria() to fill in the outer rule ID
and VNI match fields.
Nothing yet populates these match fields, nor creates outer rules.

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sfc: support offloading TC VLAN push/pop actions to the MAE

EF100 can pop and/or push up to two VLAN tags.

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
Link: https://lore.kernel.org/r/20230309115904.56442-1-edward.cree@amd.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

sfc: clean up some inconsistent indentings

Fix some indentngs and remove the warning below:
drivers/net/ethernet/sfc/mae.c:657 efx_mae_enumerate_mports() warn: inconsistent indenting

Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Link: https://bugzilla.openanolis.cn/show_bug.cgi?id=4117
Signed-off-by: Yang Li <yang.lee@linux.alibaba.com>
Acked-by: Martin Habets <habetsm.xilinx@gmail.com>
Link: https://lore.kernel.org/r/20230220065958.52941-1-yang.lee@linux.alibaba.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

sfc: add mport lookup based on driver's mport data

Obtaining mport id is based on asking the firmware about it. This is
still needed for mport initialization itself, but once the mport data is
now kept by the driver, further mport id request can be satisfied
internally without firmware interaction.

Previous function is just modified in name making clear the firmware
interaction. The new function uses the old name and looks for the data
in the mport data structure.

Signed-off-by: Alejandro Lucero <alejandro.lucero-palau@amd.com>
Acked-by: Martin Habets <habetsm.xilinx@gmail.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>

sfc: enumerate mports in ef100

MAE ports (mports) are the ports on the EF100 embedded switch such
as networking PCIe functions, the physical port, and potentially
others.

Signed-off-by: Alejandro Lucero <alejandro.lucero-palau@amd.com>
Acked-by: Martin Habets <habetsm.xilinx@gmail.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>

sfc: attach an MAE counter to TC actions that need it

The only actions that expect stats (that sfc HW supports) are gact shot
(drop), mirred redirect and mirred mirror. Since these are 'deliverish'
actions that end an action-set, we only require at most one counter per
action-set.

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sfc: add functions to allocate/free MAE counters

efx_tc_flower_get_counter_index() will create an MAE counter mapped to
the passed (TC filter) cookie, or increment the reference if one already
exists for that cookie.

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sfc: add ef100 MAE counter support functions

Start and stop MAE counter streaming, and grant credits.

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sfc: add Layer 4 matches to ef100 TC offload

Support matching on UDP/TCP source and destination ports and TCP flags,
with masking if supported by the hardware.

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

sfc: add Layer 3 flag matches to ef100 TC offload

Support matching on ip_frag and ip_firstfrag.

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

sfc: add Layer 3 matches to ef100 TC offload

Support matching on IP protocol, Type of Service, Time To Live, source
and destination addresses, with masking if supported by the hardware.

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

sfc: add Layer 2 matches to ef100 TC offload

Support matching on EtherType, VLANs and ethernet source/destination
addresses, with masking if supported by the hardware.

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

sfc: check recirc_id match caps before MAE offload

Offloaded TC rules always match on recirc_id in the MAE, so we should
check that the MAE reported support for this match before attempting
to insert the rule.
These checks allow us to fail early, avoiding the PCIe round-trip to
firmware for an MC_CMD_MAE_ACTION_RULE_INSERT that will only fail,
and more importantly providing a more informative error message that
identifies which match field is unsupported.

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

sfc: use formatted extacks instead of efx_tc_err()

Since we can now get a formatted message back to the user with
NL_SET_ERR_MSG_FMT_MOD(), there's no need for our special logging.

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

sfc: bare bones TC offload on EF100

This is the absolute minimum viable TC implementation to get traffic to
VFs and allow them to be tested; it supports no match fields besides
ingress port, no actions besides mirred and drop, and no stats.
Example usage:
tc filter add dev $PF parent ffff: flower skip_sw \
action mirred egress mirror dev $VFREP
tc filter add dev $VFREP parent ffff: flower skip_sw \
action mirred egress redirect dev $PF
gives a VF unfiltered access to the network out the physical port ($PF
acts here as a physical port representor).
More matches, actions, and counters will be added in subsequent patches.

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sfc: interrogate MAE capabilities at probe time

Different versions of EF100 firmware and FPGA bitstreams support different
matching capabilities in the Match-Action Engine. Probe for these at
start of day; subsequent patches will validate TC offload requests
against the reported capabilities.

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sfc: use a dynamic m-port for representor RX and set it promisc

Representors do not want to be subject to the PF's Ethernet address
filters, since traffic from VFs will typically have a destination
either elsewhere on the link segment or on an overlay network.
So, create a dynamic m-port with promiscuous and all-multicast
filters, and set it as the egress port of representor default rules.
Since the m-port is an alias of the calling PF's own m-port, traffic
will still be delivered to the PF's RXQs, but it will be subject to
the VNRX filter rules installed on the dynamic m-port (specified by
the v-port ID field of the filter spec).

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

sfc: insert default MAE rules to connect VFs to representors

Default rules are low-priority switching rules which the hardware uses
in the absence of higher-priority rules. Each representor requires a
corresponding rule matching traffic from its representee VF and
delivering to the PF (where a check on INGRESS_MPORT in
__ef100_rx_packet() will direct it to the representor). No rule is
required in the reverse direction, because representor TX uses a TX
override descriptor to bypass the MAE and deliver directly to the VF.
Since inserting any rule into the MAE disables the firmware's own
default rules, also insert a pair of rules to connect the PF to the
physical network port and vice-versa.

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

sfc: determine wire m-port at EF100 PF probe time

Traffic delivered to the (MAE admin) PF could be from either the wire
or a VF. The INGRESS_MPORT field of the RX prefix distinguishes these;
base_mport is the value this field will have for traffic from the wire
(which should be delivered to the PF's netdevice, not a representor).

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

sfc: determine representee m-port for EF100 representors

An MAE port, or m-port, is a port (source/destination for traffic) on
the Match-Action Engine (the internal switch on EF100).
Representors will use their representee's m-port for two purposes: as
a destination override on TX from the representor, and as a source
match in 'default rules' to steer representee traffic (when not
matched by e.g. a TC flower rule) to representor RX via the parent
PF's receive queue.

Signed-off-by: Edward Cree <ecree.xilinx@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>