#
bba2556e |
|
13-Apr-2021 |
Ong Boon Leong <boon.leong.ong@intel.com> |
net: stmmac: Enable RX via AF_XDP zero-copy This patch adds the support for receiving packet via AF_XDP zero-copy mechanism. XDP ZC uses 1:1 mapping of XDP buffer to receive packet, therefore the use of split header is not used currently. The 'xdp_buff' is declared as union together with a struct that contains 'page', 'addr' and 'page_offset' that are associated with primary buffer. RX buffers are now allocated either via page_pool or xsk pool. For RX buffers from xsk_pool they are allocated and deallocated using below functions: * stmmac_alloc_rx_buffers_zc(struct stmmac_priv *priv, u32 queue) * dma_free_rx_xskbufs(struct stmmac_priv *priv, u32 queue) With above functions now available, we then extend the following driver functions to support XDP ZC: * stmmac_reinit_rx_buffers() * __init_dma_rx_desc_rings() * init_dma_rx_desc_rings() * __free_dma_rx_desc_resources() Note: stmmac_alloc_rx_buffers_zc() may return -ENOMEM due to RX XDP buffer pool is not allocated (e.g. samples/bpf/xdpsock TX-only). But, it is still ok to let TX XDP ZC to continue, therefore, the -ENOMEM is silently ignored to let the driver succcessfully transition to XDP ZC mode for the said RX and TX queue. As XDP ZC buffer size is different, the DMA buffer size is required to be reprogrammed accordingly for RX DMA/Queue that is populated with XDP buffer from XSK pool. Next, to add or remove per-queue XSK pool, stmmac_xdp_setup_pool() will call stmmac_xdp_enable_pool() or stmmac_xdp_disable_pool() that in-turn coordinates the tearing down and setting up RX ring via RX buffers and descriptors removal and reallocation through stmmac_disable_rx_queue() and stmmac_enable_rx_queue(). In addition, stmmac_xsk_wakeup() is added to initiate XDP RX buffer replenishing by signalling user application to add available XDP frames back to FILL queue. For RX processing using XDP zero-copy buffer, stmmac_rx_zc() is introduced which is implemented with the assumption that RX split header is disabled. For XDP verdict is XDP_PASS, the XDP buffer is copied into a sk_buff allocated through stmmac_construct_skb_zc() and sent to Linux network GRO inside stmmac_dispatch_skb_zc(). Free RX buffers are then replenished using stmmac_rx_refill_zc() v2: introduce __stmmac_disable_all_queues() to contain the original code that does napi_disable() and then make stmmac_setup_tc_block_cb() to use it. Move synchronize_rcu() into stmmac_disable_all_queues() that eventually calls __stmmac_disable_all_queues(). Then, make both stmmac_release() and stmmac_suspend() to use stmmac_disable_all_queues(). Thanks David Miller for spotting the synchronize_rcu() issue in v1 patch. Signed-off-by: Ong Boon Leong <boon.leong.ong@intel.com> Signed-off-by: David S. Miller <davem@davemloft.net>
|
#
5fabb012 |
|
31-Mar-2021 |
Ong Boon Leong <boon.leong.ong@intel.com> |
net: stmmac: Add initial XDP support This patch adds the initial XDP support to stmmac driver. It supports XDP_PASS, XDP_DROP and XDP_ABORTED actions. Upcoming patches will add support for XDP_TX and XDP_REDIRECT. To support XDP headroom, this patch adds page_offset into RX buffer and change the dma_sync_single_for_device|cpu(). The DMA address used for RX operation are changed to take into page_offset too. As page_pool can handle dma_sync_single_for_device() on behalf of driver with PP_FLAG_DMA_SYNC_DEV flag, we skip doing that in stmmac driver. Current stmmac driver supports split header support (SPH) in RX but the flexibility of splitting header and payload at different position makes it very complex to be supported for XDP processing. In addition, jumbo frame is not supported in XDP to keep the initial codes simple. This patch has been tested with the sample app "xdp1" located in samples/bpf directory for both SKB and Native (XDP) mode. The burst traffic generated using pktgen_sample03_burst_single_flow.sh in samples/pktgen directory. Changes in v3: - factor in xdp header and tail adjustment done by XDP program. Thanks to Jakub Kicinski for pointing out the gap in v2. Changes in v2: - fix for "warning: variable 'len' set but not used" reported by lkp. Reported-by: kernel test robot <lkp@intel.com> Signed-off-by: Ong Boon Leong <boon.leong.ong@intel.com> Signed-off-by: David S. Miller <davem@davemloft.net>
|