liquidio: convert tasklets to use new tasklet_setup() API

In preparation for unconditionally passing the
struct tasklet_struct pointer to all tasklet
callbacks, switch to using the new tasklet_setup()
and from_tasklet() to pass the tasklet pointer explicitly.

Signed-off-by: Romain Perier <romain.perier@gmail.com>
Signed-off-by: Allen Pais <apais@linux.microsoft.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

liquidio: Per queue oom work queue

Removed oom task unconditional rescheduling every 250ms and created per
queue oom work queue for refilling buffers.

The oom task refills only if the available descriptors is fallen to 64.
There will be no packets coming in after hitting this level. So NAPI will
not run until oom task refills the buffers.

Signed-off-by: Intiyaz Basha <intiyaz.basha@cavium.com>
Acked-by: Derek Chickles <derek.chickles@cavium.com>
Signed-off-by: Felix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

liquidio: remove obsolete functions and data structures

1. Remove unused functions and data structures.
2. Change the sending of the remaining soft commands to synchronous.

Signed-off-by: Weilin Chang <weilin.chang@cavium.com>
Signed-off-by: Felix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

liquidio: make soft command calls synchronous

1. Add wait_for_sc_completion_timeout() for waiting the response and
handling common response errors
2. Send sc's synchronously: remove unused callback function,
and context structure; use wait_for_sc_completion_timeout() to wait
its response.

Signed-off-by: Weilin Chang <weilin.chang@cavium.com>
Signed-off-by: Felix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

liquidio: xmit_more support

Defer ringing the Tx doorbell if skb->xmit_more is set unless the Tx queue
is full or stopped. To keep latency low, use a deferral limit of 8
packets. We chose 8 because Octeon can fetch at most 8 packets in a single
PCI read, and our tests show that 8 results in low latency.

Signed-off-by: Intiyaz Basha <intiyaz.basha@cavium.com>
Signed-off-by: Satanand Burla <satananda.burla@cavium.com>
Signed-off-by: Felix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

liquidio: moved liquidio_msix_intr_handler to lio_core.c

Moving common liquidio_msix_intr_handler to lio_core.c

Signed-off-by: Intiyaz Basha <intiyaz.basha@cavium.com>
Signed-off-by: Raghu Vatsavayi <raghu.vatsavayi@cavium.com>
Signed-off-by: Felix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sched/wait: Rename wait_queue_t => wait_queue_entry_t

Rename:

wait_queue_t => wait_queue_entry_t

'wait_queue_t' was always a slight misnomer: its name implies that it's a "queue",
but in reality it's a queue *entry*. The 'real' queue is the wait queue head,
which had to carry the name.

Start sorting this out by renaming it to 'wait_queue_entry_t'.

This also allows the real structure name 'struct __wait_queue' to
lose its double underscore and become 'struct wait_queue_entry',
which is the more canonical nomenclature for such data types.

Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>

liquidio: improve UDP TX performance

Improve UDP TX performance by:
* reducing the ring size from 2K to 512
* replacing the numerous streaming DMA allocations for info buffers and
gather lists with one large consistent DMA allocation per ring

BQL is not effective here. We reduced the ring size because there is heavy
overhead with dma_map_single every so often. With iommu=on, dma_map_single
in PF Tx data path was taking longer time (~700usec) for every ~250
packets. Debugged intel_iommu code, and found that PF driver is utilizing
too many static IO virtual address mapping entries (for gather list entries
and info buffers): about 100K entries for two PF's each using 8 rings.
Also, finding an empty entry (in rbtree of device domain's iova mapping in
kernel) during Tx path becomes a bottleneck every so often; the loop to
find the empty entry goes through over 40K iterations; this is too costly
and was the major overhead. Overhead is low when this loop quits quickly.

Netperf benchmark numbers before and after patch:

PF UDP TX
+--------+--------+------------+------------+---------+
| | | Before | After | |
| Number | | Patch | Patch | |
| of | Packet | Throughput | Throughput | Percent |
| Flows | Size | (Gbps) | (Gbps) | Change |
+--------+--------+------------+------------+---------+
| | 360 | 0.52 | 0.93 | +78.9 |
| 1 | 1024 | 1.62 | 2.84 | +75.3 |
| | 1518 | 2.44 | 4.21 | +72.5 |
+--------+--------+------------+------------+---------+
| | 360 | 0.45 | 1.59 | +253.3 |
| 4 | 1024 | 1.34 | 5.48 | +308.9 |
| | 1518 | 2.27 | 8.31 | +266.1 |
+--------+--------+------------+------------+---------+
| | 360 | 0.40 | 1.61 | +302.5 |
| 8 | 1024 | 1.64 | 4.24 | +158.5 |
| | 1518 | 2.87 | 6.52 | +127.2 |
+--------+--------+------------+------------+---------+

VF UDP TX
+--------+--------+------------+------------+---------+
| | | Before | After | |
| Number | | Patch | Patch | |
| of | Packet | Throughput | Throughput | Percent |
| Flows | Size | (Gbps) | (Gbps) | Change |
+--------+--------+------------+------------+---------+
| | 360 | 1.28 | 1.49 | +16.4 |
| 1 | 1024 | 4.44 | 4.39 | -1.1 |
| | 1518 | 6.08 | 6.51 | +7.1 |
+--------+--------+------------+------------+---------+
| | 360 | 2.35 | 2.35 | 0.0 |
| 4 | 1024 | 6.41 | 8.07 | +25.9 |
| | 1518 | 9.56 | 9.54 | -0.2 |
+--------+--------+------------+------------+---------+
| | 360 | 3.41 | 3.65 | +7.0 |
| 8 | 1024 | 9.35 | 9.34 | -0.1 |
| | 1518 | 9.56 | 9.57 | +0.1 |
+--------+--------+------------+------------+---------+

Signed-off-by: VSR Burru <veerasenareddy.burru@cavium.com>
Signed-off-by: Felix Manlunas <felix.manlunas@cavium.com>
Signed-off-by: Derek Chickles <derek.chickles@cavium.com>
Signed-off-by: Raghu Vatsavayi <raghu.vatsavayi@cavium.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

sched/headers: Prepare to move signal wakeup & sigpending methods from <linux/sched.h> into <linux/sched/signal.h>

Fix up affected files that include this signal functionality via sched.h.

Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>

liquidio CN23XX: copyrights changes and alignment

Updated copyrights comments and also changed some other comments
alignments.

Signed-off-by: Raghu Vatsavayi <raghu.vatsavayi@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Satanand Burla <satananda.burla@caviumnetworks.com>
Signed-off-by: Felix Manlunas <felix.manlunas@caviumnetworks.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

liquidio CN23XX: code cleanup

Cleaned up unnecessary comments and added some minor macros.

Signed-off-by: Raghu Vatsavayi <raghu.vatsavayi@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Satanand Burla <satananda.burla@caviumnetworks.com>
Signed-off-by: Felix Manlunas <felix.manlunas@caviumnetworks.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

liquidio CN23XX: device states

Cleaned up resource leaks during destroy resources by
introducing more device states.

Signed-off-by: Raghu Vatsavayi <raghu.vatsavayi@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Satanand Burla <satananda.burla@caviumnetworks.com>
Signed-off-by: Felix Manlunas <felix.manlunas@caviumnetworks.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

liquidio CN23XX: Mailbox support

Adds support for mailbox communication between PF and VF.

Signed-off-by: Raghu Vatsavayi <raghu.vatsavayi@caviumnetworks.com>
Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Satanand Burla <satananda.burla@caviumnetworks.com>
Signed-off-by: Felix Manlunas <felix.manlunas@caviumnetworks.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

liquidio: RX control commands

Adds support for RX control commands on cn23xx device.

Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Satanand Burla <satananda.burla@caviumnetworks.com>
Signed-off-by: Felix Manlunas <felix.manlunas@caviumnetworks.com>
Signed-off-by: Raghu Vatsavayi <raghu.vatsavayi@caviumnetworks.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

liquidio: Consolidate common functionality

Consolidate common functionality of various devices
from different files into lio_core.c/octeon_console.c.

Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Satanand Burla <satananda.burla@caviumnetworks.com>
Signed-off-by: Felix Manlunas <felix.manlunas@caviumnetworks.com>
Signed-off-by: Raghu Vatsavayi <raghu.vatsavayi@caviumnetworks.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

liquidio: Macro replacements

This patch has minor replacements of ACCESS_ONCE macros with
WRITE_ONCE and replacement of BUG_ON with polite version WARN_ON.

Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Satanand Burla <satananda.burla@caviumnetworks.com>
Signed-off-by: Felix Manlunas <felix.manlunas@caviumnetworks.com>
Signed-off-by: Raghu Vatsavayi <raghu.vatsavayi@caviumnetworks.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

liquidio:RX queue alloc changes

This patch is to allocate rx queue's memory based on numa node and also use
page based buffers for rx traffic improvements.

Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Satanand Burla <satananda.burla@caviumnetworks.com>
Signed-off-by: Felix Manlunas <felix.manlunas@caviumnetworks.com>
Signed-off-by: Raghu Vatsavayi <raghu.vatsavayi@caviumnetworks.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

Add support of Cavium Liquidio ethernet adapters

Following patch V8 adds support for Cavium Liquidio pci express
based 10Gig ethernet adapters.
1) Consolidated all debug macros to either call dev_* or
netdev_* macros directly, feedback from previous patch.
2) Changed soft commands to avoid crash when running
in interrupt context.
3) Fixed link status not reflecting correct status when NetworkManager
is running. Added MODULE_FIRMWARE declarations.

Following were the previous patches.
Patch V7:
1) Minor comments from v6 release regarding debug statements.
2) Fix for large multicast lists.
3) Fixed lockup issue if port initialization fails.
4) Enabled MSI by default.
https://patchwork.ozlabs.org/patch/464441/

Patch V6:
1) Addressed the uint64 vs u64 issue, feedback from previous patch.
2) Consolidated some receive processing routines.
3) Removed link status polling method.
https://patchwork.ozlabs.org/patch/459514/

Patch V5:
Based on the feedback from earlier patches with regards to
consolidation of common functions like device init, register
programming for cn66xx and cn68xx devices.
https://patchwork.ozlabs.org/patch/438979/

Patch V4:
Following were the changes based on the feedback from earlier patch:
1) Added mmiowb while synchronizing queue updates and other hw
interactions.
2) Statistics will now be incremented non-atomically per each ring.
liquidio_get_stats will add stats of each ring while reporting the
total statistics counts.
3) Modified liquidio_ioctl to return proper return codes.
4) Modified device naming to use standard Ethernet naming.
5) Global function names in the driver will have lio_/liquidio_/octeon_
prefix.
6) Ethtool related changes for:
Removed redundant stats and jiffies.
Use default ethtool handler of link status.
Speed setting will make use of ethtool_cmd_speed_set.
7) Added checks for pci_map_* return codes.
8) Check for signals while waiting in interruptible mode
https://patchwork.ozlabs.org/patch/435073/

Patch v3:
Implemented feedback from previous patch like:
Removed NAPI Config and DEBUG config options, added BQL and xmit_more
support.
https://patchwork.ozlabs.org/patch/422749/

Patch V2:
Implemented feedback from previous patch.
https://patchwork.ozlabs.org/patch/413539/

First Patch:
https://patchwork.ozlabs.org/patch/412946/

Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
Signed-off-by: Satanand Burla <satananda.burla@caviumnetworks.com>
Signed-off-by: Felix Manlunas <felix.manlunas@caviumnetworks.com>
Signed-off-by: Robert Richter <Robert.Richter@caviumnetworks.com>
Signed-off-by: Aleksey Makarov <Aleksey.Makarov@caviumnetworks.com>
Signed-off-by: Raghu Vatsavayi <raghu.vatsavayi@caviumnetworks.com>
Signed-off-by: David S. Miller <davem@davemloft.net>