s390/qdio: handle deferred cc1

A deferred condition code 1 response indicates that I/O was not started
and should be retried. The current QDIO implementation handles a cc1
response as I/O error, resulting in a failed QDIO setup. This can happen
for example when a path verification request arrives at the same time
as QDIO setup I/O is started.

Fix this by retrying the QDIO setup I/O when a cc1 response is received.

Note that since

commit 2297791c92d0 ("s390/cio: dont unregister subchannel from child-drivers")
commit 5ef1dc40ffa6 ("s390/cio: fix invalid -EBUSY on ccw_device_start")

deferred cc1 responses are much more likely to occur. See the commit
message of the latter for more background information.

Fixes: 2297791c92d0 ("s390/cio: dont unregister subchannel from child-drivers")
Reviewed-by: Alexandra Winter <wintera@linux.ibm.com>
Signed-off-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>

s390/cio: use new address translation helpers

Use virt_to_dma32() and friends to properly convert virtual to physical and
physical to virtual addresses so that "make C=1" does not generate any
warnings anymore.

Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

s390/qdio: remove unneeded sanity check in qdio_do_sqbs()

All callers of set_buf_states() are already making sure that 'count'
is not 0. So don't check it an additional time.

Note that our own code also doesn't _require_ the count to be sane
(ie. we can't overrun an array or similar). So worst case HW would
simply reject the SQBS operation and report an error.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

s390/qdio: split do_QDIO()

The callers know what type of queue they want to work with. Introduce
type-specific variants to add buffers on an {Input,Output} queue, so
that we can avoid some function parameters and the de-muxing into
type-specific hot paths.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

s390/qdio: split qdio_inspect_queue()

The callers know what type of queue they want to inspect. Introduce
type-specific variants to inspect an {Input,Output} queue, so that we
can avoid one function parameter and some conditional branches in the
hot paths.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

s390/qdio: clarify handler logic for qdio_handle_activate_check()

qdio_handle_activate_check() tries to re-use one of the queue-specific
handlers to report that the ACTIVATE ccw has been terminated. But the
logic to select that handler is overly complex - in practice both
qdio drivers have at least one Input Queue, so we never take the other
paths.

Make things more obvious by removing this unused code, and clearly
spelling out that we re-use the Input Handler for generic error
reporting. This also paves the way for a world without queue-specific
error handlers.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

s390/qdio: clean up access to queue in qdio_handle_activate_check()

qdio_handle_activate_check() re-uses a queue-specific handler to report
that the ACTIVATE ccw has been terminated. It uses either the first
input or output queue, so we can hard-code q->nr as 0. Also don't
access the q->irq_ptr parent pointer, we already have a pointer to
the qdio_irq.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

s390/qdio: avoid allocating the qdio_irq with GFP_DMA

The qdio_irq contains only two fields that are directly exposed to the
HW (ccw and qib). And only the ccw needs to reside in 31-bit memory. So
allocate it separately, and remove the GFP_DMA constraint from the
qdio_irq allocation.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

s390/qdio: improve handling of CIWs

Fetch the individual CIWs when we actually need them, rather than
fetching both of them in qdio_setup_irq() and then needing to cache
them inside the qdio_irq.

Also deal with the error when a CIW is not available, instead of
silently dropping this error condition in qdio_setup_irq()'s caller.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

s390/qdio: remove unused support for SLIB parameters

Neither of the two drivers provides any SLIB parameter data, so get rid
of the dead code.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: fine-tune the queue sync

Push the sync check from qdio_inspect_queue() down into the two
get_*_buffer_frontier() code paths, where we actually need the sync to
look at the current queue state. This lets us avoid the check when we
know that there is no work on the queue (ie. when q->nr_buf_used is 0).

While at it introduce the qdio_sync_*_queue() helpers, so that we can
avoid the branch on q->is_input_q when we already know the queue type.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: clean up SIGA capability tracking

Don't bother with translating the SIGA-related capability bits into
our own internal format, just cache the full qdioac1 field instead.

Also adjust the helper macros so that they take a qdio_irq argument
and can be used everywhere, instead of taking a qdio_q and then
internally dereferencing the parent pointer.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: use absolute data address in ESTABLISH ccw

Clean up yet another path where HW wants an absolute address, and we've
been implicitly relying on V=R.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: remove unneeded siga-sync for Output Queue

get_outbound_buffer_frontier() is only reached via qdio_inspect_queue(),
and there we already call qdio_siga_sync_q() unconditionally.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

s390/qdio: remove remaining tasklet & timer code

Both qdio drivers have moved away from using qdio's internal tasklet
and timer mechanisms for Output Queues. Rip out all the leftovers.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

s390/qdio: propagate error when cancelling a ccw fails

If qdio_cancel_ccw() times out (or is interrupted) before the interrupt
for the {halt,clear} action arrives, report this back to the caller as
an error.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

s390/qdio: improve roll-back after error on ESTABLISH ccw

If the ESTABLISH ccw fails (ie. the qdio_irq is set to
QDIO_IRQ_STATE_ERR), we don't need to call qdio_shutdown() for rolling
back our earlier actions. All the needed logic is already available in
qdio_establish()'s error chain, and using it means we don't have to
temporarily drop the setup_mutex either.

This makes qdio_shutdown() a purely external function, that should only
be called by the driver if an earlier qdio_establish() succeeded.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

s390/qdio: cancel the ESTABLISH ccw after timeout

When the ESTABLISH ccw does not complete within the specified timeout,
try our best to cancel the ccw program that is still active on the
device. Otherwise the IO subsystem might be accessing it even after
the driver eg. called qdio_free().

Fixes: 779e6e1c724d ("[S390] qdio: new qdio driver.")
Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Cc: <stable@vger.kernel.org> # 2.6.27
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

s390/qdio: fix roll-back after timeout on ESTABLISH ccw

When qdio_establish() times out while waiting for the ESTABLISH ccw to
complete, it calls qdio_shutdown() to roll back all of its previous
actions. But at this point the qdio_irq's state is still
QDIO_IRQ_STATE_INACTIVE, so qdio_shutdown() will exit immediately
without doing any actual work.

Which means that eg. the qdio_irq's thinint-indicator stays registered,
and cdev->handler isn't restored to its old value. And since
commit 954d6235be41 ("s390/qdio: make thinint registration symmetric")
the qdio_irq also stays on the tiq_list, so on the next qdio_establish()
we might get a helpful BUG from the list-debugging code:

...
[ 4633.512591] list_add double add: new=00000000005a4110, prev=00000001b357db78, next=00000000005a4110.
[ 4633.512621] ------------[ cut here ]------------
[ 4633.512623] kernel BUG at lib/list_debug.c:29!
...
[ 4633.512796] [<00000001b2c6ee9a>] __list_add_valid+0x82/0xa0
[ 4633.512798] ([<00000001b2c6ee96>] __list_add_valid+0x7e/0xa0)
[ 4633.512800] [<00000001b2fcecce>] qdio_establish_thinint+0x116/0x190
[ 4633.512805] [<00000001b2fcbe58>] qdio_establish+0x128/0x498
...

Fix this by extracting a goto-chain from the existing error exits in
qdio_establish(), and check the return value of the wait_event_...()
to detect the timeout condition.

Fixes: 779e6e1c724d ("[S390] qdio: new qdio driver.")
Root-caused-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Cc: <stable@vger.kernel.org> # 2.6.27
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

s390/qdio: get rid of register asm

Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: let driver manage the QAOB

We are spending way too much effort on qdio-internal bookkeeping for
QAOB management & caching, and it's still not robust. Once qdio's
TX path has detached the QAOB from a PENDING buffer, we lost all
track of it until it shows up in a CQ notification again. So if the
device is torn down before that notification arrives, we leak the QAOB.

Just have the driver take care of it, and simply pass down a QAOB if
they want a TX with async-completion capability. For a buffer in PENDING
state that requires the QAOB for final completion, qeth can now also try
to recycle the buffer's QAOB rather than unconditionally freeing it.

This also eliminates the qdio_outbuf_state array, which was only needed
to transfer the aob->user1 tag from the driver to the qdio layer.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Acked-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

s390/qdio: remove 'merge_pending' mechanism

For non-QEBSM devices, get_buf_states() merges PENDING and EMPTY buffers
into a single group of finished buffers. To allow the upper-layer driver
to differentiate between the two states, qdio_check_pending() looks at
each buffer's state again and sets the sbal_state flag to
QDIO_OUTBUF_STATE_FLAG_PENDING accordingly.

So effectively we're spending overhead on _every_ Output Queue
inspection, just to avoid some additional TX completion calls in case
a group of buffers has completed with mixed EMPTY / PENDING state.
Given that PENDING buffers should rarely occur, this is a bad trade-off.
In particular so as the additional checks in get_buf_states() affect
_all_ device types (even those that don't use the PENDING state).

Rip it all out, and just report the PENDING completions separately as
we already do for QEBSM devices.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: improve handling of PENDING buffers for QEBSM devices

For QEBSM devices the 'merge_pending' mechanism in get_buf_states()
doesn't apply, and we can actually get SLSB_P_OUTPUT_PENDING returned.

So for this case propagating the PENDING state to the driver via the
queue's sbal_state doesn't make sense and creates unnecessary overhead.
Instead introduce a new QDIO_ERROR_* flag that gets passed to the
driver, and triggers the same processing as if the buffers were flagged
as QDIO_OUTBUF_STATE_FLAG_PENDING.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: rework q->qdio_error indication

When inspecting a queue, any error is currently returned back through
the queue's qdio_error field. Turn this into a proper variable that gets
passed through the call chain, so that the lifetime is clear and the
error state can be accessed along the way.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: inline qdio_kick_handler()

We don't kick the handler for Input Queues anymore. Move the remaining
code into its only caller.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: track time of last data IRQ for each device

We currently track the time of the most recent QDIO Adapter Interrupt.
This is a system-wide timestamp (as such interrupts are not bound to
one specific qdio device).

If interrupt processing stalls on one device but is functional for a
different device, the timestamp continues to be updated and is of no
help for problem diagnosis.
So for debugging purposes also track the time of the last Data IRQ on
a per-device level. Collect this data in the legacy non-AI path as well.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: make thinint registration symmetric

tiqdio_add_device() adds the device to the tiq_list of eligible targets
for a data IRQ, which gets walked on each QDIO Adapter Interrupt to
inspect their DSCIs.

But currently the tiqdio_add_device() / tiqdio_remove_device() calls
are not symmetric - the device is removed within qdio_shutdown(),
but only added by qdio_activate().
So depending on the call sequence and encountered errors, we might
be trying to remove a list entry in qdio_shutdown() that was never even
added to the list. This required additional INIT_LIST_HEAD() calls to
ensure that the list entry was always in a consistent state.

All drivers now fence the IRQ delivery via qdio_start_irq() /
qdio_stop_irq(), so we can nicely integrate this tiq_list management
with the other steps needed for QDIO Adapter IRQ (de-)registration
(qdio_establish_thinint() / qdio_shutdown_thinint()).
As the naming suggests these get called during qdio_establish() and
qdio_shutdown(), with proper symmetry and roll-back after errors.

With this we longer need to worry about misplaced list removals, and
thus can clean up the list API abuse (INIT_LIST_HEAD() should not be
called on list entries).

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: adopt new tasklet API

Convert the Output Queue tasklet code to take a tasklet_struct as
parameter. Then initialize the tasklet with tasklet_setup() to indicate
that we follow the new model.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: remove qdio_inbound_q_moved() wrapper

It's used in just one place, inline it.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: remove Input tasklet code

Both qeth and zfcp have fully moved to the polling-driven flow for
Input Queues with commit 0a6e634535f1 ("s390/qdio: extend polling
support to multiple queues") and commit 0b524abc2dd1 ("scsi: zfcp: Lift
Input Queue tasklet from qdio").

So remove the tasklet code for Input Queues, streamline the IRQ handlers
and push the tasklet struct into struct qdio_output_q.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: always use dev_name() for device name in QIB

Passing a custom name from the device driver is nice - but in practice
it's only zfcp who has been using this. So we might as well hard-code
a naming scheme in the qdio layer, so that qeth also benefits from it.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Steffen Maier <maier@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: make qdio_handle_aobs() more robust

When processing a PENDING buffer with no attached aob, the current code
would get stuck on this buffer (as the 'continue' causes us to not
advance the buffer index) and process it repeatedly until the loop
terminates eventually.

Luckily this should never happen - the HW must not use the PENDING state
when no aob was provided. But we can still make this code path less
fragile and protect against buggy devices.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: remove internal polling in non-thinint path

For non-thinint devices in LPAR, qdio polls an idle Input Queue for a
little while to catch more work. But platform support for thinints has
been around practically _forever_ by now, so this micro-optimization is
seeing 0 actual use. Remove it to reduce the overall complexity of the
hot path.

In the meantime we also grew support for driver-level polling
(eg. NAPI in qeth), so it's quite questionable how useful this would
actually be on current kernels.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

s390/qdio: allow to scan all 128 Input SBALs

The comment is inaccurate, qdio_inbound_q_moved() and/or its callers no
longer get confused by a count of 128 completed SBALs.

Scanning all 128 SBALs at once can improve IRQ reduction (as we now
place the ACK at the right spot), and reduce the amount of processing
needed to handle all completed SBALs.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

s390/qdio: fix statistics for 128 SBALs

Old code would only scan up to 127 SBALs at once. So the last statistics
bucket was set aside to count "discovered 127 SBALs with new work"
events.

But nowadays we allow to scan all 128 SBALs for Output Queues, and a
subsequent patch will introduce the same for Input Queues.
So fix up the accounting to use the last bucket only when all 128 SBALs
have been discovered with new work.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>

s390/qdio: warn about unexpected SLSB states

The way we produce SBALs to the device (first update q->nr_buf_used,
then update the SLSB) should ensure that we never see some of the
SLSB states when scanning the queue for progress.
So make some noise if we do, this implies a bug in our SBAL tracking.

Also tweak the WARN msg to provide more information.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: clean up usage of qdio_data

This removes the last remaining accesses to ->qdio_data from internal
code. Just pass the qdio_irq struct where needed instead.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: reduce SLSB writes during Input Queue processing

Streamline the processing of QDIO Input Queues, and remove some
intermittent SLSB updates (no deleting of old ACKs, no redundant
transitions through NOT_INIT).

Rather than counting ACKs, we now keep track of the whole batch of
SBALs that were completed during the current polling cycle.
Most completed SBALs stay in their initial state (ie. PRIMED or ERROR),
except that the most recent SBAL in each sub-run is ACKed for
IRQ reduction.

The only logic changes happen in inbound_handle_work(), the other
delta is just a renaming of the variables that track the SBAL batch.

Note that in particular we don't need to flip the _oldest_ SBAL to
an idle state (eg. NOT_INIT or ACKed) as a guard against catching our
own tail. Since get_inbound_buffer_frontier() will never scan more than
the remaining nr_buf_used SBALs, this scenario just doesn't occur.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: fine-tune SLSB update

xchg() for a single-byte location assembles to a 4-byte Compare&Swap,
wrapped into a non-trivial amount of retry code that deals with
concurrent modifications to the unaffected bytes.

Change it to a simple byte-store, but preserve the memory ordering
semantics that the CS provided.
This simplifies the generated code for a hot path, and in theory also
allows us to amortize the memory barriers over multiple SLSB updates.

CC: Andreas Krebbel <krebbel@linux.ibm.com>
Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/cio, s390/qeth: cleanup PNSO CHSC

CHSC3D (PNSO - perform network subchannel operation) is used for
OC0 (Store-network-bridging-information) as well as for
OC3 (Store-network-address-information). So common fields are renamed
from *brinfo* to *pnso*.
Also *_bridge_host_* is changed into *_addr_change_*, e.g.
qeth_bridge_host_event to qeth_addr_change_event, for the
same reasons.
The keywords in the card traces are changed accordingly.

Remove unused L3 types, as PNSO will only return Layer2 entries.

Make PNSO CHSC implementation more consistent with existing API usage:
Add new function ccw_device_pnso() to drivers/s390/cio/device_ops.c and
the function declaration to arch/s390/include/asm/ccwdev.h, which takes
a struct ccw_device * as parameter instead of schid and calls
chsc_pnso().

PNSO CHSC has no strict relationship to qdio. So move the calling
function from qdio to qeth_l2 and move the necessary structures to a
new file arch/s390/include/asm/chsc.h.

Do response code evaluation only in chsc_error_from_response() and
use return code in all other places. qeth_anset_makerc() was meant to
evaluate the PNSO response code, but never did, because pnso_rc was
already non-zero.

Indentation was corrected in some places.

Signed-off-by: Alexandra Winter <wintera@linux.ibm.com>
Reviewed-by: Peter Oberparleiter <oberpar@linux.ibm.com>
Reviewed-by: Vineeth Vijayan <vneethv@linux.ibm.com>
Reviewed-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: remove q->first_to_kick

q->first_to_kick is obsolete, and can be replaced by q->first_to_check.

Both cursors start off at 0. Out of the three code paths that update
first_to_check, the qdio_inspect_queue() path is irrelevant as it
doesn't even touch first_to_kick anymore.
This leaves us with the two tasklet-driven code paths. Here any update
to first_to_check is followed by a call to qdio_kick_handler(), which
advances first_to_kick by the same amount.

So the two cursors will differ only for a tiny moment. Drivers have no
way of deterministically observing this difference, and thus it doesn't
matter which of the cursors we use for reporting an error to q->handler.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: fix up qdio_start_irq() kerneldoc

Document the actual semantics, correcting an old copy & paste mistake.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: add IRQ reduction for error SBALs

SBALs in PRIMED or ERROR state represent new work on the Input Queue.
But while inbound_primed() does all sorts of ACK management for new
PRIMED work, the same handling is currently missing for ERROR work.
In particular the path for ERROR work doesn't clear up _old_ ACKs.

Treat ERROR work the same as PRIMED work, but consider that the QEBSM
auto-ACK feature doesn't apply here. So we need to set the ACK manually,
as if it was a non-QEBSM device.

Note that this doesn't aspire to actually improve performance, the main
goal is to just unify the code paths and have consistent behaviour.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: refactor ACK processing for primed SBALs

inbound_primed() currently has two code paths - one for QEBSM that knows
how to deal with multiple ACKs, and a non-QEBSM path that strictly
assumes a single ACK on the queue.

In preparation for a subsequent patch, slightly adjust the non-QEBSM
path so that it can manage a queue with multiple ACKs.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: simplify overlap calculation on Input refill

Refilling the Input Queue requires additional checks, as the refilled
SBALs can overlap with the ACKs that qdio maintains on the queue.

This code path is way too complex, and does a whole bunch of wrap-around
checks that the modulo arithmetic in sub_buf() takes care of by itself.
So shrink down all that code into a few lines of equivalent
functionality.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: remove always-true condition

buf_in_between() gets passed q->u.in.ack_start as 'bufnr' parameter.

The ack_start always ranges between 0 and QDIO_MAX_BUFFERS_PER_Q - 1,
so the subsequent check will always return true. Remove it.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: de-duplicate tiqdio_inbound_processing()

Except for some initial thinint-only steps, the processing is identical
to the non-thinint case. So re-use the existing helper.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: keep track of allocated queue count

Knowing how many queues we initially allocated allows us to
1) sanity-check a subsequent qdio_establish() request, and
2) walk the queue arrays without further checks. Apply this while
cleanly splitting qdio_free_queues() into two separate helpers.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Steffen Maier <maier@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: roll-back after queue allocation error

When qdio_allocate_qs() fails, have it deal with its previous
allocations.
This way qdio_allocate() doesn't need to clean up afterwards.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Steffen Maier <maier@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: do more fine-grained allocation roll-back

Instead of having a catch-all qdio_release_memory() helper, free the
individual allocations from the respective error path.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Steffen Maier <maier@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: consolidate thinint init/exit

Wrap the init/exit steps for thinint into a single helper that follows
the established naming scheme.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Steffen Maier <maier@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: tear down thinint indicator after early error

qdio_establish() calls qdio_establish_thinint(), but later has an error
exit path that doesn't roll this call back. Fix it.

Fixes: 779e6e1c724d ("[S390] qdio: new qdio driver.")
Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: consistently restore the IRQ handler

For rolling back after an error, qdio_establish() calls qdio_shutdown().
If the error occurs early enough, then the qdio_irq's state still is
QDIO_IRQ_STATE_INACTIVE and qdio_shutdown() does nothing.

But at _any_ point where qdio_establish() bails out in this way,
qdio_setup_irq() will have already replaced the IRQ handler. This then
won't be restored after an early error, and the device can end up being
returned to the device driver with qdio's IRQ handler still installed.

Slightly reorder qdio_setup_irq() so we can be 100% sure that the IRQ
handler was replaced. Then fix the bug in qdio_establish() by calling a
helper that rolls back only the IRQ handler modification.

Also use the new helper in qdio_shutdown() to keep things in sync, and
slightly clean up the locking while doing so.
This makes minor semantical changes, but holding setup_mutex gives us
sufficient leeway to eg. pull qdio_shutdown_thinint() outside of the
ccwdev lock's scope.

Fixes: 779e6e1c724d ("[S390] qdio: new qdio driver.")
Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: clear DSCI early for polling drivers

Polling drivers in a configuration with 1 Input Queue currently keep
their DSCI armed all the way through the poll cycle, until
qdio_start_irq() clears it.

_Any_ intermittent QDIO interrupt delivered to tiqdio_thinint_handler()
will thus cause
1) the 'adapter_int' statistic to be incremented,
2) a call to tiqdio_call_inq_handlers() for this device, and then
3) the 'int_discarded' statistics to be incremented.

This causes overhead & complexity in the IRQ path, along with ambiguity
in the statistics.
On the other hand the device should be in IRQ avoidance mode during a
poll cycle, so there won't be a lot of DSCI ping-pong that this
micro-optimization could prevent.

So align the DSCI handling with what we already do for devices with
multiple Input Queues: clear it right away while processing the IRQ.

For the non-polling path this means that we no longer need to handle
the 1-queue case separately.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: remove cdev from init_data

It's no longer needed.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: cleanly split alloc and establish

All that qdio_allocate() actually uses from the init_data is the cdev,
and the number of Input and Output Queues. Have the driver pass those as
parameters, and defer the init_data processing into qdio_establish().
This includes writing per-device(!) trace entries, and most of the
sanity checks.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: set qdio_irq->cdev at allocation time

Set up qdio_irq->cdev right when the qdio_irq struct is allocated, so
that all subsequent code can rely on this pointer.

Then convert two helper functions to not pass a cdev parameter around.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: extend polling support to multiple queues

When the support for polling drivers was initially added, it only
considered Input Queue 0. But as QDIO interrupts are actually for the
full device and not a single queue, this doesn't really fit for
configurations where multiple Input Queues are used.

Rework the qdio code so that interrupts for a polling driver are not
split up into actions for each queue. Instead deliver the interrupt as
a single event, and let the driver decide which queue needs what action.

When re-enabling the QDIO interrupt via qdio_start_irq(), this means
that the qdio code needs to
(1) put _all_ eligible queues back into a state where they raise IRQs,
(2) and afterwards check _all_ eligible queues for new work to bridge
the race window.

On the qeth side of things (as the only qdio polling driver), we can now
add CQ polling support to the main NAPI poll routine. It doesn't consume
NAPI budget, and to avoid hogging the CPU we yield control after
completing one full queue worth of buffers.
The subsequent qdio_start_irq() will check for any additional work, and
have us re-schedule the NAPI instance accordingly.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Acked-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

s390/qdio: reduce access to cdev->private->qdio_data

Remove all usage of cdev->private->qdio_data that's buried deep in
internal code. This should only be used by the exported driver API,
which can then pass around a proper qdio_irq pointer.

Also trivially merge some initializations with their definitions.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: simplify ACK tracking

Current code uses a 'polling' flag to keep track of whether an Input
Queue has any ACKed SBALs. QEBSM devices might have multiple ACKed
SBALs, and those are tracked separately with 'ack_count'.

By also setting ack_count for non-QEBSM devices (to a fixed value of 1),
we can use 'ack_count != 0' as replacement for the polling flag.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: implement IQD Multi-Write

This allows IQD drivers to send out multiple SBALs with a single SIGA
instruction.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Alexandra Winter <wintera@linux.ibm.com>
Acked-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

s390/qdio: don't attempt IRQ avoidance on Output SBALs

Output interrupts are not subject to SLSB-based avoidance, so remove the
gratuitous SLSB updates for Output SBALs in ERROR state.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: simplify thinint device registration

On an interrupt, tiqdio_thinint_handler() walks a list of all objects
that might require attention, and checks their DSCI. This list is
awkwardly built from Input Queues, even though the IRQs are per-device
and the queue is then only used to dereference its qdio_irq parent.

To simplify the logic, change the code so that tiq_list contains
qdio_irq entries.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: add statistics helper macro

qperf_inc() takes a queue as input, but actually updates the statistics
in its qdio_irq parent.
In some contexts we already have access to the qdio_irq struct, and can
avoid the additional dereference.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: reduce log level for EQBS partial

Partial EQBS completion is no significant event, and the WARN ends up
spamming the debug logs for no good reason.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

s390/qdio: let drivers opt-out from Output Queue scanning

If a driver wants to use the new Output Queue poll code, then the qdio
layer must disable its internal Queue scanning. Let the driver select
this mode by passing a special scan_threshold of 0.

As the scan_threshold is the same for all Output Queues, also move it
into the main qdio_irq struct. This allows for fast opt-out checking, a
driver is expected to operate either _all_ or none of its Output Queues
in polling mode.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Acked-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

s390/qdio: enable drivers to poll for Output completions

While commit d36deae75011 ("qdio: extend API to allow polling") enhanced
the qdio layer so that drivers can poll their Input Queues, we don't
have the corresponding infrastructure for Output Queues yet.

Factor out a helper that scans a single QDIO Queue, so that qeth can
implement TX NAPI on top of it.
While doing so, remove the duplicated tracking of the next-to-scan index
(q->first_to_check vs q->first_to_kick) in this code path.

qdio_handle_aobs() needs to move slightly upwards in the code hierarchy,
so that it's still called from the polling path.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Acked-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

s390/qdio: restrict QAOB usage to IQD unicast queues

The IQD mcast queue doesn't support QAOB mode, so skip the
qdio_enable_async_operation() setup call for this queue. This avoids
the allocation of an unneeded QAOB pointer array, and sets up q->use_cq
properly so that drivers are prohibited from using QAOBs for mcast
traffic.

Take this opportunity to streamline the q->use_cq and aob != 0 checks.
The path to qdio_siga_output() is straight-forward, we don't need to
worry about being called with bad operands.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>

s390/qdio: add sanity checks to the fast-requeue path

If the device driver were to send out a full queue's worth of SBALs,
current code would end up discovering the last of those SBALs as PRIMED
and erroneously skip the SIGA-w. This immediately stalls the queue.

Add a check to not attempt fast-requeue in this case. While at it also
make sure that the state of the previous SBAL was successfully extracted
before inspecting it.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>

s390/qdio: handle PENDING state for QEBSM devices

When a CQ-enabled device uses QEBSM for SBAL state inspection,
get_buf_states() can return the PENDING state for an Output Queue.
get_outbound_buffer_frontier() isn't prepared for this, and any PENDING
buffer will permanently stall all further completion processing on this
Queue.

This isn't a concern for non-QEBSM devices, as get_buf_states() for such
devices will manually turn PENDING buffers into EMPTY ones.

Fixes: 104ea556ee7f ("qdio: support asynchronous delivery of storage blocks")
Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>

s390/qdio: optimize state inspection of HW-owned SBALs

When get_buf_states() gets called with count > 1, it scans the
corresponding number of SBAL states until it encounters a mismatch.

But when these SBALs are in a HW-owned state, the callers don't actually
care _how many_ such SBALs are on the queue. If we can't process the
first SBAL, we can't process any of the following SBALs either. So when
the first SBAL is HW-owned, skip the scan of the remaining SBALs and
thus save some CPU time.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: use get_buf_state() in debug_get_buf_state()

For a 1-SBAL state inspection, use the corresponding helper.
No functional change, just reducing the number of immediate callers to
get_buf_states().

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: allow to scan all Output SBALs in one go

Old code restricted the number of inspected SBALs to
QDIO_MAX_BUFFERS_PER_Q - 1, as otherwise the first_to_check and
first_to_kick cursors could overlap. Subsequent code would then assume that
there was no progress on the queue, when in fact _all_ SBALs on the queue
were ready-to-process.

This limitation no longer applies, so allow the queue-scan code to inspect
all SBALs on the queue. Note that qeth requires an additional patch
("s390/qeth: stop/wake TX queues based on their fill level"), to avoid
potential queue stalls when all 128 SBALs are reported as ready-to-process.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: consolidate index tracking for queue scan

qdio.ko offers a small number of high-level functions to drive the
scanning of a QDIO queue for ready-to-process SBALs:
qdio_get_next_buffers(), __[ti]qdio_inbound_processing() and
__qdio_outbound_processing().

Let each of those functions maintain the 'start' index for their current
scan, and pass it to lower-level helpers as needed. This improves the
code's overall layering, and allows us to eliminate the additional
first_to_kick cursor with a follow-on patch.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: limit direct access to first_to_check cursor

Refactor all the low-level helpers to take the first_to_check cursor as
parameter, rather than accessing it directly.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: eliminate queue's last_move cursor

This cursor is used for debugging only. But since
commit "s390/qdio: pass up count of ready-to-process SBALs" it effectively
duplicates the first_to_check cursor, diverging for just a short moment
when get_*_buffer_frontier() updates q->first_to_check.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: simplify SBAL range calculation

When passing a range of ready-to-process SBALs to the upper-layer
driver, use the available 'count' instead of calculating the distance
between the first_to_check and first_to_kick cursors.

This simplifies the logic of the queue-scan path, and opens up the
possibility of scanning all 128 SBALs in one go (as determining the
reported count no longer requires wrap-around safe arithmetic on the
queue's cursors).

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: pass up count of ready-to-process SBALs

When qdio_{in,out}bound_q_moved() scans a queue for pending work, it
currently only returns a boolean to its caller. The interface to the
upper-layer-drivers (qdio_kick_handler() and qdio_get_next_buffers())
then re-calculates the number of pending SBALs from the
q->first_to_check and q->first_to_kick cursors.

Refactor this so that whenever get_{in,out}bound_buffer_frontier()
adjusted the queue's first_to_check cursor, it also returns the
corresponding count of ready-to-process SBALs (and 0 else).
A subsequent patch will then make use of this additional information.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: clean up qdio_check_outbound_after_thinint()

This helper is not thinint-specific, qdio_get_next_buffers() also calls it
for non-thinint devices. So give it a more fitting name, and while at it
adjust its parameter.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: clean up pci_out_supported()

pci_out_supported() currently takes a single queue as parameter, even
though Output IRQ support is a per-device feature. Adjust the parameter,
so that the macro can also be used in code paths with no access to a queue
struct. This allows us to remove the remaining open-coded checks for
QIB_AC_OUTBOUND_PCI_SUPPORTED.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: clean up AOB handling

I've stumbled over this too many times now... AOBs are only ever used on
Output Queues. So in qdio_kick_handler(), move the call to their handler
into the Output-only path, and get rid of the convoluted contains_aobs()
helper. No functional change.

While at it, also remove
1. the unused sbal_state->aob field. For processing an async completion,
upper-layer drivers get their AOB pointer from the CQ buffer.
2. an unused EXPORT for qdio_allocate_aob(). External users would have
no way of passing an allocated AOB back into qdio.ko anyways...

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: reset old sbal_state flags

When allocating a new AOB fails, handle_outbound() is still capable of
transmitting the selected buffer (just without async completion).

But if a previous transfer on this queue slot used async completion, its
sbal_state flags field is still set to QDIO_OUTBUF_STATE_FLAG_PENDING.
So when the upper layer driver sees this stale flag, it expects an async
completion that never happens.

Fix this by unconditionally clearing the flags field.

Fixes: 104ea556ee7f ("qdio: support asynchronous delivery of storage blocks")
Cc: <stable@vger.kernel.org> #v3.2+
Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: clear intparm during shutdown

During shutdown, qdio returns its ccw device back to control by the
upper-layer driver. But there is a remote chance that by the time where the
IRQ handler gets switched back, the interrupt for the preceding
ccw_device_{clear,halt} hasn't been presented yet.
Upper-layer drivers would then need to handle this IRQ - and since the IO
is issued with an intparm, it could very well be confused with whatever
intparm mechanism the driver uses itself (eg intparm == request address).

So when switching over the IRQ handler, also clear the intparm and have
upper-layer drivers deal with any such delayed interrupt as if it was
unsolicited.

Suggested-by: Sebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: Julian Wiedmann <jwi@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: split up CCQ handling for EQBS / SQBS

Get rid of the confusing two-stage translation in a hot path, and only
handle CCQs that we anticipate for the respective command. Any
unexpected value (such as CCQ 97 (rc == 1) for SQBS) should be
considered a severe HW/driver bug, and traced as such.

Signed-off-by: Julian Wiedmann <jwi@linux.vnet.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: don't retry EQBS after CCQ 96

Immediate retry of EQBS after CCQ 96 means that we potentially misreport
the state of buffers inspected during the first EQBS call.

This occurs when
1. the first EQBS finds all inspected buffers still in the initial state
set by the driver (ie INPUT EMPTY or OUTPUT PRIMED),
2. the EQBS terminates early with CCQ 96, and
3. by the time that the second EQBS comes around, the state of those
previously inspected buffers has changed.

If the state reported by the second EQBS is 'driver-owned', all we know
is that the previous buffers are driver-owned now as well. But we can't
tell if they all have the same state. So for instance
- the second EQBS reports OUTPUT EMPTY, but any number of the previous
buffers could be OUTPUT ERROR by now,
- the second EQBS reports OUTPUT ERROR, but any number of the previous
buffers could be OUTPUT EMPTY by now.

Effectively, this can result in both over- and underreporting of errors.

If the state reported by the second EQBS is 'HW-owned', that doesn't
guarantee that the previous buffers have not been switched to
driver-owned in the mean time. So for instance
- the second EQBS reports INPUT EMPTY, but any number of the previous
buffers could be INPUT PRIMED (or INPUT ERROR) by now.

This would result in failure to process pending work on the queue. If
it's the final check before yielding initiative, this can cause
a (temporary) queue stall due to IRQ avoidance.

Fixes: 25f269f17316 ("[S390] qdio: EQBS retry after CCQ 96")
Cc: <stable@vger.kernel.org> #v3.2+
Signed-off-by: Julian Wiedmann <jwi@linux.vnet.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: restrict buffer merging to eligible devices

Only attempt to merge PENDING into EMPTY buffers for devices where
the PENDING state is actually expected (ie. IQD with CQ).
This might speed up the hot path a little bit.

Signed-off-by: Julian Wiedmann <jwi@linux.vnet.ibm.com>
Reviewed-by: Ursula Braun <ubraun@linux.vnet.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: don't merge ERROR output buffers

On an Output queue, both EMPTY and PENDING buffer states imply that the
buffer is ready for completion-processing by the upper-layer drivers.

So for a non-QEBSM Output queue, get_buf_states() merges mixed
batches of PENDING and EMPTY buffers into one large batch of EMPTY
buffers. The upper-layer driver (ie. qeth) later distuingishes PENDING
from EMPTY by inspecting the slsb_state for
QDIO_OUTBUF_STATE_FLAG_PENDING.

But the merge logic in get_buf_states() contains a bug that causes us to
erronously also merge ERROR buffers into such a batch of EMPTY buffers
(ERROR is 0xaf, EMPTY is 0xa1; so ERROR & EMPTY == EMPTY).
Effectively, most outbound ERROR buffers are currently discarded
silently and processed as if they had succeeded.

Note that this affects _all_ non-QEBSM device types, not just IQD with CQ.

Fix it by explicitly spelling out the exact conditions for merging.

For extracting the "get initial state" part out of the loop, this relies
on the fact that get_buf_states() is never called with a count of 0. The
QEBSM path already strictly requires this, and the two callers with
variable 'count' make sure of it.

Fixes: 104ea556ee7f ("qdio: support asynchronous delivery of storage blocks")
Cc: <stable@vger.kernel.org> #v3.2+
Signed-off-by: Julian Wiedmann <jwi@linux.vnet.ibm.com>
Reviewed-by: Ursula Braun <ubraun@linux.vnet.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: simplify math in get_*_buffer_frontier()

When determining the buffer count that get_buf_states() should
be queried for, 'count' is capped at 127 buffers.
So the check
q->first_to_check == (q->first_to_check + count) % 128
can be reduced to
count == 0

This helps to emphasize that get_buf_states() is really only
called with count > 0.

Signed-off-by: Julian Wiedmann <jwi@linux.vnet.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/cio: fix kernel-doc usage

Fix the kernel-doc usage in cio to get rid of (W=1) build warnings like:
drivers/s390/cio/cio.c:1068: warning: No description found for parameter 'sch'

Signed-off-by: Sebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: restrict target-full handling to IQDIO

The 'no target buffer empty' error code only applies to HiperSockets.
If this code is reported on a different queue type, be sure to make the
same amount of noise as for any other error code.

Signed-off-by: Julian Wiedmann <jwi@linux.vnet.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: consider ERROR buffers for inbound-full condition

In the unlikely case that an ERROR buffer (presented by the HW)
consumed the last available slot on the input queue, increment the
corresponding statistics counter.

Signed-off-by: Julian Wiedmann <jwi@linux.vnet.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390: cio: add SPDX identifiers to the remaining files

It's good to have SPDX identifiers in all files to make it easier to
audit the kernel tree for correct licenses.

Update the drivers/s390/cio/ files with the correct SPDX license
identifier based on the license text in the file itself. The SPDX
identifier is a legally binding shorthand, which can be used instead of
the full boiler plate text.

This work is based on a script and data from Thomas Gleixner, Philippe
Ombredanne, and Kate Stewart.

Cc: Sebastian Ott <sebott@linux.vnet.ibm.com>
Cc: Peter Oberparleiter <oberpar@linux.vnet.ibm.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Cornelia Huck <cohuck@redhat.com>
Cc: Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Kate Stewart <kstewart@linuxfoundation.org>
Cc: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390: qdio: Convert timers to use timer_setup()

In preparation for unconditionally passing the struct timer_list pointer to
all timer callbacks, switch to using the new timer_setup() and from_timer()
to pass the timer pointer explicitly.

Cc: Peter Oberparleiter <oberpar@linux.vnet.ibm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Kees Cook <keescook@chromium.org>
[sebott: fixed compile error due to invalid struct member]
Signed-off-by: Sebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>

s390/qdio: improve some debug prints

With multiple input queues, these DBFs turned out to be not
very helpful...

Signed-off-by: Julian Wiedmann <jwi@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: avoid reschedule of outbound tasklet once killed

During qdio_shutdown the queue tasklets are killed for all inbound and
outbound queues. The queue structures might be freed after
qdio_shutdown.
Thus it must be guaranteed that these queue tasklets are not rescheduled
after that. In addition the outbound queue timers are deleted and it
must
be guaranteed that these timers are not restarted after qdio_shutdown
processing. Timer deletion should make use of del_timer_sync() to make
sure qdio_outbound_timer() is finished on other CPUs as well. Queue
tasklets should be scheduled in state QDIO_IRQ_STATE_ACTIVE only.

Signed-off-by: Ursula Braun <ubraun@linux.vnet.ibm.com>
Reviewed-by: Benjamin Block <bblock@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: remove checks for ccw device internal state

Prior to starting IO qdio checks for the internal state of the ccw
device. These checks happen without locking, so consistency between
state evaluation and starting of the IO is not guaranteed.

Since the internal state is checked during ccw_device_start it is
safe to get rid of these additional checks.

Signed-off-by: Sebastian Ott <sebott@linux.vnet.ibm.com>
Reviewed-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: fix double return code evaluation

qdio sometimes checks return codes twice. First with the ccw device's
lock held and then a 2nd time after the lock is released. Simplify
the code by releasing the lock earlier and unify the return code
evaluation.

Signed-off-by: Sebastian Ott <sebott@linux.vnet.ibm.com>
Reviewed-by: Ursula Braun <ubraun@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: get rid of spin_lock_irqsave usage

All qdio functions that use spin_lock_irqsave are never used
from irq context. Thus it is safe to convert all of them to
use spin_lock_irq.

Signed-off-by: Sebastian Ott <sebott@linux.vnet.ibm.com>
Reviewed-by: Ursula Braun <ubraun@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: obtain subchannel_id via ccw_device_get_schid()

We want to get rid of the copy of struct subchannel_id maintained in
ccw_device_private, so obtain it using ccw_device_get_schid().

Signed-off-by: Sebastian Ott <sebott@linux.vnet.ibm.com>
Reviewed-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Reviewed-by: Ursula Braun <ubraun@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: fix WARN_ON_ONCE condition

If HiperSockets Completion Queueing is enabled, qdio always
issues a warning, since the condition is always met.
This patch fixes the condition in WARN_ON_ONCE that was always
true.

Signed-off-by: Eugene Crosser <Eugene.Crosser@ru.ibm.com>
Signed-off-by: Ursula Braun <ursula.braun@de.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

qdio: Keep device-specific dbf entries

Keep the per-device dbf entries until module is removed, with
proper error checking for debug feature setup.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Reviewed-by: Steffen Maier <maier@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: replace shift loop by ilog2

account_sbals is called by get_inbound_buffer_frontier and
get_outbound_buffer_frontier with 'count' value > 0 so we can safely
convert shift loop to ilog2.

Cc: Joe Perches <joe@perches.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Fabian Frederick <fabf@skynet.be>
Signed-off-by: Sebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: correct program-controlled interruption checking

Get rid of compile warning in qdio_int_handler_pci() when checking
for program-controlled interruption on outbound queues.

Signed-off-by: Ursula Braun <ursula.braun@de.ibm.com>
Reviewed-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: bridgeport support - CHSC part

Introduce function for the "Perform network-subchannel operation"
CHSC command with operation code "bridgeport information",
and bit definitions for "characteristics" pertaning to this command.

Signed-off-by: Eugene Crosser <eugene.crosser@ru.ibm.com>
Signed-off-by: Frank Blaschka <frank.blaschka@de.ibm.com>
Reviewed-by: Sebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

s390/qdio: bridgeport support - CHSC part

Introduce function for the "Perform network-subchannel operation"
CHSC command with operation code "bridgeport information",
and bit definitions for "characteristics" pertaning to this command.

Signed-off-by: Eugene Crosser <eugene.crosser@ru.ibm.com>
Reviewed-by: Sebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: fix atomic_sub() misusage

get_inbound_buffer_frontier() makes use of the return value of atomic_sub()
which shouldn't work, since atomic_sub() is supposed to return void.
This only works on s390 because atomic_sub() gets mapped to atomic_sub_return()
with a define without changing it's return value to void.

So use atomic_sub_return() instead of atomic_sub() in qeth code before fixing
atomic ops.

Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/time: correct use of store clock fast

The result of the store-clock-fast (STCKF) instruction is a bit fuzzy.
It can happen that the value stored on one CPU is smaller than the value
stored on another CPU, although the order of the stores is the other
way around. This can cause deltas of get_tod_clock() values to become
negative when they should not be.

We need to be more careful with store-clock-fast, this patch partially
reverts git commit e4b7b4238e666682555461fa52eecd74652f36bb "time:
always use stckf instead of stck if available". The get_tod_clock()
function now uses the store-clock-extended (STCKE) instruction.
get_tod_clock_fast() can be used if the fuzziness of store-clock-fast
is acceptable e.g. for wait loops local to a CPU.

Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: remove unused variable

Fix a "set but not used" warning found via make W=1.

Signed-off-by: Sebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: remove unused function

Remove the unused function qdio_trace_aob.

Acked-by: Ursula Braun <ursula.braun@de.ibm.com>
Signed-off-by: Sebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

qdio: remove unused parameters

Remove unused function parameters.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/time: rename tod clock access functions

Fix name clash with some common code device drivers and add "tod"
to all tod clock access function names.

Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: rework BUG's and WARN_ON's

Remove or replace BUG/BUG_ON where possible and convert WARN_ON
to WARN_ON_ONCE if they can occur freqeuently as pointed out by:
https://lkml.org/lkml/2012/9/27/461

Checks have been removed if:
- the error condition leads to a hardware error which gets logged
and in most cases stops the device
- the error condition is a null pointer access
- the error condition is just pointless or already handled at
another location

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/comments: unify copyright messages and remove file names

Remove the file name from the comment at top of many files. In most
cases the file name was wrong anyway, so it's rather pointless.

Also unify the IBM copyright statement. We did have a lot of sightly
different statements and wanted to change them one after another
whenever a file gets touched. However that never happened. Instead
people start to take the old/"wrong" statements to use as a template
for new files.
So unify all of them in one go.

Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>

s390/time: always use stckf instead of stck if available

The store clock fast instruction saves a couple of instructions compared
to the store clock instruction. Always use stckf instead of stck if it
is available.

Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Reviewed-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

s390/qdio: Cleanup error handling to drivers

Various improvements of qdio error reporting to the
upper-layer drivers (qeth, zfcp):

- Split QDIO_ERROR_ACTIVATE_CHECK_CONDITION into:

QDIO_ERROR_ACTIVATE: qdio termination interrupt
QDIO_ERROR_GET_BUF_STATE: QIOASSIST eqbs error
QDIO_ERROR_SET_BUF_STATE: QIOASSIST sqbs error

Add QDIO_ERROR_FATAL / QDIO_ERROR_TEMPORARY masks
to ease recovery decision in upper-layer drivers.

- Don't (ab-)use qdio handler errors as return codes
for do_QDIO but use standard error codes:

-ENOBUFS: temporary target CC=2 condition
-EBUSY: unresolved SIGA-W CC=2 busy condition
-EIO: I/O error (CC=1, CC=3)

- Remove unneeded memory clobber from SIGA-R
- Remove EX_TABLE entry on SIGA-W, we want to see these errors

Reviewed-by: Ursula Braun <ursula.braun@de.ibm.com>
Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: log all adapter characteristics

Log all adapter characteristics in the s390dbf to ease debugging of
new features.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Reported-by: Raymond Higgs <rayhiggs@us.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] kernel: Add z/VM LGR detection

Currently the following mechanisms are available to move active
Linux on System z instances between machines:
* z/VM 6.2 SSI (Single System Image)
* Suspend/resume
For moving Linux instances in this patch the term LGR (Linux Guest
Relocation) is used. Because such an operation is critical, it
should be detectable from Linux. With this patch for both, a live
system and a kernel dump, the information about LGRs is accessible.
To identify a guest, stsi and stfle data is used. A new function
lgr_info_log() compares the current data (lgr_info_cur) with the
last recorded one (lgr_info_last). In case the two data sets differ,
lgr_info_cur is logged to the "lgr" s390dbf.

The following trigger points call lgr_info_log():
* panic
* die
* kdump
* LGR timer
* PSW restart
* QDIO recovery
* resume

This patch also changes the s390dbf hex_ascii view. Now only printable ASCII
characters are shown.

Reviewed-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Michael Holzheu <holzheu@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: fix handler function arguments for zfcp data router

Git commit 25f269f17316549e "[S390] qdio: EQBS retry after CCQ 96"
introduced a regression in regard to the zfcp data router.
Revoke the incorrect simplification of the function call arguments
for the qdio handler to make the zfcp hardware data router working
again.

This is applicable to 3.2+ kernels.

Signed-off-by: Steffen Maier <maier@linux.vnet.ibm.com>
Reviewed-by: Jan Glauber <jang@linux.vnet.ibm.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: prevent dsci access without adapter interrupts

A kernel panic may occur during sending or receiving network packets
on a machine without adapter interrupts since commit d36deae.
The bug is triggered by writing to the shared indicator address which
is set to 0 if the machine doesn't have adapter interrupts.

Make the reading and setting of the shared indicator dependent on the
adapter interrupt feature and while at it move the code to the
file containing the adapter interrupt related code.

Thanks to Jan Jaeger for tracking this down.

Reported-by: Jan Jaeger <jan.jaeger@westnet.com.au>
Tested-by: Jan Jaeger <jan.jaeger@westnet.com.au>
Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] sparse: fix sparse static warnings

Make functions and data static to avoid sparse warnings.

Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: remove multicast polling

The multicast poll check for the outbound queue is redundant since
3d6c76f "[S390] qdio: outbound tasklet scan threshold". Remove the check.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: reset outbound SBAL error states

Don't leave outbound SBALs in error state after a target full condition.
Reset the state to not initialized to make the error handling consistent
across all types of errors.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: EQBS retry after CCQ 96

Running under z/VM with QIOASSIST enabled, qdio queues could stall if EQBS
did not extract all SBAL states. Add an instant retry for EQBS and, if the
retry fails, set up a timer to ensure outstanding SBALs are processed later.

While at it, optimize qdio_do_eqbs and qdio_do_sqbs to eliminate 3 jumps on
the hot path.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: add timestamp for last queue scan time

Add a timestamp per queue and update the timestamp when the queue is
scanned. Add the queue timestamps and the timestamp of the last
adapter interrupt to the debugfs output. The timestamps are useful
for debugging stall conditions.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] fix mismatch in summation of I/O IRQ statistics

Current IRQ statistics support does not show detail counts for I/O
interrupts which are processed internally only. The result is a
summation count which is way off such as this one:

CPU0 CPU1 CPU2
I/O: 1331 710 442
[...]
QAI: 15 16 16 [I/O] QDIO Adapter Interrupt
QDI: 1 0 0 [I/O] QDIO Interrupt
DAS: 706 645 381 [I/O] DASD
C15: 26 10 0 [I/O] 3215
C70: 0 0 0 [I/O] 3270
TAP: 0 0 0 [I/O] Tape
VMR: 0 0 0 [I/O] Unit Record Devices
LCS: 0 0 0 [I/O] LCS
CLW: 0 0 0 [I/O] CLAW
CTC: 0 0 0 [I/O] CTC
APB: 0 0 0 [I/O] AP Bus

Fix this by moving I/O interrupt accounting into the common I/O layer.

Signed-off-by: Peter Oberparleiter <peter.oberparleiter@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[SCSI] qdio: base support for hardware data router with zfcp

FICON Express8S supports hardware data router, which requires an
adapted qdio request format.
This part 1/2 provides the qdio base required for exploitation in
zfcp.

Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Steffen Maier <maier@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>

qdio: support forced signal adapter indications

This patch ensures that signal adapter commands are issued if they are
indicated to be required.

Signed-off-by: Einar Lueck <elelueck@de.ibm.com>
Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Frank Blaschka <frank.blaschka@de.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

qdio: support asynchronous delivery of storage blocks

This patch introduces support for asynchronous delivery of storage blocks for
Hipersockets. Upper layers may exploit this functionality to reuse SBALs for
which the delivery status is still pending.

Signed-off-by: Einar Lueck <elelueck@de.ibm.com>
Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Frank Blaschka <frank.blaschka@de.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

[S390] qdio: 2nd stage retry on SIGA-W busy conditions

The SIGA-W may return with the busy bit set which means the device was
blocked. The busy loop which retries the SIGA-W for 100us may not be
long enough when running under a heavily loaded hypervisor.

Extend the retry mechanism by adding a longer second stage which retries
the SIGA-W for up to 10s. In difference to the first retry loop the second
stage is using mdelay to stop the cpu between the retries and thereby
avoid additional preassure in on the hypervisor.
If the second stage retry is successfull a device reset is avoided.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>

atomic: use <linux/atomic.h>

This allows us to move duplicated code in <asm/atomic.h>
(atomic_inc_not_zero() for now) to <linux/atomic.h>

Signed-off-by: Arun Sharma <asharma@fb.com>
Reviewed-by: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: David Miller <davem@davemloft.net>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Acked-by: Mike Frysinger <vapier@gentoo.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

[S390] qdio: Split SBAL entry flags

The qdio SBAL entry flag is made-up of four different values that are
independent of one another. Some of the bits are reserved by the
hardware and should not be changed by qdio. Currently all four values
are overwritten since the SBAL entry flag is defined as an u32.

Split the SBAL entry flag into four u8's as defined by the hardware
and don't touch the reserved bits.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: prevent compile warning

Prevent the following compile warning for !CONFIG_64BIT:

CC drivers/s390/cio/qdio_main.o
drivers/s390/cio/qdio_main.c: In function ‘handle_outbound’:
drivers/s390/cio/qdio_main.c:1449: warning: ‘state’ may be used uninitialized in this function

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: reset error states immediately

The qdio hardware may surpress further interrupts as long as a SBAL is in
the error state. That can lead to unnotified data in the SBALs following
the error state. To prevent this behaviour change the SBAL[s] in error
state immediately to another program owned state so interrupts are again
received for further traffic on the device.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: fix init sequence

Reorder the initialization sequence of the qdio module to avoid
writing to an uninitialized debug feature entry. Also reorder
the exit function to restore a consistent cleanup path.

Signed-off-by: Sebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: prevent handling of buffers if count is zero

Exit do_QDIO early if the buffer count is zero to prevent side effects
in the following functions.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: prevent compile warning under CONFIG_32BIT

Prevent the following compiler warning if compiling a 31 bit kernel:

drivers/s390/cio/qdio_main.c: In function ‘get_outbound_buffer_frontier’:
drivers/s390/cio/qdio_main.c:646:16: warning: ‘state’ may be used uninitialized in this function
CC lib/radix-tree.o
CC drivers/s390/scsi/zfcp_cfdc.o
drivers/s390/cio/qdio_main.c: In function ‘qdio_inbound_q_moved’:
drivers/s390/cio/qdio_main.c:479:16: warning: ‘state’ may be used uninitialized in this function
drivers/s390/cio/qdio_main.c:479:16: note: ‘state’ was declared here

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: cleanup SIGA sync

Simplify the SIGA sync code and add unlikely annotations. In polling mode
SBALs may be accessed without interrupt, so call SIGA sync before every scan.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: remove enhanced SIGA

HiperSocket devices only use one SBAL per qdio call without the enhanced SIGA
feature. Since that feature is currently not used remove it from the qdio code
so the compiler can generate better code for the HiperSocket outbound path.
While at it mark the SIGA error conditions as unlikely.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: use proper QEBSM operand for SIGA-R and SIGA-S

If QIOASSIST is enabled for a qdio device the SIGA instruction requires
a modified function code. This function code modifier was missing for
SIGA-R and SIGA-S which can lead to a kernel panic caused by an
operand exception.

Cc: stable@kernel.org
Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: outbound queue full counter

Add a counter for outbound queue full events to the qdio statistics.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: outbound tasklet scan threshold

Introduce a scan treshold for the qdio outbound queues. By setting the
threshold the driver can tell qdio after how much used SBALs qdio
should schedule the outbound tasklet that scans the queue for finished
SBALs. The threshold is specific by the drivers because a
Hipersockets device is much faster in utilizing outbound buffers than a
ZFCP or OSA device.

The default values after how many used SBALs the tasklet should run are:

OSA: > 31 SBALs
Hipersockets: > 7 SBALs
zfcp: > 55 SBALs

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: prevent race for shared indicators

If the shared indicator is used the following race leads to
an inbound stall:

Device CPU0 CPU1
========================================================

non-shared DSCI =>1
ALSI => 1
Thin INT
ALSI => 0

non-shared DSCI
tasklets scheduled

shared DSCI => 1
ALSI => 1

shared DSCI => 0
ALSI ? -> set
Thin INT
ALSI => 0
ALSI was set,
shared DSCI => 1

After that no more interrupts occur because the DSCI is still set.
Fix that race by only resetting the shared DSCI if it was actually
set so the tasklets for all shared devices are scheduled and will
run after the interrupt.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: add qdio interrupts to interrupt statistics

Count traditional qdio interrupts and adapter interrupts for qdio
in the interrupt statistics.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

qdio: extend API to allow polling

Extend the qdio API to allow polling in the upper-layer driver. This
is needed by qeth to use NAPI.

To use the new interface the upper-layer driver must specify the
queue_start_poll(). This callback is used to signal the upper-layer
driver that is has initiative and must process the inbound queue by
calling qdio_get_next_buffers(). If the upper-layer driver wants to
stop polling it calls qdio_start_irq().

Since adapter interrupts are not completely stoppable qdio implements
a software bit QDIO_QUEUE_IRQS_DISABLED to safely disable interrupts for an
input queue.

The old interface is preserved and will be used as is by zfcp.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Signed-off-by: Frank Blaschka <frank.blaschka@de.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

[S390] qdio: remove API wrappers

Remove qdio API wrappers used by qeth and replace them by calling the
appropriate functions directly.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: dont convert timestamps to microseconds

Don't convert timestamps to microseconds, use timestamps returned by
get_clock() directly.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: prevent starvation on PCI devices

If adapter interrupts are not available and traditional IO interrupts
are used for qdio the inbound tasklet continued to run if new data
arrived. That could possibly block other tasklets scheduled on the
same CPU. If new data arrives schedule the tasklet again instead of
directly processing the new data.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: count number of qdio interrupts

Add missing increment for the qdio interrupt counter.

Signed-off-by: Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h

percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.

percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.

http://userweb.kernel.org/~tj/misc/slabh-sweep.py

The script does the followings.

* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.

* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.

* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.

The conversion was done in the following steps.

1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.

2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.

3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.

4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.

5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.

6. percpu.h was updated not to include slab.h.

7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).

* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig

8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.

Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.

Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>

[S390] qdio: add missing bracket

Add a missing bracket to only log the outbound handler event in the
appropriate case.

Signed-off-by: Ursula Braun <ursula.braun@de.ibm.com>
Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] add MACHINE_IS_LPAR flag

Introduce the MACHINE_IS_LPAR flag for code that should only be
executed if Linux is running in an LPAR.

Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: account processed SBAL during queue scan

Add counters for the number of processed SBALs. The numbers summarize
how many SBALs were processed at each queue scan and indicate the
utilization of the queue. Furthermore the number of unsuccessfull
queue scans, SBAL errors and the total number of processed
SBALs are accounted.

Also regroup struct qdio_q to move read-mostly and write-mostly data
into different cachelines.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: prevent call trace if CHPID is offline

If a CHPID is offline during a device shutdown the ccw_device_halt|clear
may fail and the qdio device stays in state STOPPED until the shutdown is
finished. If an interrupt occurs before the device is set to INACTIVE
the STOPPED state triggers a WARN_ON in the interrupt handler.
Prevent this WARN_ON by catching the STOPPED state in the interrupt
handler.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: continue polling for buffer state ERROR

Inbound traffic handling may hang if next buffer to check is in
state ERROR, polling is stopped and the final check for further
available inbound buffers disregards buffers in state ERROR.
This patch includes state ERROR when checking availability of
more inbound buffers.

Cc: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Ursula Braun <ursula.braun@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: convert global statistics to per-device stats

Revamp the qdio performance statistics and move them from procfs to
debugfs using the seq_file interface. Since the statistics are not
intended for the general user the removal of /proc/qdio_perf should
not surprise anyone.

The per device statistics are disabled by default, writing 1 to
/<debugfs mountpoint>/qdio/<device bus ID>/statistics enables the
statistics for the given device.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: add counter for input queue full condition

Add a counter to the qdio performance statistics that indicates that no
free buffers were left in the input queue. If the counter gets increased
it means that the qdio adapter filled all available buffers and possibly
had more buffers ready but could not transmit them.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: change state of all primed input buffers

If input buffers stay in primed state qdio may not receive further interrupts
for the input queue depending on the firmware. That can cause a connection
hang on OSA cards.

Change the state of all primed input buffers that are not acknowledged to
not initialized.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: reduce per device debug messages

Even if turned off the debug message overhead is measurable in the hot path.
Reduce the number of debug message calls in do_QDIO and qdio_kick_handler.
Also use hex numbers to save space in the debug entries.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: continue polling if the queue is not finished

With commit c38f96080955854e54df9cb392bc674e1ae330e1 polling was
stopped for the queue even if new data is available.

Return immediately after scheduling the queue tasklet if the queue
is not done.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: Sanitize do_QDIO sanity checks

Remove unneeded sanity checks from do_QDIO since this is the hot path.
Change the type of bufnr and count to unsigned int so the check for the
maximum value works.

Reported-by: Roel Kluin <roel.kluin@gmail.com>
Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: leave inbound SBALs primed

It is not required to change the state of primed SBALs. Leaving them
primed saves a SQBS instruction under z/VM.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: extract all primed SBALs at once

For devices without QIOASSIST primed SBALS were extracted in a loop.
Remove the loop since get_buf_states can already return more than
one primed SBAL.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: fix check for running under z/VM

The check whether qdio runs under z/VM was incorrect since SIGA-Sync is not
set if the device runs with QIOASSIST. Use MACHINE_IS_VM instead to prevent
polling under z/VM.

Merge qdio_inbound_q_done and tiqdio_is_inbound_q_done.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: move adapter interrupt tasklet code

Move the adapter interrupt tasklet function to the qdio main code
since all the functions used by the tasklet are located there.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: fix access beyond ARRAY_SIZE of irq_ptr->{in,out}put_qs

Do not go beyond ARRAY_SIZE of irq_ptr->{in,out}put_qs

Signed-off-by: Roel Kluin <roel.kluin@gmail.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: simplify error handling in irq handler

The check for the device status in qdio_establish_handle_irq()
had dead code. Remove the unused code and simplify the error
handling.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: remove dead timeout handler

The QDIO ccw devices are started by ccw_device_start so no timeout
can occur for the interrupt handler. Remove the dead code.

In case of an I/O error set the device state to error and wake up
a possibly running qdio_shutdown waiter.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: merge inbound and outbound handler functions

The inbound and outbound handlers are nearly identical if the outbound
handler uses first_to_check as end index instead of last_move. Since both
values are identical at that point the handlers can be merged.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: report SIGA errors directly

Errors from SIGA instructions are stored in the per queue qdio_error
and reported back when the queue handler is called. That opens a race
when multiple error conditions occur simultanously.

Report SIGA errors immediately in the return value of do_QDIO so the
upper layer can react and SIGA errors no longer interfere with other
errors.

Move the SIGA error handling in qeth from the outbound handler to
qeth_flush_buffers.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>

[S390] qdio: seperate last move index and polling index

The index value that indicated that the input queue moved was also used to
store the index of the first acknowledged buffer. For non-qebsm only the
newest buffer is acknowledged which may be different from the last move index
so two seperate values are needed to track the input queue.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: move ACK to newest buffer for devices without QEBSM

The ACKnowledgement state should be set on the newest SBAL so an
adapter interrupt surpression check needs to scan fewer SBALs.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: call qdio_free also if qdio_shutdown fails

qdio_cleanup is a wrapper function that should call qdio_shutdown and
qdio_free. qdio_free was not called if an error occured in qdio_shutdown
resulting in a missing free of allocated resources.

Call qdio_free regardless of the return value of qdio_shutdown.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: proper kill of qdio tasklets

The queue tasklets were stopped with tasklet_disable. Although tasklet_disable
prevents the tasklet from beeing executed it is still possible that a tasklet
is scheduled on a CPU at that point. A following qdio_establish calls
tasklet_init which clears the tasklet count and the tasklet state leading to
the following Oops:

<2>kernel BUG at kernel/softirq.c:392!
<4>illegal operation: 0001 [#1] SMP
<4>Modules linked in: iptable_filter ip_tables x_tables dm_round_robin dm_multipath scsi_dh sg sd_mod crc_t10dif nfs lockd nfs
_acl sunrpc fuse loop dm_mod qeth_l3 ipv6 zfcp qeth scsi_transport_fc qdio scsi_tgt scsi_mod chsc_sch ccwgroup dasd_eckd_mod dasdm
od ext3 mbcache jbd
<4>Supported: Yes
<4>CPU: 0 Not tainted 2.6.27.13-1.1.mz13-default #1
<4>Process blast.LzS_64 (pid: 16445, task: 000000006cc02538, ksp: 000000006cb67998)
<4>Krnl PSW : 0704c00180000000 00000000001399f4 (tasklet_action+0xc8/0x1d4)
<4> R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:0 PM:0 EA:3
<4>Krnl GPRS: ffffffff00000030 0000000000000002 0000000000000002 fffffffffffffffe
<4> 000000000013aabe 00000000003b6a18 fffffffffffffffd 0000000000000000
<4> 00000000006705a8 000000007d0914a8 000000007d0914b0 000000007fecfd30
<4> 0000000000000000 00000000003b63e8 000000007fecfd90 000000007fecfd30
<4>Krnl Code: 00000000001399e8: b9200021 cgr %r2,%r1
<4> 00000000001399ec: a7740004 brc 7,1399f4
<4> 00000000001399f0: a7f40001 brc 15,1399f2
<4> >00000000001399f4: c0100027e8ee larl %r1,636bd0
<4> 00000000001399fa: bf1f1008 icm %r1,15,8(%r1)
<4> 00000000001399fe: a7840019 brc 8,139a30
<4> 0000000000139a02: c0300027e8ef larl %r3,636be0
<4> 0000000000139a08: e3c030000004 lg %r12,0(%r3)
<4>Call Trace:
<4>([<0000000000139c12>] tasklet_hi_action+0x112/0x1d4)
<4> [<000000000013aabe>] __do_softirq+0xde/0x1c4
<4> [<000000000010fa2e>] do_softirq+0x96/0xb0
<4> [<000000000013a8d8>] irq_exit+0x70/0xcc
<4> [<000000000010d1d8>] do_extint+0xf0/0x110
<4> [<0000000000113b10>] ext_no_vtime+0x16/0x1a
<4> [<000003e0000a3662>] ext3_dirty_inode+0xe6/0xe8 [ext3]
<4>([<00000000001f6cf2>] __mark_inode_dirty+0x52/0x1d4)
<4> [<000003e0000a44f0>] ext3_ordered_write_end+0x138/0x190 [ext3]
<4> [<000000000018d5ec>] generic_perform_write+0x174/0x230
<4> [<0000000000190144>] generic_file_buffered_write+0xb4/0x194
<4> [<0000000000190864>] __generic_file_aio_write_nolock+0x418/0x454
<4> [<0000000000190ee2>] generic_file_aio_write+0x76/0xe4
<4> [<000003e0000a05c2>] ext3_file_write+0x3e/0xc8 [ext3]
<4> [<00000000001cc2fe>] do_sync_write+0xd6/0x120
<4> [<00000000001ccfc8>] vfs_write+0xac/0x184
<4> [<00000000001cd218>] SyS_write+0x68/0xe0
<4> [<0000000000113402>] sysc_noemu+0x10/0x16
<4> [<0000020000043188>] 0x20000043188
<4>Last Breaking-Event-Address:
<4> [<00000000001399f0>] tasklet_action+0xc4/0x1d4
<6>qdio: 0.0.c61b ZFCP on SC f67 using AI:1 QEBSM:0 PCI:1 TDD:1 SIGA: W AOP
<4> <0>Kernel panic - not syncing: Fatal exception in interrupt

Use tasklet_kill instead of tasklet_disbale. Since tasklet_schedule must not be
called after tasklet_kill use the QDIO_IRQ_STATE_STOPPED to inidicate that a
queue is going down and prevent further tasklet schedules in that case.

Remove superflous tasklet_schedule from input queue setup, at that time
the queues are not ready so the schedule results in a NOP.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: Dont call qdio_shutdown in case qdio_activate fails

Remove the call to qdio_shutdown from qdio_activate since the upper-layer
drivers are responsible to call qdio_shutdown when qdio_activate returns
with an error.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: add missing tiq_list locking

Add a mutex to protect the tiq_list. Although reading the list is done
using RCU adding and removing elements from the list must still
happen locked since multiple qdio devices may change the list in parallel
otherwise.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

fix similar typos to successfull

When I review ocfs2 code, find there are 2 typos to "successfull". After
doing grep "successfull " in kernel tree, 22 typos found totally -- great
minds always think alike :)

This patch fixes all the similar typos. Thanks for Randy's ack and comments.

Signed-off-by: Coly Li <coyli@suse.de>
Acked-by: Randy Dunlap <randy.dunlap@oracle.com>
Acked-by: Roland Dreier <rolandd@cisco.com>
Cc: Jeremy Kerr <jk@ozlabs.org>
Cc: Jeff Garzik <jeff@garzik.org>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Theodore Ts'o <tytso@mit.edu>
Cc: Mark Fasheh <mfasheh@suse.com>
Cc: Vlad Yasevich <vladislav.yasevich@hp.com>
Cc: Sridhar Samudrala <sri@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

[S390] qdio: fix error reporting for hipersockets

Hipersocket connections can encounter temporary busy conditions.
In case of the busy bit set we retry the SIGA operation immediatelly.
If the busy condition still persists after 100 ms we fail and report
the error to the upper layer. The second stage retry logic is removed.
In case of ongoing busy conditions the upper layer needs to reset the
connection.

The reporting of a SIGA error is now done synchronously to allow the
network driver to requeue the buffers. Also no error trace is created
for the temporary SIGA errors so the error message view is not flooded.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: improve inbound buffer acknowledgement

- Use automatic acknowledgement of incoming buffers in QEBSM mode
- Move ACK for non-QEBSM mode always to the newest buffer to prevent
a race with qdio_stop_polling
- Remove the polling spinlock, the upper layer drivers return new buffers
in the same code path and could not run in parallel
- Don't flood the error log in case of no-target-buffer-empty
- In handle_inbound we check if we would overwrite an ACK'ed buffer, if so
advance the pointer to the oldest ACK'ed buffer so we don't overwrite an
empty buffer in qdio_stop_polling

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: rework debug feature logging

- make qdio_trace a per device view
- remove s390dbf exceptions
- remove CONFIG_QDIO_DEBUG, not needed anymore if we check for the level
before calling sprintf
- use snprintf for dbf entries
- add start markers to see if the dbf view wrapped
- add a global error view for all queues

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: fix compile warning under 31 bit

The QEBSM instructions are only available for CONFIG_64BIT, they are not
used under 31 bit. Make compiler happy about the false positive:

drivers/s390/cio/qdio_main.c: In function ?qdio_inbound_q_done?:
drivers/s390/cio/qdio_main.c:532: warning: ?state? may be used uninitialized in this function

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: add eqbs/sqbs instruction counters

Add counters for the eqbs and sqbs instructions that indicate how often
we issued the instructions and how often the instructions returned with
less buffers than specified.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: fix qeth port count detection

qeth needs to get the port count information before
qdio has allocated a page for the chsc operation.
Extend qdio_get_ssqd_desc() to store the data in the
specified structure.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: prevent double qdio shutdown in case of I/O errors

In case of I/O errors on a qdio subchannel qdio_shutdown may be
called twice by the qdio driver and by zfcp. Remove the
superfluous shutdown from qdio and let the upper layer driver
handle the error condition.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio enhanced SIGA (iqdio) support.

Add support for z10 HiperSockets multiwrite SBALs on output
queues. This is used on LPAR with EDDP enabled devices.

Signed-off-by: Klaus-Dieter Wacker <kdwacker@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] bus_id ->dev_name() conversions in qdio

Use dev_name() in the new qdio driver.

Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: speed up multicast traffic on full HiperSocket queue

If an asynchronous HiperSockets queue runs full, no further packet
can be sent. In this case the next initiative to give transmitted
skbs back to the stack is triggered only by a 10-seconds qdio timer.
This timer has been introduced for low multicast traffic scenarios
to guarantee freeing of skbs in a limited amount of time. For high
HiperSocket multicast traffic scenarios progress checking on the
outbound queue should be enforced by tasklet rescheduling.

Signed-off-by: Ursula Braun <ursula.braun@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: remove the module_get & module_put pair

Increasing the qdio reference count for every used subchannel
is unnecessary since unloading qdio (if build as a module) is
only possible if other modules that use qdio are unloaded.
Unloading modules that use qdio in turn requires that these
modules shut down all qdio subchannels. Therefore the additional
module_get reference is not needed.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: improve s390 debug feature usage

Improve s390 debug feature usage:
- log busy bit in dbf
- increase size of dbf views
- consistent logging of qdio api calls to setup view
- print subchannel number so one can associate the interface
with the dbf data
- only log events to one view

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

[S390] qdio: make sure qdr is aligned to page size

kzalloc does not guarantee the required alignment of qdr to page size,
use get_zeroed_page instead.

Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>

[S390] qdio: new qdio driver.

List of major changes:
- split qdio driver into several files
- seperation of thin interrupt code
- improved handling for multiple thin interrupt devices
- inbound and outbound processing now always runs in tasklet context
- significant less tasklet schedules per interrupt needed
- merged qebsm with non-qebsm handling
- cleanup qdio interface and added kerneldoc
- coding style

Reviewed-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Reviewed-by: Utz Bacher <utz.bacher@de.ibm.com>
Reviewed-by: Ursula Braun <braunu@de.ibm.com>
Signed-off-by: Jan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>