scsi: isci: Remove redundant check in isci_task_request_build()

sci_task_request_construct_ssp() has invariant return. Change this function
to void and get rid of unnecessary checks.

Found by Linux Verification Center (linuxtesting.org) with SVACE.

Signed-off-by: Artem Chernyshev <artem.chernyshev@red-soft.ru>
Link: https://lore.kernel.org/r/20231128121159.2373975-1-artem.chernyshev@red-soft.ru
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>

scsi: libsas: Delete lldd_clear_aca callback

This callback is never called, so remove support.

Link: https://lore.kernel.org/r/1645112566-115804-4-git-send-email-john.garry@huawei.com
Tested-by: Yihang Li <liyihang6@hisilicon.com>
Tested-by: Damien Le Moal <damien.lemoal@opensource.wdc.com>
Reviewed-by: Jack Wang <jinpu.wang@ionos.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>

scsi: libsas: Drop SAS_TASK_AT_INITIATOR

This flag is now only ever set, so delete it.

This also avoids a use-after-free in the pm8001 queue path, as reported in
the following:

https://lore.kernel.org/linux-scsi/c3cb7228-254e-9584-182b-007ac5e6fe0a@huawei.com/T/#m28c94c6d3ff582ec4a9fa54819180740e8bd4cfb

https://lore.kernel.org/linux-scsi/0cc0c435-b4f2-9c76-258d-865ba50a29dd@huawei.com/

[mkp: checkpatch + two SAS_TASK_AT_INITIATOR references]

Link: https://lore.kernel.org/r/1644489804-85730-3-git-send-email-john.garry@huawei.com
Reviewed-by: Damien Le Moal <damien.lemoal@wdc.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>

scsi: isci: Drop SAS_TASK_AT_INITIATOR check in isci_task_abort_task()

In the queue path, move around when we assign sas_task->lldd_task such that
this pointer and the SAS_TASK_AT_INITIATOR flag are set atomically. It is
also not required to clear SAS_TASK_AT_INITIATOR in isci_task_execute_task()
error path as it is also cleared immediately after in isci_task_refuse()
call.

Now the following items may be considered:

- SAS_TASK_STATE_DONE and SAS_TASK_AT_INITIATOR are mutually exclusive
apart from possibly when SAS_TASK_STATE_DONE is set in
sas_scsi_find_task(), but that is after .lldd_abort_task, i.e. the
considered callback, is called.

- If isci_task_refuse() is called in the queue path, then
sas_task->lldd_task and SAS_TASK_AT_INITIATOR are cleared atomically in
isci_task_refuse().

- In the completion path, SAS_TASK_STATE_DONE is set and
SAS_TASK_AT_INITIATOR is cleared atomically before the
sas_task.lldd_task is cleared later.

So in isci_task_abort_task() if SAS_TASK_STATE_DONE is not set and
sas_task.lldd_task is still set, then SAS_TASK_AT_INITIATOR must be set -
so we can drop this check on SAS_TASK_AT_INITIATOR.

[mkp: checkpatch]

Link: https://lore.kernel.org/r/1644489804-85730-2-git-send-email-john.garry@huawei.com
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>

scsi: libsas: Introduce more SAM status code aliases in enum exec_status

This patch prepares for converting SAM status codes into an enum. Without
this patch converting SAM status codes into an enumeration type would
trigger complaints about enum type mismatches for the SAS code.

Link: https://lore.kernel.org/r/20210524025457.11299-2-bvanassche@acm.org
Cc: Hannes Reinecke <hare@suse.com>
Cc: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Cc: Jason Yan <yanaijie@huawei.com>
Reviewed-by: John Garry <john.garry@huawei.com>
Reviewed-by: Himanshu Madhani <himanshu.madhani@oracle.com>
Acked-by: Jack Wang <jinpu.wang@ionos.com>
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>

scsi: isci: Use correctly sized target buffer for memcpy()

In preparation for FORTIFY_SOURCE performing compile-time and run-time
field bounds checking for memcpy(), avoid intentionally writing across
neighboring array fields.

Switch from rsp_ui to resp_buf, since resp_ui isn't SSP_RESP_IU_MAX_SIZE
bytes in length. This avoids future compile-time warnings.

Link: https://lore.kernel.org/r/20210528181337.792268-4-keescook@chromium.org
Reviewed-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>

scsi: isci: task: Demote non-conformant header and remove superfluous param

Fixes the following W=1 kernel build warning(s):

drivers/scsi/isci/task.c:383: warning: Function parameter or member 'isci_host' not described in 'isci_task_send_lu_reset_sas'
drivers/scsi/isci/task.c:383: warning: Function parameter or member 'isci_device' not described in 'isci_task_send_lu_reset_sas'
drivers/scsi/isci/task.c:677: warning: Excess function parameter 'lun' description in 'isci_task_query_task'

Link: https://lore.kernel.org/r/20210317091230.2912389-23-lee.jones@linaro.org
Cc: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Cc: "James E.J. Bottomley" <jejb@linux.ibm.com>
Cc: "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: linux-scsi@vger.kernel.org
Signed-off-by: Lee Jones <lee.jones@linaro.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>

scsi: libsas: aic94xx: hisi_sas: mvsas: pm8001: Use dev_is_expander()

Many times in libsas, and in LLDDs which use libsas, the check for an
expander device is re-implemented or open coded.

Use dev_is_expander() instead. We rename this from
sas_dev_type_is_expander() to not spill so many lines in referencing.

Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Jason Yan <yanaijie@huawei.com>
Reviewed-by: Jack Wang <jinpu.wang@cloud.ionos.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>

scsi: isci: Change sci_controller_start_task's return type to sci_status

Clang warns when an enumerated type is implicitly converted to another.

drivers/scsi/isci/request.c:3476:13: warning: implicit conversion from
enumeration type 'enum sci_task_status' to different enumeration type
'enum sci_status' [-Wenum-conversion]
status = sci_controller_start_task(ihost,
~ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/scsi/isci/host.c:2744:10: warning: implicit conversion from
enumeration type 'enum sci_status' to different enumeration type 'enum
sci_task_status' [-Wenum-conversion]
return SCI_SUCCESS;
~~~~~~ ^~~~~~~~~~~
drivers/scsi/isci/host.c:2753:9: warning: implicit conversion from
enumeration type 'enum sci_status' to different enumeration type 'enum
sci_task_status' [-Wenum-conversion]
return status;
~~~~~~ ^~~~~~

Avoid all of these implicit conversion by just making
sci_controller_start_task use sci_status. This silences
Clang and has no functional change since sci_task_status
has all of its values mapped to something in sci_status.

Link: https://github.com/ClangBuiltLinux/linux/issues/153
Signed-off-by: Nathan Chancellor <natechancellor@gmail.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>

libsas: remove task_collector mode

The task_collector mode (or "latency_injector", (C) Dan Willians) is an
optional I/O path in libsas that queues up scsi commands instead of
directly sending it to the hardware. It generall increases latencies
to in the optiomal case slightly reduce mmio traffic to the hardware.

Only the obsolete aic94xx driver and the mvsas driver allowed to use
it without recompiling the kernel, and most drivers didn't support it
at all.

Remove the giant blob of code to allow better optimizations for scsi-mq
in the future.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Acked-by: Dan Williams <dan.j.williams@intel.com>

isci: Spelling s/stucture/structure/

Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
Cc: James E.J. Bottomley <JBottomley@parallels.com>
Signed-off-by: Jiri Kosina <jkosina@suse.cz>

[SCSI] isci: fix reset timeout handling

Remove an erroneous BUG_ON() in the case of a hard reset timeout. The
reset timeout handler puts the port into the "awaiting link-up" state.
The timeout causes the device to be disconnected and we need to be in
the awaiting link-up state to re-connect the port. The BUG_ON() made
the incorrect assumption that resets never timeout and we always
complete the reset in the "resetting" state.

Testing this patch also uncovered that libata continues to attempt to
reset the port long after the driver has torn down the context. Once
the driver has committed to abandoning the link it must indicate to
libata that recovery ends by returning -ENODEV from
->lldd_I_T_nexus_reset().

Cc: <stable@vger.kernel.org>
Acked-by: Lukasz Dorau <lukasz.dorau@intel.com>
Reported-by: David Milburn <dmilburn@redhat.com>
Reported-by: Xun Ni <xun.ni@intel.com>
Tested-by: Xun Ni <xun.ni@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>

[SCSI] isci: Fix a race condition in the SSP task management path

This commit fixes a race condition in the isci driver abort task and SSP
device task management path. The race is caused when an I/O termination
in the SCU hardware is necessary because of an SSP target timeout condition,
and the check of the I/O end state races against the HW-termination-driven
end state. The failure of the race meant that no TMF was sent to the device
to clean-up the pending I/O.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Reviewed-by: Lukasz Dorau <lukasz.dorau@intel.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>

[SCSI] sas: unify the pointlessly separated enums sas_dev_type and sas_device_type

These enums have been separate since the dawn of SAS, mainly because the
latter is a procotol only enum and the former includes additional state
for libsas. The dichotomy causes endless confusion about which one you
should use where and leads to pointless warnings like this:

drivers/scsi/mvsas/mv_sas.c: In function 'mvs_update_phyinfo':
drivers/scsi/mvsas/mv_sas.c:1162:34: warning: comparison between 'enum sas_device_type' and 'enum sas_dev_type' [-Wenum-compare]

Fix by eliminating one of them. The one kept is effectively the sas.h
one, but call it sas_device_type and make sure the enums are all
properly namespaced with the SAS_ prefix.

Signed-off-by: James Bottomley <JBottomley@Parallels.com>

treewide: fix comment/printk/variable typos

Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com>
Signed-off-by: Jiri Kosina <jkosina@suse.cz>

isci: Check IDEV_GONE before performing abort path operations.

In the link fail path, set IDEV_GONE for every device on the domain
when the last link in the port fails.

In the abort path functions like isci_reset_device, make sure that
there has not already been a detected domain failure with the device
by checking IDEV_GONE, before performing any kind of hard reset, SMP
phy control, or TMF operation.

The check for IDEV_GONE makes sure that the device in the abort path
really has control of the port with which it is associated. This
prevents starting hard resets at incorrect times and scheduling
unnecessary LUN resets for SATA devices.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Directly control IREQ_ABORT_PATH_ACTIVE when completing TMFs.

TMF requests, unlike normal I/O requests, need to handle I/O management
conditions in the completion function because TMFs are not handled in the
completion tasklet.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Manage tag releases differently when aborting tasks.

When an individual request is being terminated, the request's tag
is managed in the terminate function.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Implement waiting for suspend in the abort path.

In order to prevent a device from receiving an I/O request while still
in an RNC suspending or resuming state (and therefore failing that
I/O back to libsas with a reset required status) wait for the RNC state
change before proceding in the abort path.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Make sure all TCs are terminated and cleaned in LUN reset.

In the libsas error path, SATA disks require extra handling in
libata to recover operation. However, libsas expects to be able
to immediately recover all outstanding I/O once the error handler
escalation stops. This patch fixes the condition where the libata
error handler is scheduled for operation but libsas has already
deleted the outstanding sas_tasks.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Remove isci_device reqs_in_process and dev_node from isci_device.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Device access in the error path does not depend on IDEV_GONE.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Redesign device suspension, abort, cleanup.

This commit changes the means by which outstanding I/Os are handled
for cleanup.
The likelihood is that this commit will be broken into smaller pieces,
however that will be a later revision. Among the changes:

- All completion structures have been removed from the tmf and
abort paths.
- Now using one completed I/O list, with the I/O completed in host bit being
used to select error or normal callback paths.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Escalate to I_T_Nexus_Reset when the device is gone.

If LUN reset sees that the device is gone, it returns TMF_RESP_FUNC_FAILED
to cause libsas to escalate to an I_T_Nexus_Reset.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Manage device suspensions during TC terminations.

TCs must be terminated only while the RNC is suspended. This commit
adds remote device suspensions and resumptions in the abort, reset and
termination paths.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

[SCSI] isci: remove IDEV_EH hack to disable "discovery-time" ata resets

Prior to commit 61aaff49 "isci: filter broadcast change notifications
during SMP phy resets" we borrowed the MVS_DEV_EH approach from the
mvsas driver for preventing ->lldd_I_T_nexus_reset() events during ata
discovery. This hack was protecting against the old ->phy_reset() in
ata_bus_probe(), but since the conversion to the new error handling this
hack is preventing resets from reaching ata devices.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>

[SCSI] isci: remove bus and reset handlers

Remove ->eh_device_reset_handler() and ->eh_bus_reset_handler() for the
same reason they are not implemented for libata hosts, they cannot be
implemented reliably with ata-eh. ATA error recovery wants to divert
all resets to the eh thread and wait for completion, these handlers may
be invoked from a non-blocking ioctl.

The other path they are called from is libsas-eh, and if we escalate
past I_T_nexus reset we have larger problems i.e. tear down all
in-flight commands in the domain potentially without notification to the
lldd if it has chosen not to implement ->lldd_clear_nexus_port() /
->lldd_clear_nexus_ha().

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>

[SCSI] isci: stop interpreting ->lldd_lu_reset() as an ata soft-reset

Driving resets from libsas-eh is pre-mature as libata will make a
decision about performing a softreset. Currently libata determines
whether to perform a softreset based on ata_eh_followup_srst_needed(),
and none of those conditions apply to isci.

Remove the srst implementation and translate ->lldd_lu_reset() for ata
devices as a request to drive a reset via libata-eh.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>

[SCSI] isci: fix interpretation of "hard" reset

A hard reset to isci in the direct-attached case is one where the driver
internally manages debouncing the link. In the sas-expander-attached
case a hard reset is one that clears affiliations. The driver should
not be prematurely dropping affiliations at run time, that decision (to
force expander hard resets to ata devices) is left to userspace to
manage. So, arrange for I_T_nexus resets to be sas-link-resets in the
expander-attached case and isci-hard-resets in the direct-attached case.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>

[SCSI] libsas: fix sas_find_local_phy(), take phy references

In the direct-attached case this routine returns the phy on which this
device was first discovered. Which is broken if we want to support
wide-targets, as this phy reference can become stale even though the
port is still active.

In the expander-attached case this routine tries to lookup the phy by
scanning the attached sas addresses of the parent expander, and BUG_ONs
if it can't find it. However since eh and the libsas workqueue run
independently we can still be attempting device recovery via eh after
libsas has recorded the device as detached. This is even easier to hit
now that eh is blocked while device domain rediscovery takes place, and
that libata is fed more timed out commands increasing the chances that
it will try to recover the ata device.

Arrange for dev->phy to always point to a last known good phy, it may be
stale after the port is torn down, but it will catch up for wide port
reconfigurations, and never be NULL.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>

[SCSI] libsas: remove ata_port.lock management duties from lldds

Each libsas driver (mvsas, pm8001, and isci) has invented a different
method for managing the ap->lock. The lock is held by the ata
->queuecommand() path. mvsas drops it prior to acquiring any internal
locks which allows it to hold its internal lock across calls to
task->task_done(). This capability is important as it is the only way
the driver can flush task->task_done() instances to guarantee that it no
longer has any in-flight references to a domain_device at
->lldd_dev_gone() time.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>

[SCSI] isci: remove unused 'isci_tmf->device' field

As the field was never set, isci_print_tmf() using 'isci_tmf->device'
sometimes causes a kernel crash if the dev_dbg() statement is enabled.
Remove the unused field both from isci_tmf struct definition and from
isci_print_tmf()

Signed-off-by: Maciej Trela <maciej.trela@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>

[SCSI] isci: revert bcn filtering

The initial bcn filtering implementation was validated on a kernel
baseline that predated the switch to new libata error handling. Also,
prior to that conversion we borrowed the mvsas MVS_DEV_EH approach to
prevent the unwanted extra ap->ops->phy_reset(ap) that occurred in the
ata_bus_probe() path.

After the conversion to new libata eh resets at discovery are more
frequent and get filtered prematurely by IDEV_EH. The result is that
our bcn filtering has been blocked from running and at discovery and it
appears to stall discovery completion to the point of triggering hung
task timeouts. So, revert the implementation for now. When it returns
it will go into libsas proper.

The domain rediscovery that takes place due to ->lldd_I_T_nexus_reset()
events should now be properly waited for by the ata_port_wait_eh() call
in ata_port_probe(). So the hard coded delay in the isci
->lldd_I_T_nexus_reset() and other libsas drivers should help debounce
the libsas thread from seeing temporary device removals.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>

[SCSI] isci: No need to manage the pending reset bit on pending requests.

The lldd does not need to look at or manage the pending device
reset bit in pending sas_tasks.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>

[SCSI] isci: Remove redundant isci_request.ttype field.

Use the existing IREQ_TMF flag as a request type indicator.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>

[SCSI] isci: Fix task management for SMP, SATA and on dev remove.

libsas uses the LLDD abort task interface to handle I/O timeouts
in the SATA/STP and SMP discovery paths, so this change will terminate
STP/SMP requests. Also, if the device is gone, the lldd will prevent
libsas from further escalations in the error handler.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>

[SCSI] isci: No task_done callbacks in error handler paths.

libsas will cleanup pending sas_tasks after error handler
path functions are called; do not call task_done callbacks.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>

[SCSI] isci: Handle task request timeouts correctly.

In the case where "task" requests timeout (note that this class of
requests can also include SATA/STP soft reset FIS transmissions),
handle the case where the task was being managed by some call to
terminate the task request by completing both the tmf and the aborting
process.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>

[SCSI] isci: Fix tag leak in tasks and terminated requests.

Make sure terminated requests and completed task tags are freed.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>

[SCSI] isci: Immediately fail I/O to removed devices.

In the case where an I/O fails to start in isci_request_execute,
only allow retries if the device is not already gone.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>

[SCSI] libsas: export sas_alloc_task()

Now that isci has added a 3rd open coded user of this functionality just
share the libsas version.

Acked-by: Jack Wang <jack_wang@usish.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>

[SCSI] isci: fix checkpatch errors

Signed-off-by: James Bottomley <JBottomley@Parallels.com>

isci: Device reset should request sas_phy_reset(phy, true)

The hard_reset parameter passed to the LLDD in the direct-attached
phy control case allows the LLDD to filter link failure events
while the direct-attached device reset is executing.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: pare back error messsages

The messages emitted from task.c and some from request.c likely
duplicate (in a less undertandable way) what is reported by the
midlayer.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: merge sata.[ch] into request.c

Undo some needless separation.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: kill 'get/set' macros

Most of these simple dereference macros are longer than their open coded
equivalent. Deleting enum sci_controller_mode is thrown in for good
measure.

Reported-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: retire scic_sds_ and scic_ prefixes

The distinction between scic_sds_ scic_ and sci_ are no longer relevant
so just unify the prefixes on sci_. The distinction between isci_ and
sci_ is historically significant, and useful for comparing the old
'core' to the current Linux driver. 'sci_' represents the former core as
well as the routines that are closer to the hardware and protocol than
their 'isci_' brethren. sci == sas controller interface.

Also unwind the 'sds1' out of the parameter structs.

Reported-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: unify isci_host and scic_sds_controller

Remove the distinction between these two implementations and unify on
isci_host (local instances named ihost). Hmmm, we had two
'oem_parameters' instances, one was unused... nice.

Reported-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: unify isci_remote_device and scic_sds_remote_device

Remove the distinction between these two implementations and unify on
isci_remote_device (local instances named idev).

Reported-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: unify isci_request and scic_sds_request

They are one in the same object so remove the distinction. The near
duplicate fields (owning_controller, and isci_host) will be cleaned up
after the scic_sds_contoller isci_host unification.

Reported-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: preallocate requests

the dma_pool interface is optimized for object_size << page_size which
is not the case with isci_request objects and the dma_pool routines show
up in the top of the profile.

The old io_request_table which tracked whether tci slots were in-flight
or not is replaced with an IREQ_ACTIVE flag per request.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: combine request flags

Combine three bools into one unsigned long 'flags'. Doesn't increase the
request size due to packing. (to do: optimize the structure layout).

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: unify can_queue tracking on the tci_pool, uplevel tag assignment

The tci_pool tracks our outstanding command slots which are also the 'index'
portion of our tags. Grabbing the tag early in ->lldd_execute_task let's us
drop the isci_host_can_queue() and ->was_tag_assigned_by_user infrastructure.
->was_tag_assigned_by_user required the task context to be duplicated in
request-local buffer. With the tci established early we can build the
task_context directly into its final location and skip a memcpy.

With the task context buffer at a known address at request construction we
have the opportunity/obligation to also fix sgl handling. This rework feels
like it belongs in another patch but the sgl handling and task_context are too
intertwined.
1/ fix the 'ab' pair embedded in the task context to point to the 'cd' pair in
the task context (previously we were prematurely linking to the staging
buffer).
2/ fix the broken iteration of pio sgls that assumes all sgls are relative to
the request, and does a dangerous looking reverse lookup of physical
address to virtual address.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Terminate dev requests on FIS err bit rx in NCQ

When the remote device transitions to a not-ready state because of
an NCQ error condition, all outstanding requests to that device
are terminated and completed to libsas on the normal path. The
device then waits for a READ LOG EXT command to issue on the task
management path.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: fix isci_task_execute_tmf completion

1/ fix the timeout for wait_for_completion_timeout
2/ In the tmf timeout case we need to wait for our termination callback
3/ Once the request is successfully started it will be freed according to the
normal lifetime for requests.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: kill isci_remote_device_change_state()

Now that "stopping/stopped" are one in the same and signalled by a NULL device
pointer the rest of the device status infrastructure can be removed (->status
and ->state_lock). The "not ready for i/o state" is replaced with a state
flag, and is evaluated under scic_lock so that we don't see transients from
taking the device reference to submitting the i/o.

This also fixes a potential leakage of can_queue slots in the rare case that
SAS_TASK_ABORTED is set at submission.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: atomic device lookup and reference counting

We have unsafe references to remote devices that are notified to
disappear at lldd_dev_gone. In order to clean this up we need a single
canonical source for device lookups and stable references once a lookup
succeeds. Towards that end guarantee that domain_device.lldd_dev is
NULL as soon as we start the process of stopping a device. Any code
path that wants to safely lookup a remote device must do so through
task->dev->lldd_dev (isci_lookup_device()).

For in-flight references outside of scic_lock we need reference counting
to ensure that the device is not recycled before we are done with it.
Simplify device back references to just scic_sds_request.target_device
which is now the only permissible internal reference that is maintained
relative to the reference count.

There were two occasions where we wanted new i/o's to be treated as
SAS_TASK_UNDELIVERED but where the domain_dev->lldd_dev link is still
intact. Introduce a 'gone' flag to prevent i/o while waiting for libsas
to take action on the port down event.

One 'core' leftover is that we currently call
scic_remote_device_destruct() from isci_remote_device_deconstruct()
which is called when the 'core' says the device is stopped. It would be
more natural for the final put to trigger
isci_remote_device_deconstruct() but this implementation is deferred as
it requires other changes.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: cleanup request allocation

Rather than return an error code and update a pointer that was passed by
reference just return the request object directly (or null if allocation
failed).

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: fix isci_terminate_pending() list management

Walk through the list of pending requests being careful to consider that
multiple requests can be terminated when the lock is dropped (i.e.
invalidating the 'next' reference established by
list_for_each_entry_safe).

Also noticed that all callers to isci_terminate_pending_requests()
specifying terminating, so just drop the parameter.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Handle timed-out request terminations correctly

In the situation where a termination of an I/O times-out,
make sure that the linkage from the request to the task
is severed completely. Also make sure that the selection
of tasks to terminate occurs under scic_lock.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: filter broadcast change notifications during SMP phy resets

When resetting a sata device in the domain we have seen occasions where
libsas prematurely marks a device gone in the time it takes for the
device to re-establish the link. This plays badly with software raid
arrays. Other libsas drivers have non-uniform delays in their reset
handlers to try to cover this condition, but not sufficient to close the
hole. Given that a sata device can take many seconds to recover we
filter bcns and poll for the device reattach state before notifying
libsas that the port needs the domain to be rediscovered. Once this has
been proven out at the lldd level we can think about uplevelling this
feature to a common implementation in libsas.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
[ use kzalloc instead of kmem_cache ]
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
[ use eventq and time macros ]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Move the reset delay after the remote node resumption.

Delay after bringing up the RNC to allow for resumption latency.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Removing unused variables compiler warnings

Newer gcc's are better at identifying "set, but not used" variables.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: remove isci_timer interface

Delete code which is no longer used.

Signed-off-by: Edmund Nadolski <edmund.nadolski@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Remove tmf timeout_timer

Replace the timeout_timer in the isci_tmf with a call to
wait_for_completion_timeout

Signed-off-by: Edmund Nadolski <edmund.nadolski@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: uplevel request infrastructure

* Consolidate tiny header files
* Move files out of core/ (drop core/scic_sds_ prefix)
* Merge core/scic_sds_request.[ch] into request.[ch]
* Cleanup request.c namespace (clean forward declarations and global
namespace pollution)

Reported-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: move core/controller to host

Now that the data structures are unified unify the implementation in
host.[ch] and cleanup namespace pollution.

Reported-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: unify constants

cross driver constants are spread out over multiple header files, consolidate
them into isci.h, and push some includes out to the source files that need
them.

TODO: remove SCI_MODE_SIZE infrastructure.
TODO: task.h is full of inlines that are too large

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: unify request data structures

Make scic_sds_request a proper member of isci_request. Also let's us
get rid of the dma pool object size tracking since we now know that all
requests are sizeof(isci_request). While cleaning up the construct
routine incidentally replaced SCI_FIELD_OFFSET with offsetof.

Reported-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: make command/response iu explicit request object members

Final elimination of the anonymous data at the end of the request
structure.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: make sgl explicit/aligned request object member

Towards unifying request objects we need all members to be defined in the
object and not carved out of anonymous buffer space.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: move stp request info to scic_sds_request

In preparation for unifying allocation of all request information make stp
data available in all requests. Incidentally collapse indentation.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: unify isci_host data structures

Make it explicit that isci_host and scic_sds_controller are one in the same
object.

Signed-off-by: Artur Wojcik <artur.wojcik@intel.com>
[removed ->ihost back pointer]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: implement I_T_nexus_reset

This is a requirement for 2.6.39's new libata eh.

Still some questions about lldd_dev_gone racing against dev->lldd_dev
lookups, but we are at least no more broken than mvsas in this regard.

We also short-circuit I_T_nexus_reset invocations from the device
discovery path (IDEV_EH similar to MVS_DEV_EH) to filter out the
resulting domain rediscoveries triggered by the reset.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Convert of sci_ssp_response_iu to ssp_response_iu

Converting to Linux native format. However the isci driver does a lot of
the calculation based on the max size of this data structure and the
Linux data structure only has a pointer to the response data. Thus the
sizeof(struct ssp_response_iu) will be incorrect and we need to define
the max size.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Fixup SSP command IU and task IU

Fixup of SSP command IU and SSP task IU to something that looks like Linux

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Convert SATA fis data structures to Linux native

Converting of sata_fis_reg_d2h to dev_to_host_fis
Converting of sata_fis_reg_h2d to host_to_dev_fis

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Removed sci_base_object from scic_sds_request.

The 'struct sci_base_object' was removed from the struct
scic_sds_request and was replaced by a pointer to
struct isci_request.

Signed-off-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Remove excessive log noise with expander hot-unplug

We are logging excessive output when hot unplug from expander. Moving
that to debug.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: kill smp_discover_response_protocols in favor of domain_device.dev_type

This is step 1 of removing the contortions to:
1/ unparse expander phy data into a smp discover frame
2/ open-code-parse the smp discover fram into a domain_device.dev_type equivalent

libsas has already spent cycles determining the dev_type, so now that
scic_sds_remote_device is unified with isci_remote_device we can
directly reference dev_type.

This might also change multi-level expander detection as we previously only
looked at dev_type == EDGE_DEV and we did not consider the FANOUT_DEV case.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: move remote_device handling out of the core

Now that the core/lldd remote_device data structures are nominally unified
merge the corresponding sources into the top-level directory. Also move the
remote_node_context infrastructure which has no analog at the lldd level.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: unify remote_device data structures

Make it explicit that isci_remote_device and scic_sds_remote_device are
one in the same object.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: audit usage of BUG_ON macro in isci driver

Removes unnecessary usage of BUG_ON macro, excluding core directory.
In some cases macro is unnecesary, check is done in caller function.
In other cases macro is replaced by if construction with
appropriate warning.

Signed-off-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
[changed some survivable bug conditions to WARN_ONCE]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: sparse warnings cleanup

Clean warnings and errors reported by sparse tool.

request.c:430:50: warning: mixing different enum types
remote_device.c:534:39: warning: symbol 'flags' shadows an earlier one
task.c:495:44: warning: mixing different enum types
scic_sds_controller.c:2155:24: warning: mixing different enum types
scic_sds_controller.c:2272:36: warning: mixing different enum types
scic_sds_controller.c:2911:38: warning: incorrect type in initializer (different address spaces)
scic_sds_controller.c:2913:25: warning: incorrect type in argument 2 (different address spaces)
scic_sds_request.c:875:34: warning: cast removes address space of expression
scic_sds_request.c:876:123: warning: incorrect type in argument 2 (different address spaces)
scic_sds_port.c:585:51: warning: incorrect type in assignment (different address spaces)
scic_sds_port.c:712:9: warning: incorrect type in argument 2 (different address spaces)
scic_sds_port.c:1770:25: warning: incorrect type in argument 2 (different address spaces)

Signed-off-by: Bartosz Barcinski <Bartosz.Barcinski@intel.com>
Signed-off-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
[fixed up some false positives and misconversions]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: remove scic_controller state handlers

Remove the state handler indirections for the scic_controller, and replace
them with procedural calls that check for the correct state first.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: fix fragile/conditional isci_host lookups

A domain_device can always reference back to ->lldd_ha unlike local lldd
structures. Fix up cases where the driver uses local objects to look up the
isci_host. This also changes the calling conventions of some routines to
expect a valid isci_host parameter rather than re-lookup the pointer on entry.

Incidentally cleans up some macros that are longer to type than the open-coded
equivalent:
isci_host_from_sas_ha
isci_dev_from_domain_dev

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Qualify when the host lock is managed for STP/SATA callbacks.

In the case of internal discovery related STP/SATA I/O started
through sas_execute_task the host lock is not taken by libsas before
calling lldd_execute_task, so the lock should not be managed before
calling back to libsas through task->task_done or sas_task_abort.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Properly handle requests in the "aborting" state.

When a TMF times-out, the request is set back to "aborting".
Requests in the "aborting" state must be terminated when
LUN and device resets occur.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: namespacecheck cleanups

* mark needlessly global routines static
* delete unused functions
* move kernel-doc blocks from header files to source
* reorder some functions to delete declarations
* more default handler cleanups phy

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: fix incorrect assumptions about task->dev and task->dev->port being NULL

A domain_device has the same lifetime as its related scsi_target. The
scsi_target is reference counted based on outstanding commands,
therefore it is safe to assume that if we have a valid sas_task that the
->dev pointer is also valid.

The asd_sas_port of a domain_device has the same lifetime as the driver
so it can also never be NULL as long as the sas_task is valid and the
driver is loaded.

This also cleans up isci_task_complete_for_upper_layer(), renames it to
isci_task_refuse() and notices that the isci_completion_selection
parameter was set to isci_perform_normal_io_completion by all callers.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Errors in the submit path for SATA devices manage the ap lock.

Since libsas takes the domain device sata_dev.ap->lock before submitting
a task, error completions in the submit path for SATA devices must
unlock/relock when completing the sas_task back to libsas.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Fixed BUG_ON in isci_abort_task_process_cb callback.

The request may be in the "aborted" or the "completed" state when
performing a task management operation on it.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Fix TMF build for SAS/SATA LUN reset cases.

In the case where a SAS or SATA LUN reset TMF is built a NULL pointer
dereference occurred because of the (unused) callback data pointer.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Jacek Danecki <Jacek.Danecki@intel.com>

isci: Termination handling cleanup, added termination timeouts.

Added a request "dead" state for use when a termination wait times-out.

isci_terminate_pending_requests now detaches the device's pending list
and terminates each entry on the detached list.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Code review change for completion pointer cleanup.

Since the request structure contains a pointer to the completion to be
used if the request is being aborted or terminated, there is no reason
to pass the completion as a pointer to isci_terminate_request_core().

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Jacek Danecki <Jacek.Danecki@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Cleaning up task execute path.

Made sure the device ready check accounts for all states.
Moved the aborted task check into the loop of pulling task requests
off of the submitted list.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Jacek Danecki <Jacek.Danecki@intel.com>
[remove host and device starting state checks]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: save the i/o tag outside the scic request structure.

The pointer to the core representation of a request is marked NULL at
completion, but we need to save the i/o tag for task management.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Jacek Danecki <Jacek.Danecki@intel.com>
[revise changelog]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: fix completion / abort path.

Corrected use of the request state_lock in the completion callback.

In the case where an abort (or reset) thread is trying to terminate an
I/O request, it sets the request state to "aborting" (or "terminating")
if the state is still "starting". One of the bugs was to never set the
state to "completed". Another was to not correctly recognize the
situation where the I/O had completed but the sas_task was still pending
callback to task_done - this was typically a problem in the LUN and
device reset cases.

It is now possible that we leave isci_task_abort_task() with
request->io_request_completion pointing to localy allocated
aborted_io_completion struct. It may result in a system crash.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Maciej Trela <Maciej.Trela@intel.com>
Signed-off-by: Jacek Danecki <Jacek.Danecki@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: isci_request_cleanup_completed_loiterer checks task before task_done

In the condition where outstanding I/Os are being cleaned from the device
requests in process list, the cleanup function needs to check that the
request is actually a sas-task and not a task management function.

Signed-off-by: Jeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: kill "host quiesce" mechanism

The midlayer is already throttling i/o in the places where host_quiesce
was trying to prevent further i/o to the device. It's also problematic
in that it holds a lock over GFP_KERNEL allocations.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: remove sci_device_handle

It belies the fact that isci_remote_device and scic_sds_remote_device
are one in same object with the same lifetime rules.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: rework timer api

Prepare the timer api for the arrival of dynamic creation and
destruction events from the core. It pretended to do this previously
but the core to date only used it in a static init-time only fashion.
This is an interim fix until a cleaner event queue can be developed.

1/ make all locking external to the api (add WARN_ONCE to verify)
2/ add a timer_destroy interface (to be used by the core)
3/ use del_timer_sync() prior to deallocating timer data
4/ delete the "timer_list" indirection, we only have timers allocated
for the isci_host
5/ fix detection of timer list allocation errors

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: cleanup "starting" state handling

The lldd actively disallows requests in the "starting" state. Retrying
or holding off commands in this state is sub-optimal:
1/ it adds another state check to the fast path
2/ retrying can cause libsas to give up

However, isci's ->lldd_dev_found() routine already waits for controller
start to complete before allowing further progress. Checking the
"starting" state in isci_task_execute_task and the isr is redundant and
misleading. Clean this up and introduce a controller-wide event queue
to start reeling in "completion" proliferation in the driver.

The "stopping" state cleanups are in a similar vein, rely on the the isr
and other paths being precluded from occurring rather than implementing
state checking logic.

Reported-by: Christoph Hellwig <hch@infradead.org>
Cc: Jeff Garzik <jeff@garzik.org>
Signed-off-by: Edmund Nadolski <edmund.nadolski@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

isci: Intel(R) C600 Series Chipset Storage Control Unit Driver

Support for the up to 2x4-port 6Gb/s SAS controllers embedded in the
chipset.

This is a snapshot of the first publicly available version of the driver,
commit 4c1db2d0 in the 'historical' branch.

git://git.kernel.org/pub/scm/linux/kernel/git/djbw/isci.git historical

Signed-off-by: Maciej Trela <maciej.trela@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Edmund Nadolski <edmund.nadolski@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>