Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r261535:
Add support for MegaRAID Fury cards. The main change needed to boot from a
9341-4i controller was to ensure that scatter/gather lists are ended with
an end-of-list marker. Both the mrsas and Linux megaraid_sas drivers use
this marker with Invader cards as well, so we do the same thing, though
it is apparently not strictly necessary.

MFC r242726 (ambrisko):
Add support for SCSI pass through devices to be attached and detached.

MFC r251172 (sbruno):
xpt_create_path() requires mfi_io_lock to be held, so do it.

mfi(4) doesn't panic on host startup now.

Fix two swapped definitions. Note it is directly committed to this branch
because it fixes r250496, i.e., MFC of r247369 to stable/9.

MFC r244123 Add CTLFLAG_RWTUN define
MFC r247367 Fix non-recusive mutex MFI I/O lock
MFC r247369 Fix a large amount of bugs in MFI that cause panics

Approved by: pjd (mentor)

MFC r242681 (ambrisko):

- Extend the prior commit to use the generic SCSI command building
function use that for JBOD and Thunderbolt disk write command. Now
we only have one implementation in mfi.
- Fix dumping on Thunderbolt cards. Polled IO commands do not seem to
be normally acknowledged by changing cmd_status to MFI_STAT_OK.
In order to get acknowledgement of the IO is complete, the Thunderbolt
command queue needs to be run through. I added a flag MFI_CMD_SCSI
to indicate this command is being polled and to complete the
Thunderbolt wrapper and indicate the result. This flag needs to be
set in the JBOD case in case if that us using Thunderbolt card.
When in the polling loop check for completed commands.
- Remove mfi_tbolt_is_ldio and just do the check when needed.
- Fix an issue when attaching of disk device happens when a device is
already scheduled to be attached but hasn't attached.
- add a tunable to allow raw disk attachment to CAM via:
hw.mfi.allow_cam_disk_passthrough=1
- fixup aborting of commands (AEN and LD state change). Use a generic
abort function and only wait the command being aborted not both.
Thunderbolt cards don't seem to abort commands so the abort times
out.

MFC r238373

sys/dev/mfivar.h contains references to MFI_DEBUG, but it never gets turned on unless the file also includes opt_mfi.h.

Submitted by: Andrew Boyer aboyer@averesystems.com

MFC r235318, r235321

Fix inappropriate data type for two bus_dmamap_t variables that were causing
PAE to insta-panic on startup. Remove one unused variable that was
commented out.

MFC r235014: Rebuild fix and LBA calculation fix for ThunderBolt
based cards.

MFC r235016: Some style improvements.

MFC r235040: Fix the returns in mfi_tbolt_sync_map_info that I added.

PR: 167226

MFC r233711:
Major update to driver to support for Drake Skinny and ThunderBolt cards.

MFC r233768:
Change typedef atomic_t to struct mfi_atomic to avoid name space
collision and some couple more style changes.

MFC r233805:
Move struct megasas_sge from mfi_ioctl.h to mfivar.h so we can
remove including machine/bus.h. Add some more mfi_ prefixes to
avoid name space pollution.

MFC r233877:
- Do not include machine/atomic.h. It is no longer necessary since r233768.
- Remove bogus "atomic" macros and a read-only variable from softc.

MFC 227562:
Add single-message MSI support to mfi(4). It is disabled by default but
can be enabled via the hw.mfi.msi tunable. Many mfi(4) controllers also
support MSI-X, but in testing it seems that many adapters do not work with
MSI-X but do work with MSI.

MFC r226896:
Fix an implicit dependency between the MFI driver and CAM that had grown due
to an API change in CAM. It's once again possible to link a static kernel
with 'mfi' without requiring 'scbus' as well. Ditto for KLD loading.

Approved by: re (bz)

MFC r225869:
- Add special support for the MFI_CMD ioctl with MFI_CMD_STP command,
used by present MegaCLI version. It has some special meaning for the
first s/g list entry, while the main s/g list begins from the the second
entry, and those lists should remain separate after loading to the
busdma map.
- Fix bug in 32bit ioctl compatibility shims when s/g list consists of
more then on element.

Approved by: re (kib)

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Separate the parallel scsi knowledge out of the core of the XPT, and
modularize it so that new transports can be created.

Add a transport for SATA

Add a periph+protocol layer for ATA

Add a driver for AHCI-compliant hardware.

Add a maxio field to CAM so that drivers can advertise their max
I/O capability. Modify various drivers so that they are insulated
from the value of MAXPHYS.

The new ATA/SATA code supports AHCI-compliant hardware, and will override
the classic ATA driver if it is loaded as a module at boot time or compiled
into the kernel. The stack now support NCQ (tagged queueing) for increased
performance on modern SATA drives. It also supports port multipliers.

ATA drives are accessed via 'ada' device nodes. ATAPI drives are
accessed via 'cd' device nodes. They can all be enumerated and manipulated
via camcontrol, just like SCSI drives. SCSI commands are not translated to
their ATA equivalents; ATA native commands are used throughout the entire
stack, including camcontrol. See the camcontrol manpage for further
details. Testing this code may require that you update your fstab, and
possibly modify your BIOS to enable AHCI functionality, if available.

This code is very experimental at the moment. The userland ABI/API has
changed, so applications will need to be recompiled. It may change
further in the near future. The 'ada' device name may also change as
more infrastructure is completed in this project. The goal is to
eventually put all CAM busses and devices until newbus, allowing for
interesting topology and management options.

Few functional changes will be seen with existing SCSI/SAS/FC drivers,
though the userland ABI has still changed. In the future, transports
specific modules for SAS and FC may appear in order to better support
the topologies and capabilities of these technologies.

The modularization of CAM and the addition of the ATA/SATA modules is
meant to break CAM out of the mold of being specific to SCSI, letting it
grow to be a framework for arbitrary transports and protocols. It also
allows drivers to be written to support discrete hardware without
jeopardizing the stability of non-related hardware. While only an AHCI
driver is provided now, a Silicon Image driver is also in the works.
Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware
is possible and encouraged. Help with new transports is also encouraged.

Submitted by: scottl, mav
Approved by: re

- Fix from jhb for failing I/O request when bus_dmamap_load fails.
- Fix to ioctl path in which the length could be 0 which means
no data in/out from LSI.
- Fix to ioctl path in which the data in the sense data space
of the ioctl packet is a really a pointer to some location in
user-space. From LSI re-worked a bit by me.
- Add HW support for next gen cards from LSI.

Thanks to LSI for their support!

Submitted by: jhb, LSI
MFC after: 3 days

Update the MFI driver to support new "1078" series of hardware. This
includes the upcoming Dell PERC6 series. Many thanks to LSI for
contributing this code.

Submitted by: LSI
Approved by: re

Teach the mfi(4) driver to handle requests from userland management
applications to add and remove volumes.

MFC after: 1 week
Approved by: re (bmah)
Reviewed by: ambrisko, scottl

Add in a couple of things:
- In the ioctl path let command get queued up and return
when complete _without_ blocking the driving waiting for
the response. This way the driver doesn't "lock up" for
~30s during a flash command. Submitted by scottl.
- Add a guard so that if a DCMD of 0 is sent down the ioctl
path don't send it to the controller. Return with a
status of OK. This is a little strange since MegaCli
doesn't seem to like something and will issue some DCMD
of 0. This doesn't happen under Linux. So the emulation
needs to be improved but I'm not sure what. Another strange
thing is that when a DCMD of 0 gets issued under i386 the
controller returns OK but in amd64 the context is messed
up.
- Add a guard so the context has to be with-in the legal
limit so we get a reasonable error assertion versus random
panic.

It's going to be a challenge to figure out why MegaCli is not totally
happy and then sends some bogus commands. This means that flashing
firmware via the Linux tool won't work since it generates a DCMD of
0 when it should be opening the firmware for a flash update. Without
this problem flashing works fine. This means there is no publicly
available tool to upgrade the RAID firmware under FreeBSD right now.

I plan to MFC all of the mfi changes to 6.X shortly. This might not
include the SCSI pass-through changes.

Submitted by: scottl
Reviewed by: scottl
MFC after: 3 days

Catch up to driver changes required for the mfip sub-driver.

Collapse the mfi_ld object. Add an ioctl to help management apps map
array Id's to FreeBSD device names.

- Add a command validator for use in debugging.
- Fix the locking protocol to eliminate races between normal I/O and AENs.
- Various small improvements and usability tweaks.

Sponsored by: IronPort
Portions Submitted by: Doug Ambrisko

Add a command debugging module and a periodic watchdog timer.

Sponsored by: IronPort

Change some variable names and update some comments to help clarify some
confusing issues.

Instead of using scsi probes to do device discovery, use the firmware
commands to grab the device listing. This resolves issues using
multiple volumes, where each volume was actually internally pointing
to target 0.

Add in a bunch of things to the mfi driver:
- Linux ioctl support, with the other Linux changes MegaCli
will run if you mount linprocfs & linsysfs then set
sysctl compat.linux.osrelease=2.6.12 or similar. This works
on i386. It should work on amd64 but not well tested yet.
StoreLib may or may not work. Remember to kldload mfi_linux.
- Add in AEN (Async Event Notification) support so we can
get messages from the firmware when something happens.
Not all messages are in defined in event detail. Use
event_log to try to figure out what happened.
- Try to implement something like SIGIO for StoreLib. Since
mrmonitor doesn't work right I can't fully test it. StoreLib
works best with the rh9 base. In theory mrmonitor isn't
needed due to native driver support of AEN :-)
Now we can configure and monitor the RAID better.

Submitted by: IronPort Systems.

Add a driver for the new LSI MegaRAID SAS controller family. The 'MFI' name
is derived from the phrase 'MegaRAID Firmware Interface' used by LSI. This
driver provides a block interface to logical disks on the card and a minimal
management device. It is MPSAFE, INTR_FAST, and 64-bit capable.

Thanks to Dell for providing hardware to test with and IronPort for
sponsoring the work.

Sponsored by: Dell, Ironport
MFC After: 3 days