MFC r258871:

Properly report an error instead of panicing when user tries to create
LUN backed by non-disk device, e.g. /dev/null.

Approved by: re (glebius)
Sponsored by: The FreeBSD Foundation

- Copy stable/10 (r259064) to releng/10.0 as part of the
10.0-RELEASE cycle.
- Update __FreeBSD_version [1]
- Set branch name to -RC1

[1] 10.0-CURRENT __FreeBSD_version value ended at '55', so
start releng/10.0 at '100' so the branch is started with
a value ending in zero.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r256552:
Unify periph invalidation and destruction reporting.
Print message containing device model and serial number on invalidation.

Approved by: re (hrs)

MFC r256533:
Unhide "Serial Number" lines from bootverbose. That information may
be useful for system administration to have in hard copy (in logs) if
one of several devices suddenly dies.

Approved by: re (hrs)

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Remove no longer useful debugging output and a stale comment.

Approved by: re (gjb)
Sponsored by: FreeBSD Foundation

Make the error handling more consistant. Shouldn't make any functional
difference.

Approved by: re (gjb)
Sponsored by: FreeBSD Foundation

Tidy up, cache return value of a function, and add an assertion;
shouldn't make any functional difference.

Approved by: re (gjb)
Sponsored by: FreeBSD Foundation

Close the race on path ID allocation in xpt_bus_register() if two buses are
registered simultaneously. Due to topology unlock between the ID allocation
and the bus registration there is a chance that two buses may get the
same IDs. That is supposed reason of lock assertion panic in CAM during
initial bus scanning after new iscsid initiates two sessions same time.

Reported by: trasz
Approved by: re (glebus, marius)
MFC after: 2 weeks

Fix NOP-In/NOP-Out payload handling. Previous way didn't work at all; fortunately
nothing seems to actually use this feature, but it's required by standard.

Approved by: re (glebius)
Sponsored by: FreeBSD Foundation

Properly fix out of memory handling in the iSCSI target.

Approved by: re (glebius)
Sponsored by: FreeBSD Foundation

Split cfiscsi_datamove() in two; no functional changes.

Approved by: re (glebius)
Sponsored by: FreeBSD Foundation

Don't leak memory when removing an unconnected session, and remove useless
UMA_ZONE_NOFREE that caused another leak when unloading the module.

Approved by: re (glebius)
Sponsored by: FreeBSD Foundation

Make sure the CCB xflags field is initialized to zero so that
CAM_EXTLUN_VALID is not erroneously set. Also add an XPORT_SRP
identifier to the known SCSI transports for the SCSI RDMA protocol, as
used, for example with Infiniband storage.

Reviewed by: scottl
Approved by: re (marius)

Re-do r255853. Along with adding back the API/ABI changes from the
original, this hides the contents of cam_compat.h from ktrace/kdump/truss,
avoiding problems there. There are no user-servicable parts in there, so
no need for those tools to be groping around in there.

Approved by: re

Revert r255853 pending fixes to build errors in usr.bin/kdump

Approved by: re (implicit)

Update the CAM API for FreeBSD 10:

- Remove the timeout_ch field. It's been deprecated since FreeBSD 7.0;
MPSAFE drivers should be managing their own timeout storage. The
remaining non-MPSAFE drivers have been modified to also manage their own
storage, and should be considered for updating to MPSAFE (or removal)
during the FreeBSD 10.x lifecycle.

- Add fields related to soft timeouts and quality of service, to be used
in upcoming work.

- Add room for more flags in the CCB header and path_inq structures.

- Begin support for extended 64-bit LUNs.

- Bump the CAM version number to 0x18, but add compat shims. Tested with
camcontrol and smartctl.

Reviewed by: nathanw, ken, kib
Approved by: re
Obtained from: Netflix

Properly ignore PDUs with CmdSN outside of allowed range.

Approved by: re (glebius)
Sponsored by: FreeBSD Foundation

Fix a few instances of M_WAITOK in threads marked as prohibited from sleep,
missed in r255824.

Approved by: re (kib)
Sponsored by: FreeBSD Foundation

Don't use M_WAITOK when running from context where sleeping is prohibited,
such as callout or a geom thread.

Approved by: re (marius)
Sponsored by: FreeBSD Foundation

Add some spare fields to structs used by the new iSCSI stack - some just
in case, some for future MC/S support.

This requires kernel and world rebuild.

Approved by: re (blanket)
Sponsored by: FreeBSD Foundation

Bring in the new iSCSI target and initiator.

Reviewed by: ken (parts)
Approved by: re (delphij)
Sponsored by: FreeBSD Foundation

Make SES driver adequately react on simple enclosure devices -- read Short
Enclosure status to enclosure status field, clear previous state and exit.

Add camcontrol support for the SCSI sanitize command

Reviewed by: ken, mjacob (eariler version)
Sponsored by: Netapp

Fix kernel panic if cache->nelms is zero.

MFC after: 2 weeks

Add debug trace points for freeze/release device queue.

Bring legacy CAM target implementation back into API/KPI-coherent and even
functional state. While CTL is much more superior target from all points,
there is no reason why this code should not work.

Tested with ahc(4) as target side HBA.

MFC after: 2 weeks

Fix SES_ENABLE_PASSTHROUGH kernel option, unexpectedly broken during driver
overhaul.

MFC after: 3 days

Fix targbh crash on XPT_IMMED_NOTIFY error during attach.

Fix the build with CTLFEDEBUG, broken by unmapped I/O support changes.

Bump up the default timeouts for move commands in the ch(4) driver
to 15 minutes, and 5 minutes for things like READ ELEMENT STATUS.

This is needed to account for the worst case scenarios on at least
some Spectra Logic tape libraries.

Sponsored by: Spectra Logic
MFC after: 3 days

If a drive returns ASC/ASCQ 0x04,0x11 "Logical unit not ready,
notify (enable spinup) required", instead of doing the normal
retries, poll for a change in status.

We will poll every half second for a minute for the status to
change.

Hitachi drives (and likely other SAS drives) return that ASC/ASCQ
when they are waiting to spin up. What it means is that they are
waiting for the SAS expander to send them the SAS
NOTIFY (ENABLE SPINUP) primitive.

That primitive is the mechanism expanders/enclosures use to
sequence drive spinup to avoid overloading power supplies.

Sponsored by: Spectra Logic
MFC after: 3 days

Add new attribute lunname to report only textual LUN-specific device IDs.
While lunid attribute prefers to report numeric ones, having both may be
useful in some situations.

Add support to physio(9) for devices that don't want I/O split and
configure sa(4) to request no I/O splitting by default.

For tape devices, the user needs to be able to clearly understand
what blocksize is actually being used when writing to a tape
device. The previous behavior of physio(9) was that it would split
up any I/O that was too large for the device, or too large to fit
into MAXPHYS. This means that if, for instance, the user wrote a
1MB block to a tape device, and MAXPHYS was 128KB, the 1MB write
would be split into 8 128K chunks. This would be done without
informing the user.

This has suboptimal effects, especially when trying to communicate
status to the user. In the event of an error writing to a tape
(e.g. physical end of tape) in the middle of a 1MB block that has
been split into 8 pieces, the user could have the first two 128K
pieces written successfully, the third returned with an error, and
the last 5 returned with 0 bytes written. If the user is using
a standard write(2) system call, all he will see is the ENOSPC
error. He won't have a clue how much actually got written. (With
a writev(2) system call, he should be able to determine how much
got written in addition to the error.)

The solution is to prevent physio(9) from splitting the I/O. The
new cdev flag, SI_NOSPLIT, tells physio that the driver does not
want I/O to be split beforehand.

Although the sa(4) driver now enables SI_NOSPLIT by default,
that can be disabled by two loader tunables for now. It will not
be configurable starting in FreeBSD 11.0. kern.cam.sa.allow_io_split
allows the user to configure I/O splitting for all sa(4) driver
instances. kern.cam.sa.%d.allow_io_split allows the user to
configure I/O splitting for a specific sa(4) instance.

There are also now three sa(4) driver sysctl variables that let the
users see some sa(4) driver values. kern.cam.sa.%d.allow_io_split
shows whether I/O splitting is turned on. kern.cam.sa.%d.maxio shows
the maximum I/O size allowed by kernel configuration parameters
(e.g. MAXPHYS, DFLTPHYS) and the capabilities of the controller.
kern.cam.sa.%d.cpi_maxio shows the maximum I/O size supported by
the controller.

Note that a better long term solution would be to implement support
for chaining buffers, so that that MAXPHYS is no longer a limiting
factor for I/O size to tape and disk devices. At that point, the
controller and the tape drive would become the limiting factors.

sys/conf.h: Add a new cdev flag, SI_NOSPLIT, that allows a
driver to tell physio not to split up I/O.

sys/param.h: Bump __FreeBSD_version to 1000049 for the addition
of the SI_NOSPLIT cdev flag.

kern_physio.c: If the SI_NOSPLIT flag is set on the cdev, return
any I/O that is larger than si_iosize_max or
MAXPHYS, has more than one segment, or would have
to be split because of misalignment with EFBIG.
(File too large).

In the event of an error, print a console message to
give the user a clue about what happened.

scsi_sa.c: Set the SI_NOSPLIT cdev flag on the devices created
for the sa(4) driver by default.

Add tunables to control whether we allow I/O splitting
in physio(9).

Explain in the comments that allowing I/O splitting
will be deprecated for the sa(4) driver in FreeBSD
11.0.

Add sysctl variables to display the maximum I/O
size we can do (which could be further limited by
read block limits) and the maximum I/O size that
the controller can do.

Limit our maximum I/O size (recorded in the cdev's
si_iosize_max) by MAXPHYS. This isn't strictly
necessary, because physio(9) will limit it to
MAXPHYS, but it will provide some clarity for the
application.

Record the controller's maximum I/O size reported
in the Path Inquiry CCB.

sa.4: Document the block size behavior, and explain that
the option of allowing physio(9) to split the I/O
will disappear in FreeBSD 11.0.

Sponsored by: Spectra Logic

CTL changes required for iSCSI target, most notably LUN remapping
and a mechanism to allow CTL frontends for retrieving LUN options.

Reviewed by: ken (earlier version)

Fix the (unused for now) SCSI_PROTO_iSCSI define to match style(9).

Add unmapped I/O and larger I/O support to the sa(4) driver.

We now pay attention to the maxio field in the XPT_PATH_INQ CCB,
and if it is set, propagate it up to physio via the si_iosize_max
field in the cdev structure.

We also now pay attention to the PIM_UNMAPPED capability bit in the
XPT_PATH_INQ CCB, and set the new SI_UNMAPPED cdev flag when the
underlying SIM supports unmapped I/O.

scsi_sa.c: Add unmapped I/O support and propagate the SIM's
maximum I/O size up.

Adjust scsi_tape_read_write() in the same way that
scsi_read_write() was changed to support unmapped
I/O. We overload the readop parameter with bits
that tell us whether it's an unmapped I/O, and we
need to set the CAM_DATA_BIO CCB flag. This change
should be backwards compatible in source and
binary forms.

MFC after: 1 week
Sponsored by: Spectra Logic

Turn comments about locking into actual lock assertions.

Reviewed by: ken
Tested by: ken
MFC after: 1 month

Added 4K quirks for:-
* OCZ Agility 2 SSDs
* Marvell SSDs
* Intel X25-M Series SSDs

Remove droping topology mutex after iterating 100 periphs in CAMGETPASSTHRU.
That is not so slow and so often operation to handle unneeded otherwise
xsoftc.xpt_generation and respective locking complications.

Improve r253721 by reporting detected lack of BIO_FLUSH support to GEOM.
That prevents more of such requests from coming and errors from logging.

Remove dead code.

MFprojects/camlock r249006:
Pass SIM pointer as an argument to camisr_runqueue() instead of doneq
pointer.

MFprojects/camlock r249505:
Change CCB queue resize logic to be able safely handle overallocations:
- (re)allocate queue space in power of 2 chunks with 64 elements minimum
and never shrink it; with only 4/8 bytes per element size is insignificant.
- automatically reallocate the queue to double size if it is overflowed.
- if queue reallocation failed, store extra CCBs in unsorted TAILQ,
fetching them back as soon as some queue element is freed.

To free space in CCB for TAILQ linking, change highpowerq from keeping
high-power CCBs to keeping devices frozen due to high-power CCBs.

This encloses all pieces of queue resize logic inside of cam_queue.[ch],
removing some not obvious duties from xpt_release_ccb().

Add NO_RC16 quirk to make da driver avoid using READ CAPACITY(16) command
if possible. Use it for Kingston JetFlash USB sticks, that are known to
return garbage in response to that command.

Fix returning incorrect bio_resid value with failed BIO_DELETE requests.
Neither residual length reported for ATA/SCSI command nor one from another
BIO_DELETE request are in any way related to the value to be returned.

Synchronize device cache on close only if there were some write operations.
While these operations are not really needed otherwise, at least for SCSI
they may cause extra errors if some other initiator holds write exclusive
reservation on the LUN (SYNCHRONIZE CACHE handled as "write" operation).

Oops, revert unwanted part of r253721.

Detect unsupported PREVENT ALLOW MEDIUM REMOVAL and SYNCHRONIZE CACHE(10)
to not spam devices with useless commands and logs with errors.

CAM and mps(4) driver scanning changes.

Add a PIM_NOSCAN flag to the CAM path inquiry CCB. This tells CAM
not to perform a rescan on a bus when it is registered.

We now use this flag in the mps(4) driver. Since it knows what
devices it has attached, it is more efficient for it to just issue
a target rescan on the targets that are attached.

Also, remove the private rescan thread from the mps(4) driver in
favor of the rescan thread already built into CAM. Without this
change, but with the change above, the MPS scanner could run before
or during CAM's initial setup, which would cause duplicate device
reprobes and announcements.

sys/param.h:
Bump __FreeBSD_version to 1000039 for the inclusion of the
PIM_RESCAN CAM path inquiry flag.

sys/cam/cam_ccb.h:
sys/cam/cam_xpt.c:
Added a PIM_NOSCAN flag. If a SIM sets this in the path
inquiry ccb, then CAM won't rescan the bus in
xpt_bus_regsister.

sys/dev/mps/mps_sas.c
For versions of FreeBSD that have the PIM_NOSCAN path
inquiry flag, don't freeze the sim queue during scanning,
because CAM won't be scanning this bus. Instead, hold
up the boot. Don't call mpssas_rescan_target in
mpssas_startup_decrement; it's redundant and I don't
know why it was in there.

Set PIM_NOSCAN in path inquiry CCBs.

Remove methods related to the internal rescan daemon.

Always use async events to trigger a probe for EEDP support.
In older versions of FreeBSD where AC_ADVINFO_CHANGED is
not available, use AC_FOUND_DEVICE and issue the
necessary READ CAPACITY manually.

Provide a path to xpt_register_async() so that we only
receive events for our own SCSI domain.

Improve error reporting in cases where setup for EEDP
detection fails.

sys/dev/mps/mps_sas.h:
Remove softc flags and data related to the scanner thread.

sys/dev/mps/mps_sas_lsi.c:
Unconditionally rescan the target whenever a device is added.

Sponsored by: Spectra Logic
MFC after: 1 week

Make some improvements to r253322 to really rescan target, not a bus.
Add there and in two more places checks for NULL on xpt_alloc_ccb_nowait().

Fix an argument reversal in calls to scsi_read_element_status().

Reported by: Ulrich Spoerlein <uqs@FreeBSD.org>
MFC after: 3 days

When printing opcode description, map T_NODEVICE to Direct Access Device to
handle REPORT LUNS, etc.

Improve handling of 0x3F/0x0E "Reported LUNs data has changed" and 0x25/0x00
"Logical unit not supported" errors. First initiates specific target rescan,
second -- destroys specific LUN. That allows to automatically detect changes
in list of device LUNs. This mechanism doesn't work when target is completely
idle, but probably that is all what can be done without active polling.

Reviewed by: ken
Sponsored by: iXsystems, Inc.

Const-ify the new da_delete_functions.
Remove a redundant sanity check

Submitted by: Steven Hartland
Obtained from: Netflix
MFC after: 3 days

Fix a problem with READ ELEMENT STATUS that occurs on some
changers that don't support the DVCID and CURDATA bits that were
introduced in the SMC spec.

These changers will return an Illegal Request type error if the
bits are set. This causes "chio status" to fail.

The fix is two-fold. First, for changers that claim to be SCSI-2
or older, don't set the DVCID and CURDATA bits for READ ELEMENT
STATUS. For newer changers (SCSI-3 and newer), we default to
setting the new bits, but back off and try the READ ELEMENT STATUS
without the bits if we get an Illegal Request type error.

This has been tested on a Qualstar TLS-8211, which is a SCSI-2
changer that does not support the new bits, and a Spectra T-380,
which is a SCSI-3 changer that does support the new bits. In the
absence of a SCSI-3 changer that does not support the bits, I
tested that with some error injection code. (The SMC spec says
that support for CURDATA is mandatory, and DVCID is optional.)

scsi_ch.c: Add a new quirk, CH_Q_NO_DVCID that gets set for
SCSI-2 and older libraries, or newer libraries that
report errors when the DVCID/CURDATA bits are set.

In chgetelemstatus(), use the new quirk to
determine whether or not to set DVCID and CURDATA.
If we get an error with the bits set, back off and
try without the bits. Set the quirk flag if the
read element status succeeds without the bits set.

Increase the READ ELEMENT STATUS timeout to 60
seconds after testing with a Spectra T-380. The
previous value was 10 seconds, and too short for
the T-380. This may be decreased later after
some additional testing and investigation.

Tested by: Andre Albsmeier <Andre.Albsmeier@siemens.com>
Sponsored by: Spectra Logic
MFC after: 3 days

Refactor the various delete methods out of dastart(). Cleans up a bunch
of style and adds more modularity and clarity.

Obtained from: Netflix
MFC after: 3 days

Added 4K QUIRK for OCZ Vertex 4 SSDs

Submitted by: Borja Marcos <borjam@sarenet.es>
MFC after: 2 days

Make it little bit more C++ friendly. This explicit casting fixes some
ports, emulators/virtualbox-ose and sysutils/smartmontools for example.

Bump disk(9) ABI version to signify the addition of d_delmaxsize by r249940.

Ensure that d_delmaxsize is always set, removing init to 0 which could cause
future issues if use cases change.

Allow kern.cam.da.X.delete_max (which maps to d_delmaxsize) to be increased
up to the calculated max after being reduced.

MFC after: 1 day
X-MFC-With: r249940

Fix NULL-deference kernel panic on attempt of destroying non-existing
ramdisk-backed CTL LUN.

Introduce accessors for the ccb status word. Convert one (of many more)
modules to use it, will convert the others once the appropriate shed
color is selected by consensus.

Obtained from: Netflix
MFC after: 3 days

Fix some UTF-8 chars slipped into r252204 via copy/paste.

Add bunch of names for Seagate and HGST vennor-specififc ASC/ASCQ codes.

Corrected ATA Passthrough defines from decimal to hex

Reviewed by: scottl
MFC after: 1 week

Fix an unfortunate typo with the compat shims

Obtained from: Netflix
MFC after: now

Add infrastructure for doing compatibility shims, as has been sorely
needed for the last 10 years. Far too much of the internal API is
exposed, and every small adjustment causes applications to stop working.
To kick this off, bump the API version to 0x17 as should have been done
with r246713, but add shims to compensate. Thanks to the shims, there
should be no visible change in application behavior.

I have plans to do a significant overhaul of the API to harnen it for
the future, but until then, I welcome others to add shims for older
versions of the API.

Obtained from: Netflix

Restore use of polling mode for disk cache flush in case of kernel panic.
While I am not sure that any extra hardware access is a good idea after
panic, that is an existing behaviour that should better work correctly.

Revert r251649:
ken@ noticed that with recently added d_gone() disk method GEOM already
holds reference on the periph, so we don't need another one.

Make CAM return and GEOM DISK pass through new GEOM::lunid attribute.

SPC-4 specification states that serial number may be property of device,
but not a specific logical unit. People reported about FC storages using
serial number in that way, making it unusable for purposes of LUN multipath
detection. SPC-4 states that designators associated with logical unit from
the VPD page 83h "Device Identification" should be used for that purpose.
Report first of them in the new attribute in such preference order: NAA,
EUI-64, T10 and SCSI name string.

While there, make GEOM DISK properly report GEOM::ident in XML output also
using d_getattr() method, if available. This fixes serial numbers reporting
for SCSI disks in `geom disk list` output and confxml.

Discussed with: gibbs, ken
Sponsored by: iXsystems, Inc.
MFC after: 2 weeks

Acquire periph reference when handling d_getattr() method call.

While GEOM in general has provider opened while sending BIO_GETATTR,
GEOM DISK does not really need to open disk to read medium-unrelated
attributes for own use.

Proposed by: ken

Simplify the checking of flags for cam_periph_mapmem(). This gets rid of
a lot of code redundancy and grossness at very minor expense.

Reviewed by: smh
Obtained from: Netflix
MFC after: 3 days

Added missing SCSI quirks from r241784

Re-ordered SSD quirks alphabetically so they are easier to maintain.

Removed my email and PR reference from comments on each quirk.

Added quirks for more SSDs:
* Crucial M4
* Corsair Force GT
* Intel 520 Series
* Kingston E100 Series
* Samsung 830 Series

Reviewed by: pjd (mentor)
Approved by: pjd (mentor)
MFC after: 1 week

Enforce validation on the selected delete method via sysctl.

This prevents users from selecting a delete method which may cause
corruption e.g. MPS WS16 on pre P14 firmware.

Reviewed by: pjd (mentor)
Approved by: pjd (mentor)
MFC after: 2 days

Added output of device QUIRKS for CAM and AHCI devices during boot.

Reviewed by: mav
Approved by: pjd (mentor)
MFC after: 2 weeks

Suppress error printing for "PREVENT ALLOW MEDIUM REMOVAL" on da open.
Change at r250208 exposed more errors here, hidden before. The same flag
is used in cd driver.

Intel's 320-series and 510-series SSDs advertise 512-byte sectors
sizes for both logical and physical. Add ADA_Q_4K quirks
for both.

PR: kern/178040
Submitted by: Jeremy Chadwick <jdc@koitsu.org>

Disable sending Early R_OK on SiI3726/SiI3826 port multipliers.
With "cached read" HDD testing and multiple ports busy on a SATA
host controller, 3726/3826 PMP will very rarely drop a deferred
R_OK that was intended for the host. Symptom will be all 5 drives
under test will timeout, get reset, and recover.

Submitted by: Rich Futyma <rich.futyma@sanmina.com>
MFC after: 2 weeks

Fxi a bunch of typos.

PR: misc/174625
Submitted by: Jeremy Chadwick <jdc@koitsu.org>

Rework r250298 in more correct way.

Fix byte order of ATA WWN when converting it to SCSI LUN ID.

Tune support for removable media in da driver:
- remove DA_FLAG_SAW_MEDIA flag, almost opposite to DA_FLAG_PACK_INVALID,
using the last instead.
- allow opening device with no media present, reporting zero media size
and non-zero sector size, as geom/notes suggests. That allow to read
device attributes and potentially do other things, not related to media.

Enable CAM SCSI to choice ATA TRIM during autodetection and correct method
names after increasing the priority of ATA TRIM.

Reviewed by: mav
Approved by: pjd (mentor)
MFC after: 1 week

Use the existence of ATA Information VPD to determine if we should attempt
to query ATA functionality via ATA Pass-Through (16) as this page is defined
as "must" for SATL devices, hence indicating that the device is at least
likely to support Pass-Through (16).

This eliminates errors produced by CTL when ATA Pass-Through (16) fails.

Switch ATA probe daerror call to SF_NO_PRINT to avoid errors printing out
for devices which return invalid errors.

Output details about supported and choosen delete method when verbose booted.

Reviewed by: mav
Approved by: pjd (mentor)
MFC after: 1 week

Fix probe in progress check in dareprobe

Reviewed by: mav
Approved by: pjd (mentor)
MFC after: 1 week

Update probe flow so that devices with lbp can also disable disksort.

Ensure that delete_available is reset so re-probes after a media change,
to one with different delete characteristics, will result in the correct
methods being flagged as available.

Make all ccb state changes use a consistent flow:
* free()
* xpt_release_ccb()
* softc->state = <new state>
* xpt_schedule()

Reviewed by: mav
Approved by: pjd (mentor)
MFC after: 1 week

Correct a few sizeof()s

Submitted by: swildner@DragonFlyBSD.org
Reviewed by: alfred

Correct comment typo's
Add missing comment

Reviewed by: pjd (mentor)
Approved by: pjd (mentor)
MFC after: 2 weeks

Add some cam_freeze_devq()'s missed at r249466.
This makes number of freezes match the number of releases.

Reported by: dim

MFprojects/camlock r249542:
Remove ADA_FLAG_PACK_INVALID flag. Since ATA disks have no concept of media
change it only duplicates CAM_PERIPH_INVALID flag, so we can use last one.

Slightly cleanup DA_FLAG_PACK_INVALID use.

MFprojects/camlock r249541:
Give periph validity flag own periph reference. That slightly simplifies
the release logic and covers hypothetical case if lock is dropped inside
the periph_oninval() method.

Added automatic detection of non-rotating media which disables the
use of BIO queue sorting, hence optimising performance for devices
such as SSD's

Reviewed by: scottl
Approved by: pjd (mentor)
MFC after: 2 weeks

Teach GEOM and CAM about the difference between the max "size" of r/w and delete
requests.

sys/geom/geom_disk.h:
- Added d_delmaxsize which represents the maximum size of individual
device delete requests in bytes. This can be used by devices to
inform geom of their size limitations regarding delete operations
which are generally different from the read / write limits as data
is not usually transferred from the host to physical device.

sys/geom/geom_disk.c:
- Use new d_delmaxsize to calculate the size of chunks passed through to
the underlying strategy during deletes instead of using read / write
optimised values. This defaults to d_maxsize if unset (0).

- Moved d_maxsize default up so it can be used to default d_delmaxsize

sys/cam/ata/ata_da.c:
- Added d_delmaxsize calculations for TRIM and CFA

sys/cam/scsi/scsi_da.c:
- Added re-calculation of d_delmaxsize whenever delete_method is set.

- Added kern.cam.da.X.delete_max sysctl which allows the max size for
delete requests to be limited. This is useful in preventing timeouts
on devices who's delete methods are slow. It should be noted that
this limit is reset then the device delete method is changed and
that it can only be lowered not increased from the device max.

Reviewed by: mav
Approved by: pjd (mentor)

Added available delete methods discovery during device probe, including the
maximum sizes for said methods, which are used when processing BIO_DELETE
requests. This includes updating UNMAP support discovery to be based on
SBC-3 T10/1799-D Revision 31 specification.

Added ATA TRIM support to cam scsi devices via ATA Pass-Through(16)

sys/cam/scsi/scsi_da.c:
- Added ATA Data Set Management TRIM support via ATA Pass-Through(16)
as a delete_method

- Added four new probe states used to identity available methods and their
limits for the processing of BIO_DELETE commands via both UNMAP and the
new ATA TRIM commands.

- Renamed Probe states to better indicate their use

- Added delete method descriptions used when informing user of issues.

- Added automatic calculation of the optimum delete mode based on which
method presents the largest maximum request size as this is most likely
to result in the best performance.

- Added WRITE SAME max block limits

- Updated UNMAP range generation to mirror that used by ATA TRIM, this
optimises the generation of ranges and fixes a potential overflow
issue in the count when combining multiple BIO_DELETE requests

- Added output of warnings about short deletes. This should only ever
be triggered on devices that fail to correctly advertise their supported
delete modes / max sizes.

- Fixed WS16 requests being incorrectly limited to 65535 in length.

Reviewed by: mav
Approved by: pjd (mentor)
MFC after: 2 weeks

Refactored scsi_xpt use of device_has_vpd to generic scsi_vpd_supported_page
so its available for use in generic scsi code.

This is a pre-requirement for using VPD queries to determine available SCSI
delete methods within scsi_da.

Reviewed by: mav
Approved by: pjd (mentor)
MFC after: 2 weeks

Updated TRIM calculations in cam/ata to be based off ATA_DSM_* defines

Reviewed by: mav
Approved by: pjd (mentor)
MFC after: 2 weeks

Added the ability to send ATA identify and Data Set Management (DSM) TRIM
commands to an ATA device attached via a SCSI control.

sys/cam/scsi/scsi_all.c:
- Added scsi_ata_identify, scsi_ata_trim
Which use ATA Pass-Through to send commands to the attached disk.

sys/cam/scsi/scsi_all.h:
- Added defines for all missing ATA Pass-Through commands values.

- Added scsi_ata_identify, scsi_ata_trim methods used in ATA TRIM
support.

- Added scsi_vpd_logical_block_prov structure used when querying for
the supported sizes UNMAP commands.

- Added scsi_vpd_block_limits structure used when querying for the
supported sizes of the UNMAP command.

Reviewed by: mav
Approved by: pjd (mentor)
MFC after: 2 weeks

Removed unneeded tests in dadeletemethodset changing it to return void

Reviewed by: mav
Approved by: pjd (mentor)
MFC after: 2 weeks

Fix compilation.

Pointy hat to: ken

Expose CAM_BOOT_DELAY as a kernel conf item now.

This allows users who boot without loader to adjust their environments
around slightly buggy or slow hardware.

PR: kern/161809
Submitted by: rozhuk.im@gmail.com
MFC after: 2 weeks

Update chio(1) and ch(4) to support reporting element designators.

This allows mapping a tape drive in a changer (as reported by
'chio status') to a sa(4) driver instance by comparing the
serial numbers.

The designators can be ASCII (which is printed out directly), binary
(which is printed in hex format) or UTF-8, which is printed in either
native UTF-8 format if the terminal can support it, or in %XX notation
for non-ASCII characters. Thanks to Hiroki Sato <hrs@> for the
explaining UTF-8 printing and example UTF-8 printing code.

chio.h: Modify the changer_element_status structure to add new
fields and definitions from the SMC3r16 spec.

Rename the original CHIOGSTATUS ioctl to OCHIOGTATUS and
define a new CHIOGSTATUS ioctl.

Clean up some tab/space issues.

chio.c: For the 'status' subcommand, print the designator field
if it is supplied by a device.

scsi_ch.h: Add new flags for DVCID and CURDATA to the READ
ELEMENT STATUS command structure.

Add a read_element_status_device_id structure
for the data fields in the new standard. Add new
unions, dt_or_obsolete and voltage_devid, to hold
and address data from either SCSI-2 or newer devices.

scsi_ch.c: Implement support for fetching device IDs with READ
ELEMENT STATUS data.

Add new arguments to scsi_read_element_status() to
allow the user to request the DVCID and CURDATA bits.
This isn't compiled into libcam (it's only an internal
kernel interface), so we don't need any special
handling for the API change.

If the user issues the new CHIOGSTATUS ioctl, copy all of
the available element status data out. If he issues the
OCHIOGSTATUS ioctl, we don't copy the new fields in the
structure.

Fix a bug in chopen() that would result in the peripheral
never getting unheld if chgetparams() failed.

Sponsored by: Spectra Logic
Submitted by: Po-Li Soong
MFC After: 1 week

- Corrrect mispellings of word useful

Submitted by: Christoph Mallon <christoph.mallon@gmx.de> (via private mail)

- Correct mispellings of the word occurrence

Submitted by: Christoph Mallon <christoph.mallon@gmx.de> (via private mail)

Remove some more pieces of multilevel freeze mechanism, missed in r249466.

Remove owner field from struct cam_ed, unused at least since FreeBSD 7.

MFprojects/camlock r248982:
Stop abusing xpt_periph in random plases that really have no periph related
to CCB, for example, bus scanning. NULL value is fine in such cases and it
is correctly logged in debug messages as "noperiph". If at some point we
need some real XPT periphs (alike to pmpX now), quite likely they will be
per-bus, and not a single global instance as xpt_periph now.

MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905,
r248917, r248918, r248978, r249001, r249014, r249030:

Remove multilevel freezing mechanism, implemented to handle specifics of
the ATA/SATA error recovery, when post-reset recovery commands should be
allocated when queues are already full of payload requests. Instead of
removing frozen CCBs with specified range of priorities from the queue
to provide free openings, use simple hack, allowing explicit CCBs over-
allocation for requests with priority higher (numerically lower) then
CAM_PRIORITY_OOB threshold.

Simplify CCB allocation logic by removing SIM-level allocation queue.
After that SIM-level queue manages only CCBs execution, while allocation
logic is localized within each single device.

Suggested by: gibbs

MFprojects/camlock r248894:
Use full freeze while PMP does hard reset. This is only cosmetical change.

Remove ctl(4) from GENERIC. Also remove 'options CTL_DISABLE'
and kern.cam.ctl.disable tunable; those were introduced as a workaround
to make it possible to boot GENERIC on low memory machines.

With ctl(4) being built as a module and automatically loaded by ctladm(8),
this makes CTL work out of the box.

Reviewed by: ken
Sponsored by: FreeBSD Foundation

Do not sent 120 TEST UNIT READY requests on generic NOT READY statuses.

Some failing disks tend to return vendor-specific ASC/ASCQ codes with
NOT READY sense key. It caused extremely long recovery attempts, repeating
these 120 TURs (it takes at least 1 minute) for every I/O request.
Instead of that use default error handling, doing just few retries.

Reviewed by: ken, gibbs
MFC after: 1 month

Add a callback to the ada(4) driver so that it knows when GEOM has released
references to it.

This is the functional equivalent to change r237518, which added this
functionality to the cd(4) and da(4) drivers.

This fix prevents a panic caused by GEOM calling adaopen() while the device
is going away. We now keep the device around until GEOM has finished
cleaning up its state.

ata_da.c: In adaregister(), add a d_gone callback to the GEOM disk
structure registered for the ada driver. Increment the
peripheral reference count for GEOM.

Add a new callback, adadiskgonecb(), that GEOM calls when
it is done with its resources. This callback releases the
reference acquired in adaregister().

Submitted by: Po-Li Soong
Sponsored by: Spectra Logic
MFC After: 5 days

Fix a memory leak that showed up when we delete LUNs. The memory used for
the LUN was never freed.

ctl.c: Adjust ctl_alloc_lun() to make sure we don't clear the
CTL_LUN_MALLOCED flag.

Reported by: Sreenivasa Honnur <shonnur@chelsio.com>
Sponsored by: Spectra Logic
MFC after: 3 days

Remove extra semicolons from CAM_SIM_[UN]LOCK() macros.

- With the demise of !ATA_CAM, ATA_STATIC_ID is the only ata(4) related
option left but actually consumed by ada(4), so move it to opt_ada.h
and get rid of opt_ata.h.
- Fix stand-alone build of atacore(4) by adding opt_cam.h.
- Use __FBSDID.
- Use DEVMETHOD_END.
- Use NULL instead of 0 for pointers.

Replicate r245306 from SCSI to ATA. The problem didn't appear so far,
covered by multilevel freeze mechanism, but it is better to be safe.

Unbreak ATA_NO_48BIT_DMA with ATA_CAM by treating 48-bit DMA as an
optional property with PATA transport.

Reviewed by: mav
MFC after: 3 days

Make SYNCHRONIZE CACHE work with LUNs backed by device files (as opposed
to regular files, which already worked fine). With this change, it's no
longer neccessary to use "ctladm realsync off" workaround.

MFprojects/camlock:
r249017:
Some cosmetic things:
- Unify device to target insertion inside xpt_alloc_device() instead of
duplicating it three times.
- Remove extra checks for empty lists of devices and targets on release
since zero refcount check also implies it.
- Reformat code to reduce indentation.

r249103:
- Add lock assertions to every point where reference counters are modified.
- When reference counters are reaching zero, add assertions that there are
no children items left.
- Add a bit more locking to the xptpdperiphtraverse().

MFprojects/camlock r248931:
Replace some direct mutex operations with wrappers.

MFC after: 2 weeks

MFprojects/camlock r248930:
Remove extra NULL checks. d_drv1 can never be NULL during periph life cycle.

MFC after: 2 weeks

MFprojects/camlock r248928:
Move CAM_DEBUG_CDB messages from the point of queuing to the point of
sending to SIM. That allows to inspect real requests execution order,
respecting priorities, freezing, etc.

MFC after: 2 weeks

Since the CTL version in FreeBSD does not support High Availability,
ctl_is_single should always be set to 1. Make it so. Previously
it was always 0, because ctl_isc_start() never got to run.

Suggested by: ken

Remove xpt_sim_opened(), the only consumer of which was atapicam, which is
now gone.

Use xpt_lock_buses() instead of equivalent mtx_lock(&xsoftc.xpt_topo_lock)
to unify the code.

Remove all legacy ATA code parts, not used since options ATA_CAM enabled in
most kernels before FreeBSD 9.0. Remove such modules and respective kernel
options: atadisk, ataraid, atapicd, atapifd, atapist, atapicam. Remove the
atacontrol utility and some man pages. Remove useless now options ATA_CAM.

No objections: current@, stable@
MFC after: never

Fix locking problem in ctl_maintenance_in() - one cannot use M_WAITOK or call
ctl_done() with mutex held.

Reviewed by: ken
Sponsored by: FreeBSD Foundation

Add xpt_release_ccb()'s missed at r248872. That made `shutdown -p` stuck
on controller with small number of queue slots and several disks connected.

Fix sending virtual scatter/gather lists from the CTL CAM frontend
peripheral.

Sponsored by: Spectra Logic

Don't directly dereference userland pointer; instead use kernel pointer
copied in from userspace. This fixes instant panic when creating CTL LUN
on sparc64. Not a security problem, since the API is root-only.

Reviewed by: ken
Sponsored by: FreeBSD Foundation

Fix comment formatting.

Remove unused code.

Reviewed by: ken

Make it possible to build CTL as a module.

Reviewed by: ken
Sponsored by: FreeBSD Foundation

Fix panic in the error path caused by recursive acquisition of XPT topology
lock.

Reviewed by: ken

Added ATA Pass-Through support to CAM

sys/cam/scsi/scsi_all.c:
- Added scsi_ata_pass_16 method
Which use ATA Pass-Through to send commands to the attached disk.

sys/cam/scsi/scsi_all.h:
- Added defines for all missing ATA Pass-Through commands values.

- Added scsi_ata_pass_16 method.

- Fixed a comment typo while I'm here

Reviewed by: mav
Approved by: pjd (mentor)
MFC after: 2 weeks

Adds the ability to enable / disable sorting of BIO requests queued within
CAM. This can significantly improve performance particularly for SSDs
which don't suffer from seek latencies.

The sysctl / tunable kern.cam.sort_io_queues provides the systems default
setting where:-
0 = queued BIOs are NOT sorted
1 = queued BIOs are sorted (default)

Each device gets its own sysctl kern.cam.<type>.<id>.sort_io_queue
Valid values are:-
-1 = use system default (default)
0 = queued BIOs are NOT sorted
1 = queued BIOs are sorted

Note: Additional patch will look to add automatic use of none sorted queues
for none rotating media e.g. SSD's

Reviewed by: scottl
Approved by: pjd (mentor)
MFC after: 2 weeks

Unbreak compilation after r248868.

Make pre-shutdown flush and spindown routines to not use xpt_polled_action(),
but execute the commands in regular way. There is no any reason to cook CPU
while the system is still fully operational. After this change polling in
CAM is used only for kernel dumping.

Implement CAM_PERIPH_FOREACH() macro, safely iterating over the list of
driver's periphs, acquiring and releaseing periph references while doing it.

Use it to iterate over the lists of ada and da periphs when flushing caches
and putting devices to sleep on shutdown and suspend. Previous code could
panic in theory if some device disappear in the middle of the process.

On SIM destruction free associated CCBs, preallocated inside xpt_get_ccb().
Before this change they were just leaked. Fortunately USB sticks now use
only one CCB, and so leak was only 2KB per detach, while other bigger SIMs
with much more allocated CCBs are rarely detached.

MFC after: 2 weeks

Remove two bzero()s that are erasing only few more bytes then set later.

Commit the removal of a whitespace to record the proper commit message
for the r248519:

For the cam-attached HBAs, allow the driver to specify that it accepts
the unmapped bio by the PIM_UNMAPPED flag. The CAM passes the
CAM_DATA_BIO data transfer type request for the unmapped bio, and the
driver could use the bus_dmamap_load_ccb() as a helper to
transparently handle the ccb.

Sponsored by: The FreeBSD Foundation
Reviewed by: scottl
Tested by: pho, scottl

Support unmapped i/o for the md(4).

The vnode-backed md(4) has to map the unmapped bio because VOP_READ()
and VOP_WRITE() interfaces do not allow to pass unmapped requests to
the filesystem. Vnode-backed md(4) uses pbufs instead of relying on
the bio_transient_map, to avoid usual md deadlock.

Sponsored by: The FreeBSD Foundation
Tested by: pho, scottl

Do not remap usermode pages into KVA for physio.

Sponsored by: The FreeBSD Foundation
Tested by: pho

Assert that a ccb passed to cam_periph_mapmem() for XPT_SCSI_IO and
XPT_ATA_IO holds virtual buffer address.

Sponsored by: The FreeBSD Foundation
Tested by: pho

Re-enable CTL in GENERIC on i386 and amd64, but turn on the CTL disable
tunable by default.

This will allow GENERIC configurations to boot on small memory boxes, but
not require end users who want to use CTL to recompile their kernel. They
can simply set kern.cam.ctl.disable=0 in loader.conf.

The eventual solution to the memory usage problem is to change the way
CTL allocates memory to be more configurable, but this should fix things
for small memory situations in the mean time.

UPDATING: Explain the change in the CTL configuration, and
how users can enable CTL if they would like to use
it.

sys/conf/options: Add a new option, CTL_DISABLE, that prevents CTL
from initializing.

ctl.c: If CTL_DISABLE is turned on, don't initialize.

i386/conf/GENERIC,
amd64/conf/GENERIC: Re-enable device ctl, and add the CTL_DISABLE
option.

Hide SEMB port of the SiI3826 Port Multiplier by default to avoid extra
errors while it tries to talk via I2C to usually missing external SEP.
There is tunable to enable it back when needed.

Add DA_Q_NO_PREVENT quirk for Kingston DataTraveler G3 1.00 USB flash.

PREVENT ALLOW MEDIUM REMOVAL commands return errors on these devices
without returning sense data. In some cases unrelated following commands
start to return errors too, that makes device to be dropped by CAM.

Reform the busdma API so that new types may be added without modifying
every architecture's busdma_machdep.c. It is done by unifying the
bus_dmamap_load_buffer() routines so that they may be called from MI
code. The MD busdma is then given a chance to do any final processing
in the complete() callback.

The cam changes unify the bus_dmamap_load* handling in cam drivers.

The arm and mips implementations are updated to track virtual
addresses for sync(). Previously this was done in a type specific
way. Now it is done in a generic way by recording the list of
virtuals in the map.

Submitted by: jeff (sponsored by EMC/Isilon)
Reviewed by: kan (previous version), scottl,
mjacob (isp(4), no objections for target mode changes)
Discussed with: ian (arm changes)
Tested by: marius (sparc64), mips (jmallet), isci(4) on x86 (jharris),
amd64 (Fabian Keil <freebsd-listen@fabiankeil.de>)

Format CDB output as 2 digit hex correcting the length

Approved by: pjd (mentor)
MFC after: 1 week

Sanitize the element descriptor string before using it as a device name.

Reported and tested by: Vitalij Satanivskij
Reviewed by: gibbs, mav

Do not pretend to have autosense data when no such data is available.

Make umass return an error code if SCSI sense retrieval request
has failed. Make sure scsi_error_action honors SF_NO_RETRY and
SF_NO_RECOVERY in all cases, even if it cannot parse sense bytes.

Reviewed by: hselasky (umass), scottl (cam)

- Add missig xpt_schedule() call for cases when requested immediate CCB
priority is lower then payload/TUR one.

- Reduce TUR priority and avoid sending them if there are any other
outstanding commands, alike to DA driver.

Do not schedule periph for payload/TUR requests if reprobe is in progress
to avoid sending extra READ CAPACITY requests by dastart(). Schedule periph
again on reprobe completion, or otherwise it may stuck indefinitely long.

This should fix USB explore thread hanging on device unplug, waiting for
periph destruction.

Reported by: hselasky

Changed scsi_da device requests to use the sysctl tunable value for retry_count
and da_default_timeout where their current hardcoded values matched the current
default value for said tunables.

PR: kern/169976
Reviewed by: pjd (mentor)
Approved by: mav

Updates delete_method sysctl changes to always maintain disk d_flags
DISKFLAG_CANDELETE. While this change makes this layer consistent
other layers such as UFS and ZFS BIO_DELETE support may not notice
any change made manually via these device sysctls until the device
is reopened via a mount.

Also corrected var order in dadeletemethodsysctl

PR: kern/169801
Reviewed by: pjd (mentor)
Approved by: mav
MFC after: 2 weeks

Removes essentially unused variables from scsi_da probe setups

PR: kern/169835
Reviewed by: pjd (mentor)
Approved by: mav
MFC after: 2 weeks

Make CTL work a little better with loading and unloading drivers.

Previously CTL would leave individual LUNs enabled in the target
driver, whether or not the port as a whole was enabled. It would
also leave the wildcard LUN enabled indefinitely.

This change means that CTL will enable and disable any active LUNs,
as well as the wildcard LUN, when enabling and disabling a port.

Also, fix a bug that could crop up due to an uninitialized CCB
type.

ctl.c: Before calling ctl_frontend_online(), run through
the LUN list and enable all active LUNs.

After calling ctl_frontend_offline(), run through
the LUN list and disble all active LUNs.

scsi_ctl.c: Before bringing a port online, allocate the
wildcard peripheral for that bus. And after taking
a port offline, invalidate the wildcard peripheral
for that bus.

Make sure that we hold the SIM lock around all
calls to xpt_action() and other transport layer
interfaces that require it.

Use CAM_SIM_{LOCK|UNLOCK} consistently to acquire
and release the SIM lock.

Update a number of outdated comments. Some of
these should have been fixed long ago.

Actually do LUN disbables now. The newer drivers
in the tree work correctly for this as far as I
know.

Initialize the CCB type to CTLFE_CCB_DEFAULT to
avoid a panic due to uninitialized memory.

Submitted by: Chuck Tuffli (partially)
MFC after: 1 week

Make SES driver to not fall out on some errors in Additional Status page.
This allows CAM devices still get their physical paths even if status of
later elements it corrupted.

Sponsored by: iXsystems, Inc.

Fix bug in r242720, that caused additional status page to not be used if
descriptor page is supported.

Fix a couple of CTL locking issues and clean up some duplicated code.

ctl_frontend_cam_sim.c: Coalesce cfcs_online() and cfcs_offline()
into a single function since these were
identical except for one line.

Make sure we hold the SIM lock around path
creation, and calling xpt_rescan().

scsi_ctl.c: In ctlfe_onoffline(), make sure we hold the
SIM lock around path creation and free
calls, as well as xpt_action().

In ctlfe_lun_enable(), hold the SIM lock
around path and peripheral operations that
require it.

Sponsored by: Spectra Logic Corporation
MFC after: 1 week

Make sure we hold the SIM lock when calling xpt_free_path().

Sponsored by: Spectra Logic Corporation
MFC after: 1 week

Fix the CTL OOA queue dumping code so that it does not hold a mutex
while doing a copyout. That can cause a panic, because copyout
can trigger VM faults, and we can't handle VM faults while holding
a mutex.

The solution here is to malloc a separate buffer to hold the OOA
queue entries, so that we don't risk a VM fault while filling up
the buffer and we don't have to drop the lock. The other solution
would be to wire the user's memory while filling their buffer with
copyout, but that would have been a little more complex.

Also fix a debugging parenthesis issue in ctl_abort_task() pointed
out by Chuck Tuffli.

Sponsored by: Spectra Logic Corporation
MFC after: 1 week

Fix a device departure bug for the the pass(4), enc(4), sg(4) and ch(4)
drivers.

The bug occurrs when a userland process has the driver instance
open and the underlying device goes away. We get the devfs
callback that the device node has been destroyed, but not all of
the closes necessary to fully decrement the reference count on the
CAM peripheral.

The reason is that once devfs calls back and says the device has
been destroyed, it is moved off to deadfs, and devfs guarantees
that there will be no more open or close calls. So the solution
is to keep track of how many outstanding open calls there are on
the device, and just release that many references when we get the
callback from devfs.

scsi_pass.c,
scsi_enc.c,
scsi_enc_internal.h: Add an open count to the softc in these
drivers. Increment it on open and
decrement it on close.

When we get a devfs callback to say that
the device node has gone away, decrement
the peripheral reference count by the
number of still outstanding opens.

Make sure we don't access the peripheral
with cam_periph_unlock() after what might
be the final call to
cam_periph_release_locked(). The
peripheral might have been freed, and we
will be dereferencing freed memory.

scsi_ch.c,
scsi_sg.c: For the ch(4) and sg(4) drivers, add the
same changes described above, and in
addition, fix another bug that was
previously fixed in the pass(4) and enc(4)
drivers.

These drivers were calling destroy_dev()
from their cleanup routine, but that could
cause a deadlock because the cleanup
routine could be indirectly called from
the driver's close routine. This would
cause a deadlock, because the device node
is being held open by the active close
call, and can't be destroyed.

Sponsored by: Spectra Logic Corporation
MFC after: 1 week

Fix a panic during CAM EDT traversal.

The problem was a race condition between the EDT traversal used by
things like 'camcontrol devlist', and CAM peripheral driver
removal.

The EDT traversal code holds the CAM topology lock, and wants
to show devices that have been invalidated. It acquires a
reference to the peripheral to make sure the peripheral it is
examining doesn't go away.

However, because the peripheral removal code in camperiphfree()
drops the CAM topology lock to call the peripheral's destructor
routine, we can run into a situation where the EDT traversal
increments the peripheral reference count after free process is
already in progress. At that point, the reference count is
ignored, because it was 0 when we started the process.

Fix this race by setting a flag, CAM_PERIPH_FREE, that I previously
added and checked in xptperiphtraverse() and xptpdperiphtravsere(),
but failed to use. If the EDT traversal code sees that flag,
it will know that the peripheral free process has already started,
and that it should not access that peripheral.

Also, fix an inconsistency in the locking between
xptpdperiphtraverse() and xptperiphtraverse(). They now both
hold the CAM topology lock while calling the peripheral traversal
function.

cam_xpt.c: Change xptperiphtraverse() to hold the CAM topology
lock across calls to the traversal function.

Take out the comment in xptpdperiphtraverse() that
referenced the locking inconsistency.

cam_periph.c: Set the CAM_PERIPH_FREE flag when we are in the
process of freeing a peripheral driver.

Sponsored by: Spectra Logic Corporation
MFC after: 1 week

Fix problem with the Samsung 840 PRO series SSD detection.
The device reports support for SATA Asynchronous Notification in its
IDENTIFY data, but returns error on attempt to enable that feature.
Make SATA XPT of CAM only report these errors, but not fail the device.

MFC after: 1 week

Use information about suported diagnostic pages to avoid reading optional
Element Descriptor page if it is not supported. This removes one error
message from verbose logs during boot on systems with some enclosures.

Sponsored by: iXsystems, Inc.

Use M_ZERO instead of explicit memsets and bzeros.

Fix locking problem in disk_resize(); previously it would run without
topology lock, resulting in assertion when running with DIAGNOSTIC.

Reviewed by: mav (earlier version)

Remove priority enforcement from xpt_ation(). It is not good and even not
safe in some cases to reduce CCB priority after it was scheduled with high
priority. This fixes reproducible deadlock when command sent through the
pass interface while ATA XPT recovers from command timeout.

Instead of that enforce priority at passioctl(). libcam provides no obvious
interface to specify CCB priority and so much (all?) code specifies zero
(highest) priority. This change limits pass CCBs priority to NORMAL run
level, allowing XPT to complete bus and device recovery after reset before
running any payload.

Remove several uses of numeric priorities from immediate CCB setups.

Remove one more numeric priority constant.

Remove two more 'periph == NULL' checks missed in r241404.
This condition can never be true as functions are called from single place
and the checks just pollute the code and confuse Clang Static Analyzer.

Remove the support for using non-mpsafe filesystem modules.

In particular, do not lock Giant conditionally when calling into the
filesystem module, remove the VFS_LOCK_GIANT() and related
macros. Stop handling buffers belonging to non-mpsafe filesystems.

The VFS_VERSION is bumped to indicate the interface change which does
not result in the interface signatures changes.

Conducted and reviewed by: attilio
Tested by: pho

remove duplicate semicolons where possible.

Approved by: cperciva
MFC after: 1 week

Adds 4K quirks for the some SSD's which all perform better when 4K
aligned and only except 4K deletes (TRIM).

PR: kern/169974
Submitted by: Steven Hartland <steven.hartland@multiplay.co.uk>
Tested by: ak
Reviewed by: mav
Approved by: cperciva (implicit)
MFC after: 1 week

Add support for samsung HM250JI

PR: usb/121474
Submitted by: Ben Stuyts <ben@altesco.nl>
Approved by: cperciva (implicit)
MFC after: 3 days

Add support for the USB DISK Pro PMAP.

This patch has sit for 6 years in the PR database.

PR: usb/96381
Submitted by: jhs
Reviewed by: mav
Approved by: cperciva (implicit)
MFC after: 3 days

Add explicit check for not set time inside cam_periph_freeze_after_event().

System time is set later on boot process then initial bus scan by CAM.
Until that moment microtime() is equal to microuptime(), and if system
boots quickly, the value can be close to zero. That causes settle time
waiting even for buses that don't use reset during probe.

On my test system this reduces boot time by 1 second if USB enabled, or
by 4 seconds if USB disabled. CAM waited for ctl2cam0 bus "settle".

Don't exclude XPT SIM from locking in xpt_create_path_unlocked().
We don't want xpt periph, device, target or bus disappeared because of
incorrect reference counting.

Use xpt_create_path_unlocked() for initial debug path compilation because
we are not holding respective SIM lock.

Extend SIM lock coverage during the bus registration process.

Fix XPT_DEBUG paths operations locking:
- Extend the lock to cover xpt_path_release() for the new path.
- While xpt_action() is called while holding right SIM lock for the new
bus, the old path release may require different SIM lock. So we have
to temporary drop the new lock and get the old one.

XPT_DEV_MATCH is probably the only xpt_action() method that is called
without holding SIM lock. It really doesn't need that lock, but adding it
removes that specific exception, allowing to assert locking there later.

Submitted by: ken@ (earlier version)

Get SIM lock in several places while calling CAM functions.
This fixes several use-after-free panics on systems with SAS enclosures.

Submitted by: ken@, mav@

Protect xpt_getattr() calls with the SIM lock and assert that.

Submitted by: ken@ (earlier version)

Use separate malloc buckets for CAM devices, CCBs and paths. This will
make it easier to track down the source of any use after free problems.

Submitted by: ken@

Don't duplicate path/ccb allocation code, use existing functions.

Increase device CCB queue array size by CAM_RL_VALUES - 1 (4) elements.
It is required to store extra recovery requests in case of bus resets.
On ATA/SATA this fixes assertion panics on HEAD with INVARIANTS enabled or
possible memory corruptions otherwise if timeout/reset happens when device
CCB queue is already full.

Reported by: gibbs@
MFC after: 1 week

There are SCSI conditions that are not an errors. In those cases cderror()
returns zero while request status is not CAM_REQ_CMP. That could cause
partial device attach or other unexpected results.

Found by: Clang Static Analyzer

Really handle xpt_compile_path() error in xpt_bus_register() instead of
print error message and probably crash just after it on NULL dereference.

Found by: Clang Static Analyzer

Remove 'periph == NULL' check from bunch of periph drivers.
This condition can never be true as functions are called from single place
and the checks just pollute the code and confuse Clang Static Analyzer.

ata_da: set disk::d_ident from serial number

MFC after: 10 days

Change queue overflow checks from DIAGNOSTIC+panic() to KASSERT() to make
them enabled on HEAD by default. It is probably better to do single compare
then hunt for unexpected memory corruption.

Remove useless NULL checks after M_WAITOK allocations.

Fix panic in CTL caused by trying to free invalid pointers passed
by the userland process via the IOCTL interface.

Reviewed by: ken@

Fix a panic when trying to play invalid audio tracks.

s/ is is / is /g
s/ a a / a /g

Approved by: cperciva
MFC after: 3 days

Remove some more NetBSD compat shims and other unused bits from these
drivers:
- Remove scsi_low_pisa.*, they were unused.
- Remove <compat/netbsd/physio_proc.h> and calls to the stubs in that
header. They were empty nops.
- Retire sl_xname and use device_get_nameunit() and device_printf() with
the underlying device_t instead.
- Remove unused {ct,ncv,nsp,stg}print() functions.
- Remove empty SOFT_INTR_REQUIRED() macro and the unused sl_irq member.

Remove NetBSD compat shims for drivers originally shared with NetBSD/pc98.
NetBSD/pc98 was never merged into the main NetBSD tree and is no longer
developed. Adding locking to these drivers would have made the compat
shims hard to impossible to maintain, so remove the shims to ease
future changes.

These changes were verified by md5. Some additional shims can be removed
that do affect the compiled results that I will probably do in another
round.

Approved by: nyan (tentatively)

Fix scsi_da's BIO_DELETE->SCSI_UNMAP translation to use correct local
variable when determining various sizes related to SCSI UNMAP block
descriptor lists.

Sponsored by: Intel
Reviewed by: mav
MFC after: 3 days

1. Remove SEN support. I doubt there are any working examples
of this hardware still running (close to twenty years now).

2. Quiesece and use ENC_VLOG instead of ENC_LOG for most
complaints. That is, they're visible with bootverbose, but
otherwise quiesced and not repeatedly spamming messages
with constant reminders that hardware in this space is
rarely fully compliant.

MFC after: 1 month

Add missing VERIFY_10 definition.

MFC after: 1 month

Remove opt_enc.h from files committed with r235911. enc(4) is the
'encapsulating interface' used with IPsec and has nothing to do with
storage 'enclosure' services.

MFC after: 3 days
Noticed while: debugging why enc(4) is no longer automatically created

Implement media change notification for DA and CD removable media devices.
It includes three parts:
1) Modifications to CAM to detect media media changes and report them to
disk(9) layer. For modern SATA (and potentially UAS) devices it utilizes
Asynchronous Notification mechanism to receive events from hardware.
Active polling with TEST UNIT READY commands with 3 seconds period is used
for incapable hardware. After that both CD and DA drivers work the same way,
detecting two conditions: "NOT READY: Medium not present" after medium was
detected previously, and "UNIT ATTENTION: Not ready to ready change, medium
may have changed". First one reported to disk(9) as media removal, second
as media insert/change. To reliably receive second event new
AC_UNIT_ATTENTION async added to make UAs broadcasted to all periphs by
generic error handling code in cam_periph_error().
2) Modifications to GEOM core to handle media remove and change events.
Media removal handled by spoiling all consumers attached to the provider.
Media change event also schedules provider retaste after spoiling to probe
new media. New flag G_CF_ORPHAN was added to consumers to reflect that
consumer is in process of destruction. It allows retaste to create new
geom instance of the same class, while previous one is still dying.
3) Modifications to some GEOM classes: DEV -- to report media change
events to devd; VFS -- to handle spoiling same as orphan to prevent
accessing replaced media. PART class already handles spoiling alike to
orphan.

Reviewed by: silence on geom@ and scsi@
Tested by: avg
Sponsored by: iXsystems, Inc. / PC-BSD
MFC after: 2 months

Handle a case where we had an SRR that pushed back the
data pointer. This is a temp fix that resubmits the
command, adjusted, so that the backend can fetch the
data again.

Sponsored by: Spectralogic
MFC after: 1 month

Fix off by one error in ses_enc_desc_last_byte().

Do not call ses_softc_cleanup() in case of configuration read failure.
Just free inclomplete daemon cache instead to let it retry next time.
Premature ses_softc_cleanup() caused NULL dereference when freed softc
was accessed later.

Fix some typos in r238595.

Reported by: brueffer

Add bunch of new ASC/ASCQ values from T10 site.

The call to disk_resize causes a panic if DIAGNOSTIC is set.
Coping with that while the finest minds of our generation
figure out why.

Add and utilize defines for the ATA device register.

PR: 169764
Submitted by: Steven Hartland <steven.hartland@multiplay.co.uk>
Reviewed by: mav
MFC after: 2 weeks

Apply similar same change as in r238379:

Renamed the kern.cam.ada.ada_send_ordered sysctl and tunable to
kern.cam.ada.send_ordered, more in line with the other da sysctls/tunables.

Suggested by: kib

Renamed the kern.cam.da.da_send_ordered sysctl and tunable to
kern.cam.da.send_ordered, more in line with the other da sysctls/tunables.

PR: 169765
Submitted by: Steven Hartland <steven.hartland@multiplay.co.uk>
Reviewed by: mav

- fix description of SECURITY_FREEZE_LOCK (1)
- add missing underscores to SECURITY_DISABLE_PASSWORD

PR: 169763 (1)
Submitted by: Steven Hartland <steven.hartland@multiplay.co.uk>
Reviewed by: mav
MFC after: 1 week

Make the da(4) driver notify GEOM about LUN size change.

Reviewed by: mav
Sponsored by: FreeBSD Foundation

Remove variables which are initialized but never used thereafter
reported by gcc46 warning

Reviewed by: scottl
Approved by: cperciva
MFC after: 1 week

Add a loader tunable, kern.cam.ctl.disable, that will disable
loading CTL. This may be useful in very low memory installations.

MFC after: 3 days

Add a sysctl to set the cdrom timeout. Data recovery operations from
a CD or DVD drive with a damaged disc often benefit from a shorter
timeout. Also, when retries are set to 0, an application is expecting
errors and recovering them so do not print the error into the log.
The number of expected errors can literally be in the hundreds of
thousands which significantly slows data recovery.

Reviewed by: ken@ (but quite some time ago).

Fix a typo in a panic() call.

PR: kern/169497
Submitted by: Steven Hartland <steven.hartland@multiplay.co.uk>
MFC after: 3 days

Fix an issue that caused the kernel to panic inside CTL when trying
to attach to target capable HBAs that implement the old immediate
notify (XPT_IMMED_NOTIFY) and notify acknowledge (XPT_NOTIFY_ACK)
CCBs. The new API has been in place since SVN change 196008 in
2009.

The solution is two-fold: fix CTL to handle the responses from the
HBAs, and convert the HBA drivers in question to use the new API.

These drivers have not been tested with CTL, so how well they will
interoperate with CTL is unknown.

scsi_target.c: Update the userland target example code to use the
new immediate notify API.

scsi_ctl.c: Detect when an immediate notify CCB is returned
with CAM_REQ_INVALID or CAM_PROVIDE_FAIL status,
and just free it.

Fix a duplicate assignment.

aic79xx.c,
aic79xx_osm.c: Update the aic79xx driver to use the new API.
Target mode is not enabled on for this driver, so
the changes will have no practical effect.

aic7xxx.c,
aic7xxx_osm.c: Update the aic7xxx driver to use the new API.

sbp_targ.c: Update the firewire target code to work with the
new API.

mpt_cam.c: Update the mpt(4) driver to work with the new API.
Target mode is only enabled for Fibre Channel
mpt(4) devices.

MFC after: 3 days

Fix a bug which causes a panic in daopen(). The panic is caused by
a da(4) instance going away while GEOM is still probing it.

In this case, the GEOM disk class instance has been created by
disk_create(), and the taste of the disk is queued in the GEOM
event queue.

While that event is queued, the da(4) instance goes away. When the
open call comes into the da(4) driver, it dereferences the freed
(but non-NULL) peripheral pointer provided by GEOM, which results
in a panic.

The solution is to add a callback to the GEOM disk code that is
called when all of its resources are cleaned up. This is
implemented inside GEOM by adding an optional callback that is
called when all consumers have detached from a provider, and the
provider is about to be deleted.

scsi_cd.c,
scsi_da.c: In the register routine for the cd(4) and da(4)
routines, acquire a reference to the CAM peripheral
instance just before we call disk_create().

Use the new GEOM disk d_gone() callback to register
a callback (dadiskgonecb()/cddiskgonecb()) that
decrements the peripheral reference count once GEOM
has finished cleaning up its resources.

In the cd(4) driver, clean up open and close
behavior slightly. GEOM makes sure we only get one
open() and one close call, so there is no need to
set an open flag and decrement the reference count
if we are not the first open.

In the cd(4) driver, use cam_periph_release_locked()
in a couple of error scenarios to avoid extra mutex
calls.

geom.h: Add a new, optional, providergone callback that
is called when a provider is about to be deleted.

geom_disk.h: Add a new d_gone() callback to the GEOM disk
interface.

Bump the DISK_VERSION to version 2. This probably
should have been done after a couple of previous
changes, especially the addition of the d_getattr()
callback.

geom_disk.c: Add a providergone callback for the disk class,
g_disk_providergone(), that calls the user's
d_gone() callback if it exists.

Bump the DISK_VERSION to 2.

geom_subr.c: In g_destroy_provider(), call the providergone
callback if it has been provided.

In g_new_geomf(), propagate the class's
providergone callback to the new geom instance.

blkfront.c: Callers of disk_create() are supposed to pass in
DISK_VERSION, not an explicit disk API version
number. Update the blkfront driver to do that.

disk.9: Update the disk(9) man page to include information
on the new d_gone() callback, as well as the
previously added d_getattr() callback, d_descr
field, and HBA PCI ID fields.

MFC after: 5 days

Add scsi_extract_sense_ccb() -- wrapper around scsi_extract_sense_len().
It allows to remove number of duplicate checks from several places.

Change 'camcontrol defects' to first probe a drive to find out how much
defect information it has before grabbing the full defect list.

This works around a bug with some Hitachi drives that generate data overrun
errors when they are asked for more defect data than they have.

The change is done in a spec-compliant way, so it should have no negative
impact on drives that don't have this issue.

This is based on work originally done at Sandvine.

scsi_da.h: Add a define for the maximum amount of data that can be
contained in a defect list.

camcontrol.c: Update the readdefects() function to issue an initial
command to determine the length of the defect list, and
then use that length in the request for the full defect
list.

camcontrol.8: Add a note that some drives will report 0 defects available
if you don't request either the PLIST or GLIST.

Submitted by: Mark Johnston <markjdb@gmail.com> (original version)
MFC after: 3 days

Don't print SCSI Queue Full and CAM_REQUEUE_REQ statuses as errors if they
were handled and retried. They are part of normal operation for SCSI TCQ.

MFC after: 3 days

Make cam_periph_hold() behavior consistent: drop taken reference and
return ENXIO if periph was invalidated while we were waiting for it.

MFC after: 1 week

In camisr() clear CAM_SIM_ON_DONEQ flag after camisr_runqueue() purged SIM
done queue. Clearing it before caused extra SIM queueing in some cases.
It was invisible during normal operation, but during USB device unplug and
respective SIM destruction it could keep pointer on SIM without having
counted reference and as result crash the system by use afer free.

Reported by: hselasky
MFC after: 1 week

Remove unused error variables in cdclose() and daclose().

Check status of cam_periph_hold() inside cdclose(). If cd device was
invalidated while open, cam_periph_hold() will return error and won't
get the reference. Following reference release will crash the system.

Sponsored by: iXsystems, Inc.
MFC after: 3 days

Fix several reference counting and object lifetime issues between
the pass(4) and enc(4) drivers and devfs.

The pass(4) driver uses the destroy_dev_sched() routine to
schedule its device node for destruction in a separate thread
context. It does this because the passcleanup() routine can get
called indirectly from the passclose() routine, and that would
cause a deadlock if the close routine tried to destroy its own
device node.

In any case, once a particular passthrough driver number, e.g.
pass3, is destroyed, CAM considers that unit number (3 in this
case) available for reuse.

The problem is that devfs may not be done cleaning up the previous
instance of pass3, and will panic if isn't done cleaning up the
previous instance.

The solution is to get a callback from devfs when the device node
is removed, and make sure we hold a reference to the peripheral
until that happens.

Testing exposed some other cases where we have reference counting
issues, and those were also fixed in the pass(4) driver.

cam_periph.c: In camperiphfree(), reorder some of the operations.

The peripheral destructor needs to be called before
the peripheral is removed from the peripheral is
removed from the list. This is because once we
remove the peripheral from the list, and drop the
topology lock, the peripheral number may be reused.
But if the destructor hasn't been called yet, there
may still be resources hanging around (like devfs
nodes) that haven't been fully cleaned up.

cam_xpt.c: Add an argument to xpt_remove_periph() to indicate
whether the topology lock is already held.

scsi_enc.c: Acquire an extra reference to the peripheral during
registration, and release it once we get a callback
from devfs indicating that the device node is gone.

Call destroy_dev_sched_cb() in enc_oninvalidate()
instead of calling destroy_dev() in the cleanup
routine.

scsi_pass.c: Add reference counting to handle peripheral and
devfs object lifetime issues.

Add a reference to the peripheral and the devfs
node in the peripheral registration.

Don't attempt to add a physical path alias if the
peripheral has been marked invalid.

Release the devfs reference once the initial
physical path alias taskqueue run has completed.

Schedule devfs node destruction in the
passoninvalidate(), and release our peripheral
reference in a new routine, passdevgonecb() once
the devfs node is gone. This allows the peripheral
to fully go away, and the peripheral destructor,
passcleanup(), will get called.

MFC after: 3 days
Sponsored by: Spectra Logic

Remove never used CD/DA_FLAG_TAGGED_QUEUING flags.

MFC after: 1 week

One more major cam_periph_error() rewrite to improve error handling and
reporting. It includes:
- removing of error messages controlled by bootverbose, replacing them
with more universal and informative debugging on CAM_DEBUG_INFO level,
that is now built into the kernel by default;
- more close following to the arguments submitted by caller, such as
SF_PRINT_ALWAYS, SF_QUIET_IR and SF_NO_PRINT; consumer knows better which
errors are usual/expected at this point and which are really informative;
- adding two new flags SF_NO_RECOVERY and SF_NO_RETRY to allow caller
specify how much assistance it needs at this point; previously consumers
controlled that by not calling cam_periph_error() at all, but that made
behavior inconsistent and debugging complicated;
- tuning debug messages and taken actions order to make debugging output
more readable and cause-effect relationships visible;
- making camperiphdone() (common device recovery completion handler) to
also use cam_periph_error() in most cases, instead of own dumb code;
- removing manual sense fetching code from cam_periph_error(); I was told
by number of people that it is SIM obligation to fetch sense data, so this
code is useless and only significantly complicates recovery logic;
- making ada, da and pass driver to use cam_periph_error() with new limited
recovery options to handle error recovery and debugging in common way;
as one of results, CAM_REQUEUE_REQ and other retrying statuses are now
working fine with pass driver, that caused many problems before.
- reverting r186891 by raj@ to avoid burning few seconds in tight DELAY()
loops on device probe, while device simply loads media; I think that problem
may already be fixed in other way, and even if it is not, solution must be
different.

Sponsored by: iXsystems, Inc.
MFC after: 2 weeks

Add CAM_DEBUG_INFO debug messages for periph created/invalidated/destroyed
and for asyncs sent.

To make CAM debugging easier, compile in some debug flags (CAM_DEBUG_INFO,
CAM_DEBUG_CDB, CAM_DEBUG_PERIPH and CAM_DEBUG_PROBE) by default.
List of these flags can be modified with CAM_DEBUG_COMPILE kernel option.
CAMDEBUG kernel option still enables all possible debug, if not overriden.

Additional 50KB of kernel size is a good price for the ability to debug
problems without rebuilding the kernel. In case where size is important,
debugging can be compiled out by setting CAM_DEBUG_COMPILE option to 0.

Remove declaration of scsi_interpret_sense(), removed 11 years ago.

Fix a memory leak in the kernel case in scsi_command_string().

Submitted by: Kashyap Desai <Kashyap.Desai@lsi.com>
MFC after: 3 days

ATA/SATA controllers have no idea about protocol of the connected device
until transport will do some probe actions (at least soft reset).
Make ATA/SATA SIMs to not report bogus and confusing PROTO_ATA protocol.
Make ATA/SATA transport to fill that gap by reporting protocol to SIM with
XPT_SET_TRAN_SETTINGS and patching XPT_GET_TRAN_SETTINGS results if needed.

Use default error handler when flushing disk caches.

Tune and add some more CAM_DEBUG() points for the probe sequences.

Replace #ifdef CAMDEBUG + if + panic() with single KASSERT().

Do not reinvent a wheel and let default error handler do its job.

Tune and add some missing CAM_DEBUG() points for better consistency.

Remove some dead code that I doubt will ever be implemented.

Rewrite enabling NCQ for SATA devices in a way more alike to SCSI TCQ.
This allows to control it with `camcontrol negotiate adaX -T (en|dis)able`
on the fly, same as for SCSI devices.

Sponsored by: iXsystems, Inc.

Print FC PortID as a hex number. This makes it easy to
figure out domain, etc..

Zero ATIO and INOTify allocations. It makes for much
less guesswork when looking at the structure and
seeing 'deadc0de' present.

Reviewed by: kdm
MFC after: 2 weeks
Sponsored by: Spectralogic

Use AC_GETDEV_CHANGED async to notify ada driver about DMA and NCQ status
change. Now that allows switching between PIO and DMA modes on the fly.

Add support for newer garmin devices

PR: kern/163932
Submitted by: Bartosz Fabianowski <freebsd@chillt.de>
Approved by: cperciva
MFC after: 1 week

Allow to change number of openings (used tags) for ATA/SATA devices
via `camcontrol tags ... -N ...`. There is no need to tune it in
usual cases, but some users want to have it for debugging purposes.

MFC after: 2 weeks

Plug request and references leak caused by race between invalidated
ond probe periph destruction and new incoming probe request.

This at least caused problems with SATA Port Multipliers hot-plug.

MFC after: 2 weeks

Work around a race condition in devfs by changing the way closes
are handled in most CAM peripheral drivers that are not handled by
GEOM's disk class.

The usual character driver open and close semantics are that the
driver gets N open calls, but only one close, when the last caller
closes the device.

CAM peripheral drivers expect that behavior to be honored to the
letter, and the CAM peripheral driver code (specifically
cam_periph_release_locked_busses()) panics if it is done incorrectly.

Since devfs has to drop its locks while it calls a driver's close
routine, and it does not have a way to delay or prevent open calls
while it is calling the close routine, there is a race.

The sequence of events, simplified a bit, is:

- devfs acquires a lock
- devfs checks the reference count, and if it is 1, continues to close.
- devfs releases the lock

- 2nd process open call on the device happens here

- devfs calls the driver's close routine

- devfs acquires a lock
- devfs decrements the reference count
- devfs releases the lock

- 2nd process close call on the device happens here

At the second close, we get a panic in
cam_periph_release_locked_busses(), complaining that peripheral
has been released when the reference count is already 0. This is
because we have gotten two closes in a row, which should not
happen.

The fix is to add the D_TRACKCLOSE flag to the driver's cdevsw, so
that we get a close() call for each open(). That does happen
reliably, so we can make sure that our reference counts are
correct.

Note that the sa(4) and pt(4) drivers only allow one context
through the open routine. So these drivers aren't exposed to the
same race condition.

scsi_ch.c,
scsi_enc.c,
scsi_enc_internal.h,
scsi_pass.c,
scsi_sg.c:
For these drivers, change the open() routine to
increment the reference count for every open, and
just decrement the reference count in the close.

Call cam_periph_release_locked() in some scenarios
to avoid additional lock and unlock calls.

scsi_pt.c: Call cam_periph_release_locked() in some scenarios
to avoid additional lock and unlock calls.

MFC after: 3 days

Add tunable/sysctl kern.cam.pmp.hide_special, controlling whether special
PMP ports such as PMP configuration or SEMB should be exposed or hidden.
These ports were always hidden before as useless and sometimes promatic.
But with updated ses driver supporting SEMB it is no longer so straight.
Keep ports hidden by default to avoid probe request ttimeouts if SEP is
not connected to PMP's SEMB via I2C, that is very often situation.

Remove sleep() from invalidate call in ses driver, waiting for daemon
process exit. Instead use CAM's standard reference counting to prevent
periph going away until process won't complete. I think that sleep in
single CAM SWI thread is not a good idea and may lead to deadlocks if
daemon process waits for some command completion. Combined with recent
patch avoiding use of CAM SWI for ATA it just causes panics because of
sleeps prohibited in interrupt thread context.

MFprojects/zfsd:
Revamp the CAM enclosure services driver.
This updated driver uses an in-kernel daemon to track state changes and
publishes physical path location information\for disk elements into the
CAM device database.

Sponsored by: Spectra Logic Corporation
Sponsored by: iXsystems, Inc.
Submitted by: gibbs, will, mav

MFprojects/zfsd:
- Add low-level support for SATA Enclosure Management Bridge (SEMB)
devices -- SATA equivalents of the SCSI SES/SAF-TE devices.
- Add some utility functions for SCSI SAF-TE devices access.

Sponsored by: iXsystems, Inc.

Add two functions xpt_batch_start() and xpt_batch_done() to the CAM SIM KPI
to allow drivers to handle request completion directly without passing
them to the CAM SWI thread removing extra context switch.
Modify all ATA/SATA drivers to use them.

Reviewed by: gibbs, ken
MFC after: 2 weeks

Add support for:
Olympus FE-210 camera
LG UP3S MP3 player
Laser MP3-2GA13 MP3

PR: usb/119201
Submitted by: Peter Jeremy <peterjeremy@optushome.com.au>
Approved by: cperciva
MFC after: 1 week

Alike to SCSI make adaclose() to not return error if device gone.
This fixes KASSERT panic inside GEOM if kernel built with INVARIANTS.

MFC after: 1 week

Fix panic at boot with SD/MMC readers with no media present, introduced
at r234177. Note that this is a temporary fix, until I come up with something
prettier.

Refactor da(4) to remove one of two code paths used to query capacity
data.

Reviewed by: ken, mav (earlier version)
Sponsored by: The FreeBSD Foundation

Change the SCSI INQUIRY peripheral qualifier that CTL reports for LUNs
that don't exist.

Anecdotal evidence indicates that it is better to return 011b (bad LUN)
than 001b (LUN offline). However, this change also gives the user a
sysctl/tunable, kern.cam.ctl.inquiry_pq_no_lun, to override the change
and return to the previous behavior. (The previous behavior was to
return 001b, or LUN offline.)

ctl.c: Change the default inquiry peripheral qualifier to 011b,
and add a sysctl and tunable to allow the user to change
it back to 001b if needed.

Don't insert a Copan copyright statement in the inquiry
data. The copyright statements on the files are
sufficient.

ctl_private.h: Add sysctl variable context to the CTL softc.

ctl_cmd_table.c,
ctl_frontend_internal.c,
ctl_frontend.c,
ctl_backend.c,
ctl_error.c: Include sys/sysctl.h.

MFC after: 3 days

Be more conservative in using READ CAPACITY(16) command. Previous code
checked PROTECT bit in INQUIRY data for all SPC devices, while it is defined
only since SPC-3. But there are some SPC-2 USB devices were reported, that
have PROTECT bit set, return no error for READ CAPACITY(16) command, but
return wrong sector count value in response.

MFC after: 3 days

Add LUN resizing to CTL. Also make it possible to explicitly set
size when creating file-backed or device-backed LUN.

Reviewed by: ken (earlier version)
Sponsored by: The FreeBSD Foundation

Use a better way to silence unneeded internal declaration warnings in
several sys/cam/ctl files.

Suggested by: ed
Reviewed by: ken
MFC after: 1 week

Limit the ST3146855LW U320 drive to 55 tags to avoid command timeouts
under load.

Submitted by: Gelson Borsoi
MFC after: 1 day

Avoid panic from unlocking a not locked mutex (in some error cases).

Reviewed by: ken@

Insert ordered command every 1/4 of the current command timeout, not 1/4
of the default one.

Without this change setting kern.cam.ada.default_timeout to 1 instead of 30
allowed me to trigger several false positive command timeouts under heavy
ZFS load on a SiI3132 siis(4) controller with 5 HDDs on a port multiplier.

MFC after: 1 week

Make CAM ATA honor old hw.ata.ata_dma and hw.ata.atapi_dma tunables.

Even having more specific hint.ata.X.mode controls, global ones are
still could be useful from some points, including compatibility.

PR: kern/164651
MFC after: 1 week

Add CAM infrastructure to allow reporting when a drive's long read capacity
data changes.

cam_ccb.h: Add a new advanced information type, CDAI_TYPE_RCAPLONG,
for long read capacity data.

cam_xpt_internal.h:
Add a read capacity data pointer and length to struct cam_ed.

cam_xpt.c: Free the read capacity buffer when a device goes away.
While we're here, make sure we don't leak memory for other
malloced fields in struct cam_ed.

scsi_all.c: Update the scsi_read_capacity_16() to take a uint8_t * and
a length instead of just a pointer to the parameter data
structure. This will hopefully make this function somewhat
immune to future changes in the parameter data.

scsi_all.h: Add some extra bit definitions to struct
scsi_read_capacity_data_long, and bump up the structure
size to the full size specified by SBC-3.

Change the prototype for scsi_read_capacity_16().

scsi_da.c: Register changes in read capacity data with the transport
layer. This allows the transport layer to send out an
async notification to interested parties. Update the
dasetgeom() API.

Use scsi_extract_sense_len() instead of
scsi_extract_sense().

scsi_xpt.c: Add support for the new CDAI_TYPE_RCAPLONG advanced
information type.

Make sure we set the physpath pointer to NULL after freeing
it. This allows blindly freeing it in the struct cam_ed
destructor.

sys/param.h: Bump __FreeBSD_version from 1000005 to 1000006 to make it
easier for third party drivers to determine that the read
capacity data async notification is available.

camcontrol.c,
mptutil/mpt_cam.c:
Update these for the new scsi_read_capacity_16() argument
structure.

Sponsored by: Spectra Logic

Fix a bug introduced in r230000. We were eliminating all LUNs on a target
in response to CAM_DEV_NOT_THERE, instead of just the LUN in question.

This will now just eliminate the specified LUN in response to
CAM_DEV_NOT_THERE.

Reported by: Richard Todd <rmtodd@servalan.servalan.com>
MFC after: 3 days

Quiet some clang warnings when compiling CTL.

ctl_error.c,
ctl_error.h: Take out the ctl_sense_format enumeration, and use
scsi_sense_data_type instead.

Remove ctl_get_sense_format() and switch ctl_build_ua()
over to using scsi_sense_data_type.

ctl_backend_ramdisk.c,
ctl_backend_block.c:
Use C99 structure initializers instead of GNU initializers.

ctl.c: Switch over to using the SCSI sense format enumeration
instead of the CTL-specific enumeration.

Submitted by: dim (partially)
MFC after: 1 month

dadump: don't leak the periph lock on i/o error

Reported by: az
MFC after: 1 week

Add BIO_DELETE support for SCSI Direct Access devices (da).

Depending on device capabilities use different methods to implement it.
Currently used method can be read/set via kern.cam.da.X.delete_method
sysctls. Possible values are:
NONE - no provisioning support reported by the device;
DISABLE - provisioning support was disabled because of errors;
ZERO - use WRITE SAME (10) command to write zeroes;
WS10 - use WRITE SAME (10) command with UNMAP bit set;
WS16 - use WRITE SAME (16) command with UNMAP bit set;
UNMAP - use UNMAP command (equivalent of the ATA DSM TRIM command).
The last two methods (UNMAP and WS16) are defined by SBC specification and
the UNMAP method is the most advanced one. The rest of methods I've found
supported in Linux, and as soon as they were trivial to implement, then
why not? Hope they will be useful in some cases.

Unluckily I have no devices properly reporting parameters of the logical
block provisioning support via respective VPD pages (0xB0 and 0xB2). So
all info I have/use now is the flag telling whether logical block
provisioning is supported or not. As result, specific methods chosen now
by trying different ones in order (UNMAP, WS16, DISABLE) and checking
completion status to fallback if needed. I don't expect problems from this,
as if something go wrong, it should just disable itself. It may disable
even too aggressively if only some command parameter misfit.

Unlike Linux, which executes each delete with separate request, I've
implemented here the same request aggregation as implemented in ada driver.
Tests on SSDs I have show much better results doing it this way: above
8GB/s of the linear delete on Intel SATA SSD on LSI SAS HBA (mps).

Reviewed by: silence on scsi@
MFC after: 2 month
Sponsored by: iXsystems, Inc.

Silence some unnecessary verbosity.

Reported by: mav
MFC after: 1 month

Fix a race condition in CAM peripheral free handling, locking
in the CAM XPT bus traversal code, and a number of other periph level
issues.

cam_periph.h,
cam_periph.c: Modify cam_periph_acquire() to test the CAM_PERIPH_INVALID
flag prior to allowing a reference count to be gained
on a peripheral. Callers of this function will receive
CAM_REQ_CMP_ERR status in the situation of attempting to
reference an invalidated periph. This guarantees that
a peripheral scheduled for a deferred free will not
be accessed during its wait for destruction.

Panic during attempts to drop a reference count on
a peripheral that already has a zero reference count.

In cam_periph_list(), use a local sbuf with SBUF_FIXEDLEN
set so that mallocs do not occur while the xpt topology
lock is held, regardless of the allocation policy of the
passed in sbuf.

Add a new routine, cam_periph_release_locked_buses(),
that can be called when the caller already holds
the CAM topology lock.

Add some extra debugging for duplicate peripheral
allocations in cam_periph_alloc().

Treat CAM_DEV_NOT_THERE much the same as a selection
timeout (AC_LOST_DEVICE is emitted), but forgo retries.

cam_xpt.c: Revamp the way the EDT traversal code does locking
and reference counting. This was broken, since it
assumed that the EDT would not change during
traversal, but that assumption is no longer valid.

So, to prevent devices from going away while we
traverse the EDT, make sure we properly lock
everything and hold references on devices that
we are using.

The two peripheral driver traversal routines should
be examined. xptpdperiphtraverse() holds the
topology lock for the entire time it runs.
xptperiphtraverse() is now locked properly, but
only holds the topology lock while it is traversing
the list, and not while the traversal function is
running.

The bus locking code in xptbustraverse() should
also be revisited at a later time, since it is
complex and should probably be simplified.

scsi_da.c: Pay attention to the return value from cam_periph_acquire().

Return 0 always from daclose() even if the disk is now gone.

Add some rudimentary error injection support.

scsi_sg.c: Fix reference counting in the sg(4) driver.

The sg driver was calling cam_periph_release() on close,
but never called cam_periph_acquire() (which increments
the reference count) on open.

The periph code correctly complained that the sg(4)
driver was trying to decrement the refcount when it
was already 0.

Sponsored by: Spectra Logic
MFC after: 2 weeks

Add the CAM Target Layer (CTL).

CTL is a disk and processor device emulation subsystem originally written
for Copan Systems under Linux starting in 2003. It has been shipping in
Copan (now SGI) products since 2005.

It was ported to FreeBSD in 2008, and thanks to an agreement between SGI
(who acquired Copan's assets in 2010) and Spectra Logic in 2010, CTL is
available under a BSD-style license. The intent behind the agreement was
that Spectra would work to get CTL into the FreeBSD tree.

Some CTL features:

- Disk and processor device emulation.
- Tagged queueing
- SCSI task attribute support (ordered, head of queue, simple tags)
- SCSI implicit command ordering support. (e.g. if a read follows a mode
select, the read will be blocked until the mode select completes.)
- Full task management support (abort, LUN reset, target reset, etc.)
- Support for multiple ports
- Support for multiple simultaneous initiators
- Support for multiple simultaneous backing stores
- Persistent reservation support
- Mode sense/select support
- Error injection support
- High Availability support (1)
- All I/O handled in-kernel, no userland context switch overhead.

(1) HA Support is just an API stub, and needs much more to be fully
functional.

ctl.c: The core of CTL. Command handlers and processing,
character driver, and HA support are here.

ctl.h: Basic function declarations and data structures.

ctl_backend.c,
ctl_backend.h: The basic CTL backend API.

ctl_backend_block.c,
ctl_backend_block.h: The block and file backend. This allows for using
a disk or a file as the backing store for a LUN.
Multiple threads are started to do I/O to the
backing device, primarily because the VFS API
requires that to get any concurrency.

ctl_backend_ramdisk.c: A "fake" ramdisk backend. It only allocates a
small amount of memory to act as a source and sink
for reads and writes from an initiator. Therefore
it cannot be used for any real data, but it can be
used to test for throughput. It can also be used
to test initiators' support for extremely large LUNs.

ctl_cmd_table.c: This is a table with all 256 possible SCSI opcodes,
and command handler functions defined for supported
opcodes.

ctl_debug.h: Debugging support.

ctl_error.c,
ctl_error.h: CTL-specific wrappers around the CAM sense building
functions.

ctl_frontend.c,
ctl_frontend.h: These files define the basic CTL frontend port API.

ctl_frontend_cam_sim.c: This is a CTL frontend port that is also a CAM SIM.
This frontend allows for using CTL without any
target-capable hardware. So any LUNs you create in
CTL are visible in CAM via this port.

ctl_frontend_internal.c,
ctl_frontend_internal.h:
This is a frontend port written for Copan to do
some system-specific tasks that required sending
commands into CTL from inside the kernel. This
isn't entirely relevant to FreeBSD in general,
but can perhaps be repurposed.

ctl_ha.h: This is a stubbed-out High Availability API. Much
more is needed for full HA support. See the
comments in the header and the description of what
is needed in the README.ctl.txt file for more
details.

ctl_io.h: This defines most of the core CTL I/O structures.
union ctl_io is conceptually very similar to CAM's
union ccb.

ctl_ioctl.h: This defines all ioctls available through the CTL
character device, and the data structures needed
for those ioctls.

ctl_mem_pool.c,
ctl_mem_pool.h: Generic memory pool implementation used by the
internal frontend.

ctl_private.h: Private data structres (e.g. CTL softc) and
function prototypes. This also includes the SCSI
vendor and product names used by CTL.

ctl_scsi_all.c,
ctl_scsi_all.h: CTL wrappers around CAM sense printing functions.

ctl_ser_table.c: Command serialization table. This defines what
happens when one type of command is followed by
another type of command.

ctl_util.c,
ctl_util.h: CTL utility functions, primarily designed to be
used from userland. See ctladm for the primary
consumer of these functions. These include CDB
building functions.

scsi_ctl.c: CAM target peripheral driver and CTL frontend port.
This is the path into CTL for commands from
target-capable hardware/SIMs.

README.ctl.txt: CTL code features, roadmap, to-do list.

usr.sbin/Makefile: Add ctladm.

ctladm/Makefile,
ctladm/ctladm.8,
ctladm/ctladm.c,
ctladm/ctladm.h,
ctladm/util.c: ctladm(8) is the CTL management utility.
It fills a role similar to camcontrol(8).
It allow configuring LUNs, issuing commands,
injecting errors and various other control
functions.

usr.bin/Makefile: Add ctlstat.

ctlstat/Makefile
ctlstat/ctlstat.8,
ctlstat/ctlstat.c: ctlstat(8) fills a role similar to iostat(8).
It reports I/O statistics for CTL.

sys/conf/files: Add CTL files.

sys/conf/NOTES: Add device ctl.

sys/cam/scsi_all.h: To conform to more recent specs, the inquiry CDB
length field is now 2 bytes long.

Add several mode page definitions for CTL.

sys/cam/scsi_all.c: Handle the new 2 byte inquiry length.

sys/dev/ciss/ciss.c,
sys/dev/ata/atapi-cam.c,
sys/cam/scsi/scsi_targ_bh.c,
scsi_target/scsi_cmds.c,
mlxcontrol/interface.c: Update for 2 byte inquiry length field.

scsi_da.h: Add versions of the format and rigid disk pages
that are in a more reasonable format for CTL.

amd64/conf/GENERIC,
i386/conf/GENERIC,
ia64/conf/GENERIC,
sparc64/conf/GENERIC: Add device ctl.

i386/conf/PAE: The CTL frontend SIM at least does not compile
cleanly on PAE.

Sponsored by: Copan Systems, SGI and Spectra Logic
MFC after: 1 month

Add support for CDRIOCGETBLOCKSIZE and CDRIOCSETBLOCKSIZE IOCTLs to control
sector size same as acd driver does. Together with r228808 and r228847 this
allows existing multimedia/vlc to play Audio CDs via CAM cd driver.

PR: ports/162190
MFC after: 1 week

Remove unneeded checks for CAM_DEV_QFRZN after cam_periph_runccb() call.
cam_periph_runccb() since the beginning checks it and releases device queue.
After r203108 it even clears CAM_DEV_QFRZN flag after that to avoid double
release, so removed code is unreachable now.

MFC after: 1 month

Report stripeoffset as zero not stripesize if physical block is zero
aligned, same as it is done for ATA.

Addition to r228808:
READ CD is a 12 byte command. So fill additional bytes and update CDB length
when patching READ(10).

MFC after: 1 month

Use READ CAPACITY(16) to get information about device physical sectors.
As soon as not all devices support READ CAPACITY(16), automatically fall
back to READ CAPACITY(10) if CAM_REQ_INVALID or SSD_KEY_ILLEGAL_REQUEST
status returned.

It also provides first bits of information about Logical Block Provisioning
(aka UNMAP/TRIM) support by the device.

Merge to da driver quirks hinting 4K physical sector sizes for SATA disks
connected via SAS or USB. Unluckily I've found that SAS (mps) and USB-SATA
I have translate models in different ways, requiring twice more quirks.
Unluckily for Hitachi, their model names are trimmed on SAS, making
impossible to identify 4K sector drives that way.

Update list of 4K physical sector hard drives.

Make cd driver to handle Audio CDs, reporting their 2352 bytes sectors to
GEOM and using READ CD command for reading data, same as acd driver does.
Audio CDs identified by checking respective bit of the control field of
the first track in TOC.

This fixes bunch of error messages during boot (GEOM taste) with Audio CD
inserted and allows to grab Audio CD image using just dd.

MFC after: 1 month

Change targ(4) to use cdevpriv, instead of multiple character devices.

Also update the manpage and the scsi_target example program accordingly.

Discussed on: scsi@
Tested by: Chuck Tuffli <chuck tuffli net>

Do not use the sometimes-reserved work 'bool' for a variable name.

MFC after: 2 weeks
Sponsored by: Isilon Systems, LLC

- Add support for Support SEAGATE DAT Scopion 130

PR: kern/141934
Submitted by: HASHI Hiroaki <hashiz@stenmark.meridiani.jp>
Approved by: sbruno@
MFC after: 1 week

Move the scsi_da_bios_params() prototype from pc98_machdep.h to md_var.h
where the prototype for pc98_ata_disk_firmware_geom_adjust() also lives
in order to avoid an #ifdef'ed include in cam(4).

For sparc64 also adjust the geometry of da(4) driven disks to not overflow
the 16-bit cylinders field of the VTOC8 disk label (at around 502GB). The
geometry chosen for disks above that limit allows to use disks up to 2TB,
which is the limit of the extended VTOC8 format. The geometry used for
disks smaller than the 16-bit cylinders limit stays the same as used by
cam_calc_geometry(9) for extended translation.
Thanks to Hans-Joerg Sirtl for providing hardware for testing this change.

MFC after: 3 days

Introduce CAM_SIM_POLLED SIM flag, indicating that it works in polling mode.
It blocks CAM SWI usage on requests completion, unneeded because of polling
and denied during kernel dumping because of blocked scheduler.

Before r198899 there was periph flag CAM_PERIPH_POLLED, but that was wrong,
because there is whole SIM is polled or handled by SWI, not a single periph.

Tested by: kib
MFC after: 1 month

- fix duplicate "a a" in some comments

Submitted by: eadler
Approved by: simon
MFC after: 3 days

Mark all SYSCTL_NODEs static that have no corresponding SYSCTL_DECLs.

The SYSCTL_NODE macro defines a list that stores all child-elements of
that node. If there's no SYSCTL_DECL macro anywhere else, there's no
reason why it shouldn't be static.

Mark MALLOC_DEFINEs static that have no corresponding MALLOC_DECLAREs.

This means that their use is restricted to a single C file.

Add descriptor sense support to CAM, and honor sense residuals properly in
CAM.

Desriptor sense is a new sense data format that originated in SPC-3. Among
other things, it allows for an 8-byte info field, which is necessary to
pass back block numbers larger than 4 bytes.

This change adds a number of new functions to scsi_all.c (and therefore
libcam) that abstract out most access to sense data.

This includes a bump of CAM_VERSION, because the CCB ABI has changed.
Userland programs that use the CAM pass(4) driver will need to be
recompiled.

camcontrol.c: Change uses of scsi_extract_sense() to use
scsi_extract_sense_len().

Use scsi_get_sks() instead of accessing sense key specific
data directly.

scsi_modes: Update the control mode page to the latest version (SPC-4).

scsi_cmds.c,
scsi_target.c: Change references to struct scsi_sense_data to struct
scsi_sense_data_fixed. This should be changed to allow the
user to specify fixed or descriptor sense, and then use
scsi_set_sense_data() to build the sense data.

ps3cdrom.c: Use scsi_set_sense_data() instead of setting sense data
manually.

cam_periph.c: Use scsi_extract_sense_len() instead of using
scsi_extract_sense() or accessing sense data directly.

cam_ccb.h: Bump the CAM_VERSION from 0x15 to 0x16. The change of
struct scsi_sense_data from 32 to 252 bytes changes the
size of struct ccb_scsiio, but not the size of union ccb.
So the version must be bumped to prevent structure
mis-matches.

scsi_all.h: Lots of updated SCSI sense data and other structures.

Add function prototypes for the new sense data functions.

Take out the inline implementation of scsi_extract_sense().
It is now too large to put in a header file.

Add macros to calculate whether fields are present and
filled in fixed and descriptor sense data

scsi_all.c: In scsi_op_desc(), allow the user to pass in NULL inquiry
data, and we'll assume a direct access device in that case.

Changed the SCSI RESERVED sense key name and description
to COMPLETED, as it is now defined in the spec.

Change the error recovery action for a number of read errors
to prevent lots of retries when the drive has said that the
block isn't accessible. This speeds up reconstruction of
the block by any RAID software running on top of the drive
(e.g. ZFS).

In scsi_sense_desc(), allow for invalid sense key numbers.
This allows calling this routine without checking the input
values first.

Change scsi_error_action() to use scsi_extract_sense_len(),
and handle things when invalid asc/ascq values are
encountered.

Add a new routine, scsi_desc_iterate(), that will call the
supplied function for every descriptor in descriptor format
sense data.

Add scsi_set_sense_data(), and scsi_set_sense_data_va(),
which build descriptor and fixed format sense data. They
currently default to fixed format sense data.

Add a number of scsi_get_*() functions, which get different
types of sense data fields from either fixed or descriptor
format sense data, if the data is present.

Add a number of scsi_*_sbuf() functions, which print
formatted versions of various sense data fields. These
functions work for either fixed or descriptor sense.

Add a number of scsi_sense_*_sbuf() functions, which have a
standard calling interface and print the indicated field.
These functions take descriptors only.

Add scsi_sense_desc_sbuf(), which will print a formatted
version of the given sense descriptor.

Pull out a majority of the scsi_sense_sbuf() function and
put it into scsi_sense_only_sbuf(). This allows callers
that don't use struct ccb_scsiio to easily utilize the
printing routines. Revamp that function to handle
descriptor sense and use the new sense fetching and
printing routines.

Move scsi_extract_sense() into scsi_all.c, and implement it
in terms of the new function, scsi_extract_sense_len().
The _len() version takes a length (which should be the
sense length - residual) and can indicate which fields are
present and valid in the sense data.

Add a couple of new scsi_get_*() routines to get the sense
key, asc, and ascq only.

mly.c: Rename struct scsi_sense_data to struct
scsi_sense_data_fixed.

sbp_targ.c: Use the new sense fetching routines to get sense data
instead of accessing it directly.

sbp.c: Change the firewire/SCSI sense data transformation code to
use struct scsi_sense_data_fixed instead of struct
scsi_sense_data. This should be changed later to use
scsi_set_sense_data().

ciss.c: Calculate the sense residual properly. Use
scsi_get_sense_key() to fetch the sense key.

mps_sas.c,
mpt_cam.c: Set the sense residual properly.

iir.c: Use scsi_set_sense_data() instead of building sense data by
hand.

iscsi_subr.c: Use scsi_extract_sense_len() instead of grabbing sense data
directly.

umass.c: Use scsi_set_sense_data() to build sense data.

Grab the sense key using scsi_get_sense_key().

Calculate the sense residual properly.

isp_freebsd.h: Use scsi_get_*() routines to grab asc, ascq, and sense key
values.

Calculate and set the sense residual.

MFC after: 3 days
Sponsored by: Spectra Logic Corporation

Fixes for sure bus reference miscounting and potential device and
target reference miscounts. It also adds a helper function to get
the current reference counts for components of cam_path for debug
aid. One minor style(9) change.

Partially Obtained from: Chuck Tuffli (Emulex)
Reviewed by: scsi@ (ken)
Approved by: re (kib)
MFC after: 1 month

Higher-priority initialization request can eat request scheduling done from
adaclose(). Add immediate_priority check into adaschedule() to restore it.

Approved by: re (kib)
MFC after: 1 week

Add control for ATA disk read-ahead, alike to the previously added write
cache control. Some controller BIOS'es tend to disable read-ahead, that
dramatically reduces read performance. Previously ata(4) always enabled
read-ahead unconditionally.

Approved by: re (kib)

In some cases failed SATA disks may report their presence, but don't
respond to any commands. I've found that because of multiple command
retries, each of which cause 30s timeout, bus reset and another retry or
requeue for many commands, it may take ages to eventually drop the
failed device. The odd thing is that those retries continue even after
XPT considered device as dead and invalidated it.

This patch makes cam_periph_error() to block any command retries after
periph was marked as invalid. With that patch all activity completes in
1-2 minutes, just after several timeouts, required to consider device
death. This should make ZFS, gmirror, graid, etc. operation more robust.

Reviewed by: mjacob@ on scsi@

Approved by: re (kib)

Do not try to execute FLUSHCACHE on close and print extra messages for
invalidated (considered lost) ada device. Exactly same already done for
the da devices.

Approved by: re (kib)
MFC after: 1 week

cam/scsi/scsi_cd.c:
In cdregister(), hold the periph lock semaphore during changer
probe/configuration. This removes a window where an open of the
cd device may succeed before probe processing has completed.

cam/cam_xpt.c:
In camisr_runqueue(), we need to run the sims queue regardless of
whether or not the current peripheral has more work to do. This
reverts a change mistakenly made in revision 223081.

Reported by: ache

Fix ATAPI breakage introduced by r223443. It made SCSI commands to ATAPI
device to never complete, that caused probe process (system boot) stuck.

Return CAM_REQ_INVALID if the SCSI XPT receives an unsupported operation
via the XPT_DEV_ADVINFO CCB.

Reviewed by: ken

Plumb support for the device advanced information CCB in the ATA XPT.

This was previously done only for SCSI XPT in r223081, on which the change
in r223089 depended in order to respond to serial number requests. As a
result of r223089, da(4) and ada(4) devices register a d_getattr for geom to
use to obtain the information.

Reported by: ache
Reviewed by: ken

Fix a typo in adagetattr() from r223089. In particular, this restores
the ability to use ahci(4) for kernel dumps.

Plumb device physical path reporting from CAM devices, through GEOM and
DEVFS, and make it accessible via the diskinfo utility.

Extend GEOM's generic attribute query mechanism into generic disk consumers.
sys/geom/geom_disk.c:
sys/geom/geom_disk.h:
sys/cam/scsi/scsi_da.c:
sys/cam/ata/ata_da.c:
- Allow disk providers to implement a new method which can override
the default BIO_GETATTR response, d_getattr(struct bio *). This
function returns -1 if not handled, otherwise it returns 0 or an
errno to be passed to g_io_deliver().

sys/cam/scsi/scsi_da.c:
sys/cam/ata/ata_da.c:
- Don't copy the serial number to dp->d_ident anymore, as the CAM XPT
is now responsible for returning this information via
d_getattr()->(a)dagetattr()->xpt_getatr().

sys/geom/geom_dev.c:
- Implement a new ioctl, DIOCGPHYSPATH, which returns the GEOM
attribute "GEOM::physpath", if possible. If the attribute request
returns a zero-length string, ENOENT is returned.

usr.sbin/diskinfo/diskinfo.c:
- If the DIOCGPHYSPATH ioctl is successful, report physical path
data when diskinfo is executed with the '-v' option.

Submitted by: will
Reviewed by: gibbs
Sponsored by: Spectra Logic Corporation

Add generic attribute change notification support to GEOM.

sys/sys/geom/geom.h:
Add a new attrchanged method field to both g_class
and g_geom.

sys/sys/geom/geom.h:
sys/geom/geom_event.c:
- Provide the g_attr_changed() function that providers
can use to advertise attribute changes.
- Perform delivery of attribute change notifications
from a thread context via the standard GEOM event
mechanism.

sys/geom/geom_subr.c:
Inherit the attrchanged method from class to geom (class instance).

sys/geom/geom_disk.c:
Provide disk_attr_changed() to provide g_attr_changed() access
to consumers of the disk API.

sys/cam/scsi/scsi_pass.c:
sys/cam/scsi/scsi_da.c:
sys/geom/geom_dev.c:
sys/geom/geom_disk.c:
Use attribute changed events to track updates to physical path
information.

sys/cam/scsi/scsi_da.c:
Add AC_ADVINFO_CHANGED to the registered asynchronous CAM
events for this driver. When this event occurs, and
the updated buffer type references our physical path
attribute, emit a GEOM attribute changed event via the
disk_attr_changed() API.

sys/cam/scsi/scsi_pass.c:
Add AC_ADVINFO_CHANGED to the registered asynchronous CAM
events for this driver. When this event occurs, update
the physical patch devfs alias for this pass instance.

Submitted by: gibbs
Sponsored by: Spectra Logic Corporation

sys/cam/scsi/scsi_da.c:
- Only attempt the closing synchronize cache on a disk
if it is still there.
- When a device is lost, report the number of outstanding
I/Os as they are drained.
- When a device is lost, return any unprocessed bios with
ENXIO instead of EIO.
- Filter asynchronous events, but always allow cam_periph_async()
to see them too.

Sponsored by: Spectra Logic Corporation

Lay groundwork in CAM for recording and reporting physical path and
other device attributes stored in the CAM Existing Device Table (EDT).
This includes some infrastructure requried by the enclosure services
driver to export physical path information.

Make the CAM device advanced info interface accept store requests.

sys/cam/scsi/scsi_all.c:
sys/cam/scsi/scsi_all.h:
- Replace scsi_get_sas_addr() with a scsi_get_devid() which takes
a callback that decides whether to accept a particular descriptor.
Provide callbacks for NAA IEEE Registered addresses and for SAS
addresses, replacing the old function. This is needed because
the old function doesn't work for an enclosure address for a SAS
device, which is not flagged as a SAS address, but is NAA IEEE
Registered. It may be worthwhile merging this interface with the
devid match interface.
- Add a few more defines for some device ID fields.

sbin/camcontrol/camcontrol.c:
- Update for the CCB_DEV_ADVINFO interface change.

cam/cam_xpt_internal.h:
- Add the new fields for the physical path string to the CAM EDT.
cam/cam_ccb.h:
- Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb
structure to ccb_dev_advinfo.
- Add a flag that changes this CCB's action to store, rather than
the default, retrieve.
- Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT
physpath field.
- Remove the never-implemented transport & proto flags.
cam/cam_xpt.c:
cam/cam_xpt.h:
- Add xpt_getattr(), which provides a wrapper for fetching a device's
attribute using the GEOM strings as key. This method currently
supports "GEOM::ident" and "GEOM::physpath".

Submitted by: will
Reviewed by : gibbs

Extend the XPT_DEV_MATCH api to allow a device search by device ID.
As far as the API is concerned, device ID is a binary blob to be
interpreted by the transport layer. The SCSI implementation assumes
it is an array of VPD device ID descriptors.

sys/cam/cam_ccb.h:
Create a new structure, device_id_match_pattern, and
update the XPT_DEV_MATCH datastructures and flags so
that this pattern type can be used.

sys/cam/cam_xpt.c:
- A single pattern matching on both inquiry data and device
ID is invalid. Report any violators.
- Pass device ID match requests through to the new routine
scsi_devid_match(). The direct call of a SCSI routine is
a layering violation, but no worse than the one a few
lines up that checks inquiry data. Defer cleaning this
up until our future, larger, rototilling of CAM.
- Zero out cam_ed and cam_et nodes on allocation. Prior to
this change, device_id_len and device_id were not inialized,
preventing proper detection of the presence of this
information.

sys/cam/scsi/scsi_all.c:
sys/cam/scsi/scsi_all.h:
Add the scsi_match_devid() routine.

Add a helper function for extracting peripherial driver names

sys/cam/cam_periph.c:
sys/cam/cam_periph.h:
Add the cam_periph_list() method which fills an sbuf
with a comma delimited list of the peripheral instances
associated with a given CAM path.

Add a helper functions for SCSI commands used by the SES driver.

sys/cam/scsi/scsi_all.c:
sys/cam/scsi/scsi_all.h:
Add structure definitions and csio filling functions for
the receive diagnostic results and send diagnostic commands.

Misc CAM XPT cleanups.

sys/cam/cam_xpt.c:
Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED
events at the time async event handlers are attached
even when registering just for events on a partitular
SIM. Previously, you had to register for these
events on all SIMs in the system in order to get
the initial broadcast even though subsequent device
and path arrivals would be delivered.

sys/cam/cam_xpt.c:
Remove SIM mutex held asserts from path accessors.
CAM paths are reference counted and it is this
reference count, not the sim mutex, that garantees
they are stable.

Sponsored by: Spectra Logic Corporation

Do not report CFA devices as ATAPI, even though IDENTIFY data look alike.

Increase maximum supported number of ranges per TRIM command from 256 to 512
to use full potential of Intel X25-M SSDs. On synthetic test with 32K ranges
it gives about 20% speedup, which probably costs more then 2K of RAM.

When possible, join ranges of subsequest BIO_DELETE requests to handle more
(up to 2048 instead of 256 or even 64) of them with single TRIM request.

OCZ Vertex2/Vertex3 SSDs can handle no more then 64 ranges per TRIM request.
Due to lack of BIO_DELETE clustering now, it means that we could delete no
more then 2MB per request (on FS with 32K block) with limited request rate.
This change increases delete rate on Vertex2 from 250MB/s to 950MB/s.

Add quirks to hint 4K physical sector (Advanced Format) for ATA disks not
reporting it properly (none? of known disks now).

Hitachi and WDC AF disks seem could be identified more or less formally.
For Seagate and Samsung enumerate some found models/series.
For other disks it can be forced with kern.cam.ada.X.quirks=1 tunable.

Change new constant names to ones used by OpenSolaris.

Add names for few more SES element types according SES-2 specification.

scsi_cd: silence READ_TOC errors in CDIOREADTOCHEADER ioctl

An optical disk may not have a TOC (e.g. for blank media) and userland
software may legitimately try to use CDIOREADTOCHEADER to find out about
the TOC.

Silence from: scsi@
MFC after: 10 days

Do not report legacy unit numbers (do not create legacy aliases) for disks
on port multiplier ports above first two. They don't fit into ATA_STATIC_ID
scheme and so can't be mapped properly. No need to pollute dev.

Make CAM_DEBUG_CDB also dump ATA commands in addition to SCSI.

- Add shim to simplify migration to the CAM-based ATA. For each new adaX
device in /dev/ create symbolic link with adY name, trying to mimic old ATA
numbering. Imitation is not complete, but should be enough in most cases to
mount file systems without touching /etc/fstab.
- To know what behavior to mimic, restore ATA_STATIC_ID option in cases
where it was present before.
- Add some more details to UPDATING.

Add basic support for DMA-capable ATA disks on DMA-incapable controller.
This is really rare situation these days, but still may happen in embedded.

Remove always false "< 0" check for unsgined int variable. This check is
also duplicate, as the value was already checked for 0 before decrementing.

Reported by: rpaulo

Remove some used variables.

Found with: Clang Static Analyzer

Remove some used variables.

Found with: Clang Static Analyzer

Move a comment to the right place. The rearrange done in r208928 left
the comment to a wrong place.

Move TUNABLE_INT_FETCH() out of the SIM lock to avoid sleeping while
holding the lock. The fix is analogous to r220618 for ada(4).

Reviewed by: mav

Make ada(4) driver put ATA disks into sleep state on suspend.

Submitted by: jkim (original version)

Make CAM report devices with ATA/SATA transport to devstat(9) as IDE.

Fix typo in write_cache tunable name and move TUNABLE_INT_FETCH() out of
the SIM lock to fix WITNESS warning.

Reported by: jh

Properly log few more ATA commands used by the kernel.

Improve SATA Asynchronous Notification feature support in CAM:
- make SATA SIMs announce capabilities to handle SDB with Notification bit;
- make PMP driver honor this SIMs capability;
- make SATA XPT to negotiate and enable this feature for ATAPI devices.

This feature allows supporting SATA ATAPI devices to inform system about
some events happened, that may require attention. In my case this allows
LG GH22LS50 SATA DVR-RW drive to report tray open/close events. Events
reported to CAM in form of AC_SCSI_AEN async. Further they could be used
as a hints for checking device status and reporting media change to upper
layers, for example, via spoiling mechanism of GEOM.

- Add kern.cam.ada.X.write_cache tunables/sysctls to control write caching
on per-device basis.
- While adding support for per-device sysctls, merge from graid branch
support for ADA_TEST_FAILURE kernel option, which opens few more sysctl,
allowing to simulate read and write errors for testing purposes.

Make ada(4) driver to control device write cache, same as ata(4) does.
Add kern.cam.ada.write_cache sysctl/tunable to control it alike hw.ata.wc.

Don't automatically send a START UNIT to sequential access devices-
this might cause them to load the tape unintentionally.

Reviewed by: gibbs
MFC after: 1 month

Missed a file in r219056: add disk description for da(4).

Add the disk ident and a human-meaningful description (here, the disk model
string) to the geom_disk config XML so that they are easily accessible from
userland.

MFC after: 1 week

Add some FEATURE macros for various features (AUDIT/CAM/IPC/KTR/MAC/NFS/NTP/
PMC/SYSV/...).

No FreeBSD version bump, the userland application to query the features will
be committed last and can serve as an indication of the availablility if
needed.

Sponsored by: Google Summer of Code 2010
Submitted by: kibab
Reviewed by: arch@ (parts by rwatson, trasz, jhb)
X-MFC after: to be determined in last commit with code from this project

Fix typos - remove duplicate "the".

PR: bin/154928
Submitted by: Eitan Adler <lists at eitanadler.com>
MFC after: 3 days

In addition to r217444 ignore also ATA status errors on DMA Auto-Activation
enabling request. Some HP disks reported to return ABORT error there while
declaring support for this feature.

Make device initialization sequence shorter when possible. Do not enable/
disable already enabled/disabled SATA features.

sysctl(8) should use the CTLTYPE to determine the type of data when
reading. (This was already done for writing to a sysctl). This
requires all SYSCTL setups to specify a type. Most of them are now
checked at compile-time.

Remove SYSCTL_*X* sysctl additions as the print being in hex should be
controlled by the -x flag to sysctl(8).

Succested by: bde

Some old WD SATA disks report supported and enabled device-initiated
interface power management, but return ABORT error on attempt to disable
it. Make CAM SATA probe sequence ignore this error, as it is not fatal.

Add a 64-bit hex-printed sysctl(9) since there is at least one place in
the code that wanted it. It is named X64 rather than XQUAD since the
quad name is a historical abomination that should not be perpetuated.

Fix up a few more sysctl(9) mis-typing found in various LINT builds.

Remove unneeded includes of <sys/linker_set.h>. Other headers that use
it internally contain nested includes.

Reviewed by: bde

Fix a few issues related to the XPT_GDEV_ADVINFO CCB.

camcontrol.c: In buildbusdevlist(), don't attempt to get call
getdevid() for an unconfigured device, even when the
verbose flag is set. The cam_open_btl() call will almost
certainly fail.

Probe for the buffer size when issuing the XPT_GDEV_ADVINFO
CCB. Probing for the buffer size first helps us avoid
allocating the maximum buffer size when it really may not
be necessary. This also helps avoid errors from
cam_periph_mapmem() if we attempt to map more than MAXPHYS.

cam_periph.c: In cam_periph_mapmem(), if the XPT_GDEV_ADVINFO CCB
shows a bufsiz of 0, we don't have anything to map,
so just return.

Also, set the maximum mapping size to MAXPHYS
instead of DFLTPHYS for XPT_GDEV_ADVINFO CCBs,
since they don't actually go down to the hardware.

scsi_pass.c: Don't bother mapping the buffer in XPT_GDEV_ADVINFO
CCBs if bufsiz is 0.

Add Serial Management Protocol (SMP) passthrough support to CAM.

This includes support in the kernel, camcontrol(8), libcam and the mps(4)
driver for SMP passthrough.

The CAM SCSI probe code has been modified to fetch Inquiry VPD page 0x00
to determine supported pages, and will now fetch page 0x83 in addition to
page 0x80 if supported.

Add two new CAM CCBs, XPT_SMP_IO, and XPT_GDEV_ADVINFO. The SMP CCB is
intended for SMP requests and responses. The ADVINFO is currently used to
fetch cached VPD page 0x83 data from the transport layer, but is intended
to be extensible to fetch other types of device-specific data.

SMP-only devices are not currently represented in the CAM topology, and so
the current semantics are that the SIM will route SMP CCBs to either the
addressed device, if it contains an SMP target, or its parent, if it
contains an SMP target. (This is noted in cam_ccb.h, since it will change
later once we have the ability to have SMP-only devices in CAM's topology.)

smp_all.c,
smp_all.h: New helper routines for SMP. This includes
SMP request building routines, response parsing
routines, error decoding routines, and structure
definitions for a number of SMP commands.

libcam/Makefile: Add smp_all.c to libcam, so that SMP functionality
is available to userland applications.

camcontrol.8,
camcontrol.c: Add smp passthrough support to camcontrol. Several
new subcommands are now available:

'smpcmd' functions much like 'cmd', except that it
allows the user to send generic SMP commands.

'smprg' sends the SMP report general command, and
displays the decoded output. It will automatically
fetch extended output if it is available.

'smppc' sends the SMP phy control command, with any
number of potential options. Among other things,
this allows the user to reset a phy on a SAS
expander, or disable a phy on an expander.

'smpmaninfo' sends the SMP report manufacturer
information and displays the decoded output.

'smpphylist' displays a list of phys on an
expander, and the CAM devices attached to those
phys, if any.

cam.h,
cam.c: Add a status value for SMP errors
(CAM_SMP_STATUS_ERROR).

Add a missing description for CAM_SCSI_IT_NEXUS_LOST.

Add support for SMP commands to cam_error_string().

cam_ccb.h: Rename the CAM_DIR_RESV flag to CAM_DIR_BOTH. SMP
commands are by nature bi-directional, and we may
need to support bi-directional SCSI commands later.

Add the XPT_SMP_IO CCB. Since SMP commands are
bi-directional, there are pointers for both the
request and response.

Add a fill routine for SMP CCBs.

Add the XPT_GDEV_ADVINFO CCB. This is currently
used to fetch cached page 0x83 data from the
transport later, but is extensible to fetch many
other types of data.

cam_periph.c: Add support in cam_periph_mapmem() for XPT_SMP_IO
and XPT_GDEV_ADVINFO CCBs.

cam_xpt.c: Add support for executing XPT_SMP_IO CCBs.

cam_xpt_internal.h: Add fields for VPD pages 0x00 and 0x83 in struct
cam_ed.

scsi_all.c: Add scsi_get_sas_addr(), a function that parses
VPD page 0x83 data and pulls out a SAS address.

scsi_all.h: Add VPD page 0x00 and 0x83 structures, and a
prototype for scsi_get_sas_addr().

scsi_pass.c: Add support for mapping buffers in XPT_SMP_IO and
XPT_GDEV_ADVINFO CCBs.

scsi_xpt.c: In the SCSI probe code, first ask the device for
VPD page 0x00. If any VPD pages are supported,
that page is required to be implemented. Based on
the response, we may probe for the serial number
(page 0x80) or device id (page 0x83).

Add support for the XPT_GDEV_ADVINFO CCB.

sys/conf/files: Add smp_all.c.

mps.c: Add support for passing in a uio in mps_map_command(),
so we can map a S/G list at once.

Add support for SMP passthrough commands in
mps_data_cb(). SMP is a special case, because the
first buffer in the S/G list is outbound and the
second buffer is inbound.

Add support for warning the user if the busdma code
comes back with more buffers than will work for the
command. This will, for example, help the user
determine why an SMP command failed if busdma comes
back with three buffers.

mps_pci.c: Add sys/uio.h.

mps_sas.c: Add the SAS address and the parent handle to the
list of fields we pull from device page 0 and cache
in struct mpssas_target. These are needed for SMP
passthrough.

Add support for the XPT_SMP_IO CCB. For now, this
CCB is routed to the addressed device if it supports
SMP, or to its parent if it does not and the parent
does. This is necessary because CAM does not
currently support SMP-only nodes in the topology.

Make SMP passthrough support conditional on
__FreeBSD_version >= 900026. This will make it
easier to MFC this change to the driver without
MFCing the CAM changes as well.

mps_user.c: Un-staticize mpi_init_sge() so we can use it for
the SMP passthrough code.

mpsvar.h: Add a uio and iovecs into struct mps_command for
SMP passthrough commands.

Add a cm_max_segs field to struct mps_command so
that we can warn the user if busdma comes back with
too many segments.

Clear the cm_reply when a command gets freed. If
it is not cleared, reply frames will eventually get
freed into the pool multiple times and corrupt the
pool. (This fix is from scottl.)

Add a prototype for mpi_init_sge().

sys/param.h: Bump __FreeBSD_version to 900026 for the for the
inclusion of the XPT_GDEV_ADVINFO and XPT_SMP_IO
CAM CCBs.

If HBA doesn't report user-enabled SATA capabilies (like ATA_CAM wrapper) -
handle all of them as disabled. This was original cause of the problem,
workarounded by r215453.

MFC after: 1 week

When requesting sense data for SIM not doing it automatically (such as
ATAPI or USB), request only as much data as requested by consumer.
On the way back -- report how much sense data we have actually received.

Make da driver to handle some probably broken Android devices, returning
zero media and sector size instead of "Medium not present" error,
until some confirmation button is tapped on device.

Mostly revert r203420, and add similar functionality into ada(4) since the
existing code caused problems with some SCSI controllers.

A new sysctl kern.cam.ada.spindown_shutdown has been added that controls
whether or not to spin-down disks when shutting down.
Spinning down the disks unloads/parks the heads - this is
much better than removing power when the disk is still
spinning because otherwise an Emergency Unload occurs which may cause damage
to the actuator.

PR: kern/140752
Submitted by: olli
Reviewed by: arundel
Discussed with: mav
MFC after: 2 weeks

Decrease poll interval from 1000 to 100us. This significantly reduces
kernel dump time, especially with minidump.

Use destroy_dev_sched(9) instead of destroy_dev(9) in passcleanup() as
it is indirectly a d_close method.

Prompted by: kib
Reviewed by: mav
MFC after: 2 weeks

Correct bioq_disksort so that bioq_insert_tail() offers barrier semantic.
Add the BIO_ORDERED flag for struct bio and update bio clients to use it.

The barrier semantics of bioq_insert_tail() were broken in two ways:

o In bioq_disksort(), an added bio could be inserted at the head of
the queue, even when a barrier was present, if the sort key for
the new entry was less than that of the last queued barrier bio.

o The last_offset used to generate the sort key for newly queued bios
did not stay at the position of the barrier until either the
barrier was de-queued, or a new barrier (which updates last_offset)
was queued. When a barrier is in effect, we know that the disk
will pass through the barrier position just before the
"blocked bios" are released, so using the barrier's offset for
last_offset is the optimal choice.

sys/geom/sched/subr_disk.c:
sys/kern/subr_disk.c:
o Update last_offset in bioq_insert_tail().

o Only update last_offset in bioq_remove() if the removed bio is
at the head of the queue (typically due to a call via
bioq_takefirst()) and no barrier is active.

o In bioq_disksort(), if we have a barrier (insert_point is non-NULL),
set prev to the barrier and cur to it's next element. Now that
last_offset is kept at the barrier position, this change isn't
strictly necessary, but since we have to take a decision branch
anyway, it does avoid one, no-op, loop iteration in the while
loop that immediately follows.

o In bioq_disksort(), bypass the normal sort for bios with the
BIO_ORDERED attribute and instead insert them into the queue
with bioq_insert_tail(). bioq_insert_tail() not only gives
the desired command order during insertion, but also provides
barrier semantics so that commands disksorted in the future
cannot pass the just enqueued transaction.

sys/sys/bio.h:
Add BIO_ORDERED as bit 4 of the bio_flags field in struct bio.

sys/cam/ata/ata_da.c:
sys/cam/scsi/scsi_da.c
Use an ordered command for SCSI/ATA-NCQ commands issued in
response to bios with the BIO_ORDERED flag set.

sys/cam/scsi/scsi_da.c
Use an ordered tag when issuing a synchronize cache command.

Wrap some lines to 80 columns.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c
sys/geom/geom_io.c
Mark bios with the BIO_FLUSH command as BIO_ORDERED.

Sponsored by: Spectra Logic Corporation
MFC after: 1 month

Revert r211434. Offline discussions have convinced me that this should
be left alone for now.

Now is as good a time as any to find out if we induce breakage
by issueing aborts for any pending commands when we're decommssioning
a disk.

MFC after: 3 months

Fix two (very common) occurrences of s/sytem/system in comments.

Approved by: mav
MFC after: 3 days

Export PCI IDs of ATA/SATA controllers through CAM and ata(4) layers to
GEOM. This information needed for proper soft-RAID's on-disk metadata
reading and writing.

ATA device reset starts probe sequence from the beginning. If reset caused
by timeout/error of one of probe commands, process may continue infinitely.
Make CAM ATA more robust to faulty devices and false positive detections,
abort probe after two restarts on timeouts or ten on other errors.

Don't lock buses around a call to xptperiphlistmatch- the buses will be
locked at appropriate places.

MFC after: 1 week
X-MFC: 208752

For the target port groups structures, don't allocate the initial element.
This makes things easier for target implementations to calculate how many
elements they need to allocate.

Discussed with: mjacob, gibbs
MFC after: 1 week

Rearrange how things are done to avoid dereferencing stale pointers in
the case of immediate unconfigure after configure. Hold the periph an
extra count while we have the task to create sysctl context outstanding
so that the periph doesn't go away unexpectedly.

Sponsored by: Panasas
Reviewed by: scsi@
MFC after: 1 month

One byte off for scsi_target_group cdb.

MFC after: 2 weeks

Implement the usage of Report Luns as part of SCSI probing for SCP3 or
better devices. This can be disabled on a per-device basis using quirks as
well.

This also handles the case where there is actually no connected LUN 0
(which can definitely be the case for storage arrays).

Reviewed by: scsi@
MFC after: 1 month

Make additional definitions up to and including SPC-4. Add in definitions
for REPORT and SET TARGET PORT GROUP commands (foundations for future work).

Regularize opcodes to be upper case hex.

Pick *one* of tab or space after #define (tab) and stick with that.

MFC after: 2 weeks

redfacedly noting a missing part of a patch.

Pointy Hat To: Me
MFC after: 2 Week
X-MFC: 208896

Do a minor amount of stylifying. Also, get a Fibre Channel WWPN if one exists
for a da unit and create a sysctl OID for it.

Add allocation error hadling.

Found with: Coverity Prevent(tm)
CID: 3897

Fix double free on error.

Found with: Coverity Prevent(tm)
CID: 4573

scsi_cd: pass correct pointer to free()

Found with: Coverity Prevent(tm)
CID: 2986
MFC after: 4 days

Protect periph drivers list and rearrange things to minimize the chance of
stepping oneself during probing.

Don't blindly decrement a periph probe count.

Reviewed by: scsi@
Obtained from: Alexander Motin, Atillio Rao, Others
MFC after: 1 month

Add a new primitive, XPT_SCAN_TGT, to cover the range between scanning a
whole bus (XPT_SCAN_BUS) and a single lun on that bus (XPT_SCAN_LUN).

It's less resource comsumptive than scanning a whole bus when the
caller knows only one target has changes.

Reviewed by: scsi@
Sponsored by: Panasas
MFC after: 1 month

Change ad_firmware_geom_adjust() to operate on a struct disk * only and
hook it up to ada(4) also. While at it, rename *ad_firmware_geom_adjust()
to *ata_disk_firmware_geom_adjust() etc now that these are no longer
limited to ad(4).

Reviewed by: mav
MFC after: 3 days

Remove unnecessary headers which were inherited from cam_xpt.c without
reason.

Reviewed by: mav
MFC after: 3 days

Pick up the right change, not it's close cousin. The one
previously submitted was wrong.

Point hat: mjacob
X-MFC: 207933
MFC after: 1 week

Deal sensibly with more than 26 sg devices. It isn't a complete
solution.

Sponsored by: Panasas
MFC after: 1 week

Make SATA XPT negotiate and enable some additional SATA features, such as:
- device initiated power management (some devices support only this way);
- Automatic Partial to Slumber Transition (more power saving);
- DMA auto-activation (expected to slightly improve performance).
More features could be added later, when hardware supports.

Add xpt_schedule_dev_sendq() call, lost at r203108. It is not needed in
usual operation, but required in some conditions to make queue running
after being shrinked.

MFC after: 3 days

Revert r198705.
As scottl@ noticed, max_target/max_lun was intended to be only a hint for
existing bus scanner. Some FC/SAS SIMs report fake values there, that are
smaller then maximum supported IDs. In that case this check makes impossible
manual scan outside hinted range.

For ATA/SATA SIMs respective check was instead implemented at SIM level.
Newer SCSI SIMs expected to have these checks at driver or firmware level.
Some older SCSI SIMs have no this check and the issues will get back there.

Report PMP absence using target 15, same as for precence (not a wildcard),
to not confuse target ID checks at SIMs.

Update device identify data and serial number when device change detected.
Reprobe immediately following this should have fresh data.

MFp4:
Move PI_TAG_ABLE check from ada driver to ATA XPT.

scsi_cd: CD_FLAG_VALID_MEDIA is sufficient to set d_sectorsize and
d_mediasize

CD_FLAG_VALID_TOC is not required for setting those media properties.

PR: kern/145385
Submitted by: Juergen Lock <nox@jelal.kn-bremen.de>
a slightly different version
Tested by: Pavel Sukhoy <sukhoy@ripn.net>,
Markus Wild <m.wild@cybernet.ch>,
Juergen Lock <nox@jelal.kn-bremen.de>,
uqs
MFC after: 1 week

Nullify our periph pointer in dacleanup to try and catch
race conditions in callbacks/downcalls.

Add a couple missing basic mode page codes.

MFC after: 1 week

For unhandled actions in xpt_action_default, remember to call
xpt_done for queued requests. This solves the problem of
indefinite hangs for unspecified transports when XPT_SCAN_BUS
is called.

A few minor cosmetics elsewhere.

MFC after: 1 week

We actually can generate a host number.

MFC after: 1 month

Make PUIS detection more strict. Previous implementation caused false
positives on VMWare's virtual CD-ROMs.

Fix newlines broken at r204220.

Make xpt_rescan() more correct, as it was before r197208: do not use
XPT_SCAN_LUN for wildcard LUN, fall back to XPT_SCAN_BUS.

Virtualize transport part of periph announcement.

Improve output for controllers that doesn't report SATA speed.

Make CD driver a bit more robust and predictable to unreported errors.

On probe error, if restart requested, skip any retries and recovery.
Just restart probe from the beginning immediately.

Disable kern.cam.power_down ATM. It doesn't work fine on some controllers.

MFp4:
Add Power Up In Stand-by feature support. Device with PUIS enabled
require explicit command to do initial spin-up. Mark that command
with CAM_HIGH_POWER flag, to allow CAM manage staggered spin-up.

MFp4:
Make CAM to stop all attached devices on system shutdown.
It allows devices to park heads, reducing stress on power loss.
Add `kern.cam.power_down` tunable and sysctl to controll it.

Change the way in which fake async events generated. Do not use
taskqueue for lock decoupling, as it causes unwanted races.

- Use separate buffer for identify data fetching. We can't use main buffer
here if device already running, as data need to be formatted before use.
- Remove some saved_ccb variables. They are unused now.

Some missed parts for r203376.

- Give ATA/SATA SIMs info about ATAPI packet size, supported by device.
- Make ATA XPT to reject longer SCSI CDBs then supported by device, or
any SCSI CDBs, if device doesn't support ATAPI.

MFp4: Large set of CAM inprovements.

- Unify bus reset/probe sequence. Whenever bus attached at boot or later,
CAM will automatically reset and scan it. It allows to remove duplicate
code from many drivers.
- Any bus, attached before CAM completed it's boot-time initialization,
will equally join to the process, delaying boot if needed.
- New kern.cam.boot_delay loader tunable should help controllers that
are still unable to register their buses in time (such as slow USB/
PCCard/ CardBus devices), by adding one more event to wait on boot.
- To allow synchronization between different CAM levels, concept of
requests priorities was extended. Priorities now split between several
"run levels". Device can be freezed at specified level, allowing higher
priority requests to pass. For example, no payload requests allowed,
until PMP driver enable port. ATA XPT negotiate transfer parameters,
periph driver configure caching and so on.
- Frozen requests are no more counted by request allocation scheduler.
It fixes deadlocks, when frozen low priority payload requests occupying
slots, required by higher levels to manage theit execution.
- Two last changes were holding proper ATA reinitialization and error
recovery implementation. Now it is done: SATA controllers and Port
Multipliers now implement automatic hot-plug and should correctly
recover from timeouts and bus resets.
- Improve SCSI error recovery for devices on buses without automatic sense
reporting, such as ATAPI or USB. For example, it allows CAM to wait, while
CD drive loads disk, instead of immediately return error status.
- Decapitalize diagnostic messages and make them more readable and sensible.
- Teach PMP driver to limit maximum speed on fan-out ports.
- Make boot wait for PMP scan completes, and make rescan more reliable.
- Fix pass driver, to return CCB to user level in case of error.
- Increase number of retries in cd driver, as device may return several UAs.

- Report SATA in legacy emulation mode still as SATA.
- Make ATA XPT able to handle such case.

Remove extraneous semicolons, no functional changes.

Submitted by: Marc Balmer <marc@msys.ch>
MFC after: 1 week

Add BIO_DELETE support to ada(4):
- For SSDs use TRIM feature of DATA SET MANAGEMENT command, as defined by
ACS-2 specification working draft.
- For CompactFlash use CFA ERASE command, same as ad(4) does.

With this patch, `newfs -E /dev/ada1` was able to restore write speed of
my heavily weared OCZ Vertex SSD (firmware 1.4) up to the initial level
for the most part of it's capacity. Previous 1.3 firmware, even reportiong
TRIM capabilty bit set, was not working, reporting ABORT error for every
DSM command.

I have no idea whether it is normal, but for some reason it takes 200ms
to handle any TRIM command on this drive, that was making delete extremely
slow. But TRIM command is able to accept long list of LBAs and the length of
that list seems doesn't affect it's execution time. Implemented request
clusting algorithm allowed me to rise delete rate up to reasonable numbers,
when many parallel DELETE requests running.

Report stripe size only if physical sector size is not equal to logical.

Remove duplicate devstat_start_transaction_bio() call. It is already called
from geom_disk. Dulicate call causes wrong queue depth and busy accounting.

Clear result before requesting XPT_PATH_INQ.
Many SIMs doesn't fill maxio field yet.

MFp4;
- Cleanup kernel messages, mostly PMP.
- Took references on devices, while PMP reinitializes them, to not let them
go and distort freeze reference counting.

MFp4:
If we panicked with SIM lock held, do not try to flush caches.
Extra lock recursing will not make debugging easier.

MFp4:
Introduce ATA_CAM kernel option, turning ata(4) controller drivers into
cam(4) interface modules. When enabled, this options deprecates all ata(4)
peripheral drivers (ad, acd, ...) and interfaces and allows cam(4) drivers
(ada, cd, ...) and interfaces to be natively used instead.

As side effect of this, ata(4) mode setting code was completely rewritten
to make controller API more strict and permit above change. While doing
this, SATA revision was separated from PATA mode. It allows DMA-incapable
SATA devices to operate and makes hw.ata.atapi_dma tunable work again.

Also allow ata(4) controller drivers (except some specific or broken ones)
to handle larger data transfers. Previous constraint of 64K was artificial
and is not really required by PCI ATA BM specification or hardware.

Submitted by: nwitehorn (powerpc part)

Fix several cases where the periph lock was held over malloc.

Submitted by: Jaakko Heinonen

Add CAM_ATAIO_DMA ATA command flag to mark DMA protocol commands.
It is not needed for SATA controllers, but required for PATA.

MFp4:
Improve ATA mode/SATA revision control.

Fix small copu-paste bug.

MFp4:
- Extend XPT-SIM transfer settings control API. Now it allows to report to
SATA SIM number of tags supported by each device, implement ATA mode and
SATA revision negotiation for both SATA and PATA SIMs.
- Make ahci(4) and siis(4) to use submitted maximum tag number, when
scheduling requests. It allows to support NCQ on devices with lower tags
count then controller supports.
- Make PMP driver to report attached devices connection speeds.
- Implement ATA mode negotiation between user settings, device and
controller capabilities.

Disable PortMultiplier Async Notifications for time of ports reset.
They are useless at that time, but confuse Marvell AHCI.

Add quirk for SiI57XX Port Multipliers, to hide extra port.

MFp4:
Some more missed parts from previous commits.

MFp4:
Remove code that years ago was closing race between request submission
to SIM and device/SIM freeze. That race become impossible after moving from
spl to mutex locking, while this workaround causes some unexpected effects.

MFp4:
Fix several device freeze counting bugs.

MFp4:
Do not enable tagged queueing if controller reports 0 tags support.

MFp4:
- Move tagged queueing control from ADA to ATA XPT. It allows to control
device command queue length correctly. First step to support < 32 tags.
- Limit queue for non-tagged devices by 2 slots for ahci(4) and siis(4).
- Implement quirk matching for ATA devices.
- Move xpt_schedule_dev_sendq() from header to source file.
- Move delayed queue shrinking to the more expected place - element freeing.
- Remove some SCSIsms in ATA.

MFp4:
Implement device stats accounting for ATA commands.

Fix protype.

PMP commands use short format. PMP write doesn't return result.

MFp4:
- Remove CAM_PERIPH_POLLED flag. It is broken by design. Polling can't be
periph flag. May be SIM, may be CCB, but now it works fine just without it.
- Remove check unused for at least five years. If we will ever have non-BIO
devices in CAM, this check is smallest of what we will need.
- If several controllers complete requests same time, call swi_sched()
only once.

MFp4:
- Add support for sector size > 512 bytes and physical sector of several
logical sectors, introduced by ATA-7 specification.
- Remove some obsoleted code.

MFp4:
Improve reporting ATA Status error details.

Provide the same sanity check on the sector size in dagetcapacity as when the
disk is first probed. dagetcapacity is called whenever the disk is opened from
geom via d_open(), a zero sector size will cause geom to panic later on.

The async callback could free the device. If it is a broadcast async,
it doesn't hold device reference, so take our own reference.

Submitted by: thompsa

MFp4:
Fix reference counting bug, when device unreferenced before then
invalidated. To do it, do not handle validity flag as another
reference, but explicitly modify reference count each time flag is
modified.

Discovered by: thompsa

MFp4:
- Reduce code duplication in ATA XPT and PMP driver.
- Move PIO size setting from ada driver to ATA XPT. It is XPT business
to negotiate transfer details. ada driver is now stateless.
- Report PIO size to SIM. It is required for correct PATA SIM operation.
- Tune PMP scan timings. It workarounds some problems with SiI.
- If reset hapens during PMP initialization - restart it.
- Introduce early-initialized periph drivers, which are used during initial
scan process. Use it for xpt, probe, aprobe and pmp. It gives pmp chance
to finish scan before mountroot and numerate devices in right order.

MFp4:
Ensure target/lun passed from user-level supported on this bus.
Scanning unsupported IDs causes different issues from duplicate
devices to system crash.

Make "Retrying Command" to be printed before actual retrying.
It should make debug/error log a bit more readable.

MFp4:
Move Port Multiplier support code out of ATA XPT into pmp periph driver.
This is convinient, as PMP itself is a bus target and has own state.

Fix the build.

Replace most of priority numbers with defines. No logical changes.

Remove some obsoleted comments.

After thinking again, implement cam_ccbq_fini().
This is effectively NULL change, but makes this API a bit more consistent.

cam_ccbq_fini() declared for 11 years, but never implemented. Remove it.

Fix a memory leak in an error case.

PR: 138376
Submitted by: Patroklos Argyroudis <argp@census-labs.com>
Reviewed by: scottl
MFC after: 1 week

MFp4:
Do not search for bus when it is not needed,

MFp4:
Separate CAM_DEV_IDENTIFY_DATA_VALID flag from CAM_DEV_INQUIRY_DATA_VALID.
Add workaround for very old devices without support for mode setting.
Add some PATA bus scanning support.
Remove some SCSIsms.

MFp4:
Add support for PIO-only devices.
Fix maxio values and 256 sectors transactions for 28bits commands.
Implement periodic ordered commands insertion, sames as da driver does.
Remove some SCSIsms.

MFp4:
Report real max_target = 15. SIM doesn't need to know that target 15 is PMP.
It is XPT business.

Export disk serial numbers for adaX disks.

Reviewed by: mav
MFC after: 3 days

Report SATA 3.x devices.

MFp4:
If on sense request device returns no sence, give up and return,
or we may loop forever.

Remember to unlock the peripheral prior to notifying the user. Make some
allocations M_NOWAIT so that we don't try and sleep with a nested non-sleepable
lock.

This makes the userland scsi_target begin to function again.

Obtained from: Sean Bruno
MFC after: 1 month

Fix some instances where CAM rescans get hung up or take a long time to
complete.

Also, allow xpt_rescan() to rescan a LUN instead of a full bus.

MFC after: 3 days
Sponsored by: Copan Systems, Inc.

Use C99 initialization for struct filterops.

Obtained from: Mac OS X
Sponsored by: Apple Inc.
MFC after: 3 weeks

Free the correct buffer in an error case.

Submitted by: phk

Revert previous commit and add myself to the list of people who should
know better than to commit with a cat in the area.

Add necessary include.

MFp4:
Remove duplicate qfrozen_cnt variable from struct cam_ed.
ccbq.queue.qfrozen_cnt should be used instead.

Remove unneeded CAM_SIM_MPSAFE check.