MFC r326645 (by imp): Define xpt_path_inq.

This provides a nice wrarpper around the XPT_PATH_INQ ccb creation and
calling.

MFC r349283 (by scottl):
Refactor xpt_getattr() to make it more readable. No outwardly
visible functional changes, though code flow was modified a bit
internally to lessen the need for goto jumps and chained if
conditionals.

MFC r349243: Optimize xpt_getattr().

Do not allocate temporary buffer for attributes we are going to return
as-is, just make sure to NUL-terminate them. Do not zero temporary 64KB
buffer for CDAI_TYPE_SCSI_DEVID, XPT tells us how much data it filled
and there are also length fields inside the returned data also.

MFC r335934: remove unneeded inclusion of sys/interrupt.h from several files

MFC r314624:

Reject userland CCBs that have CAM_UNLOCKED set.

CAM_UNLOCKED is internal flag and cannot correctly be set by userland.
Return EINVAL from CAMIOCOMMAND and CAMIOQUEUE if it is set.

Also fix leaks in some of the error paths for CAMIOQUEUE.

PR: 215356

MFC r303468 (by imp):
Move protocol specific stuff into a linker set object that's
per-protocol. This reduces the number scsi symbols references by
cam_xpt significantly, and eliminates all ata / nvme symbols. There's
still some NVME / ATA specific code for dealing with XPT_NVME_IO and
XPT_ATA_IO respectively, and a bunch of scsi-specific code, but this
is progress.

MFC r303467 (by imp):
Switch to linker sets to find the xport callback object. This
eliminates the need to special case everything in cam_xpt for new
transports. It is now a failure to not have a transport object when
registering the bus as well. You can still, however, create a
transport that's unspecified (XPT_)

MFC r322999 (by imp): Fix NVMe's use of XPT_GDEV_TYPE

This patch changes the way XPT_GDEV_TYPE works for NVMe. The current
ccb_getdev structure includes pointers to the NVMe Identify Controller
and Namespace structures, but these are kernel virtual addresses which
are not accessible from user space.

As an alternative, the patch changes the pointers into padding in
ccb_getdev and adds two new types to ccb_dev_advinfo to retrieve the
Identify Controller (CDAI_TYPE_NVME_CNTRL) and Namespace
(CDAI_TYPE_NVME_NS) data structures.

MFC r320984 (by imp):
This adds CAM pass(4) support for NVMe IO's. Applications indicate
the IO type (Admin or NVM) using XPT op-codes XPT_NVME_ADMIN or
XPT_NVME_IO.

MFC r303123 (by imp): Fix mismerge and include the nvme support.
Also, print out the name of any CCB's functions that's not supported.

MFC r326100:

Always null-terminate CAM periph_name and dev_name

Reported by: Coverity
CID: 1010039, 1010040, 1010041, 1010043
Reviewed by: ken, imp
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D13194

MFC r325857:

Remove a double free(9) in xpt_bus_register

In xpt_bus_register(), remove superfluous call to free(). This was mostly
benign since free(9) checks for NULL before doing anything, and
xpt_create_path() is nice enough to NULL out the pointer on failure.
However, it could've segfaulted if malloc(9) failed during
xpt_create_path().

Submitted by: gibbs
Sponsored by: Spectra Logic Corp

MFC r315673, r315674: Make CAM SIM lock optional.

For three years now CAM does not use SIM lock, but still enforces SIM to
use it. Remove this requirement, allowing SIMs to have any locking they
prefer, if they pass no mutex to cam_sim_alloc().

MFC r315082: Allow XPT_GDEV_STATS for UNCONFIGURED devices.

Queue statistics has nothing to do with presence or absence of INQUIRY
data, etc. Target mode devices are never configured, but have queues.

MFC r311305 (by asomers):
Always null-terminate ccb_pathinq.(sim_vid|hba_vid|dev_name)

The sim_vid, hba_vid, and dev_name fields of struct ccb_pathinq are
fixed-length strings. AFAICT the only place they're read is in
sbin/camcontrol/camcontrol.c, which assumes they'll be null-terminated.
However, the kernel doesn't null-terminate them. A bunch of copy-pasted code
uses strncpy to write them, and doesn't guarantee null-termination. For at
least 4 drivers (mpr, mps, ciss, and hyperv), the hba_vid field actually
overflows. You can see the result by doing "camcontrol negotiate da0 -v".

This change null-terminates those fields everywhere they're set in the
kernel. It also shortens a few strings to ensure they'll fit within the
16-character field.

PR: 215474
Reported by: Coverity
CID: 1009997 1010000 1010001 1010002 1010003 1010004 1010005
CID: 1331519 1010006 1215097 1010007 1288967 1010008 1306000
CID: 1211924 1010009 1010010 1010011 1010012 1010013 1010014
CID: 1147190 1010017 1010016 1010018 1216435 1010020 1010021
CID: 1010022 1009666 1018185 1010023 1010025 1010026 1010027
CID: 1010028 1010029 1010030 1010031 1010033 1018186 1018187
CID: 1010035 1010036 1010042 1010041 1010040 1010039

MFC r310360, r310361: Report UUID and MD5 LUN IDs.

MFC r309282: Explicitly initialize cdai.flags.

In SES driver uninitialized value caused unreliable physpath reporting.

MFC r306061:
Protect ccbq access with devq->send_mtx in the XPT_ABORT handler.

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Add place holder for SDIO CAM stuff for CCB XPT type.

Sponsored by: Netflix

Add NVME IO type.

Improve debugging of xpt.

Sponsored by: Netflix

Add "camcontrol reprobe" subcommand, and implement it for da(4).
This makes it possible to manually force updating capacity data
after the disk got resized. Without it it might be neccessary to
reboot before FreeBSD notices updated disk size under eg VMWare.

Discussed with: imp@
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D6108

sys/cam: spelling fixes in comments.

No functional change.

cam: unsign some types to match their definitions and avoid overflows.

numpatterns is u_int.

ctl:
CTL_NUM_MODE_PAGES comes from sizeof().
In struct:ctl_scsiio, kern_sg_entries is uint32_t.

MFC after: 2 weeks

New CAM I/O scheduler for FreeBSD. The default I/O scheduler is the same
as before. The common scheduling bits have moved from inline code in
each of the CAM periph drivers into a library that implements the
default scheduling.

In addition, a number of rate-limiting and I/O preference options can
be enabled by adding CAM_IOSCHED_NETFLIX to your config file. A number
of extra stats are also maintained. CAM_IOSCHED_NETFLIX isn't on by
default because it uses a separate BIO_READ and BIO_WRITE queue, so
doesn't honor BIO_ORDERED between these two types of operations. We
already didn't honor it for BIO_DELETE, and we don't depend on
BIO_ORDERED between reads and writes anywhere in the system (it is
currently used with BIO_FLUSH in ZFS to make sure some writes are
complete before others start and as a poor-man's soft dependency in
one place in UFS where we won't be issuing READs until after the
operation completes). However, out of an abundance of caution, it
isn't enabled by default.

Plus, this also brings in NCQ TRIM support for those SSDs that support
it. A black list is also provided for known rogues that use NCQ trim
as an excuse to corrupt the drive. It was difficult to separate out
into a separate commit.

This code has run in production at Netflix for over a year now.

Sponsored by: Netflix, Inc
Differential Revision: https://reviews.freebsd.org/D4609

Move to new value for XPT_GET_SIM_KNOB to avoid clash with XPT_ATA_IO.

Improve cam tracing a little by including the function code in the
traces for xpt_action. Note up-calls (down-calls?) to the SIM as well.

Differential Review: https://reviews.freebsd.org/D4382

Add asynchronous command support to the pass(4) driver, and the new
camdd(8) utility.

CCBs may be queued to the driver via the new CAMIOQUEUE ioctl, and
completed CCBs may be retrieved via the CAMIOGET ioctl. User
processes can use poll(2) or kevent(2) to get notification when
I/O has completed.

While the existing CAMIOCOMMAND blocking ioctl interface only
supports user virtual data pointers in a CCB (generally only
one per CCB), the new CAMIOQUEUE ioctl supports user virtual and
physical address pointers, as well as user virtual and physical
scatter/gather lists. This allows user applications to have more
flexibility in their data handling operations.

Kernel memory for data transferred via the queued interface is
allocated from the zone allocator in MAXPHYS sized chunks, and user
data is copied in and out. This is likely faster than the
vmapbuf()/vunmapbuf() method used by the CAMIOCOMMAND ioctl in
configurations with many processors (there are more TLB shootdowns
caused by the mapping/unmapping operation) but may not be as fast
as running with unmapped I/O.

The new memory handling model for user requests also allows
applications to send CCBs with request sizes that are larger than
MAXPHYS. The pass(4) driver now limits queued requests to the I/O
size listed by the SIM driver in the maxio field in the Path
Inquiry (XPT_PATH_INQ) CCB.

There are some things things would be good to add:

1. Come up with a way to do unmapped I/O on multiple buffers.
Currently the unmapped I/O interface operates on a struct bio,
which includes only one address and length. It would be nice
to be able to send an unmapped scatter/gather list down to
busdma. This would allow eliminating the copy we currently do
for data.

2. Add an ioctl to list currently outstanding CCBs in the various
queues.

3. Add an ioctl to cancel a request, or use the XPT_ABORT CCB to do
that.

4. Test physical address support. Virtual pointers and scatter
gather lists have been tested, but I have not yet tested
physical addresses or scatter/gather lists.

5. Investigate multiple queue support. At the moment there is one
queue of commands per pass(4) device. If multiple processes
open the device, they will submit I/O into the same queue and
get events for the same completions. This is probably the right
model for most applications, but it is something that could be
changed later on.

Also, add a new utility, camdd(8) that uses the asynchronous pass(4)
driver interface.

This utility is intended to be a basic data transfer/copy utility,
a simple benchmark utility, and an example of how to use the
asynchronous pass(4) interface.

It can copy data to and from pass(4) devices using any target queue
depth, starting offset and blocksize for the input and ouptut devices.
It currently only supports SCSI devices, but could be easily extended
to support ATA devices.

It can also copy data to and from regular files, block devices, tape
devices, pipes, stdin, and stdout. It does not support queueing
multiple commands to any of those targets, since it uses the standard
read(2)/write(2)/writev(2)/readv(2) system calls.

The I/O is done by two threads, one for the reader and one for the
writer. The reader thread sends completed read requests to the
writer thread in strictly sequential order, even if they complete
out of order. That could be modified later on for random I/O patterns
or slightly out of order I/O.

camdd(8) uses kqueue(2)/kevent(2) to get I/O completion events from
the pass(4) driver and also to send request notifications internally.

For pass(4) devcies, camdd(8) uses a single buffer (CAM_DATA_VADDR)
per CAM CCB on the reading side, and a scatter/gather list
(CAM_DATA_SG) on the writing side. In addition to testing both
interfaces, this makes any potential reblocking of I/O easier. No
data is copied between the reader and the writer, but rather the
reader's buffers are split into multiple I/O requests or combined
into a single I/O request depending on the input and output blocksize.

For the file I/O path, camdd(8) also uses a single buffer (read(2),
write(2), pread(2) or pwrite(2)) on reads, and a scatter/gather list
(readv(2), writev(2), preadv(2), pwritev(2)) on writes.

Things that would be nice to do for camdd(8) eventually:

1. Add support for I/O pattern generation. Patterns like all
zeros, all ones, LBA-based patterns, random patterns, etc. Right
Now you can always use /dev/zero, /dev/random, etc.

2. Add support for a "sink" mode, so we do only reads with no
writes. Right now, you can use /dev/null.

3. Add support for automatic queue depth probing, so that we can
figure out the right queue depth on the input and output side
for maximum throughput. At the moment it defaults to 6.

4. Add support for SATA device passthrough I/O.

5. Add support for random LBAs and/or lengths on the input and
output sides.

6. Track average per-I/O latency and busy time. The busy time
and latency could also feed in to the automatic queue depth
determination.

sys/cam/scsi/scsi_pass.h:
Define two new ioctls, CAMIOQUEUE and CAMIOGET, that queue
and fetch asynchronous CAM CCBs respectively.

Although these ioctls do not have a declared argument, they
both take a union ccb pointer. If we declare a size here,
the ioctl code in sys/kern/sys_generic.c will malloc and free
a buffer for either the CCB or the CCB pointer (depending on
how it is declared). Since we have to keep a copy of the
CCB (which is fairly large) anyway, having the ioctl malloc
and free a CCB for each call is wasteful.

sys/cam/scsi/scsi_pass.c:
Add asynchronous CCB support.

Add two new ioctls, CAMIOQUEUE and CAMIOGET.

CAMIOQUEUE adds a CCB to the incoming queue. The CCB is
executed immediately (and moved to the active queue) if it
is an immediate CCB, but otherwise it will be executed
in passstart() when a CCB is available from the transport layer.

When CCBs are completed (because they are immediate or
passdone() if they are queued), they are put on the done
queue.

If we get the final close on the device before all pending
I/O is complete, all active I/O is moved to the abandoned
queue and we increment the peripheral reference count so
that the peripheral driver instance doesn't go away before
all pending I/O is done.

The new passcreatezone() function is called on the first
call to the CAMIOQUEUE ioctl on a given device to allocate
the UMA zones for I/O requests and S/G list buffers. This
may be good to move off to a taskqueue at some point.
The new passmemsetup() function allocates memory and
scatter/gather lists to hold the user's data, and copies
in any data that needs to be written. For virtual pointers
(CAM_DATA_VADDR), the kernel buffer is malloced from the
new pass(4) driver malloc bucket. For virtual
scatter/gather lists (CAM_DATA_SG), buffers are allocated
from a new per-pass(9) UMA zone in MAXPHYS-sized chunks.
Physical pointers are passed in unchanged. We have support
for up to 16 scatter/gather segments (for the user and
kernel S/G lists) in the default struct pass_io_req, so
requests with longer S/G lists require an extra kernel malloc.

The new passcopysglist() function copies a user scatter/gather
list to a kernel scatter/gather list. The number of elements
in each list may be different, but (obviously) the amount of data
stored has to be identical.

The new passmemdone() function copies data out for the
CAM_DATA_VADDR and CAM_DATA_SG cases.

The new passiocleanup() function restores data pointers in
user CCBs and frees memory.

Add new functions to support kqueue(2)/kevent(2):

passreadfilt() tells kevent whether or not the done
queue is empty.

passkqfilter() adds a knote to our list.

passreadfiltdetach() removes a knote from our list.

Add a new function, passpoll(), for poll(2)/select(2)
to use.

Add devstat(9) support for the queued CCB path.

sys/cam/ata/ata_da.c:
Add support for the BIO_VLIST bio type.

sys/cam/cam_ccb.h:
Add a new enumeration for the xflags field in the CCB header.
(This doesn't change the CCB header, just adds an enumeration to
use.)

sys/cam/cam_xpt.c:
Add a new function, xpt_setup_ccb_flags(), that allows specifying
CCB flags.

sys/cam/cam_xpt.h:
Add a prototype for xpt_setup_ccb_flags().

sys/cam/scsi/scsi_da.c:
Add support for BIO_VLIST.

sys/dev/md/md.c:
Add BIO_VLIST support to md(4).

sys/geom/geom_disk.c:
Add BIO_VLIST support to the GEOM disk class. Re-factor the I/O size
limiting code in g_disk_start() a bit.

sys/kern/subr_bus_dma.c:
Change _bus_dmamap_load_vlist() to take a starting offset and
length.

Add a new function, _bus_dmamap_load_pages(), that will load a list
of physical pages starting at an offset.

Update _bus_dmamap_load_bio() to allow loading BIO_VLIST bios.
Allow unmapped I/O to start at an offset.

sys/kern/subr_uio.c:
Add two new functions, physcopyin_vlist() and physcopyout_vlist().

sys/pc98/include/bus.h:
Guard kernel-only parts of the pc98 machine/bus.h header with
#ifdef _KERNEL.

This allows userland programs to include <machine/bus.h> to get the
definition of bus_addr_t and bus_size_t.

sys/sys/bio.h:
Add a new bio flag, BIO_VLIST.

sys/sys/uio.h:
Add prototypes for physcopyin_vlist() and physcopyout_vlist().

share/man/man4/pass.4:
Document the CAMIOQUEUE and CAMIOGET ioctls.

usr.sbin/Makefile:
Add camdd.

usr.sbin/camdd/Makefile:
Add a makefile for camdd(8).

usr.sbin/camdd/camdd.8:
Man page for camdd(8).

usr.sbin/camdd/camdd.c:
The new camdd(8) utility.

Sponsored by: Spectra Logic
MFC after: 1 week

Remove compatibility shims for legacy ATA device names.

We got new ATA stack in FreeBSD 8.x, switched to it at 9.x, completely
removed old stack at 10.x, so at 11.x it is time to remove compat shims.

Make pass, sg and targ drivers respect HBA's maxio.

Previous limitation of 64K (DFLTPHYS) is quite annoying.

Initialize async_arg_ptr in xpt_async when called with async_code
AC_ADVINFO_CHANGED.

Without this change, newly inserted hard disks won't always have their
physical path device nodes created. The problem reproduces most readily
when attaching a large number of disks at once.

Differential Revision: https://reviews.freebsd.org/D2290
Reviewed by: mav, imp
MFC after: 2 weeks
Sponsored by: Spectra Logic

Be consistent with M_ZERO when allocating ccbs.

There are four places, all in cam_xpt.c, where ccbs are malloc'ed. Two of
these use M_ZERO, two don't. The two that don't meant that allocated ccbs
had trash in them making it hard to debug errors where they showed up. Due
to this, use M_ZERO all the time when allocating ccbs.

Submitted by: Scott Ferris <scott.ferris@isilon.com>
Sponsored by: EMC/Isilon Storage Division
Reviewed by: scottl, imp
Differential: https://reviews.freebsd.org/D2120

Make periphdriver_register() take XPT lock when modifying the periph_drivers
array.

This fixes a panic that sometimes occured when kldloading ctl.ko.

Reviewed by: mav@
MFC after: 1 month
Sponsored by: The FreeBSD Foundation

Add support for probing the SCSI VPD Extended Inquiry page (0x86).

This VPD page is effectively an extension of the standard Inquiry
data page, and includes lots of additional bits.

This commit includes support for probing the page in the SCSI probe code,
and an additional request type for the XPT_DEV_ADVINFO CCB. CTL already
supports the Extended Inquiry page.

Support for querying this page in the sa(4) driver will come later.

sys/cam/scsi/scsi_xpt.c:
Probe the Extended Inquiry page, if the device supports it, and
return it in response to a XPT_DEV_ADVINFO CCB if it is requested.

sys/cam/scsi/cam_ccb.h:
Define a new advanced information CCB data type, CDAI_TYPE_EXT_INQ.

sys/cam/cam_xpt.c:
Free the extended inquiry data in a device when the device goes
away.

sys/cam/cam_xpt_internal.h:
Add an extended inquiry data pointer and length to struct cam_ed.

sys/sys/param.h
Bump __FreeBSD_version for the addition of the new
CDAI_TYPE_EXT_INQ advanced information type.

Sponsored by: Spectra Logic
MFC after: 1 week

Restore the CAM XPT peripheral generation counter, and export it via sysctl.

Define it as an atomic uint32_t. These increments happen infrequently
enough for the atomic overhead to be a problem, and since they're now
independent atomics, they won't contend with xpt_lock_buses().

This counter is useful as a means of cheaply identifying whether any changes
have been made to the CAM peripheral list. Userland programs have no guarantee
that the counter won't change on them while being returned or while processing
the information, so they must be written accordingly.

Discussed with: ken, mav (in general)
MFC after: 1 week
Sponsored by: Spectra Logic

Remove extra mtx_unlock().

Submitted by: Dmitry Luhtionov <dmitryluhtionov@gmail.com>
MFC after: 1 week

When passing LUN IDs through treat ASCII values as fixed-length, not
interpreating NULLs as EOLs, but converting them to spaces.

SPC-4 does not tell that T10-based IDs should be NULL-terminated/padded.
And while it tells that it should include only ASCII chars (0x20-0x7F),
there are some USB sticks (SanDisk Ultra Fit), that have NULLs inside
the value. Treating NULLs as EOLs there made those LUN IDs non-unique.

MFC after: 1 week

Prevent overflow issues in timeout processing

Previously, any timeout value for which (timeout * hz) will overflow the
signed integer, will give weird results, since callout(9) routines will
convert negative values of ticks to '1'. For unsigned integer overflow we
will get sufficiently smaller timeout values than expected.

Switch from callout_reset, which requires conversion to int based ticks
to callout_reset_sbt to avoid this.

Also correct isci to correctly resolve ccb timeout.

This was based on the original work done by Eygene Ryabinkin
<rea@freebsd.org> back in 5 Aug 2011 which used a macro to help avoid
the overlow.

Differential Revision: https://reviews.freebsd.org/D1157
Reviewed by: mav, davide
MFC after: 1 month
Sponsored by: Multiplay

Use proper variable when looping through periphs with CAM_PERIPH_FREE.

PR: 194256
Submitted by: Scott M. Ferris <smferris@gmail.com>
MFC after: 3 days
Sponsored by: EMC/Isilon Storage Division

Correct a comment

Update CAM CCB accounting for the new status quo.

devq_openings counter lost its meaning after allocation queues has gone.
held counter is still meaningful, but problematic to update due to separate
locking of CCB allocation and queuing.

To fix that replace devq_openings counter with allocated counter. held is
now calculated on request as difference between number of allocated, queued
and active CCBs.

MFC after: 1 month

Pull in r267961 and r267973 again. Fix for issues reported will follow.

Revert r267961, r267973:

These changes prevent sysctl(8) from returning proper output,
such as:

1) no output from sysctl(8)
2) erroneously returning ENOMEM with tools like truss(1)
or uname(1)
truss: can not get etype: Cannot allocate memory

Extend the meaning of the CTLFLAG_TUN flag to automatically check if
there is an environment variable which shall initialize the SYSCTL
during early boot. This works for all SYSCTL types both statically and
dynamically created ones, except for the SYSCTL NODE type and SYSCTLs
which belong to VNETs. A new flag, CTLFLAG_NOFETCH, has been added to
be used in the case a tunable sysctl has a custom initialisation
function allowing the sysctl to still be marked as a tunable. The
kernel SYSCTL API is mostly the same, with a few exceptions for some
special operations like iterating childrens of a static/extern SYSCTL
node. This operation should probably be made into a factored out
common macro, hence some device drivers use this. The reason for
changing the SYSCTL API was the need for a SYSCTL parent OID pointer
and not only the SYSCTL parent OID list pointer in order to quickly
generate the sysctl path. The motivation behind this patch is to avoid
parameter loading cludges inside the OFED driver subsystem. Instead of
adding special code to the OFED driver subsystem to post-load tunables
into dynamically created sysctls, we generalize this in the kernel.

Other changes:
- Corrected a possibly incorrect sysctl name from "hw.cbb.intr_mask"
to "hw.pcic.intr_mask".
- Removed redundant TUNABLE statements throughout the kernel.
- Some minor code rewrites in connection to removing not needed
TUNABLE statements.
- Added a missing SYSCTL_DECL().
- Wrapped two very long lines.
- Avoid malloc()/free() inside sysctl string handling, in case it is
called to initialize a sysctl from a tunable, hence malloc()/free() is
not ready when sysctls from the sysctl dataset are registered.
- Bumped FreeBSD version to indicate SYSCTL API change.

MFC after: 2 weeks
Sponsored by: Mellanox Technologies

Report more readable state "-" for idle CAM scan thread.

Fix memory and references leak due to unfreed path in case we can't
allocate bus scan CCB.

MFC after: 2 weeks

Move xpt_run_devq() call before request completion callback where it was
originally.

I am not sure why exactly have I moved it during one of many refactorings
during camlock project, but obviously it opens race window that may cause
use after free panics during SIM (in reported cases umass(4)) detach.

MFC after: 2 weeks

Take additional reference on SCSI probe periph to cover its freeze count.

Otherwise periph may be invalidated and freed before single-stepping freeze
is dropped, causing use after free panic.

Replace several instances of -1 with appropriate CAM_*_WILDCARD and types.

It was equal before r259397, but for good or bad, not any more for LUNs.

This change fixes at least CAM debugging.

Some CAM locks polishing:
- Fix LOR and possible lock recursion when handling high-power commands.
Introduce new lock to protect left power quota and list of frozen devices.
- Correct locking around xpt periph creation.
- Remove seems never used XPT_FLAG_OPEN xpt periph flag.

Fix lock recursion, triggered by `smartctl -a /dev/adaX`.

printf() specifier updates to CAM to handle either 32-bit or 64-bit lun_id_t.

MFC after: 2 weeks

Minor (mostly cosmetical) addition to r256960.

Move CAM_UNQUEUED_INDEX setting to the last moment and under the periph lock.
This fixes race condition with cam_periph_ccbwait(), causing use-after-free.

Unconditionally acquire periph reference on CCB allocation failure.

cam_periph_acquire() can return error if periph already invalidated, but
that may be unacceptable and cause deadlock if the invalidated periph can't
be destroyed without "executing" the scheduled request.

Coverity CID: 1109822
MFC after: 2 months

Merge CAM locking changes from the projects/camlock branch to radically
reduce lock congestion and improve SMP scalability of the SCSI/ATA stack,
preparing the ground for the coming next GEOM direct dispatch support.

Replace big per-SIM locks with bunch of smaller ones:
- per-LUN locks to protect device and peripheral drivers state;
- per-target locks to protect list of LUNs on target;
- per-bus locks to protect reference counting;
- per-send queue locks to protect queue of CCBs to be sent;
- per-done queue locks to protect queue of completed CCBs;
- remaining per-SIM locks now protect only HBA driver internals.

While holding LUN lock it is allowed (while not recommended for performance
reasons) to take SIM lock. The opposite acquisition order is forbidden.
All the other locks are leaf locks, that can be taken anywhere, but should
not be cascaded. Many functions, such as: xpt_action(), xpt_done(),
xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM
lock to be held.

To keep compatibility and solve cases where SIM lock can't be dropped, all
xpt_async() calls in addition to xpt_done() calls are queued to completion
threads for async processing in clean environment without SIM lock held.

Instead of single CAM SWI thread, used for commands completion processing
before, use multiple (depending on number of CPUs) threads. Load balanced
between them using "hash" of the device B:T:L address.

HBA drivers that can drop SIM lock during completion processing and have
sufficient number of completion threads to efficiently scale to multiple
CPUs can use new function xpt_done_direct() to avoid extra context switch.
Make ahci(4) driver to use this mechanism depending on hardware setup.

Sponsored by: iXsystems, Inc.
MFC after: 2 months

Unify periph invalidation and destruction reporting.
Print message containing device model and serial number on invalidation.

Requested by: glebius
MFC after: 1 week

Unhide "Serial Number" lines from bootverbose. That information may be
useful for system administration to have in hard copy (in logs) if one of
several devices suddenly dies.

Requested by: glebius
MFC after: 1 week

Close the race on path ID allocation in xpt_bus_register() if two buses are
registered simultaneously. Due to topology unlock between the ID allocation
and the bus registration there is a chance that two buses may get the
same IDs. That is supposed reason of lock assertion panic in CAM during
initial bus scanning after new iscsid initiates two sessions same time.

Reported by: trasz
Approved by: re (glebus, marius)
MFC after: 2 weeks

Make sure the CCB xflags field is initialized to zero so that
CAM_EXTLUN_VALID is not erroneously set. Also add an XPORT_SRP
identifier to the known SCSI transports for the SCSI RDMA protocol, as
used, for example with Infiniband storage.

Reviewed by: scottl
Approved by: re (marius)

Re-do r255853. Along with adding back the API/ABI changes from the
original, this hides the contents of cam_compat.h from ktrace/kdump/truss,
avoiding problems there. There are no user-servicable parts in there, so
no need for those tools to be groping around in there.

Approved by: re

Revert r255853 pending fixes to build errors in usr.bin/kdump

Approved by: re (implicit)

Update the CAM API for FreeBSD 10:

- Remove the timeout_ch field. It's been deprecated since FreeBSD 7.0;
MPSAFE drivers should be managing their own timeout storage. The
remaining non-MPSAFE drivers have been modified to also manage their own
storage, and should be considered for updating to MPSAFE (or removal)
during the FreeBSD 10.x lifecycle.

- Add fields related to soft timeouts and quality of service, to be used
in upcoming work.

- Add room for more flags in the CCB header and path_inq structures.

- Begin support for extended 64-bit LUNs.

- Bump the CAM version number to 0x18, but add compat shims. Tested with
camcontrol and smartctl.

Reviewed by: nathanw, ken, kib
Approved by: re
Obtained from: Netflix

Add debug trace points for freeze/release device queue.

Add new attribute lunname to report only textual LUN-specific device IDs.
While lunid attribute prefers to report numeric ones, having both may be
useful in some situations.

Remove droping topology mutex after iterating 100 periphs in CAMGETPASSTHRU.
That is not so slow and so often operation to handle unneeded otherwise
xsoftc.xpt_generation and respective locking complications.

MFprojects/camlock r249006:
Pass SIM pointer as an argument to camisr_runqueue() instead of doneq
pointer.

MFprojects/camlock r249505:
Change CCB queue resize logic to be able safely handle overallocations:
- (re)allocate queue space in power of 2 chunks with 64 elements minimum
and never shrink it; with only 4/8 bytes per element size is insignificant.
- automatically reallocate the queue to double size if it is overflowed.
- if queue reallocation failed, store extra CCBs in unsorted TAILQ,
fetching them back as soon as some queue element is freed.

To free space in CCB for TAILQ linking, change highpowerq from keeping
high-power CCBs to keeping devices frozen due to high-power CCBs.

This encloses all pieces of queue resize logic inside of cam_queue.[ch],
removing some not obvious duties from xpt_release_ccb().

CAM and mps(4) driver scanning changes.

Add a PIM_NOSCAN flag to the CAM path inquiry CCB. This tells CAM
not to perform a rescan on a bus when it is registered.

We now use this flag in the mps(4) driver. Since it knows what
devices it has attached, it is more efficient for it to just issue
a target rescan on the targets that are attached.

Also, remove the private rescan thread from the mps(4) driver in
favor of the rescan thread already built into CAM. Without this
change, but with the change above, the MPS scanner could run before
or during CAM's initial setup, which would cause duplicate device
reprobes and announcements.

sys/param.h:
Bump __FreeBSD_version to 1000039 for the inclusion of the
PIM_RESCAN CAM path inquiry flag.

sys/cam/cam_ccb.h:
sys/cam/cam_xpt.c:
Added a PIM_NOSCAN flag. If a SIM sets this in the path
inquiry ccb, then CAM won't rescan the bus in
xpt_bus_regsister.

sys/dev/mps/mps_sas.c
For versions of FreeBSD that have the PIM_NOSCAN path
inquiry flag, don't freeze the sim queue during scanning,
because CAM won't be scanning this bus. Instead, hold
up the boot. Don't call mpssas_rescan_target in
mpssas_startup_decrement; it's redundant and I don't
know why it was in there.

Set PIM_NOSCAN in path inquiry CCBs.

Remove methods related to the internal rescan daemon.

Always use async events to trigger a probe for EEDP support.
In older versions of FreeBSD where AC_ADVINFO_CHANGED is
not available, use AC_FOUND_DEVICE and issue the
necessary READ CAPACITY manually.

Provide a path to xpt_register_async() so that we only
receive events for our own SCSI domain.

Improve error reporting in cases where setup for EEDP
detection fails.

sys/dev/mps/mps_sas.h:
Remove softc flags and data related to the scanner thread.

sys/dev/mps/mps_sas_lsi.c:
Unconditionally rescan the target whenever a device is added.

Sponsored by: Spectra Logic
MFC after: 1 week

Make some improvements to r253322 to really rescan target, not a bus.
Add there and in two more places checks for NULL on xpt_alloc_ccb_nowait().

Add infrastructure for doing compatibility shims, as has been sorely
needed for the last 10 years. Far too much of the internal API is
exposed, and every small adjustment causes applications to stop working.
To kick this off, bump the API version to 0x17 as should have been done
with r246713, but add shims to compensate. Thanks to the shims, there
should be no visible change in application behavior.

I have plans to do a significant overhaul of the API to harnen it for
the future, but until then, I welcome others to add shims for older
versions of the API.

Obtained from: Netflix

Make CAM return and GEOM DISK pass through new GEOM::lunid attribute.

SPC-4 specification states that serial number may be property of device,
but not a specific logical unit. People reported about FC storages using
serial number in that way, making it unusable for purposes of LUN multipath
detection. SPC-4 states that designators associated with logical unit from
the VPD page 83h "Device Identification" should be used for that purpose.
Report first of them in the new attribute in such preference order: NAA,
EUI-64, T10 and SCSI name string.

While there, make GEOM DISK properly report GEOM::ident in XML output also
using d_getattr() method, if available. This fixes serial numbers reporting
for SCSI disks in `geom disk list` output and confxml.

Discussed with: gibbs, ken
Sponsored by: iXsystems, Inc.
MFC after: 2 weeks

Added output of device QUIRKS for CAM and AHCI devices during boot.

Reviewed by: mav
Approved by: pjd (mentor)
MFC after: 2 weeks

Expose CAM_BOOT_DELAY as a kernel conf item now.

This allows users who boot without loader to adjust their environments
around slightly buggy or slow hardware.

PR: kern/161809
Submitted by: rozhuk.im@gmail.com
MFC after: 2 weeks

Remove owner field from struct cam_ed, unused at least since FreeBSD 7.

MFprojects/camlock r248982:
Stop abusing xpt_periph in random plases that really have no periph related
to CCB, for example, bus scanning. NULL value is fine in such cases and it
is correctly logged in debug messages as "noperiph". If at some point we
need some real XPT periphs (alike to pmpX now), quite likely they will be
per-bus, and not a single global instance as xpt_periph now.

MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905,
r248917, r248918, r248978, r249001, r249014, r249030:

Remove multilevel freezing mechanism, implemented to handle specifics of
the ATA/SATA error recovery, when post-reset recovery commands should be
allocated when queues are already full of payload requests. Instead of
removing frozen CCBs with specified range of priorities from the queue
to provide free openings, use simple hack, allowing explicit CCBs over-
allocation for requests with priority higher (numerically lower) then
CAM_PRIORITY_OOB threshold.

Simplify CCB allocation logic by removing SIM-level allocation queue.
After that SIM-level queue manages only CCBs execution, while allocation
logic is localized within each single device.

Suggested by: gibbs

MFprojects/camlock:
r249017:
Some cosmetic things:
- Unify device to target insertion inside xpt_alloc_device() instead of
duplicating it three times.
- Remove extra checks for empty lists of devices and targets on release
since zero refcount check also implies it.
- Reformat code to reduce indentation.

r249103:
- Add lock assertions to every point where reference counters are modified.
- When reference counters are reaching zero, add assertions that there are
no children items left.
- Add a bit more locking to the xptpdperiphtraverse().

MFprojects/camlock r248928:
Move CAM_DEBUG_CDB messages from the point of queuing to the point of
sending to SIM. That allows to inspect real requests execution order,
respecting priorities, freezing, etc.

MFC after: 2 weeks

Remove xpt_sim_opened(), the only consumer of which was atapicam, which is
now gone.

Use xpt_lock_buses() instead of equivalent mtx_lock(&xsoftc.xpt_topo_lock)
to unify the code.

Reform the busdma API so that new types may be added without modifying
every architecture's busdma_machdep.c. It is done by unifying the
bus_dmamap_load_buffer() routines so that they may be called from MI
code. The MD busdma is then given a chance to do any final processing
in the complete() callback.

The cam changes unify the bus_dmamap_load* handling in cam drivers.

The arm and mips implementations are updated to track virtual
addresses for sync(). Previously this was done in a type specific
way. Now it is done in a generic way by recording the list of
virtuals in the map.

Submitted by: jeff (sponsored by EMC/Isilon)
Reviewed by: kan (previous version), scottl,
mjacob (isp(4), no objections for target mode changes)
Discussed with: ian (arm changes)
Tested by: marius (sparc64), mips (jmallet), isci(4) on x86 (jharris),
amd64 (Fabian Keil <freebsd-listen@fabiankeil.de>)

Fix a panic during CAM EDT traversal.

The problem was a race condition between the EDT traversal used by
things like 'camcontrol devlist', and CAM peripheral driver
removal.

The EDT traversal code holds the CAM topology lock, and wants
to show devices that have been invalidated. It acquires a
reference to the peripheral to make sure the peripheral it is
examining doesn't go away.

However, because the peripheral removal code in camperiphfree()
drops the CAM topology lock to call the peripheral's destructor
routine, we can run into a situation where the EDT traversal
increments the peripheral reference count after free process is
already in progress. At that point, the reference count is
ignored, because it was 0 when we started the process.

Fix this race by setting a flag, CAM_PERIPH_FREE, that I previously
added and checked in xptperiphtraverse() and xptpdperiphtravsere(),
but failed to use. If the EDT traversal code sees that flag,
it will know that the peripheral free process has already started,
and that it should not access that peripheral.

Also, fix an inconsistency in the locking between
xptpdperiphtraverse() and xptperiphtraverse(). They now both
hold the CAM topology lock while calling the peripheral traversal
function.

cam_xpt.c: Change xptperiphtraverse() to hold the CAM topology
lock across calls to the traversal function.

Take out the comment in xptpdperiphtraverse() that
referenced the locking inconsistency.

cam_periph.c: Set the CAM_PERIPH_FREE flag when we are in the
process of freeing a peripheral driver.

Sponsored by: Spectra Logic Corporation
MFC after: 1 week

Remove priority enforcement from xpt_ation(). It is not good and even not
safe in some cases to reduce CCB priority after it was scheduled with high
priority. This fixes reproducible deadlock when command sent through the
pass interface while ATA XPT recovers from command timeout.

Instead of that enforce priority at passioctl(). libcam provides no obvious
interface to specify CCB priority and so much (all?) code specifies zero
(highest) priority. This change limits pass CCBs priority to NORMAL run
level, allowing XPT to complete bus and device recovery after reset before
running any payload.

Don't exclude XPT SIM from locking in xpt_create_path_unlocked().
We don't want xpt periph, device, target or bus disappeared because of
incorrect reference counting.

Use xpt_create_path_unlocked() for initial debug path compilation because
we are not holding respective SIM lock.

Fix XPT_DEBUG paths operations locking:
- Extend the lock to cover xpt_path_release() for the new path.
- While xpt_action() is called while holding right SIM lock for the new
bus, the old path release may require different SIM lock. So we have
to temporary drop the new lock and get the old one.

XPT_DEV_MATCH is probably the only xpt_action() method that is called
without holding SIM lock. It really doesn't need that lock, but adding it
removes that specific exception, allowing to assert locking there later.

Submitted by: ken@ (earlier version)

Protect xpt_getattr() calls with the SIM lock and assert that.

Submitted by: ken@ (earlier version)

Use separate malloc buckets for CAM devices, CCBs and paths. This will
make it easier to track down the source of any use after free problems.

Submitted by: ken@

Don't duplicate path/ccb allocation code, use existing functions.

Really handle xpt_compile_path() error in xpt_bus_register() instead of
print error message and probably crash just after it on NULL dereference.

Found by: Clang Static Analyzer

Implement media change notification for DA and CD removable media devices.
It includes three parts:
1) Modifications to CAM to detect media media changes and report them to
disk(9) layer. For modern SATA (and potentially UAS) devices it utilizes
Asynchronous Notification mechanism to receive events from hardware.
Active polling with TEST UNIT READY commands with 3 seconds period is used
for incapable hardware. After that both CD and DA drivers work the same way,
detecting two conditions: "NOT READY: Medium not present" after medium was
detected previously, and "UNIT ATTENTION: Not ready to ready change, medium
may have changed". First one reported to disk(9) as media removal, second
as media insert/change. To reliably receive second event new
AC_UNIT_ATTENTION async added to make UAs broadcasted to all periphs by
generic error handling code in cam_periph_error().
2) Modifications to GEOM core to handle media remove and change events.
Media removal handled by spoiling all consumers attached to the provider.
Media change event also schedules provider retaste after spoiling to probe
new media. New flag G_CF_ORPHAN was added to consumers to reflect that
consumer is in process of destruction. It allows retaste to create new
geom instance of the same class, while previous one is still dying.
3) Modifications to some GEOM classes: DEV -- to report media change
events to devd; VFS -- to handle spoiling same as orphan to prevent
accessing replaced media. PART class already handles spoiling alike to
orphan.

Reviewed by: silence on geom@ and scsi@
Tested by: avg
Sponsored by: iXsystems, Inc. / PC-BSD
MFC after: 2 months

In camisr() clear CAM_SIM_ON_DONEQ flag after camisr_runqueue() purged SIM
done queue. Clearing it before caused extra SIM queueing in some cases.
It was invisible during normal operation, but during USB device unplug and
respective SIM destruction it could keep pointer on SIM without having
counted reference and as result crash the system by use afer free.

Reported by: hselasky
MFC after: 1 week

Fix several reference counting and object lifetime issues between
the pass(4) and enc(4) drivers and devfs.

The pass(4) driver uses the destroy_dev_sched() routine to
schedule its device node for destruction in a separate thread
context. It does this because the passcleanup() routine can get
called indirectly from the passclose() routine, and that would
cause a deadlock if the close routine tried to destroy its own
device node.

In any case, once a particular passthrough driver number, e.g.
pass3, is destroyed, CAM considers that unit number (3 in this
case) available for reuse.

The problem is that devfs may not be done cleaning up the previous
instance of pass3, and will panic if isn't done cleaning up the
previous instance.

The solution is to get a callback from devfs when the device node
is removed, and make sure we hold a reference to the peripheral
until that happens.

Testing exposed some other cases where we have reference counting
issues, and those were also fixed in the pass(4) driver.

cam_periph.c: In camperiphfree(), reorder some of the operations.

The peripheral destructor needs to be called before
the peripheral is removed from the peripheral is
removed from the list. This is because once we
remove the peripheral from the list, and drop the
topology lock, the peripheral number may be reused.
But if the destructor hasn't been called yet, there
may still be resources hanging around (like devfs
nodes) that haven't been fully cleaned up.

cam_xpt.c: Add an argument to xpt_remove_periph() to indicate
whether the topology lock is already held.

scsi_enc.c: Acquire an extra reference to the peripheral during
registration, and release it once we get a callback
from devfs indicating that the device node is gone.

Call destroy_dev_sched_cb() in enc_oninvalidate()
instead of calling destroy_dev() in the cleanup
routine.

scsi_pass.c: Add reference counting to handle peripheral and
devfs object lifetime issues.

Add a reference to the peripheral and the devfs
node in the peripheral registration.

Don't attempt to add a physical path alias if the
peripheral has been marked invalid.

Release the devfs reference once the initial
physical path alias taskqueue run has completed.

Schedule devfs node destruction in the
passoninvalidate(), and release our peripheral
reference in a new routine, passdevgonecb() once
the devfs node is gone. This allows the peripheral
to fully go away, and the peripheral destructor,
passcleanup(), will get called.

MFC after: 3 days
Sponsored by: Spectra Logic

Add CAM_DEBUG_INFO debug messages for periph created/invalidated/destroyed
and for asyncs sent.

To make CAM debugging easier, compile in some debug flags (CAM_DEBUG_INFO,
CAM_DEBUG_CDB, CAM_DEBUG_PERIPH and CAM_DEBUG_PROBE) by default.
List of these flags can be modified with CAM_DEBUG_COMPILE kernel option.
CAMDEBUG kernel option still enables all possible debug, if not overriden.

Additional 50KB of kernel size is a good price for the ability to debug
problems without rebuilding the kernel. In case where size is important,
debugging can be compiled out by setting CAM_DEBUG_COMPILE option to 0.

Replace #ifdef CAMDEBUG + if + panic() with single KASSERT().

Rewrite enabling NCQ for SATA devices in a way more alike to SCSI TCQ.
This allows to control it with `camcontrol negotiate adaX -T (en|dis)able`
on the fly, same as for SCSI devices.

Sponsored by: iXsystems, Inc.

Use AC_GETDEV_CHANGED async to notify ada driver about DMA and NCQ status
change. Now that allows switching between PIO and DMA modes on the fly.

Allow to change number of openings (used tags) for ATA/SATA devices
via `camcontrol tags ... -N ...`. There is no need to tune it in
usual cases, but some users want to have it for debugging purposes.

MFC after: 2 weeks

MFprojects/zfsd:
- Add low-level support for SATA Enclosure Management Bridge (SEMB)
devices -- SATA equivalents of the SCSI SES/SAF-TE devices.
- Add some utility functions for SCSI SAF-TE devices access.

Sponsored by: iXsystems, Inc.

Add two functions xpt_batch_start() and xpt_batch_done() to the CAM SIM KPI
to allow drivers to handle request completion directly without passing
them to the CAM SWI thread removing extra context switch.
Modify all ATA/SATA drivers to use them.

Reviewed by: gibbs, ken
MFC after: 2 weeks

Add CAM infrastructure to allow reporting when a drive's long read capacity
data changes.

cam_ccb.h: Add a new advanced information type, CDAI_TYPE_RCAPLONG,
for long read capacity data.

cam_xpt_internal.h:
Add a read capacity data pointer and length to struct cam_ed.

cam_xpt.c: Free the read capacity buffer when a device goes away.
While we're here, make sure we don't leak memory for other
malloced fields in struct cam_ed.

scsi_all.c: Update the scsi_read_capacity_16() to take a uint8_t * and
a length instead of just a pointer to the parameter data
structure. This will hopefully make this function somewhat
immune to future changes in the parameter data.

scsi_all.h: Add some extra bit definitions to struct
scsi_read_capacity_data_long, and bump up the structure
size to the full size specified by SBC-3.

Change the prototype for scsi_read_capacity_16().

scsi_da.c: Register changes in read capacity data with the transport
layer. This allows the transport layer to send out an
async notification to interested parties. Update the
dasetgeom() API.

Use scsi_extract_sense_len() instead of
scsi_extract_sense().

scsi_xpt.c: Add support for the new CDAI_TYPE_RCAPLONG advanced
information type.

Make sure we set the physpath pointer to NULL after freeing
it. This allows blindly freeing it in the struct cam_ed
destructor.

sys/param.h: Bump __FreeBSD_version from 1000005 to 1000006 to make it
easier for third party drivers to determine that the read
capacity data async notification is available.

camcontrol.c,
mptutil/mpt_cam.c:
Update these for the new scsi_read_capacity_16() argument
structure.

Sponsored by: Spectra Logic

Fix a race condition in CAM peripheral free handling, locking
in the CAM XPT bus traversal code, and a number of other periph level
issues.

cam_periph.h,
cam_periph.c: Modify cam_periph_acquire() to test the CAM_PERIPH_INVALID
flag prior to allowing a reference count to be gained
on a peripheral. Callers of this function will receive
CAM_REQ_CMP_ERR status in the situation of attempting to
reference an invalidated periph. This guarantees that
a peripheral scheduled for a deferred free will not
be accessed during its wait for destruction.

Panic during attempts to drop a reference count on
a peripheral that already has a zero reference count.

In cam_periph_list(), use a local sbuf with SBUF_FIXEDLEN
set so that mallocs do not occur while the xpt topology
lock is held, regardless of the allocation policy of the
passed in sbuf.

Add a new routine, cam_periph_release_locked_buses(),
that can be called when the caller already holds
the CAM topology lock.

Add some extra debugging for duplicate peripheral
allocations in cam_periph_alloc().

Treat CAM_DEV_NOT_THERE much the same as a selection
timeout (AC_LOST_DEVICE is emitted), but forgo retries.

cam_xpt.c: Revamp the way the EDT traversal code does locking
and reference counting. This was broken, since it
assumed that the EDT would not change during
traversal, but that assumption is no longer valid.

So, to prevent devices from going away while we
traverse the EDT, make sure we properly lock
everything and hold references on devices that
we are using.

The two peripheral driver traversal routines should
be examined. xptpdperiphtraverse() holds the
topology lock for the entire time it runs.
xptperiphtraverse() is now locked properly, but
only holds the topology lock while it is traversing
the list, and not while the traversal function is
running.

The bus locking code in xptbustraverse() should
also be revisited at a later time, since it is
complex and should probably be simplified.

scsi_da.c: Pay attention to the return value from cam_periph_acquire().

Return 0 always from daclose() even if the disk is now gone.

Add some rudimentary error injection support.

scsi_sg.c: Fix reference counting in the sg(4) driver.

The sg driver was calling cam_periph_release() on close,
but never called cam_periph_acquire() (which increments
the reference count) on open.

The periph code correctly complained that the sg(4)
driver was trying to decrement the refcount when it
was already 0.

Sponsored by: Spectra Logic
MFC after: 2 weeks

Move the scsi_da_bios_params() prototype from pc98_machdep.h to md_var.h
where the prototype for pc98_ata_disk_firmware_geom_adjust() also lives
in order to avoid an #ifdef'ed include in cam(4).

For sparc64 also adjust the geometry of da(4) driven disks to not overflow
the 16-bit cylinders field of the VTOC8 disk label (at around 502GB). The
geometry chosen for disks above that limit allows to use disks up to 2TB,
which is the limit of the extended VTOC8 format. The geometry used for
disks smaller than the 16-bit cylinders limit stays the same as used by
cam_calc_geometry(9) for extended translation.
Thanks to Hans-Joerg Sirtl for providing hardware for testing this change.

MFC after: 3 days

Introduce CAM_SIM_POLLED SIM flag, indicating that it works in polling mode.
It blocks CAM SWI usage on requests completion, unneeded because of polling
and denied during kernel dumping because of blocked scheduler.

Before r198899 there was periph flag CAM_PERIPH_POLLED, but that was wrong,
because there is whole SIM is polled or handled by SWI, not a single periph.

Tested by: kib
MFC after: 1 month

Fixes for sure bus reference miscounting and potential device and
target reference miscounts. It also adds a helper function to get
the current reference counts for components of cam_path for debug
aid. One minor style(9) change.

Partially Obtained from: Chuck Tuffli (Emulex)
Reviewed by: scsi@ (ken)
Approved by: re (kib)
MFC after: 1 month

cam/cam_xpt.c:
In camisr_runqueue(), we need to run the sims queue regardless of
whether or not the current peripheral has more work to do. This
reverts a change mistakenly made in revision 223081.

Reported by: ache

Lay groundwork in CAM for recording and reporting physical path and
other device attributes stored in the CAM Existing Device Table (EDT).
This includes some infrastructure requried by the enclosure services
driver to export physical path information.

Make the CAM device advanced info interface accept store requests.

sys/cam/scsi/scsi_all.c:
sys/cam/scsi/scsi_all.h:
- Replace scsi_get_sas_addr() with a scsi_get_devid() which takes
a callback that decides whether to accept a particular descriptor.
Provide callbacks for NAA IEEE Registered addresses and for SAS
addresses, replacing the old function. This is needed because
the old function doesn't work for an enclosure address for a SAS
device, which is not flagged as a SAS address, but is NAA IEEE
Registered. It may be worthwhile merging this interface with the
devid match interface.
- Add a few more defines for some device ID fields.

sbin/camcontrol/camcontrol.c:
- Update for the CCB_DEV_ADVINFO interface change.

cam/cam_xpt_internal.h:
- Add the new fields for the physical path string to the CAM EDT.
cam/cam_ccb.h:
- Rename CCB_GDEV_ADVINFO to simply CCB_DEV_ADVINFO, and the ccb
structure to ccb_dev_advinfo.
- Add a flag that changes this CCB's action to store, rather than
the default, retrieve.
- Add a new buffer type, CDAI_TYPE_PHYS_PATH, for the new CAM EDT
physpath field.
- Remove the never-implemented transport & proto flags.
cam/cam_xpt.c:
cam/cam_xpt.h:
- Add xpt_getattr(), which provides a wrapper for fetching a device's
attribute using the GEOM strings as key. This method currently
supports "GEOM::ident" and "GEOM::physpath".

Submitted by: will
Reviewed by : gibbs

Extend the XPT_DEV_MATCH api to allow a device search by device ID.
As far as the API is concerned, device ID is a binary blob to be
interpreted by the transport layer. The SCSI implementation assumes
it is an array of VPD device ID descriptors.

sys/cam/cam_ccb.h:
Create a new structure, device_id_match_pattern, and
update the XPT_DEV_MATCH datastructures and flags so
that this pattern type can be used.

sys/cam/cam_xpt.c:
- A single pattern matching on both inquiry data and device
ID is invalid. Report any violators.
- Pass device ID match requests through to the new routine
scsi_devid_match(). The direct call of a SCSI routine is
a layering violation, but no worse than the one a few
lines up that checks inquiry data. Defer cleaning this
up until our future, larger, rototilling of CAM.
- Zero out cam_ed and cam_et nodes on allocation. Prior to
this change, device_id_len and device_id were not inialized,
preventing proper detection of the presence of this
information.

sys/cam/scsi/scsi_all.c:
sys/cam/scsi/scsi_all.h:
Add the scsi_match_devid() routine.

Add a helper function for extracting peripherial driver names

sys/cam/cam_periph.c:
sys/cam/cam_periph.h:
Add the cam_periph_list() method which fills an sbuf
with a comma delimited list of the peripheral instances
associated with a given CAM path.

Add a helper functions for SCSI commands used by the SES driver.

sys/cam/scsi/scsi_all.c:
sys/cam/scsi/scsi_all.h:
Add structure definitions and csio filling functions for
the receive diagnostic results and send diagnostic commands.

Misc CAM XPT cleanups.

sys/cam/cam_xpt.c:
Broadcast AC_FOUND_DEVICE and AC_PATH_REGISTERED
events at the time async event handlers are attached
even when registering just for events on a partitular
SIM. Previously, you had to register for these
events on all SIMs in the system in order to get
the initial broadcast even though subsequent device
and path arrivals would be delivered.

sys/cam/cam_xpt.c:
Remove SIM mutex held asserts from path accessors.
CAM paths are reference counted and it is this
reference count, not the sim mutex, that garantees
they are stable.

Sponsored by: Spectra Logic Corporation

Do not report legacy unit numbers (do not create legacy aliases) for disks
on port multiplier ports above first two. They don't fit into ATA_STATIC_ID
scheme and so can't be mapped properly. No need to pollute dev.

Make CAM_DEBUG_CDB also dump ATA commands in addition to SCSI.

- Add shim to simplify migration to the CAM-based ATA. For each new adaX
device in /dev/ create symbolic link with adY name, trying to mimic old ATA
numbering. Imitation is not complete, but should be enough in most cases to
mount file systems without touching /etc/fstab.
- To know what behavior to mimic, restore ATA_STATIC_ID option in cases
where it was present before.
- Add some more details to UPDATING.

Remove some used variables.

Found with: Clang Static Analyzer

Fix typos - remove duplicate "the".

PR: bin/154928
Submitted by: Eitan Adler <lists at eitanadler.com>
MFC after: 3 days

Fix up a few more sysctl(9) mis-typing found in various LINT builds.

Add Serial Management Protocol (SMP) passthrough support to CAM.

This includes support in the kernel, camcontrol(8), libcam and the mps(4)
driver for SMP passthrough.

The CAM SCSI probe code has been modified to fetch Inquiry VPD page 0x00
to determine supported pages, and will now fetch page 0x83 in addition to
page 0x80 if supported.

Add two new CAM CCBs, XPT_SMP_IO, and XPT_GDEV_ADVINFO. The SMP CCB is
intended for SMP requests and responses. The ADVINFO is currently used to
fetch cached VPD page 0x83 data from the transport layer, but is intended
to be extensible to fetch other types of device-specific data.

SMP-only devices are not currently represented in the CAM topology, and so
the current semantics are that the SIM will route SMP CCBs to either the
addressed device, if it contains an SMP target, or its parent, if it
contains an SMP target. (This is noted in cam_ccb.h, since it will change
later once we have the ability to have SMP-only devices in CAM's topology.)

smp_all.c,
smp_all.h: New helper routines for SMP. This includes
SMP request building routines, response parsing
routines, error decoding routines, and structure
definitions for a number of SMP commands.

libcam/Makefile: Add smp_all.c to libcam, so that SMP functionality
is available to userland applications.

camcontrol.8,
camcontrol.c: Add smp passthrough support to camcontrol. Several
new subcommands are now available:

'smpcmd' functions much like 'cmd', except that it
allows the user to send generic SMP commands.

'smprg' sends the SMP report general command, and
displays the decoded output. It will automatically
fetch extended output if it is available.

'smppc' sends the SMP phy control command, with any
number of potential options. Among other things,
this allows the user to reset a phy on a SAS
expander, or disable a phy on an expander.

'smpmaninfo' sends the SMP report manufacturer
information and displays the decoded output.

'smpphylist' displays a list of phys on an
expander, and the CAM devices attached to those
phys, if any.

cam.h,
cam.c: Add a status value for SMP errors
(CAM_SMP_STATUS_ERROR).

Add a missing description for CAM_SCSI_IT_NEXUS_LOST.

Add support for SMP commands to cam_error_string().

cam_ccb.h: Rename the CAM_DIR_RESV flag to CAM_DIR_BOTH. SMP
commands are by nature bi-directional, and we may
need to support bi-directional SCSI commands later.

Add the XPT_SMP_IO CCB. Since SMP commands are
bi-directional, there are pointers for both the
request and response.

Add a fill routine for SMP CCBs.

Add the XPT_GDEV_ADVINFO CCB. This is currently
used to fetch cached page 0x83 data from the
transport later, but is extensible to fetch many
other types of data.

cam_periph.c: Add support in cam_periph_mapmem() for XPT_SMP_IO
and XPT_GDEV_ADVINFO CCBs.

cam_xpt.c: Add support for executing XPT_SMP_IO CCBs.

cam_xpt_internal.h: Add fields for VPD pages 0x00 and 0x83 in struct
cam_ed.

scsi_all.c: Add scsi_get_sas_addr(), a function that parses
VPD page 0x83 data and pulls out a SAS address.

scsi_all.h: Add VPD page 0x00 and 0x83 structures, and a
prototype for scsi_get_sas_addr().

scsi_pass.c: Add support for mapping buffers in XPT_SMP_IO and
XPT_GDEV_ADVINFO CCBs.

scsi_xpt.c: In the SCSI probe code, first ask the device for
VPD page 0x00. If any VPD pages are supported,
that page is required to be implemented. Based on
the response, we may probe for the serial number
(page 0x80) or device id (page 0x83).

Add support for the XPT_GDEV_ADVINFO CCB.

sys/conf/files: Add smp_all.c.

mps.c: Add support for passing in a uio in mps_map_command(),
so we can map a S/G list at once.

Add support for SMP passthrough commands in
mps_data_cb(). SMP is a special case, because the
first buffer in the S/G list is outbound and the
second buffer is inbound.

Add support for warning the user if the busdma code
comes back with more buffers than will work for the
command. This will, for example, help the user
determine why an SMP command failed if busdma comes
back with three buffers.

mps_pci.c: Add sys/uio.h.

mps_sas.c: Add the SAS address and the parent handle to the
list of fields we pull from device page 0 and cache
in struct mpssas_target. These are needed for SMP
passthrough.

Add support for the XPT_SMP_IO CCB. For now, this
CCB is routed to the addressed device if it supports
SMP, or to its parent if it does not and the parent
does. This is necessary because CAM does not
currently support SMP-only nodes in the topology.

Make SMP passthrough support conditional on
__FreeBSD_version >= 900026. This will make it
easier to MFC this change to the driver without
MFCing the CAM changes as well.

mps_user.c: Un-staticize mpi_init_sge() so we can use it for
the SMP passthrough code.

mpsvar.h: Add a uio and iovecs into struct mps_command for
SMP passthrough commands.

Add a cm_max_segs field to struct mps_command so
that we can warn the user if busdma comes back with
too many segments.

Clear the cm_reply when a command gets freed. If
it is not cleared, reply frames will eventually get
freed into the pool multiple times and corrupt the
pool. (This fix is from scottl.)

Add a prototype for mpi_init_sge().

sys/param.h: Bump __FreeBSD_version to 900026 for the for the
inclusion of the XPT_GDEV_ADVINFO and XPT_SMP_IO
CAM CCBs.

Mostly revert r203420, and add similar functionality into ada(4) since the
existing code caused problems with some SCSI controllers.

A new sysctl kern.cam.ada.spindown_shutdown has been added that controls
whether or not to spin-down disks when shutting down.
Spinning down the disks unloads/parks the heads - this is
much better than removing power when the disk is still
spinning because otherwise an Emergency Unload occurs which may cause damage
to the actuator.

PR: kern/140752
Submitted by: olli
Reviewed by: arundel
Discussed with: mav
MFC after: 2 weeks

Decrease poll interval from 1000 to 100us. This significantly reduces
kernel dump time, especially with minidump.

Don't lock buses around a call to xptperiphlistmatch- the buses will be
locked at appropriate places.

MFC after: 1 week
X-MFC: 208752

Implement the usage of Report Luns as part of SCSI probing for SCP3 or
better devices. This can be disabled on a per-device basis using quirks as
well.

This also handles the case where there is actually no connected LUN 0
(which can definitely be the case for storage arrays).

Reviewed by: scsi@
MFC after: 1 month

Protect periph drivers list and rearrange things to minimize the chance of
stepping oneself during probing.

Don't blindly decrement a periph probe count.

Reviewed by: scsi@
Obtained from: Alexander Motin, Atillio Rao, Others
MFC after: 1 month

Add a new primitive, XPT_SCAN_TGT, to cover the range between scanning a
whole bus (XPT_SCAN_BUS) and a single lun on that bus (XPT_SCAN_LUN).

It's less resource comsumptive than scanning a whole bus when the
caller knows only one target has changes.

Reviewed by: scsi@
Sponsored by: Panasas
MFC after: 1 month

Add xpt_schedule_dev_sendq() call, lost at r203108. It is not needed in
usual operation, but required in some conditions to make queue running
after being shrinked.

MFC after: 3 days

Revert r198705.
As scottl@ noticed, max_target/max_lun was intended to be only a hint for
existing bus scanner. Some FC/SAS SIMs report fake values there, that are
smaller then maximum supported IDs. In that case this check makes impossible
manual scan outside hinted range.

For ATA/SATA SIMs respective check was instead implemented at SIM level.
Newer SCSI SIMs expected to have these checks at driver or firmware level.
Some older SCSI SIMs have no this check and the issues will get back there.

For unhandled actions in xpt_action_default, remember to call
xpt_done for queued requests. This solves the problem of
indefinite hangs for unspecified transports when XPT_SCAN_BUS
is called.

A few minor cosmetics elsewhere.

MFC after: 1 week

Fix newlines broken at r204220.

Make xpt_rescan() more correct, as it was before r197208: do not use
XPT_SCAN_LUN for wildcard LUN, fall back to XPT_SCAN_BUS.

Virtualize transport part of periph announcement.

Improve output for controllers that doesn't report SATA speed.

Disable kern.cam.power_down ATM. It doesn't work fine on some controllers.

MFp4:
Make CAM to stop all attached devices on system shutdown.
It allows devices to park heads, reducing stress on power loss.
Add `kern.cam.power_down` tunable and sysctl to controll it.

Change the way in which fake async events generated. Do not use
taskqueue for lock decoupling, as it causes unwanted races.

Some missed parts for r203376.

MFp4: Large set of CAM inprovements.

- Unify bus reset/probe sequence. Whenever bus attached at boot or later,
CAM will automatically reset and scan it. It allows to remove duplicate
code from many drivers.
- Any bus, attached before CAM completed it's boot-time initialization,
will equally join to the process, delaying boot if needed.
- New kern.cam.boot_delay loader tunable should help controllers that
are still unable to register their buses in time (such as slow USB/
PCCard/ CardBus devices), by adding one more event to wait on boot.
- To allow synchronization between different CAM levels, concept of
requests priorities was extended. Priorities now split between several
"run levels". Device can be freezed at specified level, allowing higher
priority requests to pass. For example, no payload requests allowed,
until PMP driver enable port. ATA XPT negotiate transfer parameters,
periph driver configure caching and so on.
- Frozen requests are no more counted by request allocation scheduler.
It fixes deadlocks, when frozen low priority payload requests occupying
slots, required by higher levels to manage theit execution.
- Two last changes were holding proper ATA reinitialization and error
recovery implementation. Now it is done: SATA controllers and Port
Multipliers now implement automatic hot-plug and should correctly
recover from timeouts and bus resets.
- Improve SCSI error recovery for devices on buses without automatic sense
reporting, such as ATAPI or USB. For example, it allows CAM to wait, while
CD drive loads disk, instead of immediately return error status.
- Decapitalize diagnostic messages and make them more readable and sensible.
- Teach PMP driver to limit maximum speed on fan-out ports.
- Make boot wait for PMP scan completes, and make rescan more reliable.
- Fix pass driver, to return CCB to user level in case of error.
- Increase number of retries in cd driver, as device may return several UAs.

MFp4:
- Extend XPT-SIM transfer settings control API. Now it allows to report to
SATA SIM number of tags supported by each device, implement ATA mode and
SATA revision negotiation for both SATA and PATA SIMs.
- Make ahci(4) and siis(4) to use submitted maximum tag number, when
scheduling requests. It allows to support NCQ on devices with lower tags
count then controller supports.
- Make PMP driver to report attached devices connection speeds.
- Implement ATA mode negotiation between user settings, device and
controller capabilities.

MFp4:
Remove code that years ago was closing race between request submission
to SIM and device/SIM freeze. That race become impossible after moving from
spl to mutex locking, while this workaround causes some unexpected effects.

MFp4:
- Move tagged queueing control from ADA to ATA XPT. It allows to control
device command queue length correctly. First step to support < 32 tags.
- Limit queue for non-tagged devices by 2 slots for ahci(4) and siis(4).
- Implement quirk matching for ATA devices.
- Move xpt_schedule_dev_sendq() from header to source file.
- Move delayed queue shrinking to the more expected place - element freeing.
- Remove some SCSIsms in ATA.

MFp4:
- Remove CAM_PERIPH_POLLED flag. It is broken by design. Polling can't be
periph flag. May be SIM, may be CCB, but now it works fine just without it.
- Remove check unused for at least five years. If we will ever have non-BIO
devices in CAM, this check is smallest of what we will need.
- If several controllers complete requests same time, call swi_sched()
only once.

The async callback could free the device. If it is a broadcast async,
it doesn't hold device reference, so take our own reference.

Submitted by: thompsa

MFp4:
Fix reference counting bug, when device unreferenced before then
invalidated. To do it, do not handle validity flag as another
reference, but explicitly modify reference count each time flag is
modified.

Discovered by: thompsa

MFp4:
- Reduce code duplication in ATA XPT and PMP driver.
- Move PIO size setting from ada driver to ATA XPT. It is XPT business
to negotiate transfer details. ada driver is now stateless.
- Report PIO size to SIM. It is required for correct PATA SIM operation.
- Tune PMP scan timings. It workarounds some problems with SiI.
- If reset hapens during PMP initialization - restart it.
- Introduce early-initialized periph drivers, which are used during initial
scan process. Use it for xpt, probe, aprobe and pmp. It gives pmp chance
to finish scan before mountroot and numerate devices in right order.

MFp4:
Ensure target/lun passed from user-level supported on this bus.
Scanning unsupported IDs causes different issues from duplicate
devices to system crash.

Fix the build.

Replace most of priority numbers with defines. No logical changes.

After thinking again, implement cam_ccbq_fini().
This is effectively NULL change, but makes this API a bit more consistent.

MFp4:
Do not search for bus when it is not needed,

Fix some instances where CAM rescans get hung up or take a long time to
complete.

Also, allow xpt_rescan() to rescan a LUN instead of a full bus.

MFC after: 3 days
Sponsored by: Copan Systems, Inc.

MFp4:
Remove duplicate qfrozen_cnt variable from struct cam_ed.
ccbq.queue.qfrozen_cnt should be used instead.

Remove unneeded CAM_SIM_MPSAFE check.

s/bus %d/scbus%d/ in some messages to correct terminology.

MFp4:
Report scbusX in xpt_announce_periph() to less confuse users by two
different bus addressing schemes.

Avoid extra swi_sched() call, if this SIM is already queued.
It reduces overhead for coalesced command completions.

Fix iSCSI initiator and vpo driver operation, broken by CAM changes.

Reviewed by: scottl, Danny Braniss
Approved by: re (rwatson)

Add 8Gb support (isp_2500). Fix a fair number of configuration and
firmware loading bugs.

Target mode support has received some serious attention to make it
more usable and stable.

Some backward compatible additions to CAM have been made that make
target mode async events easier to deal with have also been put
into place.

Further refinement and better support for NP-IV (N-port Virtualization)
is now in place.

Code for release prior to RELENG_7 has been stripped away for code clarity.

Sponsored by: Copan Systems

Reviewed by: scottl, ken, jung-uk kim
Approved by: re

Separate the parallel scsi knowledge out of the core of the XPT, and
modularize it so that new transports can be created.

Add a transport for SATA

Add a periph+protocol layer for ATA

Add a driver for AHCI-compliant hardware.

Add a maxio field to CAM so that drivers can advertise their max
I/O capability. Modify various drivers so that they are insulated
from the value of MAXPHYS.

The new ATA/SATA code supports AHCI-compliant hardware, and will override
the classic ATA driver if it is loaded as a module at boot time or compiled
into the kernel. The stack now support NCQ (tagged queueing) for increased
performance on modern SATA drives. It also supports port multipliers.

ATA drives are accessed via 'ada' device nodes. ATAPI drives are
accessed via 'cd' device nodes. They can all be enumerated and manipulated
via camcontrol, just like SCSI drives. SCSI commands are not translated to
their ATA equivalents; ATA native commands are used throughout the entire
stack, including camcontrol. See the camcontrol manpage for further
details. Testing this code may require that you update your fstab, and
possibly modify your BIOS to enable AHCI functionality, if available.

This code is very experimental at the moment. The userland ABI/API has
changed, so applications will need to be recompiled. It may change
further in the near future. The 'ada' device name may also change as
more infrastructure is completed in this project. The goal is to
eventually put all CAM busses and devices until newbus, allowing for
interesting topology and management options.

Few functional changes will be seen with existing SCSI/SAS/FC drivers,
though the userland ABI has still changed. In the future, transports
specific modules for SAS and FC may appear in order to better support
the topologies and capabilities of these technologies.

The modularization of CAM and the addition of the ATA/SATA modules is
meant to break CAM out of the mold of being specific to SCSI, letting it
grow to be a framework for arbitrary transports and protocols. It also
allows drivers to be written to support discrete hardware without
jeopardizing the stability of non-related hardware. While only an AHCI
driver is provided now, a Silicon Image driver is also in the works.
Drivers for ICH1-4, ICH5-6, PIIX, classic IDE, and any other hardware
is possible and encouraged. Help with new transports is also encouraged.

Submitted by: scottl, mav
Approved by: re

Revert r190676,190677

The geom and CAM changes for root_hold are the wrong solution for USB design
quirks.

Requested by: scottl

Add interleaving root hold tokens from the CAM probe to disk_create and geom
provider tasting. This is needed for disk attachments that happen after threads
are running in the boot process.

Tested by: rnoland

GCC attacks!

Fix parallel SCSI negotiation in the CAM_NEW_TRAN_CODE world order.
Overzealous sanity checks were locking the sync_rate and offset values to
zero, thanks to a twisty maze of recursive code.

Instrument the probe state machine so that things like DV can be tracked.

In the case that the probe has determined that it can't query the device for
a serial number, fall through to the next case so that initial negotiation
still happens. Without this, devices were showing up with only 1 available
tag opening, leading to observations of very poor I/O performance.

This should fix problems reported with VMWare Fusion and ESX. Early
generation MPT-SAS controllers with SATA disks might also be affected.
HP CISS controllers are also likely affected, as are many other
pseudo-scsi disk subsystems.

Remove an overzealous check.

Submitted by: das
Reviewed by: scottl
Approved by: rwatson (mentor, implicit)
Sponsored by: FreeBSD Foundation

Protect against NULL pointer dereference.

Reviewed by: scottl
Approved by: rwatson (mentor)
Sponsored by: FreeBSD Foundation

Guard against NULL pointer dereference.

Reviewed by: scottl
Approved by: rwatson (mentor)
Sponsored by: FreeBSD Foundation
Found with: Coverity Prevent(tm)
CID: 1847

Fix use after free.

Reviewed by: scottl
Approved by: rwatson (mentor)
Sponsored by: FreeBSD Foundation
Found with: Coverity Prevent(tm)
CID: 3712

Make "kldunload atapicam" return EBUSY instead of deadlocking when a device
created by atapicam is being kept opened or mounted. This is probably just
a temporary solution until we invent something better.

Reviewed by: scottl
Approved by: rwatson (mentor)
Sponsored by: FreeBSD Foundation
Reported by: Jaakko Heinonen

cam: Retry TEST UNIT READY command if not successful.

This fixes problems with discovering some USB devices that are very slow to
respond during initialisation.

When a USB device is inserted, CAM performs the sequence:
1) INQUIRY
2) INQUIRY (second time with other parameters)
3) TEST UNIT READY
4) READ CAPACITY

Before this change CAM didn't check if TEST UNIT READY was successful and went
on blindly to the next state and sent READ CAPACITY. If the device was still
not ready by then, CAM ended with error message. This patch adds checking for the
status of TEST UNIT READY command and retrying up to 10 times with 0.5 sec
interval.

Submitted by: Grzegorz Bernacki gjb ! semihalf dot com
Reviewed by: scottl

Periph driver fixes, second try.

Reviewed by: scottl
Approved by: rwatson (mentor)
Sponsored by: FreeBSD Foundation

Revert r186186 for now; it breaks stuff.

Approved by: rwatson (mentor)

Fix locking in periph drivers - don't try to unlock periph
that was already deallocated.

Reviewed by: scottl
Approved by: rwatson (mentor)
Sponsored by: FreeBSD Foundation

Add SIM refcounting. This is slightly different from what DragonFly
does - in DragonFly, it's cam_sim_release() what actually frees the
SIM; cam_sim_free does nothing more than calling cam_sim_release().
Here, we drain in cam_sim_free, waiting for refcount to drop to zero.
We cannot do the same think DragonFly does, because after cam_sim_free
returns, client would destroy the sim->mtx, and CAM would trip over
an initialized mutex.

Reviewed by: scottl
Approved by: rwatson (mentor)
Sponsored by: FreeBSD Foundation

Get rid of dead_sim. There is no way to make it work - any attempt
to actually use it would panic on mtx operation, as dead_sim doesn't
have a proper mutex. Even if it had a properly initialized mutex,
it wouldn't have properly locked and owned one.

Reviewed by: scottl
Approved by: rwatson (mentor)
Sponsored by: FreeBSD Foundation

Fix whitespace.

When probing a newly found device, don't automatically assume that the
device supports retrieving a serial number. Instead, first query the
list of VPD pages it does support, and only query the serial number if
it's supported, else silently move on. This eliminates a lot of noise
during verbose booting, and will likely eliminate the need for most
NOSERIAL quirks.

Add a missing unlock to an error path fro XPT_DEBUG handling.

Forced commit to note that the previous revision also covered:

PR: 119683
Submitted by: Adam McDougall <mcdouga9@egr.msu.edu>

Correct the kernel config hint in a printf statement.

Submitted by: R.Mahmatkhanov <cvs-src@yandex.ru>
Reviewed by: scottl
Approved by: rwatson (mentor)
MFC after: 3 days

style(9)

Add a missing mutex unlock.

Reported by: Michael Jung

Rename the kthread_xxx (e.g. kthread_create()) calls
to kproc_xxx as they actually make whole processes.
Thos makes way for us to add REAL kthread_create() and friends
that actually make theads. it turns out that most of these
calls actually end up being moved back to the thread version
when it's added. but we need to make this cosmetic change first.

I'd LOVE to do this rename in 7.0 so that we can eventually MFC the
new kthread_xxx() calls.

Quirk for WD Mybook. Seems they do not like serial number queries.

Submitted by: Jason Harris
PR: 107495
Approved by: re@ (blanket)

Prepare for future integration between CAM and newbus. xpt_bus_register
now takes a device_t to be the parent of the bus that is being created.
Most SIMs have been updated with a reasonable argument, but a few exceptions
just pass NULL for now. This argument isn't used yet and the newbus
integration likely won't be ready until after 7.0-RELEASE.

Despite several examples in the kernel, the third argument of
sysctl_handle_int is not sizeof the int type you want to export.
The type must always be an int or an unsigned int.

Remove the instances where a sizeof(variable) is passed to stop
people accidently cut and pasting these examples.

In a few places this was sysctl_handle_int was being used on 64 bit
types, which would truncate the value to be exported. In these
cases use sysctl_handle_quad to export them and change the format
to Q so that sysctl(1) can still print them.

Release the bus reference that is acquired when doing a CAMIOCOMMAND
ioctl. This should make SIM removal a bit more robust.

Remove an assertion in xpt_print_path() that makes debugging code much more
difficult.

Add a helper function for registering async callbacks. Besides
eliminating a lot of duplicated code, this also fixes a locking edge case.

Eliminate the use of M_TEMP.

Only schedule the xpt_finishconfig_task once. This fixes some potential
panics on boot.

Retire the spl() markers. Add in some minor missed locking as a result.

Inline cam_periph_lock|unlock to make debugging easier. Use more
CAM_SIM_LOCK() more uniformly.

Zero the CCBs when mallocing them.

Split the camisr into per-SIM done queues. This optimizes the locking a
little bit and allows for direct dispatch of the doneq from certain
contexts that would otherwise face recursive locking problems.

Revert a driver API change to xpt_alloc_ccb that isn't necessary. Fix a
couple of associated error checks.

For the XPT_SASYNC_CB operation, only decouple the broadcast to the bus
and device lists instead of decoupling the whole operation. This avoids
problems with SIMs going away.

Remove Giant from CAM. Drivers (SIMs) now register a mutex that CAM will
use to synchornize and protect all data objects that are used for that
SIM. Drivers that are not yet MPSAFE register Giant and operate as
usual. RIght now, no drivers are MPSAFE, though a few will be changed
in the coming week as this work settles down.

The driver API has changed, so all CAM drivers will need to be recompiled.
The userland API has not changed, so tools like camcontrol do not need to
be recompiled.

A silly buglet found by Coverity- check the return value from
cam_periph_acquire.

Add an xpt_rescan function and a thread that will field
rescan requests. The purpose of this is to allow a SIM
(or other entities) to request a bus rescan and have it
then fielded in a different (process) context from the
caller.

There are probably better ways to accomplish this, but
it's a very small change that helps solve a number of
problems.

Reviewed by: Justin, Ken and Scott.
MFC after: 2 weeks

fix botch

Pay attention to return value from xpt_bus_register in xpt_init.

Obtained from: Xin Li (Coverity)
MFC after: 3 days

Make cam_xpt's pronouncements match camcontrol
(Tagged -> Command) Queueing.

Add an xpt_print function to reduce most of the xpt_print_path/printf
pairs. Convert the core code to use it.

Reviewed by: scsi (various folks weighing in with good advice)
MFC after: 2 weeks

Initial cut at Basic Domain Validation- just add some states to
traverse through during probing.

Reviewed by: scsi (scottl)

XPT_GET_TRAN_SETTINGS can fail- so check for an error and
don't blindly assume it succeeded.

Fix the hanging chad if the NEW_TRAN_CODE change and change the
qualifier to force async from cur_spi to spi.

2nd and final commit that moves us to CAM_NEW_TRAN_CODE
as the default.

Reviewed by multitudes.

New Dell 1950/2950 SES backplane drops off the bus if you poke
at greater then lun 0.

MFC after: 1 week

null commit to provide commit message to previous
at the request of Sam Leffler: The previous commit
established min and maxtags for VMware pseudo disks
to fix a submitted PR.

PR: 103130
Submitted by: Shusuke Shinomiya
MFC after: 1 day

Grumble. VMWare ESX and VMWare WorkStation have *slightly* different
inquiry data (pointlessly different I might add). Pick the common
parts.

MFC after: 1 day

VMware disk volumes are only on LUN 0, and
some VMware HBAs have known bugs with commands
sent to other than LUN 0.

MFC after: 1 day

Add PIM_SEQSCAN for HBA misc flags and code that understands
what to do with it.

This forces us to scan targets sequentially, not in parallel.
The reason we might want to do this is that SPI negotiation
might not work right at the SIM level if we try to do it
in parallel. We *could* fix this for each SIM where this is
broken, but it's a lot harder to do that when we can simply
ask CAM to probe sequentially.

If PIM_SEQSCAN is not set (default), the original behaviour for
probing is unchanged.

LUN probing is still done in parallel for each target in either
case.

While we're at it, clean up some resource leakage for error
cases.

Reviewed by: ken, scott, scsi@
MFC after: 1 week

Handle some of the inquiry flags that have come into
usage as of SPC2r20. Specifically, handle the BQueue
flag which will indicate that a device supports the
Basic Queueing model (no Head of Queue or Ordered tags).
When this flag is set, SID_CmdQueue is clear. This has
causes FreeBSD to assume that the device did not support
tagged operations.

MFC after: 1 month

CAM_NEW_TRAN_CODE placeholders for SAS support.

Reviewed by: nate, ken, justin
MFC after: 1 month

When deregistering a bus, attempt to flush out all outstanding
operations before returning. Point the bus at a dummy cam_sim
structure so that any CCBs will complete immediately with a
CAM_DEV_NOT_THERE status, and ensure that any xpt_schedule() calls
on the bus's devices will immediately call the peripheral's
periph_start() routine. Also repeat the async messages because
devices that were part of the way through being probed may appear
after the original AC_LOST_DEVICE was sent, and would otherwise
never go away.

These changes make it possible to deregister a bus and free the SIM
at most stages during bus probing without the usual crashes in
camisr(). In particular, plugging in a umass device and then
unplugging it as soon as the first probe messages appeared would
almost always result in a crash. Now the device just goes away with
a few CAM errors and all references to the CAM bus, target and
device are dropped correctly.

Make the exploring of all luns supported by an HBA more of a
tunable (until we get REPORT LUNS in place).

If we're probing luns, and each probe succeeds, we keep going past
lun 7 if we're a SCSI3 or better device (until we fail to probe).

If we're probing luns, and a probe fails, we only keep going if
we're quirked *for* it (CAM_QUIRK_HILUNS), and if we're not quirked
*against* it (CAM_QUIRK_NOHILUNS), or we're a SCSI3 or better device
and the tunable (kern.cam.cam_srch_hi) is set non-zero.

Reviewed by: nate@rootlabs.org, gibbs@scsiguy.com, ken@kdm.com, scottl@samsco.org
MFC after: 1 week

- Providing fine-grained malloc statistic by replacing M_DEVBUF with
module-specific malloc types. These should help us to pinpoint the
possible memory leakage in the future.
- Implementing xpt_alloc_ccb_nowait() and replacing all malloc/free based
CCB memory management with xpt_alloc_ccb[_nowait]/xpt_free_ccb. Hopefully
this would be helpful if someday we move the CCB allocator to use UMA
instead of malloc().

Encouraged by: jeffr, rwatson
Reviewed by: gibbs, scottl
Approved by: re (scottl)

Fixing a memory leak in xpt_release_device(), which can be quickly
(depends on how many memory you have) observed through "tar -tvf /dev/sa0."

Without this patch, RELENG_5 and HEAD panics with something like:

kmem_malloc(4096): kmem_map too small: 42258432 total allocated

RELENG_4 doesn't panic but spews following errors:

camq_init: - cannot malloc array!

Reviewed by: gibbs, scottl
Approved by: re (scottl)
MFC after: 3 days

Do not initialize path variable with useless value just before
xpt_create_path overwrites it anyway.

Noticed by: Coverity Prevent analysis tool

avoid use after free

Submitted by: gibbs
Reviewed by: mdodd
Noticed by: Coverity Prevent analysis tool

Restore queue depth settings across tag disable events. The system often
disables tag queuing temporarily in order to allow controllers a window
to safely perform transfer negotiation with non-compliant devices. Before
this change, CAM would restore the queue depth to the controller specified
maximum or device quirk level rather than any depth determined by reactions
to QUEUE FULL/BUSY events or an explicit user setting.

During device probe, initialize the flags field for XPT_SCAN_BUS.
The uninitialized value often confused CAM into not bothering to
issue an AC_FOUND_DEVICE async event for new devices. The reason
this bug wasn't reported earlier is that CAM manually announces
devices after the initial system bus scans.

MFC: 3 days

Check the correct periph pointer after search for it.

Submitted by: Coverity Prevent analysis tool

Provide locking for the ccb_bioq. This allows xpt_done() to be called without
Giant held. In camisr(), move the ccb_bioq elements to a temporary local list
and then process the elements off of that list. This enables the list to be
processed by only taking the ccb_bioq_lock once and only for a very short
time.

ccb_bioq_lock is a leaf mutex, so it's fine to call xpt_done() with other
locks held. This is just a very minor step in the work to lock CAM, but
it allows us to avoid some messy locking/unlock dances in certain drivers.

Add quirk for easyRAID ER5 alias areca ARC-6010.
It reports itself as SCSI-3 but doesnt like getting probed on high luns
because it hangs hard after finding itself again on lun 32...

Suggested by: Kenneth Merry

This is a somewhat imperfect means to try and bring FreeBSD forward in
its ability to automatically scan and attach luns for modern storage
which has luns in the 0..1000 range, not 0..7.

The correct thing would be to do REPORT LUNS for devices whose LUN0
version shows a version >= SCSI3, but lacking that we should be able
to search higher than LUN 7 if we're >= SCSI3 with no ill effects.

This change keeps all of the QUIRK_HILUNS quirks, obeys the QUIRK_NOLUNS,
and introduces a QUIRK_NOHILUNS which will keep searches above LUN 7
happening for devices that report >= SCSI3 compliance. I doubt the latter
will be needed, but you never know.

This allowed me to randomly scan and attach > 500 disks at a time in
a situation where quirking for QUIRK_HILUNS wasn't practical (the
vendor id and product id changes of the virtualization changes
constantly).

Reviewed by: ken@freebsd.org, scottl@freebsd.org, gibbs@freebsd.org
MFC after: 2 weeks

Start each of the license/copyright comments with /*-

Correct another location where inquiry length as calculated from the
inquiry additional length field is off by one.

MFC: 3 days

Correct an off-by-one in the calculation of the inquiry buffer length
to request from devices during the "long inquiry" portion of our probe.
This same bug was fixed in the 4.x stream a few years ago, but the fix
was never propogated to -current.

This fix is slightly different than in -stable:

o Use offsetof() instead of a hard coded constant so as the make
the code more self-explainatory.

o Round odd long inquiry lengths up so as to avoid tickling ignore
wide residue bugs in broken parallel SCSI devices running with a
wide transfer negotiation.

MFC: 3 days

Remove the camnet swi and CAM_PERIPH_NET. It has never been used, and given
that netowrk-over-scsi never really took off, there is little chance that
it will ever be needed.

Do a pass over all modules in the kernel and make them return EOPNOTSUPP
for unknown events.

A number of modules return EINVAL in this instance, and I have left
those alone for now and instead taught MOD_QUIESCE to accept this
as "didn't do anything".

Revert rev 1.140, it was a bit pre-mature.

Add a quirk for ciss to stop trying to read the serial number off
of logical volumes.

Do the dreaded s/dev_t/struct cdev */
Bump __FreeBSD_version accordingly.

Compare the *number* of patterns to zero, not the *pointer* to the
patterns. (These lines are correct the other two times they appear.)

Reported by: "Ted Unangst" <tedu@coverity.com>
Approved by: rwatson (mentor), ken (scsi)

Device megapatch 4/6:

Introduce d_version field in struct cdevsw, this must always be
initialized to D_VERSION.

Flip sense of D_NOGIANT flag to D_NEEDGIANT, this involves removing
four D_NOGIANT flags and adding 145 D_NEEDGIANT flags.

Device megapatch 1/6:

Free approx 86 major numbers with a mostly automatically generated patch.

A number of strategic drivers have been left behind by caution, and a few
because they still (ab)use their major number.

Sprinkle GIANT_REQUIRED asserts around the xpt layer to aid with locking the
SCSI drivers.

Use __FBSDID().

Add /* FALLTHROUGH */
Move /* FALLTHROUGH */ to correct location.
Remove unused variable(s).

Found by: FlexeLint

Centralize the devstat handling for all GEOM disk device drivers
in geom_disk.c.

As a side effect this makes a lot of #include <sys/devicestat.h>
lines not needed and some biofinish() calls can be reduced to
biodone() again.

Gigacommit to improve device-driver source compatibility between
branches:

Initialize struct cdevsw using C99 sparse initializtion and remove
all initializations to default values.

This patch is automatically generated and has been tested by compiling
LINT with all the fields in struct cdevsw in reverse order on alpha,
sparc64 and i386.

Approved by: re(scottl)

Back out M_* changes, per decision of the TRB.

Approved by: trb

Remove M_TRYWAIT/M_WAITOK/M_WAIT. Callers should use 0.
Merge M_NOWAIT/M_DONTWAIT into a single flag M_NOWAIT.

Note that the large Hitachi's are also CAM_QUIRK_HILUNS

Make SCSI_DELAY setable at boot time and runtime via the
kern.cam.scsi_delay tunable/sysctl.

Reviewed by: mdodd, njl

Add \n to various printfs. grep shows these should be the last ones.

Replace various spelling with FALLTHROUGH which is lint()able

Make Veritas Storage Appliance a HILUNS device.

MFC after: 1 day

Maxtor Personal Storage 3000XT (Firewire) hangs upon
serial number probing.

MFC after: 1 week

Define the kern.cam sysctl in the cam layer, rather than multiply in several
peripheral drivers. Remove Ken's comment to the effect that this needed
to be done.

Staticise camnet_ih and cambio_ih.

Change the preemption code for software interrupt thread schedules and
mutex releases to not require flags for the cases when preemption is
not allowed:

The purpose of the MTX_NOSWITCH and SWI_NOSWITCH flags is to prevent
switching to a higher priority thread on mutex releease and swi schedule,
respectively when that switch is not safe. Now that the critical section
API maintains a per-thread nesting count, the kernel can easily check
whether or not it should switch without relying on flags from the
programmer. This fixes a few bugs in that all current callers of
swi_sched() used SWI_NOSWITCH, when in fact, only the ones called from
fast interrupt handlers and the swi_sched of softclock needed this flag.
Note that to ensure that swi_sched()'s in clock and fast interrupt
handlers do not switch, these handlers have to be explicitly wrapped
in critical_enter/exit pairs. Presently, just wrapping the handlers is
sufficient, but in the future with the fully preemptive kernel, the
interrupt must be EOI'd before critical_exit() is called. (critical_exit()
can switch due to a deferred preemption in a fully preemptive kernel.)

I've tested the changes to the interrupt code on i386 and alpha. I have
not tested ia64, but the interrupt code is almost identical to the alpha
code, so I expect it will work fine. PowerPC and ARM do not yet have
interrupt code in the tree so they shouldn't be broken. Sparc64 is
broken, but that's been ok'd by jake and tmm who will be fixing the
interrupt code for sparc64 shortly.

Reviewed by: peter
Tested on: i386, alpha

Bring the probe inquiry code in line with the SCSI spec.

It is legal to have a device with device type 0x1f, that just means
that the device is of unknown type. Instead, only check the peripheral
qualifier when deciding whether or not to reject a device based on its
inquiry information.

Tested by: julian
MFC after: 3 weeks

Disabled tagged commands for Hitachi Jura-C series with J8A8 firmware.

PR: 23536
Submitted by: amagai@nue.org

Back out revision 1.117; the correct solution is to upgrade the
firmware on the drive in question. The pattern also matched far
too many Hitachi drives.

Spotted by: dg

Hitachi DK32 disks have problems with tagged queuing under load.

PR: kern/23536
Submitted by: Y.Amagai <amagai@nue.org>
MFC after: 1 week

KSE Milestone 2
Note ALL MODULES MUST BE RECOMPILED
make the kernel aware that there are smaller units of scheduling than the
process. (but only allow one thread per process at this time).
This is functionally equivalent to teh previousl -current except
that there is a thread associated with each process.

Sorry john! (your next MFC will be a doosie!)

Reviewed by: peter@freebsd.org, dillon@freebsd.org

X-MFC after: ha ha ha ha

cleanup some of the XPORT_FC code

Hints overhaul:
- Replace some very poorly thought out API hacks that should have been
fixed a long while ago.
- Provide some much more flexible search functions (resource_find_*())
- Use strings for storage instead of an outgrowth of the rather
inconvenient temporary ioconf table from config(). We already had a
fallback to using strings before malloc/vm was running anyway.

first blush at some FC path inquiry settings

Do NOLUNS dance for oddball Exabyte. We *really* need to do this as hints.

Correct match for A5000 SES instance.

PR: 19887
MFC after: 2 weeks

Remove unneeded includes of sys/ipl.h and machine/ipl.h.

Rewrite of the CAM error recovery code.

Some of the major changes include:

- The SCSI error handling portion of cam_periph_error() has
been broken out into a number of subfunctions to better
modularize the code that handles the hierarchy of SCSI errors.
As a result, the code is now much easier to read.

- String handling and error printing has been significantly
revamped. We now use sbufs to do string formatting instead
of using printfs (for the kernel) and snprintf/strncat (for
userland) as before.

There is a new catchall error printing routine,
cam_error_print() and its string-based counterpart,
cam_error_string() that allow the kernel and userland
applications to pass in a CCB and have errors printed out
properly, whether or not they're SCSI errors. Among other
things, this helped eliminate a fair amount of duplicate code
in camcontrol.

We now print out more information than before, including
the CAM status and SCSI status and the error recovery action
taken to remedy the problem.

- sbufs are now available in userland, via libsbuf. This
change was necessary since most of the error printing code
is shared between libcam and the kernel.

- A new transfer settings interface is included in this checkin.
This code is #ifdef'ed out, and is primarily intended to aid
discussion with HBA driver authors on the final form the
interface should take. There is example code in the ahc(4)
driver that implements the HBA driver side of the new
interface. The new transfer settings code won't be enabled
until we're ready to switch all HBA drivers over to the new
interface.

src/Makefile.inc1,
lib/Makefile: Add libsbuf. It must be built before libcam,
since libcam uses sbuf routines.

libcam/Makefile: libcam now depends on libsbuf.

libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the
sbuf sources from sys/kern.

bsd.libnames.mk: Add LIBSBUF.

camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically
linked, we can't depend on the dynamic linker
to pull in libsbuf.

camcontrol.c: Use cam_error_print() instead of checking for
CAM_SCSI_STATUS_ERROR on every failed CCB.

sbuf.9: Change the prototypes for sbuf_cat() and
sbuf_cpy() so that the source string is now a
const char *. This is more in line wth the
standard system string functions, and helps
eliminate warnings when dealing with a const
source buffer.

Fix a typo.

cam.c: Add description strings for the various CAM
error status values, as well as routines to
look up those strings.

Add new cam_error_string() and
cam_error_print() routines for userland and
the kernel.

cam.h: Add a new CAM flag, CAM_RETRY_SELTO.

Add enumerated types for the various options
available with cam_error_print() and
cam_error_string().

cam_ccb.h: Add new transfer negotiation structures/types.

Change inq_len in the ccb_getdev structure to
be "reserved". This field has never been
filled in, and will be removed when we next
bump the CAM version.

cam_debug.h: Fix typo.

cam_periph.c: Modularize cam_periph_error(). The SCSI error
handling part of cam_periph_error() is now
in camperiphscsistatuserror() and
camperiphscsisenseerror().

In cam_periph_lock(), increase the reference
count on the periph while we wait for our lock
attempt to succeed so that the periph won't go
away while we're sleeping.

cam_xpt.c: Add new transfer negotiation code. (ifdefed
out)

Add a new function, xpt_path_string(). This
is a string/sbuf analog to xpt_print_path().

scsi_all.c: Revamp string handing and error printing code.
We now use sbufs for much of the string
formatting code. More of that code is shared
between userland the kernel.

scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly
useful in the first place.

Add a new error action, SS_REQSENSE. (Send a
request sense and then retry the command.)
This is useful when the controller hasn't
performed autosense for some reason.

Change the default actions around a bit.

scsi_cd.c,
scsi_da.c,
scsi_pt.c,
scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection
timeouts shouldn't be covered by a sense flag.

scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO.

Get rid of the last vestiges of a read/write
interface.

libkern/bsearch.c,
sys/libkern.h,
conf/files: Add bsearch.c, which is needed for some of the
new table lookup routines.

aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if
CAM_NEW_TRAN_CODE is defined.

sbuf.h,
subr_sbuf.c: Add the appropriate #ifdefs so sbufs can
compile and run in userland.

Change sbuf_printf() to use vsnprintf()
instead of kvprintf(), which is only available
in the kernel.

Change the source string for sbuf_cpy() and
sbuf_cat() to be a const char *.

Add __BEGIN_DECLS and __END_DECLS around
function prototypes since they're now exported
to userland.

kdump/mkioctls: Include stdio.h before cam.h since cam.h now
includes a function with a FILE * argument.

Submitted by: gibbs (mostly)
Reviewed by: jdp, marcel (libsbuf makefile changes)
Reviewed by: des (sbuf changes)
Reviewed by: ken

Send the remains (such as I have located) of "block major numbers" to
the bit-bucket.

In xpt_set_transfer_settings(), force async if either the period or the
offset is set to 0.

Re-arrange the DT limiting code so that we don't end up setting the period
to 0xa if the user really wants async. The previous behavior seemed to
confuse the aic(4) driver.

PR: kern/22733
Reviewed by: gibbs

- Catch up to the new swi API changes:
- Use swi_* function names.
- Use void * to hold cookies to handlers instead of struct intrhand *.
- In sio.c, use 'driver_name' instead of "sio" as the name of the driver
lock to minimize diffs with cy(4).

Change the peripheral driver list from a linker set to module driven
driver registration. This should allow things like da, sa, cd etc to be
in seperate KLD's to the cam core and make them preloadable.

Mechanical change to use <sys/queue.h> macro API instead of
fondling implementation details.

Created with: sed(1)
Reviewed by: md5(1)

Back out rev 1.102. The IBM DNES drives work fine for me, jedgar and other
folks.

My guess is that reducing the number of tags is just masking the real
problem for the PR submitter. I'll re-open the PR and see if I can work
with the submitter to diagnose the problem.

PR: 21139

Mark the (ancient) MaxOptix Tahiti 1 drive as a known rogue that would
respond to all LUNs.

Reviewed by: ken

Add tags quirk for IBM DNES drives.

PR: 21139
Obtained from: campt@miralink.com

Fix async notifications for listners registered to wildcard nodes. For
example, a client registered to receive specific events for bus 0, target *,
lun *, was not receiving notifications.

Reviewed by: ken@FreeBSD.org

Treat engine inquiry the same as path inquiry in xptioctl().
Return ENOTSUP for any opcode that is not supported by the XPT
device.

Add back a missing local declaration that seems to have been deleted
by my last commit.

Allow async transfer negotiation updates to modify tagging behavior.
The XPT uses this to prevent tags from being used on parallel SCSI
interfaces immediately after a bus reset or BDR so that controllers
have an oportunity to renegotiate without tag messages in the way.
Somehow this got disabled... the functionality has been here for
quite some time.

Noticed by: my SCSI bus analyzer

Allow XPT_PATH_INQ to be performed on the XPT device, as per the CAM spec.
Return ENOTSUP for XPT_ENG_INQ and XPT_ENG_EXEC when performed on the XPT
device.

Reviewed by: gibbs

Add the use of M_ZERO to some malloc calls.

Submitted by: josh@zipperup.org
Submitted by: Robert Drehmel <robd@gmx.net>
Approved by: gibbs

- Overhaul the software interrupt code to use interrupt threads for each
type of software interrupt. Roughly, what used to be a bit in spending
now maps to a swi thread. Each thread can have multiple handlers, just
like a hardware interrupt thread.
- Instead of using a bitmask of pending interrupts, we schedule the specific
software interrupt thread to run, so spending, NSWI, and the shandlers
array are no longer needed. We can now have an arbitrary number of
software interrupt threads. When you register a software interrupt
thread via sinthand_add(), you get back a struct intrhand that you pass
to sched_swi() when you wish to schedule your swi thread to run.
- Convert the name of 'struct intrec' to 'struct intrhand' as it is a bit
more intuitive. Also, prefix all the members of struct intrhand with
'ih_'.
- Make swi_net() a MI function since there is now no point in it being
MD.

Submitted by: cp

Remove unneeded #include <machine/clock.h>

Slight cleanup - this is logically equivalent code but means one less
use of the evil resource_locate() function.

- Heavyweight interrupt threads on the alpha for device I/O interrupts.
- Make softinterrupts (SWI's) almost completely MI, and divorce them
completely from the x86 hardware interrupt code.
- The ihandlers array is now gone. Instead, there is a MI shandlers array
that just contains SWI handlers.
- Most of the former machine/ipl.h files have moved to a new sys/ipl.h.
- Stub out all the spl*() functions on all architectures.

Submitted by: dfr

- Remove the inthand2_t type and use the equivalent driver_intr_t type from
newbus for referencing device interrupt handlers.
- Move the 'struct intrec' type which describes interrupt sources into
sys/interrupt.h instead of making it just be a x86 structure.
- Don't create 'ithd' and 'intrec' typedefs, instead, just use 'struct ithd'
and 'struct intrec'
- Move the code to translate new-bus interrupt flags into an interrupt thread
priority out of the x86 nexus code and into a MI ithread_priority()
function in sys/kern/kern_intr.c.
- Remove now-uneeded x86-specific headers from sys/dev/ata/ata-all.c and
sys/pci/pci_compat.c.

CAM, the module: scbus, da, cd, and st wrapped in one module.

Make the umass driver depend on this module.

Makes it possible to compile the kernel without SCSI support and load it
when for example a USB floppy is conencted.

The Yamaha CDR100 doesn't seem to like multi-LUN probing.

PR: kern/20347
Submitted by: Andre Albsmeier <andre.albsmeier@mchp.siemens.de>

cam_ccb.h:
Bring back the CAM_NEGOTIATE ccb flag. This flag indicates
that SPI transfer negotiation should occur concurrently with the
execution of this CCB. The flag is not yet used by the XPT but
is required for proper support of multi-initiator configurations
where topology scans cannot rely on a bus reset to invalidate
prior negotiations.

cam_xpt.c:
Don't allow DT transmission rates to be specified for devices
that don't have the DT feature listed in their inquiry data.

Whoops- forgot to commit this other pearl from Justin- only set or
clear CAM_TAG_ACTION_VALID if this is an XPT_SCSI_IO CCB (otherwise,
the peripheral driver knows best...)..
Obtained from:gibbs@freebsd.org

Quirk entry for TeraSolutions TRC-22 RAID controller to keep mintags
from dropping below 55 due to temporary resource shortages.

Disable multi-lun probing on Hitachi DK31* drives.

PR: misc/18793
Submitted by: Paul Haddad <paul@pth.com>

Back out the previous change to the queue(3) interface.
It was not discussed and should probably not happen.

Requested by: msmith and others

Change the way that the queue(3) structures are declared; don't assume that
the type argument to *_HEAD and *_ENTRY is a struct.

Suggested by: phk
Reviewed by: phk
Approved by: mdodd

*sigh* I must have been on something that night. Make xpt_periph an
extern with the original in cam_xpt.c instead of replicating
xpt_periph in all the sources using it (and hence not initialising it)

Add a hack to cam that makes the cam_xpt available to the rest of the
kernel. Justin agress that there is no other reasonable alternative to
do automatic rescans on connect.

The problem is that when a new device attaches to a SIM (SCSI host
controller) we need to send a XPT_SCAN_BUS command to the SIM using
xpt_action. This requires however that there is a peripheral available
to take the command (otherwise xpt_done and later bomb). The RESCAN
ioctl uses the same periph.

This enables a USB mass storage drive to do an automatic rescan on
connection of the drive.

The automatic dropping of a CAM entry on disconnection was already
working (asynchronous event).

The next thing to do is find someone to commit a change to vpo to do the
same thing. Just port umass_cam_rescan and friends across to that
driver.

Approved by: gibbs

Various typo's.

One minor nit. The speed was displayed wrong when below 1Mb/s.

Redo the SHORT INQUIRY length stuff to be a bit cleaner.
Obtained from:gibbs@freebsd.org

A minimal update to the device wiring code so that it looks at the
dynamic resource tables rather than relying on a duplicated cam-specific
table generated by config(8) in ioconf.c. This was a major holdup to
getting loader / userconfig driven configuration of scsi wiring.

Remove garbage collected tags from their usage in the ccb_getdev
structure. Remove usage of the (now gone) pd_type tag of same.

Add an extra probing state such that if we successfully run an
initial inquiry (36 bytes), rerun another one with a longer data
size as informed by the 'additional length' field in the first
returned inquiry data (making sure not to get bigger than the
actual scsi_inquiry_data structure- which has also been modified-
see separate checkin of scsi_all.h). This allows devices such
as SAF-TE devices (which have identifying marks in offsets 48-53
in inquiry data) to be successfully found without special case
inquiry commands. There are also a lot of other things such as
version codes that are coming in in the SPC2 specification that it
would be useful to get our hands on.

Reviewed by: gibbs@freebsd.org, ken@freebsd.org

Clean up the xpt_sim creation to mirror how HBA drivers perform this task.

Clean up node creation in the EDT so that initialization follows member
declaration.

Sort registered paths by pathid so that we probe busses in order of
ascending pathid. This makes hardwiring of busses without wiring
individual targets do what the user expects. (submitted by tegge@FreeBSD.org)

Fix an EDT node leak. Target nodes would never go away.

Implement xpt_bus_deregister().
(prodded by some patches from T. Ichinoseki, but implemented differently.)

Make sure we don't look at an edt entry that could go away when we try
and figure out whether to probe higher than logical unit 7.
Obtained from:gibbs@freebsd.org

minor comment update

Convert the xpt, pass, pt and target drivers to use the new
make_dev()/destroy_dev() interface.

Fix Bus Device Resets from userland, and specifically from camcontrol.

camcontrol was setup to use the old scheme of going through the xpt(4)
device, which never worked properly (and has been disabled for a while).

camcontrol now sends BDRs through the pass(4) device, and XPT_RESET_DEV
CCBs are put on the device queue in the transport layer, as they should be.

Submitted by: luoqi
Reviewed by: ken

Protect xpt_run_dev_sendq in xpt_release_devq_device with splsoftcam. This
seems to handle the case of timeouts firing during probe but after a device
has gone away. It really does help.
Obtained from:gibbs@freebsd.org

Removal of sys/device.h

- Move intrhook stuff into kernel.h
- Remove all occurrences of #device <device.h>
- Add kernel.h were necessary (nowhere)
- delete device.h

This file contained the structures for cfdata (old style config) and is no
longer used. It was included by most drivers.

It confuses the remote debugger as the definition of 'struct device' in
device.h is found before the one in bus_private.h.

Remove five now unused fields from struct cdevsw. They should never
have been there in the first place. A GENERIC kernel shrinks almost 1k.

Add a slightly different safetybelt under nostop for tty drivers.

Add some missing FreeBSD tags

$Id$ -> $FreeBSD$

Convert DEVFS hooks in (most) drivers to make_dev().

Diskslice/label code not yet handled.

Vinum, i4b, alpha, pc98 not dealt with (left to respective Maintainers)

Add the correct hook for devfs to kern_conf.c

The net result of this excercise is that a lot less files depends on DEVFS,
and devtoname() gets more sensible output in many cases.

A few drivers had minor additional cleanups performed relating to cdevsw
registration.

A few drivers don't register a cdevsw{} anymore, but only use make_dev().

Implement the guts of the XPT_ABORT function code.

Clear residuals for XPT_TARGET_IO/XPT_CONT_TARGET_IO calls just
as we do for XPT_SCSI_IO calls.

Make xpt_release_devq a proper reciprocal of xpt_freeze_devq by
allowing the caller to decrement the count more than once per call.

Change the semantics of xpt_path_cmp to return 1 for a match
with a wildcard from path1 and 2 for a match with wildcards
from path2. This change doesn't affect most callers as they
only check for >=0 (exact match or one containing a wildcard).

When cleaning up a device node, ensure that there are no timeouts
outstanding that reference it.

Put in a limit (8) for lun width- do not exceed probing this unless a device
is quirked for allowing it. This will allow SCSI cards to specify their true
correct lun width (32 for Qlogic SCSI, 16 or 65536 for Qlogic Fibre Channel)
without crippling the system because of f/w breakage of a lot of drives.
Reviewed by: gibbs@freebsd.org

disable tag queing for the MICROP 4421-07 drive. It's an old 2gig drive.

convert MICROP to a static string

Quirk NOLUNS cabletron device.
PR: 11677
Submitted by: "Matthew N. Dodd" <winter@jurai.net>

Simplify cdevsw registration.

The cdevsw_add() function now finds the major number(s) in the
struct cdevsw passed to it. cdevsw_add_generic() is no longer
needed, cdevsw_add() does the same thing.

cdevsw_add() will print an message if the d_maj field looks bogus.

Remove nblkdev and nchrdev variables. Most places they were used
bogusly. Instead check a dev_t for validity by seeing if devsw()
or bdevsw() returns NULL.

Move bdevsw() and devsw() functions to kern/kern_conf.c

Bump __FreeBSD_version to 400006

This commit removes:
72 bogus makedev() calls
26 bogus SYSINIT functions

if_xe.c bogusly accessed cdevsw[], author/maintainer please fix.

I4b and vinum not changed. Patches emailed to authors. LINT
probably broken until they catch up.

This commit should be a extensive NO-OP:

Reformat and initialize correctly all "struct cdevsw".

Initialize the d_maj and d_bmaj fields.

The d_reset field was not removed, although it is never used.

I used a program to do most of this, so all the files now use the
same consistent format. Please keep it that way.

Vinum and i4b not modified, patches emailed to respective authors.

Run xpt_action at splsoftcam() so that it cannot be re-entered by our SWI.
The XPT doesn't have a problem with this itself, but some controllers
drivers may have been caught off guard by the old behavior.

XPT_CONT_TARGET_IO is also a valid ccb type for cam_periph_unmapmem.

Rely on the statistics in XPT_GDEV_STATS instead of the versions still
retained in XPT_GDEV_TYPE for binary compatibility. Mark the legacy
structure values for removal when we bump the major CAM revision.

Add the XPT_PATH_STATS and XPT_GDEV_STATS function codes. These ccb
types allow the reporting of error counts and other statistics. Currently
we provide information on the last BDR or bus reset as well as active
transaction inforamtion, but this will be expanded as more information is
added to aid in error recovery.

Use the 'last reset' information to better handle bus settle delays.
Peripheral drivers now control whether a bus settle delay occurs and
for how long. This allows target mode peripheral drivers to avoid
having their device queue frozen by the XPT for what shoudl only be
initiator type behavior.

Don't perform a bus reset if the target device is incapable of performing
transfer negotiation (e.g. Fiber Channel).

If we don't perform a bus reset but the controller is capable of transfer
negotiations, force negotiations on the first transaction to go to the
device. This ensures that we aren't tripped up by a left over negotiation
from the prom, BIOS, loader, etc.

Add a default async handler funstion to cam_periph.c to remove duplicated
code in all initiator type peripheral drivers.

Allow mapping of XPT_CONT_TARGET_IO ccbs from userland. They are
itentical to XPT_SCSI_IO ccbs as far as data mapping is concerned.

Make xpt_path_comp work on wildcarded paths again.

Quirk this drive- all luns show up the same.
Submitted by: Stephen McKay <syssgm@detir.qld.gov.au>

Quirk NOLUNS CALIPER CP150.

Add a number of interrelated CAM feature enhancements and bug fixes.

NOTE: These changes will require recompilation of any userland
applications, like cdrecord, xmcd, etc., that use the CAM passthrough
interface. A make world is recommended.

camcontrol.[c8]:
- We now support two new commands, "tags" and "negotiate".

- The tags commands allows users to view the number of tagged
openings for a device as well as a number of other related
parameters, and it allows users to set tagged openings for
a device.

- The negotiate command allows users to enable and disable
disconnection and tagged queueing, set sync rates, offsets
and bus width. Note that not all of those features are
available for all controllers. Only the adv, ahc, and ncr
drivers fully support all of the features at this point.
Some cards do not allow the setting of sync rates, offsets and
the like, and some of the drivers don't have any facilities to
do so. Some drivers, like the adw driver, only support enabling
or disabling sync negotiation, but do not support setting sync
rates.

- new description in the camcontrol man page of how to format a disk
- cleanup of the camcontrol inquiry command
- add support in the 'devlist' command for skipping unconfigured devices if
-v was not specified on the command line.
- make use of the new base_transfer_speed in the path inquiry CCB.
- fix CCB bzero cases

cam_xpt.c, cam_sim.[ch], cam_ccb.h:

- new flags on many CCB function codes to designate whether they're
non-immediate, use a user-supplied CCB, and can only be passed from
userland programs via the xpt device. Use these flags in the transport
layer and pass driver to categorize CCBs.

- new flag in the transport layer device matching code for device nodes
that indicates whether a device is unconfigured

- bump the CAM version from 0x10 to 0x11

- Change the CAM ioctls to use the version as their group code, so we can
force users to recompile code even when the CCB size doesn't change.

- add + fill in a new value in the path inquiry CCB, base_transfer_speed.
Remove a corresponding field from the cam_sim structure, and add code to
every SIM to set this field to the proper value.

- Fix the set transfer settings code in the transport layer.

scsi_cd.c:

- make some variables volatile instead of just casting them in various
places
- fix a race condition in the changer code
- attach unless we get a "logical unit not supported" error. This should
fix all of the cases where people have devices that return weird errors
when they don't have media in the drive.

scsi_da.c:

- attach unless we get a "logical unit not supported" error

scsi_pass.c:

- for immediate CCBs, just malloc a CCB to send the user request in. This
gets rid of the 'held' count problem in camcontrol tags.

scsi_pass.h:

- change the CAM ioctls to use the CAM version as their group code.

adv driver:

- Allow changing the sync rate and offset separately.

adw driver

- Allow changing the sync rate and offset separately.

aha driver:

- Don't return CAM_REQ_CMP for SET_TRAN_SETTINGS CCBs.

ahc driver:

- Allow setting offset and sync rate separately

bt driver:

- Don't return CAM_REQ_CMP for SET_TRAN_SETTINGS CCBs.

NCR driver:

- Fix the ultra/ultra 2 negotiation bug
- allow setting both the sync rate and offset separately

Other HBA drivers:
- Put code in to set the base_transfer_speed field for
XPT_GET_TRAN_SETTINGS CCBs.

Reviewed by: gibbs, mjacob (isp), imp (aha)

Release bus entries in the EDT when a path is released.
Null out path entries when the path is released to catch bugs.
Free our configuration hook when we're done with it.

Stage 1 of a cleanup of the i386 interrupt registration mechanism.
Interrupts under the new scheme are managed by the i386 nexus with the
awareness of the resource manager. There is further room for optimizing
the interfaces still. All the users of register_intr()/intr_create()
should be gone, with the exception of pcic and i386/isa/clock.c.

cam_periph.c:
Move handling of CAM_AUTOSENSE_FAIL into block dealing with
all other scsi status errors.

cam_queue.c:
cam_queue.h:
Fix 'off by one' heap bug in a more efficient manner. Since
heap algorithms like to deal with indexes started from 1,
offset our heap array pointer at allocation time to make this
so for a C environment. This makes the implementation of the
algorithm a bit more efficient.

cam_xpt.c:
Use macros for accessing the head of the heap so that code
is isolated from implementation details of the heap.

Well folks, this is it - The second stage of the removal for build support
for LKM's..

Remove camq_regen(). We already perform modular comparisons
for generation counts, so no further steps to deal with generation
count wrap are required.

Fix an off by one problem in the camq heap code.

Disable tagged queueing for the IBM DCAS drives. These drives have poor
write performance when tagged queueing is enabled.

Although the PR was submitted for the 4 gig version of this drive, the
assumption is that the 2 gig version has the same problem. Therefore
tagged queueing is disabled for both.

Also, update the comment for the Western Digital Enterprise drives to note
that the best performance for those drives is achieved when tagged queueing
is disabled and write caching is enabled.

PR: kern/10398
Submitted by: Hideaki Okada <hokada@isl.melco.co.jp>