MFC r322302: Do not loose CCB flags after r320493.

There is at least CAM_UNLOCKED that should be kept.

MFC r320493: Unify INOT/ATIO setup.

MFC r314870: Add mechanism to unload CAM periph drivers.

For now it allows to unload CTL kernel module if there are no target-capable
SIMs in CAM. As next step full teardown of CAM targets can be implemented.

MFC r315084: Increase device openings to tagged maximum.

Some SIMs report much less untagged device openings then tagged ones.
Target mode devices are not handled by regular probing routines, and so
there is nothing to increase queue size for them to the SIM's maximum.
To fix that resize the queue explicitly on ctl periph registration.
This radically improves performance of mpt(4) in target mode.

Also fetch and report device queue statistics in `ctladm dumpstructs`,
since regular way of `camcontrol tags` is not usable in target mode.

MFC r315022: Request change of SIM target role only when it is different.

Separate WWNs change into separate request to know what actually failed.

MFC r315030: Abort all ATIOs and INOTs queued to SIM on LUN disable.

Some SIMs may not abort them implicitly, that either fail the LUN disable
request or just make us wait for those CCBs forever. With this change
I can successfully disable LUNs on mpt(4). For isp(4), which aborts them
implicitly, this change should be irrelevant.

MFC r315025: Switch work_queue from TAILQ to STAILQ.

It is mostly FIFO and we don't need random removal there.

MFC r314307: Add support for SIMs without autosense.

If we asked to send sense data by setting CAM_SEND_SENSE, but SIM didn't
confirm transmission by setting CAM_SENT_SENSE, assume it was not sent.
Queue the I/O back to CTL for later REQUEST SENSE with ctl_queue_sense().
This is needed for error reporting on SPI HBAs like ahc(4)/ahd(4).

MFC r314299, r314300: Fix residual length reporting in target mode.

This allows to properly handle cases when target wants to receive or send
more data then initiator wants to send or receive. Previously in such
cases isp(4) returned CAM_DATA_RUN_ERR, while now it returns resid > 0.

MFC r314247: Axe out some forever disabled questionable functionality.

This code is complicated enough even in its base shape.

MFC r314246: Improve CAM target frontend reference counting.

Before this change it was possible to trigger some use-after-free panics
by disabling LUNs/ports under heavy load.

MFC r314204: Explicitly abort ATIO if CTIO sending status has failed.

This helps SIM to free related resources in questionable cases.

MFC r314200: We can't access periph after ctlfe_free_ccb().

MFC r314196: Unify ATIO/INOT CCBs requeuing.

MFC r314193: Some code cleanup.

MFC r314027: Do not blindly free completed ATIOs/INOTs on invalidation.

When LUN is disabled, SIM starts returning queued ATIOs/INOTs. But at the
same time there can be some ATIOs/INOTs still carrying real new requests.
If we free those, SIM may leak some resources, forever expecting for any
response from us. So try to be careful, separating ATIOs/INOTs carrying
requests which still must be processed, from ATIOs/INOTs completed with
errors which can be freed.

MFC r312603: Add initial support for CTL module unloading.

It is only a first step and not perfect, but better then nothing.
The main blocker is CAM target frontend, that can not be unloaded,
since CAM does not have mechanism to unregister periph driver now.

MFC r312291, r312669:
Make CTL frontends report kern_data_resid for under-/overruns.

It seems like kern_data_resid was never really implemented. This change
finally does it. Now frontends update this field while transferring data,
while CTL/backends getting it can more flexibly handle the result.
At this point behavior should not change significantly, still reporting
errors on write overrun, but that may be changed later, if we decide so.

CAM target frontend still does not properly handle overruns due to CAM API
limitations. We may need to add some fields to struct ccb_accept_tio to
pass information about initiator requested transfer size(s).

MFC r312026: Improve CAM_CDB_POINTER support.

MFC r310649: Allow more efficient use of private area.

There are 16 bytes of space, so we may store two pointers in one.

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Pass task management response information from CTL through CAM to isp(4),
utilizing previously unused arg field of struct ccb_notify_acknowledge.

This makes new QUERY TASK, QUERY TASK SET and QUERY ASYNC EVENT requests
really functional for CAM target mode drivers.

Move to new value for XPT_GET_SIM_KNOB to avoid clash with XPT_ATA_IO.

Give CTL support for PIM_EXTLUNS when talking to CAM.

CTL itself still lives in flat LUN space, but it can generate extended
numbers if CAM SIM reports such capability.

Add partial support for QUERY TMF to CAM and isp(4).

This change allows to decode respective functions in isp(4) in target mode
and pass them through CAM to CTL. Unfortunately neither CAM nor isp(4)
support returning response info for those task management functions now.

On the other side I just have no initiator to test this functionality.

Fix LUN disable in CAM broken at r285155.

MFC after: 1 week

Update WRITE ATOMIC(16) support to sbc4r8 draft.

This is only a cosmetic change. We still don't support atomic boundary
field in the CDB, but at least now we do it formally.

Reimplement CTL High Availability.

CTL HA functionality was originally implemented by Copan many years ago,
but large part of the sources was never published. This change includes
clean room implementation of the missing code and fixes for many bugs.

This code supports dual-node HA with ALUA in four modes:
- Active/Unavailable without interlink between nodes;
- Active/Standby with second node handling only basic LUN discovery and
reservation, synchronizing with the first node through the interlink;
- Active/Active with both nodes processing commands and accessing the
backing storage, synchronizing with the first node through the interlink;
- Active/Active with second node working as proxy, transfering all
commands to the first node for execution through the interlink.

Unlike original Copan's implementation, depending on specific hardware,
this code uses simple custom TCP-based protocol for interlink. It has
no authentication, so it should never be enabled on public interfaces.

The code may still need some polishing, but generally it is functional.

Relnotes: yes
Sponsored by: iXsystems, Inc.

Remove unused target and initiator IDs.

Remove 600 bytes of port_priv from struct ctl_io_hdr.

This field used only for camtgt frontend, and once it any way preallocates
all requests, let it preallocate this memory too, not bothering core code.

Make first step toward supporting target and initiator roles same time.

To avoid conflicts between target and initiator devices in CAM, make
CTL use target ID reported by HBA as its initiator_id in XPT_PATH_INQ.
That target ID is known to never be used for initiator role, so it won't
conflict. For Fibre Channel and FireWire HBAs this specific ID choice
is irrelevant since all target IDs there are virtual. Same time for SPI
HBAs it seems could be even requirement to use same target ID for both
initiator and target roles.

While there are some more things to polish in isp(4) driver, first tests
of using both roles same time on the same port appeared successfull:

# camcontrol devlist -v
scbus0 on isp0 bus 0:
<FREEBSD CTLDISK 0001> at scbus0 target 1 lun 0 (da20,pass21)
<> at scbus0 target 256 lun 0 (ctl0)
<> at scbus0 target -1 lun ffffffff (ctl1)

Bring per-port LUN enable/disable code up to date:
- remove last remnants of never implemented multiple targets support;
- implement missing support for LUN mapping in this area.

Due to existing locking constraints LUN mapping code is practically
unlocked at this point. Hopefully it is not racy enough to live until
somebody get idea how to call sleeping fronend methods under lock also
taken by the same frontend in non-sleepable context. :(

Remove device queue freeze handling and replace it with dummy.

At this point CTL has no known use case for device queue freezes.
Same time existing (considered to be broken) code was found to cause
modify-after-free issues.

Discussed with: ken
MFC after: 1 week

Change ATIO/INOT counting to prevent periph destruction while requests
are still running inside CTL.

MFC after: 1 week

Allow setting only WWNN or only WWPN.

MFC after: 2 weeks

Don't count status as sent until CTIO completes successfully.

If we aggregated status sending with data move and got error, allow status
to be updated and resent again separately. Without this command may stuck
without status sent at all.

MFC after: 2 weeks

Reduce number of places where global control_softc is used.

At some point we may want to have several CTL instances, and that is not
really impossible.

MFC after: 2 weeks

Move ctlfe_onoffline() out of lock to let it sleep when needed.

Do some more other polishing while there.

MFC after: 2 weeks

Coalesce last data move and command status for read commands.

Make CTL core and block backend set success status before initiating last
data move for read commands. Make CAM target and iSCSI frontends detect
such condition and send command status together with data. New I/O flag
allows to skip duplicate status sending on later fe_done() call.

For Fibre Channel this change saves one of three interrupts per read command,
increasing performance from 126K to 160K IOPS. For iSCSI this change saves
one of three PDUs per read command, increasing performance from 1M to 1.2M
IOPS.

MFC after: 1 month
Sponsored by: iXsystems, Inc.

Decouple datamove/done logic from CTL status set.

Replace home-grown CTL IO allocator with UMA.

Old allocator created significant lock congestion protecting its lists
of preallocated I/Os, while UMA provides much better SMP scalability.
The downside of UMA is lack of reliable preallocation, that could guarantee
successful allocation in non-sleepable environments. But careful code
review shown, that only CAM target frontend really has that requirement.
Fix that making that frontend preallocate and statically bind CTL I/O for
every ATIO/INOT it preallocates any way. That allows to avoid allocations
in hot I/O path. Other frontends either may sleep in allocation context
or can properly handle allocation errors.

On 40-core server with 6 ZVOL-backed LUNs and 7 iSCSI client connections
this change increases peak performance from ~700K to >1M IOPS! Yay! :)

MFC after: 1 month
Sponsored by: iXsystems, Inc.

Partially reconstruct Active/Standby clusting.

In this mode one head is in Active state, supporting all commands, while
another is in Standby state, supporting only minimal LUN discovery subset.

It is still incomplete since Standby state requires reservation support,
which is impossible to do right without having interlink between heads.
But it allows to run some basic experiments.

Fix residual copy/paste in r274080.

MFC after: 1 week

Improve error handling around duplicate lun and port enable.

This fixes kernel panic if port enabled twice and then disabled.

MFC after: 1 week

Remove stale comments.

Give physical and virtual ports numbers some more meaning.

Shorten frontend name.

Filter out duplicate AC_PATH_REGISTERED async events.

Queued async events handling in CAM opened race, that may lead to duplicate
AC_PATH_REGISTERED events delivery during boot. That was not happening
before r272935 because the driver was initialized later. After that change
it started create duplicate ports in CTL.

Mark CTL frontend's CAM driver as CAM_PERIPH_DRV_EARLY.

Target mode operation does not depend on the initiator mode scan process.
This change allows the target driver to attach earlier and receive some
async events (like AC_CONTRACT) that could be lost otherwise.

MFC after: 1 week

Add support for WRITE ATOMIC (16) command and report SBC-4 compliance.

Atomic writes are only supported for ZVOLs in "dev" mode. In other cases
atomicity can not be guarantied and so the command is blocked.

Add support for VMWare dialect of EXTENDED COPY command, aka VAAI Clone.

This allows to clone VMs and move them between LUNs inside one storage
host without generating extra network traffic to the initiator and back,
and without being limited by network bandwidth.

LUNs participating in copy operation should have UNIQUE NAA or EUI IDs set.
For LUNs without these IDs VMWare will use traditional copy operations.

Beware: the above LUN IDs explicitly set to values non-unique from the VM
cluster point of view may cause data corruption if wrong LUN is addressed!

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

Enable TAS feature: notify initiator if its command was aborted by other.

That should make operation more kind to multi-initiator environment.
Without this, other initiators may find out that something bad happened
to their commands only via command timeout.

Add support for READ FULL STATUS action of PERSISTENT RESERVE IN command.

Teach ctl_add_initiator() to dynamically allocate IIDs from pool.

If port passed negative IID value, the function will try to allocate IID
from the pool of unused, based on passed wwpn or name arguments. It does
all its best to make IID unique and persistent across reconnects.

This makes persistent reservation properly work for iSCSI. Previously,
in case of reconnects, reservation could be unexpectedly lost, or even
migrate between intiators.

Burry devid port method, which was a gross hack.

Instead make ports provide wanted port and target IDs, and LUNs provide
wanted LUN IDs. After that core Device ID VPD code only had to link all
of them together and add relative port and port group numbers.

LUN ID for iSCSI LUNs no longer created by CTL, but by ctld, and passed
to CTL as "scsiname" LUN option. This makes LUNs to report the same set
of IDs, independently from the port through which it is accessed, as
required by SCSI specifications.

Separate concepts of frontend and port.

Before iSCSI implementation CTL had no knowledge about frontend drivers,
it had only frontends, which really were ports (alike to LUNs, if comparing
to backends). But iSCSI added there ioctl() method, which does not belong
to frontend as a port, but belongs to a frontend driver.

Remove targ_enable()/targ_disable() frontend methods.

Those methods were never implemented, and I believe that their concept is
wrong, since single frontend (SCSI port) can not handle several targets.

Make CAM target CTL frontend respect SIM I/O size limitations.

If datamove size is bigger then SIM can handle, or it has more segments
then this code can handle -- split it into several CTIO requests.

Merge CAM locking changes from the projects/camlock branch to radically
reduce lock congestion and improve SMP scalability of the SCSI/ATA stack,
preparing the ground for the coming next GEOM direct dispatch support.

Replace big per-SIM locks with bunch of smaller ones:
- per-LUN locks to protect device and peripheral drivers state;
- per-target locks to protect list of LUNs on target;
- per-bus locks to protect reference counting;
- per-send queue locks to protect queue of CCBs to be sent;
- per-done queue locks to protect queue of completed CCBs;
- remaining per-SIM locks now protect only HBA driver internals.

While holding LUN lock it is allowed (while not recommended for performance
reasons) to take SIM lock. The opposite acquisition order is forbidden.
All the other locks are leaf locks, that can be taken anywhere, but should
not be cascaded. Many functions, such as: xpt_action(), xpt_done(),
xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM
lock to be held.

To keep compatibility and solve cases where SIM lock can't be dropped, all
xpt_async() calls in addition to xpt_done() calls are queued to completion
threads for async processing in clean environment without SIM lock held.

Instead of single CAM SWI thread, used for commands completion processing
before, use multiple (depending on number of CPUs) threads. Load balanced
between them using "hash" of the device B:T:L address.

HBA drivers that can drop SIM lock during completion processing and have
sufficient number of completion threads to efficiently scale to multiple
CPUs can use new function xpt_done_direct() to avoid extra context switch.
Make ahci(4) driver to use this mechanism depending on hardware setup.

Sponsored by: iXsystems, Inc.
MFC after: 2 months

Fix the build with CTLFEDEBUG, broken by unmapped I/O support changes.

Fxi a bunch of typos.

PR: misc/174625
Submitted by: Jeremy Chadwick <jdc@koitsu.org>

MFprojects/camlock r248890, r248897, r248898, r248900, r248903, r248905,
r248917, r248918, r248978, r249001, r249014, r249030:

Remove multilevel freezing mechanism, implemented to handle specifics of
the ATA/SATA error recovery, when post-reset recovery commands should be
allocated when queues are already full of payload requests. Instead of
removing frozen CCBs with specified range of priorities from the queue
to provide free openings, use simple hack, allowing explicit CCBs over-
allocation for requests with priority higher (numerically lower) then
CAM_PRIORITY_OOB threshold.

Simplify CCB allocation logic by removing SIM-level allocation queue.
After that SIM-level queue manages only CCBs execution, while allocation
logic is localized within each single device.

Suggested by: gibbs

Remove ctl(4) from GENERIC. Also remove 'options CTL_DISABLE'
and kern.cam.ctl.disable tunable; those were introduced as a workaround
to make it possible to boot GENERIC on low memory machines.

With ctl(4) being built as a module and automatically loaded by ctladm(8),
this makes CTL work out of the box.

Reviewed by: ken
Sponsored by: FreeBSD Foundation

Fix sending virtual scatter/gather lists from the CTL CAM frontend
peripheral.

Sponsored by: Spectra Logic

Remove unused code.

Reviewed by: ken

Make it possible to build CTL as a module.

Reviewed by: ken
Sponsored by: FreeBSD Foundation

Reform the busdma API so that new types may be added without modifying
every architecture's busdma_machdep.c. It is done by unifying the
bus_dmamap_load_buffer() routines so that they may be called from MI
code. The MD busdma is then given a chance to do any final processing
in the complete() callback.

The cam changes unify the bus_dmamap_load* handling in cam drivers.

The arm and mips implementations are updated to track virtual
addresses for sync(). Previously this was done in a type specific
way. Now it is done in a generic way by recording the list of
virtuals in the map.

Submitted by: jeff (sponsored by EMC/Isilon)
Reviewed by: kan (previous version), scottl,
mjacob (isp(4), no objections for target mode changes)
Discussed with: ian (arm changes)
Tested by: marius (sparc64), mips (jmallet), isci(4) on x86 (jharris),
amd64 (Fabian Keil <freebsd-listen@fabiankeil.de>)

Make CTL work a little better with loading and unloading drivers.

Previously CTL would leave individual LUNs enabled in the target
driver, whether or not the port as a whole was enabled. It would
also leave the wildcard LUN enabled indefinitely.

This change means that CTL will enable and disable any active LUNs,
as well as the wildcard LUN, when enabling and disabling a port.

Also, fix a bug that could crop up due to an uninitialized CCB
type.

ctl.c: Before calling ctl_frontend_online(), run through
the LUN list and enable all active LUNs.

After calling ctl_frontend_offline(), run through
the LUN list and disble all active LUNs.

scsi_ctl.c: Before bringing a port online, allocate the
wildcard peripheral for that bus. And after taking
a port offline, invalidate the wildcard peripheral
for that bus.

Make sure that we hold the SIM lock around all
calls to xpt_action() and other transport layer
interfaces that require it.

Use CAM_SIM_{LOCK|UNLOCK} consistently to acquire
and release the SIM lock.

Update a number of outdated comments. Some of
these should have been fixed long ago.

Actually do LUN disbables now. The newer drivers
in the tree work correctly for this as far as I
know.

Initialize the CCB type to CTLFE_CCB_DEFAULT to
avoid a panic due to uninitialized memory.

Submitted by: Chuck Tuffli (partially)
MFC after: 1 week

Fix a couple of CTL locking issues and clean up some duplicated code.

ctl_frontend_cam_sim.c: Coalesce cfcs_online() and cfcs_offline()
into a single function since these were
identical except for one line.

Make sure we hold the SIM lock around path
creation, and calling xpt_rescan().

scsi_ctl.c: In ctlfe_onoffline(), make sure we hold the
SIM lock around path creation and free
calls, as well as xpt_action().

In ctlfe_lun_enable(), hold the SIM lock
around path and peripheral operations that
require it.

Sponsored by: Spectra Logic Corporation
MFC after: 1 week

Make sure we hold the SIM lock when calling xpt_free_path().

Sponsored by: Spectra Logic Corporation
MFC after: 1 week

Remove several uses of numeric priorities from immediate CCB setups.

Remove useless NULL checks after M_WAITOK allocations.

Handle a case where we had an SRR that pushed back the
data pointer. This is a temp fix that resubmits the
command, adjusted, so that the backend can fetch the
data again.

Sponsored by: Spectralogic
MFC after: 1 month

Add a loader tunable, kern.cam.ctl.disable, that will disable
loading CTL. This may be useful in very low memory installations.

MFC after: 3 days

Fix an issue that caused the kernel to panic inside CTL when trying
to attach to target capable HBAs that implement the old immediate
notify (XPT_IMMED_NOTIFY) and notify acknowledge (XPT_NOTIFY_ACK)
CCBs. The new API has been in place since SVN change 196008 in
2009.

The solution is two-fold: fix CTL to handle the responses from the
HBAs, and convert the HBA drivers in question to use the new API.

These drivers have not been tested with CTL, so how well they will
interoperate with CTL is unknown.

scsi_target.c: Update the userland target example code to use the
new immediate notify API.

scsi_ctl.c: Detect when an immediate notify CCB is returned
with CAM_REQ_INVALID or CAM_PROVIDE_FAIL status,
and just free it.

Fix a duplicate assignment.

aic79xx.c,
aic79xx_osm.c: Update the aic79xx driver to use the new API.
Target mode is not enabled on for this driver, so
the changes will have no practical effect.

aic7xxx.c,
aic7xxx_osm.c: Update the aic7xxx driver to use the new API.

sbp_targ.c: Update the firewire target code to work with the
new API.

mpt_cam.c: Update the mpt(4) driver to work with the new API.
Target mode is only enabled for Fibre Channel
mpt(4) devices.

MFC after: 3 days

Print FC PortID as a hex number. This makes it easy to
figure out domain, etc..

Zero ATIO and INOTify allocations. It makes for much
less guesswork when looking at the structure and
seeing 'deadc0de' present.

Reviewed by: kdm
MFC after: 2 weeks
Sponsored by: Spectralogic

Silence some unnecessary verbosity.

Reported by: mav
MFC after: 1 month

Add the CAM Target Layer (CTL).

CTL is a disk and processor device emulation subsystem originally written
for Copan Systems under Linux starting in 2003. It has been shipping in
Copan (now SGI) products since 2005.

It was ported to FreeBSD in 2008, and thanks to an agreement between SGI
(who acquired Copan's assets in 2010) and Spectra Logic in 2010, CTL is
available under a BSD-style license. The intent behind the agreement was
that Spectra would work to get CTL into the FreeBSD tree.

Some CTL features:

- Disk and processor device emulation.
- Tagged queueing
- SCSI task attribute support (ordered, head of queue, simple tags)
- SCSI implicit command ordering support. (e.g. if a read follows a mode
select, the read will be blocked until the mode select completes.)
- Full task management support (abort, LUN reset, target reset, etc.)
- Support for multiple ports
- Support for multiple simultaneous initiators
- Support for multiple simultaneous backing stores
- Persistent reservation support
- Mode sense/select support
- Error injection support
- High Availability support (1)
- All I/O handled in-kernel, no userland context switch overhead.

(1) HA Support is just an API stub, and needs much more to be fully
functional.

ctl.c: The core of CTL. Command handlers and processing,
character driver, and HA support are here.

ctl.h: Basic function declarations and data structures.

ctl_backend.c,
ctl_backend.h: The basic CTL backend API.

ctl_backend_block.c,
ctl_backend_block.h: The block and file backend. This allows for using
a disk or a file as the backing store for a LUN.
Multiple threads are started to do I/O to the
backing device, primarily because the VFS API
requires that to get any concurrency.

ctl_backend_ramdisk.c: A "fake" ramdisk backend. It only allocates a
small amount of memory to act as a source and sink
for reads and writes from an initiator. Therefore
it cannot be used for any real data, but it can be
used to test for throughput. It can also be used
to test initiators' support for extremely large LUNs.

ctl_cmd_table.c: This is a table with all 256 possible SCSI opcodes,
and command handler functions defined for supported
opcodes.

ctl_debug.h: Debugging support.

ctl_error.c,
ctl_error.h: CTL-specific wrappers around the CAM sense building
functions.

ctl_frontend.c,
ctl_frontend.h: These files define the basic CTL frontend port API.

ctl_frontend_cam_sim.c: This is a CTL frontend port that is also a CAM SIM.
This frontend allows for using CTL without any
target-capable hardware. So any LUNs you create in
CTL are visible in CAM via this port.

ctl_frontend_internal.c,
ctl_frontend_internal.h:
This is a frontend port written for Copan to do
some system-specific tasks that required sending
commands into CTL from inside the kernel. This
isn't entirely relevant to FreeBSD in general,
but can perhaps be repurposed.

ctl_ha.h: This is a stubbed-out High Availability API. Much
more is needed for full HA support. See the
comments in the header and the description of what
is needed in the README.ctl.txt file for more
details.

ctl_io.h: This defines most of the core CTL I/O structures.
union ctl_io is conceptually very similar to CAM's
union ccb.

ctl_ioctl.h: This defines all ioctls available through the CTL
character device, and the data structures needed
for those ioctls.

ctl_mem_pool.c,
ctl_mem_pool.h: Generic memory pool implementation used by the
internal frontend.

ctl_private.h: Private data structres (e.g. CTL softc) and
function prototypes. This also includes the SCSI
vendor and product names used by CTL.

ctl_scsi_all.c,
ctl_scsi_all.h: CTL wrappers around CAM sense printing functions.

ctl_ser_table.c: Command serialization table. This defines what
happens when one type of command is followed by
another type of command.

ctl_util.c,
ctl_util.h: CTL utility functions, primarily designed to be
used from userland. See ctladm for the primary
consumer of these functions. These include CDB
building functions.

scsi_ctl.c: CAM target peripheral driver and CTL frontend port.
This is the path into CTL for commands from
target-capable hardware/SIMs.

README.ctl.txt: CTL code features, roadmap, to-do list.

usr.sbin/Makefile: Add ctladm.

ctladm/Makefile,
ctladm/ctladm.8,
ctladm/ctladm.c,
ctladm/ctladm.h,
ctladm/util.c: ctladm(8) is the CTL management utility.
It fills a role similar to camcontrol(8).
It allow configuring LUNs, issuing commands,
injecting errors and various other control
functions.

usr.bin/Makefile: Add ctlstat.

ctlstat/Makefile
ctlstat/ctlstat.8,
ctlstat/ctlstat.c: ctlstat(8) fills a role similar to iostat(8).
It reports I/O statistics for CTL.

sys/conf/files: Add CTL files.

sys/conf/NOTES: Add device ctl.

sys/cam/scsi_all.h: To conform to more recent specs, the inquiry CDB
length field is now 2 bytes long.

Add several mode page definitions for CTL.

sys/cam/scsi_all.c: Handle the new 2 byte inquiry length.

sys/dev/ciss/ciss.c,
sys/dev/ata/atapi-cam.c,
sys/cam/scsi/scsi_targ_bh.c,
scsi_target/scsi_cmds.c,
mlxcontrol/interface.c: Update for 2 byte inquiry length field.

scsi_da.h: Add versions of the format and rigid disk pages
that are in a more reasonable format for CTL.

amd64/conf/GENERIC,
i386/conf/GENERIC,
ia64/conf/GENERIC,
sparc64/conf/GENERIC: Add device ctl.

i386/conf/PAE: The CTL frontend SIM at least does not compile
cleanly on PAE.

Sponsored by: Copan Systems, SGI and Spectra Logic
MFC after: 1 month