MFC r312622:

Add SCSI descriptors for USB Mass Storage.

Sponsored by: The FreeBSD Foundation

MFC r310644: Fix/synchronize field types in struct ctl_modepage_header.

MFC r310640, r310643:
Add support for revert to defaults (RTD) bit in MODE SELECT.

MFC r310257: Improve support for informational exceptions.

While CTL still has no real events to report in this way (like SMART),
it is possible to trigger false event by manually setting TEST bit in
Informational Exceptions Control mode page, that can be useful for
initiator testing. This code supports all flavours of IE reporting:
UNIT ATTENTION, RECOVERED ERROR and NO SENSE sense keys, REQUEST SENSE
command and Informational Exceptions log page.

MFC r307350: Add LUN options to limit UNMAP and WRITE SAME sizes.

CTL itself has no limits on on UNMAP and WRITE SAME sizes. But depending
on backends large requests may take too much time. To avoid that new
configuration options allow to hint initiator maximal sizes it should not
exceed.

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Give CTL support for PIM_EXTLUNS when talking to CAM.

CTL itself still lives in flat LUN space, but it can generate extended
numbers if CAM SIM reports such capability.

Umplement media load/eject support for removable devices.

In case of block backend eject really closes the backing store, while
load tries to open it back. Failed store open is reported as no media.

Synchronize mode pages between HA peers.

We allow to modify only few fields in mode pages now, but still it is
not good if they unexpectedly change during failover. Also this fixes
reporting of "Mode parameters changed" UAs on secondary node.

Relax serseq option operation for reads.

Previously, with serseq enabled, next command was unblocked only after
previous completed. With this change, for read operations, next command
is unblocked as soon as last media read completed. This is important
for frontends that actually wait for data move completion (like camtgt),
or when data are moved through the HA link, or especially when both.

Replicate initiators WWPNs and names between HA peers.

Report INQUIRY DATA HAS CHANGED for related LUNs on port on-/offline.

Reimplement CTL High Availability.

CTL HA functionality was originally implemented by Copan many years ago,
but large part of the sources was never published. This change includes
clean room implementation of the missing code and fixes for many bugs.

This code supports dual-node HA with ALUA in four modes:
- Active/Unavailable without interlink between nodes;
- Active/Standby with second node handling only basic LUN discovery and
reservation, synchronizing with the first node through the interlink;
- Active/Active with both nodes processing commands and accessing the
backing storage, synchronizing with the first node through the interlink;
- Active/Active with second node working as proxy, transfering all
commands to the first node for execution through the interlink.

Unlike original Copan's implementation, depending on specific hardware,
this code uses simple custom TCP-based protocol for interlink. It has
no authentication, so it should never be enabled on public interfaces.

The code may still need some polishing, but generally it is functional.

Relnotes: yes
Sponsored by: iXsystems, Inc.

Allow LUN options modification via CTL_LUNREQ_MODIFY.

Not all changes take effect, but that is a different question.

Small UA cleanup.

Move "ioctl" CAM frontend into separate file.

It has nothing to share with too huge ctl.c other then device descriptor,
but even that may be counted as design error that may be fixed later.
At some point we may even want to have several ioctl ports.

Add more ifdefs to fix build with GCC after r286406.

Add support for General Statistics and Performance log page.

CTL already collects most of statistics reported there, so why not.

MFC after: 2 weeks

Replace ctl_min() macro with MIN().

MFC after: 1 week

Add configuration options to override physical and UNMAP blocks geometry.

While in most cases CTL should correctly fetch those values from backing
storages, there are some initiators (like MS SQL), that may not like large
physical block sizes, even if they are true. For such cases allow override
fetched values with supported ones (like 4K).

MFC after: 1 week

Add GET LBA STATUS command support to CTL.

It is implemented for LUNs backed by ZVOLs in "dev" mode and files.
GEOM has no such API, so for LUNs backed by raw devices all LBAs will
be reported as mapped/unknown.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

Add to CTL support for logical block provisioning threshold notifications.

For ZVOL-backed LUNs this allows to inform initiators if storage's used or
available spaces get above/below the configured thresholds.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

Fix printing non-terminated strings in devlist XML.

MFC after: 1 week

Remove couple Copan's vendor-specific mode pages.

Those pages are highly system-/hardware-specific, the code is incomplete,
and so they hardly can be useful for anybody else.

When reporting some major UNIT ATTENTION condition, like POWER ON OCCURRED
or I_T NEXUS LOSS, clear all minor UAs for the LUN, redundant in this case.

All SAM specifications tell that target MAY do it, but libiscsi initiator
seems require it to be done, terminating connection with error if some more
UAs happen to be reported during iSCSI connection.

MFC after: 3 days

Improve cache control support, including DPO/FUA flags and the mode page.

At this moment it works only for files and ZVOLs in device mode since BIOs
have no respective respective cache control flags (DPO/FUA).

MFC after: 1 month
Sponsored by: iXsystems, Inc.

Add support for READ FULL STATUS action of PERSISTENT RESERVE IN command.

Implement ABORT TASK SET and I_T NEXUS RESET task management functions.

Use the last one to terminate active commands on iSCSI session termination.
Previous code was aborting only commands doing some data moves.

Create separate CTL port for every iSCSI target (and maybe portal group).

Having single port for all iSCSI connections makes problematic implementing
some more advanced SCSI functionality in CTL, that require proper ports
enumeration and identification.

This change extends CTL iSCSI API, making ctld daemon to control list of
iSCSI ports in CTL. When new target is defined in config fine, ctld will
create respective port in CTL. When target is removed -- port will be
also removed after all active commands through that port properly aborted.
This change require ctld to be rebuilt to match the kernel.

As a minor side effect, this allows to have iSCSI targets without LUNs.
While that may look odd and not very useful, that is not incorrect.

Make options KPI more generic to allow it to be used for ports too,
not only for LUNs.

Introduce fine-grained CTL locking to improve SMP scalability.

Split global ctl_lock, historically protecting most of CTL context:
- remaining ctl_lock now protects lists of fronends and backends;
- per-LUN lun_lock(s) protect LUN-specific information;
- per-thread queue_lock(s) protect request queues.
This allows to radically reduce congestion on ctl_lock.

Create multiple worker threads, depending on number of CPUs, and assign
each LUN to one of them. This allows to spread load between multiple CPUs,
still avoiging congestion on queues and LUNs locks.

On 40-core server, exporting 5 LUNs, each backed by gstripe of SATA SSDs,
accessed via 6 iSCSI connections, this change improves peak request rate
from 250K to 680K IOPS.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

Increase CTL_DEVID_LEN from 16 to 64 bytes.

SPC-4 recommends T10 vendor ID based LUN ID was created by concatenating
product name and serial number (and istgt follows that). But product name
is 16 bytes long by itself, so 16 bytes total length is clearly not enough
to fit both.

To keep compatibility with existing configurations, pad short device IDs
to old length of 16, same as before.

This change probably breaks CTL user-level ABI, so control tools should
be rebuilt after this change.

MFC after: 2 weeks

Add support for VERIFY(10/12/16) and COMPARE AND WRITE SCSI commands.

Make data_submit backends method support not only read and write requests,
but also two new ones: verify and compare. Verify just checks readability
of the data in specified location without transferring them outside.
Compare reads the specified data and compares them to received data,
returning error if they are different.

VERIFY(10/12/16) commands request either verify or compare from backend,
depending on BYTCHK CDB field. COMPARE AND WRITE command executed in two
stages: first it requests compare, and then, if succeesed, requests write.
Atomicity of operation is guarantied by CTL request ordering code.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

CTL changes required for iSCSI target, most notably LUN remapping
and a mechanism to allow CTL frontends for retrieving LUN options.

Reviewed by: ken (earlier version)

Add LUN resizing to CTL. Also make it possible to explicitly set
size when creating file-backed or device-backed LUN.

Reviewed by: ken (earlier version)
Sponsored by: The FreeBSD Foundation

Add the CAM Target Layer (CTL).

CTL is a disk and processor device emulation subsystem originally written
for Copan Systems under Linux starting in 2003. It has been shipping in
Copan (now SGI) products since 2005.

It was ported to FreeBSD in 2008, and thanks to an agreement between SGI
(who acquired Copan's assets in 2010) and Spectra Logic in 2010, CTL is
available under a BSD-style license. The intent behind the agreement was
that Spectra would work to get CTL into the FreeBSD tree.

Some CTL features:

- Disk and processor device emulation.
- Tagged queueing
- SCSI task attribute support (ordered, head of queue, simple tags)
- SCSI implicit command ordering support. (e.g. if a read follows a mode
select, the read will be blocked until the mode select completes.)
- Full task management support (abort, LUN reset, target reset, etc.)
- Support for multiple ports
- Support for multiple simultaneous initiators
- Support for multiple simultaneous backing stores
- Persistent reservation support
- Mode sense/select support
- Error injection support
- High Availability support (1)
- All I/O handled in-kernel, no userland context switch overhead.

(1) HA Support is just an API stub, and needs much more to be fully
functional.

ctl.c: The core of CTL. Command handlers and processing,
character driver, and HA support are here.

ctl.h: Basic function declarations and data structures.

ctl_backend.c,
ctl_backend.h: The basic CTL backend API.

ctl_backend_block.c,
ctl_backend_block.h: The block and file backend. This allows for using
a disk or a file as the backing store for a LUN.
Multiple threads are started to do I/O to the
backing device, primarily because the VFS API
requires that to get any concurrency.

ctl_backend_ramdisk.c: A "fake" ramdisk backend. It only allocates a
small amount of memory to act as a source and sink
for reads and writes from an initiator. Therefore
it cannot be used for any real data, but it can be
used to test for throughput. It can also be used
to test initiators' support for extremely large LUNs.

ctl_cmd_table.c: This is a table with all 256 possible SCSI opcodes,
and command handler functions defined for supported
opcodes.

ctl_debug.h: Debugging support.

ctl_error.c,
ctl_error.h: CTL-specific wrappers around the CAM sense building
functions.

ctl_frontend.c,
ctl_frontend.h: These files define the basic CTL frontend port API.

ctl_frontend_cam_sim.c: This is a CTL frontend port that is also a CAM SIM.
This frontend allows for using CTL without any
target-capable hardware. So any LUNs you create in
CTL are visible in CAM via this port.

ctl_frontend_internal.c,
ctl_frontend_internal.h:
This is a frontend port written for Copan to do
some system-specific tasks that required sending
commands into CTL from inside the kernel. This
isn't entirely relevant to FreeBSD in general,
but can perhaps be repurposed.

ctl_ha.h: This is a stubbed-out High Availability API. Much
more is needed for full HA support. See the
comments in the header and the description of what
is needed in the README.ctl.txt file for more
details.

ctl_io.h: This defines most of the core CTL I/O structures.
union ctl_io is conceptually very similar to CAM's
union ccb.

ctl_ioctl.h: This defines all ioctls available through the CTL
character device, and the data structures needed
for those ioctls.

ctl_mem_pool.c,
ctl_mem_pool.h: Generic memory pool implementation used by the
internal frontend.

ctl_private.h: Private data structres (e.g. CTL softc) and
function prototypes. This also includes the SCSI
vendor and product names used by CTL.

ctl_scsi_all.c,
ctl_scsi_all.h: CTL wrappers around CAM sense printing functions.

ctl_ser_table.c: Command serialization table. This defines what
happens when one type of command is followed by
another type of command.

ctl_util.c,
ctl_util.h: CTL utility functions, primarily designed to be
used from userland. See ctladm for the primary
consumer of these functions. These include CDB
building functions.

scsi_ctl.c: CAM target peripheral driver and CTL frontend port.
This is the path into CTL for commands from
target-capable hardware/SIMs.

README.ctl.txt: CTL code features, roadmap, to-do list.

usr.sbin/Makefile: Add ctladm.

ctladm/Makefile,
ctladm/ctladm.8,
ctladm/ctladm.c,
ctladm/ctladm.h,
ctladm/util.c: ctladm(8) is the CTL management utility.
It fills a role similar to camcontrol(8).
It allow configuring LUNs, issuing commands,
injecting errors and various other control
functions.

usr.bin/Makefile: Add ctlstat.

ctlstat/Makefile
ctlstat/ctlstat.8,
ctlstat/ctlstat.c: ctlstat(8) fills a role similar to iostat(8).
It reports I/O statistics for CTL.

sys/conf/files: Add CTL files.

sys/conf/NOTES: Add device ctl.

sys/cam/scsi_all.h: To conform to more recent specs, the inquiry CDB
length field is now 2 bytes long.

Add several mode page definitions for CTL.

sys/cam/scsi_all.c: Handle the new 2 byte inquiry length.

sys/dev/ciss/ciss.c,
sys/dev/ata/atapi-cam.c,
sys/cam/scsi/scsi_targ_bh.c,
scsi_target/scsi_cmds.c,
mlxcontrol/interface.c: Update for 2 byte inquiry length field.

scsi_da.h: Add versions of the format and rigid disk pages
that are in a more reasonable format for CTL.

amd64/conf/GENERIC,
i386/conf/GENERIC,
ia64/conf/GENERIC,
sparc64/conf/GENERIC: Add device ctl.

i386/conf/PAE: The CTL frontend SIM at least does not compile
cleanly on PAE.

Sponsored by: Copan Systems, SGI and Spectra Logic
MFC after: 1 month