Revert r330897:

This was intended to be a non-functional change. It wasn't. The commit
message was thus wrong. In addition it broke arm, and merged crypto
related code.

Revert with prejudice.

This revert skips files touched in r316370 since that commit was since
MFCed. This revert also skips files that require $FreeBSD$ property
changes.

Thank you to those who helped me get out of this mess including but not
limited to gonzo, kevans, rgrimes.

Requested by: gjb (re)

Partial merge of the SPDX changes

These changes are incomplete but are making it difficult
to determine what other changes can/should be merged.

No objections from: pfg

MFC r311305 (by asomers):
Always null-terminate ccb_pathinq.(sim_vid|hba_vid|dev_name)

The sim_vid, hba_vid, and dev_name fields of struct ccb_pathinq are
fixed-length strings. AFAICT the only place they're read is in
sbin/camcontrol/camcontrol.c, which assumes they'll be null-terminated.
However, the kernel doesn't null-terminate them. A bunch of copy-pasted code
uses strncpy to write them, and doesn't guarantee null-termination. For at
least 4 drivers (mpr, mps, ciss, and hyperv), the hba_vid field actually
overflows. You can see the result by doing "camcontrol negotiate da0 -v".

This change null-terminates those fields everywhere they're set in the
kernel. It also shortens a few strings to ensure they'll fit within the
16-character field.

PR: 215474
Reported by: Coverity
CID: 1009997 1010000 1010001 1010002 1010003 1010004 1010005
CID: 1331519 1010006 1215097 1010007 1288967 1010008 1306000
CID: 1211924 1010009 1010010 1010011 1010012 1010013 1010014
CID: 1147190 1010017 1010016 1010018 1216435 1010020 1010021
CID: 1010022 1009666 1018185 1010023 1010025 1010026 1010027
CID: 1010028 1010029 1010030 1010031 1010033 1018186 1018187
CID: 1010035 1010036 1010042 1010041 1010040 1010039

MFC r313949: Remove dead mentions of CAM target mode APIs from drivers.

This makes grepping kernel for target mode implementation much easier.

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

sys/dev: minor spelling fixes.

Most affect comments, very few have user-visible effects.

Implement Auxiliary register. Add PIM_ATA_EXT flag to flag that a SIM
can handle it, and add the code to add it to the FIS that's sent to
the drive. The mvs driver is the only other ATA driver in the system,
and its hardware doesn't appear to support setting the Auxiliary
register.

Differential Revision: https://reviews.freebsd.org/D5598

Fix some more long -> rman_res_t

Reported by: Michael Butler (siis breakage)

Convert rman to use rman_res_t instead of u_long

Summary:
Migrate to using the semi-opaque type rman_res_t to specify rman resources. For
now, this is still compatible with u_long.

This is step one in migrating rman to use uintmax_t for resources instead of
u_long.

Going forward, this could feasibly be used to specify architecture-specific
definitions of resource ranges, rather than baking a specific integer type into
the API.

This change has been broken out to facilitate MFC'ing drivers back to 10 without
breaking ABI.

Reviewed By: jhb
Sponsored by: Alex Perez/Inertial Computing
Differential Revision: https://reviews.freebsd.org/D5075

Reduce priority of ATA/SATA drivers.

Legacy ata(4) -> BUS_PROBE_LOW_PRIORITY; more functional ahci(4), siis(4),
mvs(4) -> BUS_PROBE_DEFAULT; BUS_PROBE_VENDOR leave for vendor drivers.

MFC after: 2 weeks

Prevent overflow issues in timeout processing

Previously, any timeout value for which (timeout * hz) will overflow the
signed integer, will give weird results, since callout(9) routines will
convert negative values of ticks to '1'. For unsigned integer overflow we
will get sufficiently smaller timeout values than expected.

Switch from callout_reset, which requires conversion to int based ticks
to callout_reset_sbt to avoid this.

Also correct isci to correctly resolve ccb timeout.

This was based on the original work done by Eygene Ryabinkin
<rea@freebsd.org> back in 5 Aug 2011 which used a macro to help avoid
the overlow.

Differential Revision: https://reviews.freebsd.org/D1157
Reviewed by: mav, davide
MFC after: 1 month
Sponsored by: Multiplay

Initialize variables before resource_int_value().

Submitted by: Dmitry Luhtionov <dmitryluhtionov@gmail.com>

Merge CAM locking changes from the projects/camlock branch to radically
reduce lock congestion and improve SMP scalability of the SCSI/ATA stack,
preparing the ground for the coming next GEOM direct dispatch support.

Replace big per-SIM locks with bunch of smaller ones:
- per-LUN locks to protect device and peripheral drivers state;
- per-target locks to protect list of LUNs on target;
- per-bus locks to protect reference counting;
- per-send queue locks to protect queue of CCBs to be sent;
- per-done queue locks to protect queue of completed CCBs;
- remaining per-SIM locks now protect only HBA driver internals.

While holding LUN lock it is allowed (while not recommended for performance
reasons) to take SIM lock. The opposite acquisition order is forbidden.
All the other locks are leaf locks, that can be taken anywhere, but should
not be cascaded. Many functions, such as: xpt_action(), xpt_done(),
xpt_async(), xpt_create_path(), etc. are no longer require (but allow) SIM
lock to be held.

To keep compatibility and solve cases where SIM lock can't be dropped, all
xpt_async() calls in addition to xpt_done() calls are queued to completion
threads for async processing in clean environment without SIM lock held.

Instead of single CAM SWI thread, used for commands completion processing
before, use multiple (depending on number of CPUs) threads. Load balanced
between them using "hash" of the device B:T:L address.

HBA drivers that can drop SIM lock during completion processing and have
sufficient number of completion threads to efficiently scale to multiple
CPUs can use new function xpt_done_direct() to avoid extra context switch.
Make ahci(4) driver to use this mechanism depending on hardware setup.

Sponsored by: iXsystems, Inc.
MFC after: 2 months

Make siis(4) and mvs(4) send bus_get_dma_tag() requests to parent buses
passing real bus' child pointers instead of grandchilds.

Requested by: kib

ahci(4) and siis(4) are ready to process the unmapped i/o requests

Sponsored by: The FreeBSD Foundation
Tested by: pho
Submitted by: bf (siis patch)

Reform the busdma API so that new types may be added without modifying
every architecture's busdma_machdep.c. It is done by unifying the
bus_dmamap_load_buffer() routines so that they may be called from MI
code. The MD busdma is then given a chance to do any final processing
in the complete() callback.

The cam changes unify the bus_dmamap_load* handling in cam drivers.

The arm and mips implementations are updated to track virtual
addresses for sync(). Previously this was done in a type specific
way. Now it is done in a generic way by recording the list of
virtuals in the map.

Submitted by: jeff (sponsored by EMC/Isilon)
Reviewed by: kan (previous version), scottl,
mjacob (isp(4), no objections for target mode changes)
Discussed with: ian (arm changes)
Tested by: marius (sparc64), mips (jmallet), isci(4) on x86 (jharris),
amd64 (Fabian Keil <freebsd-listen@fabiankeil.de>)

Partially revert r236666:
Return PROTO_ATA protocol in response to XPT_PATH_INQ.

smartmontools uses it to identify ATA devices and I don't know any other
place now where it is important. It could probably use XPT_GDEV_TYPE
instead for more accurate protocol information, but let it live for now.

Reported by: matthew
MFC after: 3 days

ATA/SATA controllers have no idea about protocol of the connected device
until transport will do some probe actions (at least soft reset).
Make ATA/SATA SIMs to not report bogus and confusing PROTO_ATA protocol.
Make ATA/SATA transport to fill that gap by reporting protocol to SIM with
XPT_SET_TRAN_SETTINGS and patching XPT_GET_TRAN_SETTINGS results if needed.

Add two functions xpt_batch_start() and xpt_batch_done() to the CAM SIM KPI
to allow drivers to handle request completion directly without passing
them to the CAM SWI thread removing extra context switch.
Modify all ATA/SATA drivers to use them.

Reviewed by: gibbs, ken
MFC after: 2 weeks

Rename device_delete_all_children() into device_delete_children().

Suggested by: jhb @ and marius @
MFC after: 1 week

Move the device_delete_all_children() function from usb_util.c
to kern/subr_bus.c. Simplify this function so that it no longer
depends on malloc() to execute. Identify a few other places where
it makes sense to use device_delete_all_children().

MFC after: 1 week

Mark MALLOC_DEFINEs static that have no corresponding MALLOC_DECLAREs.

This means that their use is restricted to a single C file.

According to SATA specification, when Serial ATA Enclosure Management Bridge
(SEMB) is unable to communicate to Storage Enclosure Processor (SEP), in
response to hard and soft resets it should among other things return value
0x7F in Status register. The weird side is that it means DRQ bit set, which
tells that reset request is not completed. It would be fine if SEMB was the
only device on port. But if SEMB connected to PMP or built into it, it may
block access to other devices sharing same SATA port.

Make some tunings/fixes to soft-reset handling to workaround the issue:
- ahci(4): request CLO on the port after soft reset to ignore DRQ bit;
- siis(4): gracefully reinitialize port after soft reset timeout (hardware
doesn't detect reset request completion in this case);
- mvs(4): if PMP is used, send dummy soft-reset to the PMP port to make it
clear DRQ bit for us.

For now this makes quirks in ata_pmp.c, hiding SEMB ports of SiI3726/SiI4726
PMPs, less important. Further, if hardware permit, I hope to implement real
SEMB support.

Fix some English grammar.

According to specification. device should respond to COMRESET with COMINIT
in no more then 10ms. If we detected no device presence within that time,
there is no reason to wait longer.

Properly handle memory allocation errors during error recovery.

Improve SATA Asynchronous Notification feature support in CAM:
- make SATA SIMs announce capabilities to handle SDB with Notification bit;
- make PMP driver honor this SIMs capability;
- make SATA XPT to negotiate and enable this feature for ATAPI devices.

This feature allows supporting SATA ATAPI devices to inform system about
some events happened, that may require attention. In my case this allows
LG GH22LS50 SATA DVR-RW drive to report tray open/close events. Events
reported to CAM in form of AC_SCSI_AEN async. Further they could be used
as a hints for checking device status and reporting media change to upper
layers, for example, via spoiling mechanism of GEOM.

As soon as siis_reset() doesn't waits for device readiness, but only for
controller port readiness (that should set just after PHY ready signal),
reduce wait time from 10s to 1s before trying more aggressive reset method.

This should improve system responsibility in some failure conditions.

Implement automatic SCSI sense fetching for siis(4).

Fix device freeze leak on recovery request (READ LOG, REQUEST SENSE)
failure.

Hardware supported by siis(4) allows software control over activity LEDs.
Expose that functionality to led(4) OR-ing it with regular LED activity.

Teach ahci(4), siis(4) and ATA_CAM ata(4) wrapper report to CAM residual
I/O length on underruns, that often happens for some SCSI commands.

Add missing mtx_destroy() on channel attach failure.

Fix panic, when due to some kind of congestion on FIS-based switching
port multiplier some command triggers false positive timeout, but then
completes normally.

MFC after: 2 weeks

Export PCI IDs of ATA/SATA controllers through CAM and ata(4) layers to
GEOM. This information needed for proper soft-RAID's on-disk metadata
reading and writing.

Plug memory leak to silent Coverity. Error is still not really handled.

Found with: Coverity Prevent(tm)
CID: 4196

Fix attach errors handling.

Found with: Coverity Prevent(tm)
CID: 3477

Fill rman range start/end values. It makes devinfo output more readable.

Report ATA/SATA channel number to NewBus at location string.

Improve suspend/resume support. Make sure controller is idle on suspend
and reset it on resume.

Make SATA XPT negotiate and enable some additional SATA features, such as:
- device initiated power management (some devices support only this way);
- Automatic Partial to Slumber Transition (more power saving);
- DMA auto-activation (expected to slightly improve performance).
More features could be added later, when hardware supports.

Add Target/LUN ID checks and deny access to targets 1-14 when PMP absent.

Explicitly enable PCI busmastering on attach.
Now SiI3124 with siis(4) successfully works on sparc64 (SunBlade 100).

H/W donated by: Gheorghe Ardelean

Enable MSI by default for SiI3124.

MFp4:
After last running command completed, give commands in timeout state
second time.

Use new helper functions to set PCIe max read request size.

- Give ATA/SATA SIMs info about ATAPI packet size, supported by device.
- Make ATA XPT to reject longer SCSI CDBs then supported by device, or
any SCSI CDBs, if device doesn't support ATAPI.

Reset port on disconnect event, to abort any running requests.

MFp4: Large set of CAM inprovements.

- Unify bus reset/probe sequence. Whenever bus attached at boot or later,
CAM will automatically reset and scan it. It allows to remove duplicate
code from many drivers.
- Any bus, attached before CAM completed it's boot-time initialization,
will equally join to the process, delaying boot if needed.
- New kern.cam.boot_delay loader tunable should help controllers that
are still unable to register their buses in time (such as slow USB/
PCCard/ CardBus devices), by adding one more event to wait on boot.
- To allow synchronization between different CAM levels, concept of
requests priorities was extended. Priorities now split between several
"run levels". Device can be freezed at specified level, allowing higher
priority requests to pass. For example, no payload requests allowed,
until PMP driver enable port. ATA XPT negotiate transfer parameters,
periph driver configure caching and so on.
- Frozen requests are no more counted by request allocation scheduler.
It fixes deadlocks, when frozen low priority payload requests occupying
slots, required by higher levels to manage theit execution.
- Two last changes were holding proper ATA reinitialization and error
recovery implementation. Now it is done: SATA controllers and Port
Multipliers now implement automatic hot-plug and should correctly
recover from timeouts and bus resets.
- Improve SCSI error recovery for devices on buses without automatic sense
reporting, such as ATAPI or USB. For example, it allows CAM to wait, while
CD drive loads disk, instead of immediately return error status.
- Decapitalize diagnostic messages and make them more readable and sensible.
- Teach PMP driver to limit maximum speed on fan-out ports.
- Make boot wait for PMP scan completes, and make rescan more reliable.
- Fix pass driver, to return CCB to user level in case of error.
- Increase number of retries in cd driver, as device may return several UAs.

Usually these controllers are able to automatically decode command code to
get required command protocol. But they have no idea about new commands,
such as DATA SET MANAGEMENT (TRIM). As soon as this info any way provided
by CAM, give controller specific instructions.

Increase Max Read Request Size for PCIe chips from 512 to 1024 bytes.
It gives those beasts additional 10% of write bandwidth.

Explicitly acknowledge MSI completion, as required by SiI3124 datasheet.
It makes MSI working there. Later (and cheaper) PCIe chips (3132/3531)
still randomly crashing system in few seconds of high MSI rates, generating
something inaporopriate, like NMI or "Fatal trap 30".

SiI3124 has no SNotification register. Handle Asynchronous Notifications
there without it as good as possible.

MFp4:
Improve ATA mode/SATA revision control.

MFp4:
- Extend XPT-SIM transfer settings control API. Now it allows to report to
SATA SIM number of tags supported by each device, implement ATA mode and
SATA revision negotiation for both SATA and PATA SIMs.
- Make ahci(4) and siis(4) to use submitted maximum tag number, when
scheduling requests. It allows to support NCQ on devices with lower tags
count then controller supports.
- Make PMP driver to report attached devices connection speeds.
- Implement ATA mode negotiation between user settings, device and
controller capabilities.

Do not require payload data to be aligned. It is not mentioned in datasheet
and works fine in practice.

MFp4:
- Move tagged queueing control from ADA to ATA XPT. It allows to control
device command queue length correctly. First step to support < 32 tags.
- Limit queue for non-tagged devices by 2 slots for ahci(4) and siis(4).
- Implement quirk matching for ATA devices.
- Move xpt_schedule_dev_sendq() from header to source file.
- Move delayed queue shrinking to the more expected place - element freeing.
- Remove some SCSIsms in ATA.

MFp4:
Organize device IDs and add some more of them.

Do not unarm callout on request completion and change slot selection
algorithm as done in ahci(4). This saves some CPU time on high request
rates.

MFp4:
- Rework timeout handling, to make it more graceful for devices sharing
controller port (with PMP). Wait for other commands completion/timeout
before initiating recovery.
- Handle timeouts and fatal errors with port hard-reset. The rest of
recovery will be done by XPT on receiving async event. More gracefull
per-device soft-reset recovery can be implemented later.

Reimplement device reset sequence in more controller-specific way.

MFp4:
Report real max_target = 15. SIM doesn't need to know that target 15 is PMP.
It is XPT business.

MFp4:
Freeze device queue on error to permit periph driver to do proper recovery.

On command timeout handle frozen command first, to not run it inside
XXX_end_transaction().

Submitted by: avg

MFp4:
- Add SNTF support.
- Do not report meaningless transport/protocol versions.

Add `siis` CAM driver for SiliconImage SiI3124/3132/3531 SATA2 controllers.

Driver supports Serial ATA and ATAPI devices, Port Multipliers
(including FIS-based switching), hardware command queues (31 command
per port) and Native Command Queuing. This is probably the second on
popularity, after AHCI, type of SATA2 controllers, that benefits from
using CAM, because of hardware command queuing support.

Approved by: re (kib)