MFC r330876, r330945

r330876:
Fix ISP_FC_LIP and ISP_RESCAN on big-endian 64-bit systems.

For _IO() ioctls, addr is a pointer to uap->data which is a caddr_t.
When the caddr_t stores an int, dereferencing addr as an (int *) results
in truncation on little-endian 64-bit systems and corruption (owing to
extracting top bits) on big-endian 64-bit systems. In practice the
value of chan was probably always zero on systems of the latter type as
all such FreeBSD platforms use a register-based calling convention.

Reviewed by: mav
Obtained from: CheriBSD
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14673

r330945:
Add opt_compat.h to isp(4) as required by r330876.

MFC r317740:

Correct loop mode CRN resets to adhere to FCP-4 section 4.10

Prior to this change, the CRN (Command Reference Number) is reset on any
firmware LIP, LOOP DOWN, or LOOP RESET event in violation of FCP-4 which
specifies that the CRN should only be reset in response to a LIP Reset
(LIPyx) primitive. FCP-4 also indicates PLOGI/LOGO and PRLI/PRLO ELS
actions as conditions for resetting the CRN for the associated initiator
port.

These violations manifest themselves when the HBA is removed from the
loop, or a target device is removed (especially during an outstanding
command) without power cycling. If the HBA and and the target device
determine upon re-establishing the loop that no PLOGI or PRLI is
required, and the target does not issue a LIPxy to the initiator, the
CRN for the target will have been improperly reset by the isp driver. As
a result, the target port will silently ignore all FCP commands issued
during the device probe (which will time out) preventing the device from
attaching.

This change corrects thie CRN reset behavior in response to loop state
changes, also introduces CRN resets for the above mentioned ELS actions
as encountered through async PDB change events.

This change also adds cleanup of outstanding commands in isp_loop_dead()
that was previously missing.

sys/dev/isp/isp.c
Add the last login state to debug output when syncing the pdb

sys/dev/isp/isp_freebsd.c
Replace binary statement setting aborted ccb status in
isp_watchdog() with the XS_SETERR macro used elsewhere

In isp_loop_dead(), abort or complete pending commands as done
in isp_watchdog()

In isp_async(), segregate the ISPASYNC_LOOP_RESET action from
ISPASYNC_LIP, ISPASYNC_LOOP_DOWN, and ISPASYNC_LOOP_UP
fallthroughs, and only reset the CRN in the RESET case. Also add
checks to handle false LOOP RESET actions that do not have a
proper associated LIP primitive, and log the primitive in the
debug messages

In isp_async(), remove the goto from ISP_ASYNC_DEV_STAYED, and
only reset the CRN in the DEV_CHANGED action

In isp_async(), when processing an ISPASYNC_CHANGE_PDB status,
reset CRN(s) for the associated nphdl (or all ports) if the
change reason is some form of ELS login/logout. Also remove
assignment to fc since it is not used in the scope

sys/dev/isp/ispmbox.h
Add macro definition for the global N-Port handle, and correct a
macro typo 'PDB24XX_AE_PRLI_DONJE'

sys/dev/isp/ispvar.h
Add macros FCP_AL_DA_ALL, FCP_AL_PA, and FCP_IS_DEST_ALPD for
more legible code when determining if an AL_PD port matches the
portid for a given struct fcparam* by value or by virtue of the
AL_PD port being 0xFF

Submitted by: Reid Linnemann
Sponsored by: Spectra Logic

MFC r315908: Unify initiator and target DMA setup and command sending.

The code is so alike that it is pointless to keep it separate.

MFC r315869: Remove write-only crn field from struct isp_pcmd.

MFC r315708: Cleanup response queue processing.

MFC r316652: Fix few minor issues found by Clang Analyzer.

MFC r315709: Switch from using periph_links to sim_links.

periph_links field belongs to periph drivers and must not be used here.

MFC r315681: Improve command timeout handling.

Let firmware do its best first, and if it can't, try software recovery.
I would remove software timeout handler completely, but found bunch of
complains on command timeout on sparc64 mailing list few years ago, so
better be safe in case of interrupt loss.

MFC r315677: Clean/unify some macro usage.

MFC r315587, r315652: Remove some dead/useless code.

MFC r315579, r315670: Add initial support for multiple MSI-X vectors.

For 24xx and above use 2 vectors (default and response queue).
For 26xx and above use 3 vectors (default, response and ATIO queues).
Due to global lock interrupt hardlers never run simultaneously now, but
at least this allows to save one regitster read per interrupt.

Fix build broken by different size of lun_id_t.

MFC r315545: Remove hackish code delaying ATIOs to unknown virtual port.

Since we support RQSTYPE_RPT_ID_ACQ, that functionality is only useful
in loop mode, which probably doesn't worth having this hack in 2017.

MFC r315536: Move <= 23xx PDB workaround to generic code.

It is chip-specific and has nothing to do with platform.

MFC r315533: Move 24xx RQSTYPE_NOTIFY handling to generic code.

This code has nothing to do with specific platform.

MFC r315507: Reorganize RQSTYPE_NOTIFY handling for chips <= 23xx.

There were two copies of the code: one in generic code was half-broken, and
another in platform code was never called. Leave only one in generic code
and working.

MFC r315489: Move RQSTYPE_ABTS_RCVD parsing into generic code.

MFC r315488: Extend nt_lun to full 8 byte.

MFC r315482: Use isp_target_put_entry() in places where it can be.

This unifies the code and removes some duplication.

MFC r315478: Do some notify acks cleanup.

ISPASYNC_TARGET_NOTIFY_ACK makes no sense without argument.

MFC r315327: Remove not very useful ATIO/INOT stats.

While there polish respective debug logging.

MFC r315307: Refactor interrupt handling.

Instead of single isp_intr() function doing all possible magic, introduce
four different functions to handle mailbox operation completions, async
events, response and ATIO queues. The goal is to isolate different code
paths to make code more readable, and to make easier support for multiple
interrupt vectors. Even oldest hardware in many cases can identify what
code path it should run on interrupt. Contemporary hardware can assign
them to different interrupt vectors.

MFC r315303: Fix panic when SIM dereferenced before allocation.

MFC r315279: Remove some dead/broken code paths around async handling

MFC r315273: Remove tangled isp_mbox_continue() mechanism.

It was implemented to reduce context switches when uploading firmware to
card's RAM. But this mechanism is not used last 10 years since all mbox
operations are now polled, and it was never used for cards produced in
last 15 years. Newer cards can use DMA to upload firmware.

MFC r315234: Improvements around attach, reset and detach.

This change fixes DMA resource leak on driver unload. Also it removes
DMA resources allocation for hardcoded number of requests before fetching
the real number from firmware. Also it prepares ground for more flexible
IRQs allocation according to firmware capabilities.

MFC 315229: Remove remnant of r315163.

MFC r315161: Try to slight untangle I/O and loop status handling.

MFC r315160: Remove code for unsupported FreeBSD versions.

MFC r315163: Remove strange config_intrhook_establish() magic.

Interrupts are enabled as part of chip reset just during driver attach.
Later "enabling" of already enabled interrupts is useless.

MFC r311305 (by asomers):
Always null-terminate ccb_pathinq.(sim_vid|hba_vid|dev_name)

The sim_vid, hba_vid, and dev_name fields of struct ccb_pathinq are
fixed-length strings. AFAICT the only place they're read is in
sbin/camcontrol/camcontrol.c, which assumes they'll be null-terminated.
However, the kernel doesn't null-terminate them. A bunch of copy-pasted code
uses strncpy to write them, and doesn't guarantee null-termination. For at
least 4 drivers (mpr, mps, ciss, and hyperv), the hba_vid field actually
overflows. You can see the result by doing "camcontrol negotiate da0 -v".

This change null-terminates those fields everywhere they're set in the
kernel. It also shortens a few strings to ensure they'll fit within the
16-character field.

PR: 215474
Reported by: Coverity
CID: 1009997 1010000 1010001 1010002 1010003 1010004 1010005
CID: 1331519 1010006 1215097 1010007 1288967 1010008 1306000
CID: 1211924 1010009 1010010 1010011 1010012 1010013 1010014
CID: 1147190 1010017 1010016 1010018 1216435 1010020 1010021
CID: 1010022 1009666 1018185 1010023 1010025 1010026 1010027
CID: 1010028 1010029 1010030 1010031 1010033 1018186 1018187
CID: 1010035 1010036 1010042 1010041 1010040 1010039

MFC r314326: Send TERMINATE to firmware when aborting active ATIO.

MFC r314299, r314300: Fix residual length reporting in target mode.

This allows to properly handle cases when target wants to receive or send
more data then initiator wants to send or receive. Previously in such
cases isp(4) returned CAM_DATA_RUN_ERR, while now it returns resid > 0.

MFC r314208: Respecting r314204 tighten ATIO cleanup requirements.

Every ATIO must complete with either successfully sent status or XPT_ABORT.

MFC r314203: Fix missing xpt_done() for ATIO/INOT on missing LUN.

MFC r314088: Slightly polish isp_dump_atpd().

MFC r314086: Fix multiple problems around LUN disable under load.

- Move private data about ATIOs/INOTs from per-LUN to per-channel data.
This allows active commands to continue operation after LUN destruction.
This also simplifies lookup of the data by tag in some situations.
- Unify three restart_queue processing implementations.
- Complete all ATIOs from restart_queue on LUN disable.
- Delete ATIO private data when command completed or aborted, not depending
on the ATIO being requeued, that was ugly hack and could never happen. CAM
should always call ether XPT_CONT_TARGET_IO with status or XPT_ABORT.
- Implement XPT_ABORT for queued ATIOs/INOTs to allow CAM do graceful
shutdown, not depending on LUN disable, as it is done in ahd(4)/targ(4).
- Unify isp_endcmd() arguments to make it more usable in generic code.
- Remove never really used LUN state reference counter.

MFC r314045: Remove duplicate INOT allocation.

For some reason isp_handle_platform_notify_fc() allocated INOT just
before calling isp_handle_platform_target_tmf(), which also allocates
INOT. It seems to be a braino introduced in r196008.

MFC r314038: Remove ancient __FreeBSD_version checks.

MFC r313936, r313937: Move CTIO waitq from per-LUN to per-channel.

All resources lack of which may put CTIO into the queue are either
per-channel or potentially per-queue, but none of them are per-LUN.

This is a first step to fix live LUN disabling. Before this change
any CTIOs held in a queue in time of disabling were just leaked.

MFC r313945: Remove broken remnants of obsolete INOT API.

MFC r300296: Pass proper for 23xx arguments to isp_endcmd().

MFC r300293:
Pass task management response information from CTL through CAM to isp(4),
utilizing previously unused arg field of struct ccb_notify_acknowledge.

This makes new QUERY TASK, QUERY TASK SET and QUERY ASYNC EVENT requests
really functional for CAM target mode drivers.

MFC r300222: Fix delaying requests to unknown virtual ports 2s after init.

This code was originally implemented 7 years ago, but never really worked
due to trivial error. I think this functionality may be not required.
Initiators supporting optional periodic command status checks detected
those terminated commands and retried them 3 seconds later. But thinking
about less featured initiators and the fact that it is our race makes
virtual ports "unknown" it may be good to have this feature.

MFC r300218: Add proper reporting for early task management errors.

This covers unknown requests and requests to unknown virtual ports.
Previously it "worked" only because of timeout handling on initiator.

MFC r299957: Reduce verbosity of "now sending synthesized status" message.

MFC r297856: Reimplement ISP_TSK_MGMT IOCTL via asynchronous request.

I am not sure this code is not completely dead, but it used DMA scratch
are without good reason and asked to be refactored.

MFC r292725: Unify handles allocation for initiator and target IOCBs.

I am not sure why this was split long ago, but I see no reason for it.
At this point this unification just slightly reduces memory usage, but
as next step I plan to reuse shared handle space for other IOCB types.

MFC r292690: Some polishing for command timeouts handling.

MFC r291868: Rework WWNs generation to make cards without NVRAM more useful.

MFC r291365, r291369: One more round of port scanner rewrite.

- Make scan aborted by event restart immediately and infinitely.
- Improve handling of some loop events from firmware.
- Remove loop down timer, adding its functionality to scanner thread.
- Some more unification and simplification.

MFC r291209: Fix target mode support for Qlogic 2200 FC adapters.

Now target mode works for all supported FC adapters except ancient 2100,
which is not tested.

MFC r291188: Rip off target mode support for parallel SCSI QLogic adapters.

Hacks to enable target mode there complicated code, while didn't really
work. And for outdated hardware fixing it is not really interesting.

Initiator mode tested with Qlogic 1080 adapter is still working fine.

MFC r291162: Generate fake ISPASYNC_CHANGE_PDB on fake login on pre-24xx.

This makes port scanner fix absent port ID for added initiator.

MFC r291144: Fix target mode with fabric for pre-24xx chips.

For those chips we are not receiving login events, adding initiators
based on ATIO requests. But there is no port ID in that structure, so
in fabric mode we have to explicitly fetch it from firmware to be able
to do normal scan after that.

MFC r291080: Another round of port scanner rewrite.

This change simplifies and unifies port adding/updating for loop and
fabric scanners. It also fixes problems with scanning restarts due to
concurrent port databases changes. It also fixes many cosmetic issues.

MFC r291013: Remove some confusions between loopid and nphdl.

Modern cards in most cases operate abstract port handles, that have no
any relation to real loop IDs. Leave loopid used only where it really
goes about local loop IDs.

While there, fix few more cases where LUNs were still printed in decimal.

MFC r290830: Fix/improve CRN tracking.

MFC r290118: Change the way how target mode is enabled on 23xx chips.

Without docs I am not completely sure about this, but on my tests new
method works better then previous, at least with our latest firmware.

MFC r289933, r289939: Improve Port Database Changed handling and reporting.

MFC r289930: Formalize/unify chip (re-)inits.

MFC r289882: Add PIM_EXTLUNS support to isp(4) driver.

Now 24xx and above chips support full 8-byte LUN address space.
Older FC chips may support up to 16K LUNs when firmware allows.
Tested in both initiator and target modes for 23xx, 24xx and 25xx.

MFC r289877: Remove ISP_INTERNAL_TARGET code.

We have CTL now, which is real and much more functional then this joke.

MFC r289838: Improve INOTs handling for 24xx and above chips.

MFC r289817: Disable full bus scan by CAM for FC adapters.

FC port database code already notifies CAM about all devices. Additional
full scan is just a waste of time, that by definition won't find anything
that is not present in port database.

MFC r289812, r289852: Some polishing and unification in ISR code.

MFC r289219: Export bunch of state variables as sysctls.

MFC r289843: Add partial support for QUERY TMF to CAM and isp(4).

This change allows to decode respective functions in isp(4) in target mode
and pass them through CAM to CTL. Unfortunately neither CAM nor isp(4)
support returning response info for those task management functions now.

On the other side I just have no initiator to test this functionality.

MFC r285600: MULTI_ID supported does not mean it is used.

MFC r285510:
Switch initiator IDs in target mode to the same address space as target
IDs in initiator mode -- index in port database instead of handlers.

This makes initiator IDs persist across role changes and firmware resets,
when handlers previously assigned by firmware are lost and reused.

MFC r285481: Make role sysctl handling from r284727 less strict.

MFC r285459: Unify port database use for target and initiator roles.

Aside from cleaner and more consistent code, this allows ports to be both
target and initiator same time, and easily switch from any role to any.

MFC r285155:
Make first step toward supporting target and initiator roles same time.

To avoid conflicts between target and initiator devices in CAM, make
CTL use target ID reported by HBA as its initiator_id in XPT_PATH_INQ.
That target ID is known to never be used for initiator role, so it won't
conflict. For Fibre Channel and FireWire HBAs this specific ID choice
is irrelevant since all target IDs there are virtual. Same time for SPI
HBAs it seems could be even requirement to use same target ID for both
initiator and target roles.

While there are some more things to polish in isp(4) driver, first tests
of using both roles same time on the same port appeared successfull:

# camcontrol devlist -v
scbus0 on isp0 bus 0:
<FREEBSD CTLDISK 0001> at scbus0 target 1 lun 0 (da20,pass21)
<> at scbus0 target 256 lun 0 (ctl0)
<> at scbus0 target -1 lun ffffffff (ctl1)

MFC r285154: Remove extra level of target ID indirection (isp_dev_map).

FreeBSD never had limitation on number of target IDs, and there is no
any other requirement to allocate them densely. Since slots of port
database already populated just sequentially, there is no much need
for another indirection to allocate sequentially too.

MFC r285147: Change comment added in r284540.

This appeared to be not card's issue, but driver's, though solution is
the same so far.

MFC r285146: Drop discovered targets when initiator role is disabled.

MFC r271731 (by will): Fix a kernel panic when unloading isp(4).

In the current implementation, the isp_kthread() threads never exit.

The target threads do have an exit mode from isp_attach(), but it is
not invoked from isp_detach().

Ensure isp_detach() notifies threads started for each channel, such
that they exit before their parent device softc detaches, and thus
before the module does. Otherwise, a page fault panic occurs later in:

sysctl_kern_proc
sysctl_out_proc
kern_proc_out
fill_kinfo_proc
fill_kinfo_thread
strlcpy(kp->ki_wmesg, td->td_wmesg, sizeof(kp->ki_wmesg));

For isp_kthread() (and isp(4) target threads), td->td_wmesg references
now-unmapped memory after the module has been unloaded. These threads
are typically msleep()ing at the time of unload, but they could also
attempt to execute now-unmapped code segments.

MFC r284748: Fix reported_gone setting, missed in some cases.

This makes driver better track reported CAM_SEL_TIMEOUTs to properly
report device reappearance later. This fixes target 0 not reappearing
after initiator mode disabled and then reenabled.

MFC r284727: Add dev.isp.X.role sysctl in addition to tunable.

It (mostly) allows to enable/disable initiator mode in run time.
Target mode control is blocked here to force coordination with CTL.

While there, add separate tunables/sysctls for virtual channels.

MFC r284681: Rewrite port database handling for target mode.

Previous implementation was too fragile to initiator parameters changes.
In case of port role change it could not survive different handle assigned
to the same initiator by firmware, even though initiator was logged out.
The new implementation should be more resillient to this kind of problems,
trying to work in any situation and only warn user about suspisious events.

MFC r284540:
In case of target mode disable at least ISP2532 return invalid zero
ct_rxid value on CTIO completion. Try to workaround that using tag_id
from the CCB, pointed by still valid ct_syshandle.

I don't know whether this is valid fix or dirty hack, but considering that
alternative is indefinitely stuck command -- it worth trying.

MFC r274675 (by jhb), r274708:
Convert the refire_notify_ack timer from timeout(9) to callout(9).

MFC isp(4) driver changes:

r276839, r276842, r277513, r277514, r277515

------------------------------------------------------------------------
r276839 | ken | 2015-01-08 10:41:28 -0700 (Thu, 08 Jan 2015) | 49 lines

Fix Fibre Channel Command Reference Number handling in the isp(4) driver.

The Command Reference Number is used for precise delivery of
commands, and is part of the FC-Tape functionality set. (This is
only enabled for devices that support precise delivery of commands.)
It is an 8-bit unsigned number that increments from 1 to 255. The
commands sent by the initiator must be processed by the target in
CRN order if the CRN is non-zero.

There are certain scenarios where the Command Reference Number
sequence needs to be reset. When the target is power cycled, for
instance, the initiator needs to reset the CRN to 1. The initiator
will know this because it will see a LIP (when directly connected)
or get a logout/login event (when connected to a switch).

The isp(4) driver was not resetting the CRN when a target
went away and came back. When it saw the target again after a
power cycle, it would continue the CRN sequence where it left off.
The target would ignore the command because the CRN sequence is
supposed to be reset to 1 after a power cycle or other similar
event.

The symptom that the user would see is that there would be lots of
aborted INQUIRY commands after a tape library was power cycled, and
the library would fail to probe. The INQUIRY commands were being
ignored by the tape drive due to the CRN issue mentioned above.

isp_freebsd.c:
Add a new function, isp_fcp_reset_crn(). This will reset
all of the CRNs for a given port, or the CRNs for all LUNs
on a target.

Reset the CRNs for all targets on a port when we get a LIP,
loop reset, or loop down event.

Reset the CRN for a particular target when it arrives, is changed
or departs. This is less precise behavior than the
clearing behavior specified in the FCP-4 spec (which says
that it should be reset for PRLI, PRLO, PLOGI and LOGO),
but this is the level of information we have here. If this
is insufficient, then we will need to add more precise
notification from the lower level isp(4) code.

isp_freebsd.h:
Add a prototype for isp_fcp_reset_crn().

Sponsored by: Spectra Logic
MFC after: 1 week

------------------------------------------------------------------------
r276842 | ken | 2015-01-08 10:51:12 -0700 (Thu, 08 Jan 2015) | 44 lines

Close a race in the isp(4) driver that caused devices to disappear
and not automatically come back if they were gone for a short
period of time.

The isp(4) driver has a 30 second gone device timer that gets
activated whenever a device goes away. If the device comes back
before the timer expires, we don't send a notification to CAM that
it has gone away. If, however, there is a command sent to the
device while it is gone and before it comes back, the isp(4) driver
sends the command back with CAM_SEL_TIMEOUT status.

CAM responds to the CAM_SEL_TIMEOUT status by removing the device.
In the case where a device comes back within the 30 second gone
device timer window, though, we weren't telling CAM the device
came back.

So, fix this by tracking whether we have told CAM the device is
gone, and if we have, send a rescan if it comes back within the 30
second window.

ispvar.h:
In the fcportdb_t structure, add a new bitfield,
reported_gone. This gets set whenever we return a command
with CAM_SEL_TIMEOUT status on a Fibre Channel device.

isp_freebsd.c:
In isp_done(), if we're sending CAM_SEL_TIMEOUT for for a
command sent to a FC device, set the reported_gone bit.

In isp_async(), in the ISPASYNC_DEV_STAYED case, rescan the
device in question if it is mapped to a target ID and has
been reported gone.

In isp_make_here(), take a port database entry argument,
and clear the reported_gone bit when we send a rescan to
CAM.

In isp_make_gone(), take a port database entry as an
argument, and set the reported_gone bit when we send an
async event telling CAM consumers that the device is gone.

Sponsored by: Spectra Logic
MFC after: 1 week

------------------------------------------------------------------------
r277514 | will | 2015-01-21 13:27:11 -0700 (Wed, 21 Jan 2015) | 18 lines

Force commit to record the correct log for r277513.

If the user sends an XPT_RESET_DEV CCB, make sure to reset the
Fibre Channel Command Reference Number if we're running on a FC
controller.

We send a SCSI Target Reset when we get this CCB, and as a result
need to reset the CRN to 1 on the next command.

isp_freebsd.c:
In the XPT_RESET_DEV implementation in isp_action(), reset
the CRN if we're on a FC controller.

Submitted by: ken
MFC after: 1 week
Sponsored by: Spectra Logic
MFSpectraBSD: 1112787 on 2015/01/15

------------------------------------------------------------------------
r277515 | will | 2015-01-21 13:32:36 -0700 (Wed, 21 Jan 2015) | 25 lines

Fix SCSI status byte reporting on 4Gb and 8Gb Qlogic boards.

The newer boards don't have the response field that indicates
whether the SCSI status byte is present. You have to just look to
see whether it is non-zero.

The code was looking to see whether the sense length was valid
before propagating the SCSI status byte (and sense information) up
the stack. With a status like Reservation Conflict, there is no
sense information, only the SCSI status byte. So it wasn't getting
correctly returned.

isp.c:
In isp_intr(), if we are on a 2400 or 2500 type board and
get a response, look at the actual contents of the
SCSI status value and set the RQSF_GOT_STATUS flag
accordingly so that return any SCSI status value we get. The
RQSF_GOT_SENSE flag will get set later on if there is
actual sense information returned.

Submitted by: ken
MFC after: 1 week
Sponsored by: Spectra Logic
MFSpectraBSD: 1112791 on 2015/01/15

------------------------------------------------------------------------

Sponsored by: Spectra Logic

MFC r275118: Some microoptimizations.

MFC r268395:
Pass correct command that should be aborted to ISPCTL_ABORT_CMD.

This makes XPT_ABORT to work for me on initiator side of isp(4).
Previous code was trying to abort the XPT_ABORT itself and failed.

MFC r275124: Fix WWNN/WWPN generation for virtual channels.

MFC r259860 (by mjacob):

Harvest one no longer used constant string.

Remove another and place it into play in the
normally ifdef protected zone it would be used
int.

Noticed by: dim

MFC Alexander Motin's direct dispatch, multi-queue, and finer-grained
locking support for CAM

r256826:
Fix several target mode SIMs to not blindly clear ccb_h.flags field of
ATIO CCBs. Not all CCB flags there belong to them.

r256836:
Remove hard limit on number of BIOs handled with one ATA TRIM request.

r256843:
Merge CAM locking changes from the projects/camlock branch to radically
reduce lock congestion and improve SMP scalability of the SCSI/ATA stack,
preparing the ground for the coming next GEOM direct dispatch support.

r256888:
Unconditionally acquire periph reference on CCB allocation failure.

r256895:
Fix memory and references leak due to unfreed path.

r256960:
Move CAM_UNQUEUED_INDEX setting to the last moment and under the periph lock.
This fixes race condition with cam_periph_ccbwait(), causing use-after-free.

r256975:
Minor (mostly cosmetical) addition to r256960.

r257054:
Some microoptimizations for da and ada drivers:
- Replace ordered_tag_count counter with single flag;
- From da remove outstanding_cmds counter, duplicating pending_ccbs list;
- From da_softc remove unused links field.

r257482:
Fix lock recursion, triggered by `smartctl -a /dev/adaX`.

r257501:
Make getenv_*() functions and respectively TUNABLE_*_FETCH() macros not
allocate memory and so not require sleepable environment. getenv() has
already used on-stack temporary storage, so just use it more rationally.
getenv_string() receives buffer as argument, so don't need another one.

r257914:
Some CAM locks polishing:
- Fix LOR and possible lock recursion when handling high-power commands.
Introduce new lock to protect left power quota and list of frozen devices.
- Correct locking around xpt periph creation.
- Remove seems never used XPT_FLAG_OPEN xpt periph flag.

Again, Netflix assisted with testing the merge, but all of the credit goes
to Alexander and iX Systems.

Submitted by: mav
Sponsored by: iX Systems

MFC r256826:
Fix several target mode SIMs to not blindly clear ccb_h.flags field of
ATIO CCBs. Not all CCB flags there belong to them.

MFC r256705:
Optimize isp(4) to reduce CPU usage, especially in target mode:
- Remove two excessive and slow register reads from isp_intr(). Instead
of rereading value every time, assume that registers contain what we have
written there.
- Avoid sequential search through 4096 array elements when looking for
command tag. Use hash of lists to store active tags separately from free
ones and so greatly speedup the searches.

MFC r285600: MULTI_ID supported does not mean it is used.

MFC r285510:
Switch initiator IDs in target mode to the same address space as target
IDs in initiator mode -- index in port database instead of handlers.

This makes initiator IDs persist across role changes and firmware resets,
when handlers previously assigned by firmware are lost and reused.

MFC r285481: Make role sysctl handling from r284727 less strict.

MFC r285459: Unify port database use for target and initiator roles.

Aside from cleaner and more consistent code, this allows ports to be both
target and initiator same time, and easily switch from any role to any.

MFC r285155:
Make first step toward supporting target and initiator roles same time.

To avoid conflicts between target and initiator devices in CAM, make
CTL use target ID reported by HBA as its initiator_id in XPT_PATH_INQ.
That target ID is known to never be used for initiator role, so it won't
conflict. For Fibre Channel and FireWire HBAs this specific ID choice
is irrelevant since all target IDs there are virtual. Same time for SPI
HBAs it seems could be even requirement to use same target ID for both
initiator and target roles.

While there are some more things to polish in isp(4) driver, first tests
of using both roles same time on the same port appeared successfull:

# camcontrol devlist -v
scbus0 on isp0 bus 0:
<FREEBSD CTLDISK 0001> at scbus0 target 1 lun 0 (da20,pass21)
<> at scbus0 target 256 lun 0 (ctl0)
<> at scbus0 target -1 lun ffffffff (ctl1)

MFC r285154: Remove extra level of target ID indirection (isp_dev_map).

FreeBSD never had limitation on number of target IDs, and there is no
any other requirement to allocate them densely. Since slots of port
database already populated just sequentially, there is no much need
for another indirection to allocate sequentially too.

MFC r285147: Change comment added in r284540.

This appeared to be not card's issue, but driver's, though solution is
the same so far.

MFC r285146: Drop discovered targets when initiator role is disabled.

MFC r271731 (by will): Fix a kernel panic when unloading isp(4).

In the current implementation, the isp_kthread() threads never exit.

The target threads do have an exit mode from isp_attach(), but it is
not invoked from isp_detach().

Ensure isp_detach() notifies threads started for each channel, such
that they exit before their parent device softc detaches, and thus
before the module does. Otherwise, a page fault panic occurs later in:

sysctl_kern_proc
sysctl_out_proc
kern_proc_out
fill_kinfo_proc
fill_kinfo_thread
strlcpy(kp->ki_wmesg, td->td_wmesg, sizeof(kp->ki_wmesg));

For isp_kthread() (and isp(4) target threads), td->td_wmesg references
now-unmapped memory after the module has been unloaded. These threads
are typically msleep()ing at the time of unload, but they could also
attempt to execute now-unmapped code segments.

MFC r284748: Fix reported_gone setting, missed in some cases.

This makes driver better track reported CAM_SEL_TIMEOUTs to properly
report device reappearance later. This fixes target 0 not reappearing
after initiator mode disabled and then reenabled.

MFC r284727: Add dev.isp.X.role sysctl in addition to tunable.

It (mostly) allows to enable/disable initiator mode in run time.
Target mode control is blocked here to force coordination with CTL.

While there, add separate tunables/sysctls for virtual channels.

MFC r284681: Rewrite port database handling for target mode.

Previous implementation was too fragile to initiator parameters changes.
In case of port role change it could not survive different handle assigned
to the same initiator by firmware, even though initiator was logged out.
The new implementation should be more resillient to this kind of problems,
trying to work in any situation and only warn user about suspisious events.

MFC r284540:
In case of target mode disable at least ISP2532 return invalid zero
ct_rxid value on CTIO completion. Try to workaround that using tag_id
from the CCB, pointed by still valid ct_syshandle.

I don't know whether this is valid fix or dirty hack, but considering that
alternative is indefinitely stuck command -- it worth trying.

MFC r274675 (by jhb), r274708:
Convert the refire_notify_ack timer from timeout(9) to callout(9).

MFC isp(4) driver changes:

r276839, r276842, r277513, r277514, r277515

------------------------------------------------------------------------
r276839 | ken | 2015-01-08 10:41:28 -0700 (Thu, 08 Jan 2015) | 49 lines

Fix Fibre Channel Command Reference Number handling in the isp(4) driver.

The Command Reference Number is used for precise delivery of
commands, and is part of the FC-Tape functionality set. (This is
only enabled for devices that support precise delivery of commands.)
It is an 8-bit unsigned number that increments from 1 to 255. The
commands sent by the initiator must be processed by the target in
CRN order if the CRN is non-zero.

There are certain scenarios where the Command Reference Number
sequence needs to be reset. When the target is power cycled, for
instance, the initiator needs to reset the CRN to 1. The initiator
will know this because it will see a LIP (when directly connected)
or get a logout/login event (when connected to a switch).

The isp(4) driver was not resetting the CRN when a target
went away and came back. When it saw the target again after a
power cycle, it would continue the CRN sequence where it left off.
The target would ignore the command because the CRN sequence is
supposed to be reset to 1 after a power cycle or other similar
event.

The symptom that the user would see is that there would be lots of
aborted INQUIRY commands after a tape library was power cycled, and
the library would fail to probe. The INQUIRY commands were being
ignored by the tape drive due to the CRN issue mentioned above.

isp_freebsd.c:
Add a new function, isp_fcp_reset_crn(). This will reset
all of the CRNs for a given port, or the CRNs for all LUNs
on a target.

Reset the CRNs for all targets on a port when we get a LIP,
loop reset, or loop down event.

Reset the CRN for a particular target when it arrives, is changed
or departs. This is less precise behavior than the
clearing behavior specified in the FCP-4 spec (which says
that it should be reset for PRLI, PRLO, PLOGI and LOGO),
but this is the level of information we have here. If this
is insufficient, then we will need to add more precise
notification from the lower level isp(4) code.

isp_freebsd.h:
Add a prototype for isp_fcp_reset_crn().

Sponsored by: Spectra Logic
MFC after: 1 week

------------------------------------------------------------------------
r276842 | ken | 2015-01-08 10:51:12 -0700 (Thu, 08 Jan 2015) | 44 lines

Close a race in the isp(4) driver that caused devices to disappear
and not automatically come back if they were gone for a short
period of time.

The isp(4) driver has a 30 second gone device timer that gets
activated whenever a device goes away. If the device comes back
before the timer expires, we don't send a notification to CAM that
it has gone away. If, however, there is a command sent to the
device while it is gone and before it comes back, the isp(4) driver
sends the command back with CAM_SEL_TIMEOUT status.

CAM responds to the CAM_SEL_TIMEOUT status by removing the device.
In the case where a device comes back within the 30 second gone
device timer window, though, we weren't telling CAM the device
came back.

So, fix this by tracking whether we have told CAM the device is
gone, and if we have, send a rescan if it comes back within the 30
second window.

ispvar.h:
In the fcportdb_t structure, add a new bitfield,
reported_gone. This gets set whenever we return a command
with CAM_SEL_TIMEOUT status on a Fibre Channel device.

isp_freebsd.c:
In isp_done(), if we're sending CAM_SEL_TIMEOUT for for a
command sent to a FC device, set the reported_gone bit.

In isp_async(), in the ISPASYNC_DEV_STAYED case, rescan the
device in question if it is mapped to a target ID and has
been reported gone.

In isp_make_here(), take a port database entry argument,
and clear the reported_gone bit when we send a rescan to
CAM.

In isp_make_gone(), take a port database entry as an
argument, and set the reported_gone bit when we send an
async event telling CAM consumers that the device is gone.

Sponsored by: Spectra Logic
MFC after: 1 week

------------------------------------------------------------------------
r277514 | will | 2015-01-21 13:27:11 -0700 (Wed, 21 Jan 2015) | 18 lines

Force commit to record the correct log for r277513.

If the user sends an XPT_RESET_DEV CCB, make sure to reset the
Fibre Channel Command Reference Number if we're running on a FC
controller.

We send a SCSI Target Reset when we get this CCB, and as a result
need to reset the CRN to 1 on the next command.

isp_freebsd.c:
In the XPT_RESET_DEV implementation in isp_action(), reset
the CRN if we're on a FC controller.

Submitted by: ken
MFC after: 1 week
Sponsored by: Spectra Logic
MFSpectraBSD: 1112787 on 2015/01/15

------------------------------------------------------------------------
r277515 | will | 2015-01-21 13:32:36 -0700 (Wed, 21 Jan 2015) | 25 lines

Fix SCSI status byte reporting on 4Gb and 8Gb Qlogic boards.

The newer boards don't have the response field that indicates
whether the SCSI status byte is present. You have to just look to
see whether it is non-zero.

The code was looking to see whether the sense length was valid
before propagating the SCSI status byte (and sense information) up
the stack. With a status like Reservation Conflict, there is no
sense information, only the SCSI status byte. So it wasn't getting
correctly returned.

isp.c:
In isp_intr(), if we are on a 2400 or 2500 type board and
get a response, look at the actual contents of the
SCSI status value and set the RQSF_GOT_STATUS flag
accordingly so that return any SCSI status value we get. The
RQSF_GOT_SENSE flag will get set later on if there is
actual sense information returned.

Submitted by: ken
MFC after: 1 week
Sponsored by: Spectra Logic
MFSpectraBSD: 1112791 on 2015/01/15

------------------------------------------------------------------------

Sponsored by: Spectra Logic

MFC r275118: Some microoptimizations.

MFC r268395:
Pass correct command that should be aborted to ISPCTL_ABORT_CMD.

This makes XPT_ABORT to work for me on initiator side of isp(4).
Previous code was trying to abort the XPT_ABORT itself and failed.

MFC r275124: Fix WWNN/WWPN generation for virtual channels.

MFC r259860 (by mjacob):

Harvest one no longer used constant string.

Remove another and place it into play in the
normally ifdef protected zone it would be used
int.

Noticed by: dim

MFC Alexander Motin's direct dispatch, multi-queue, and finer-grained
locking support for CAM

r256826:
Fix several target mode SIMs to not blindly clear ccb_h.flags field of
ATIO CCBs. Not all CCB flags there belong to them.

r256836:
Remove hard limit on number of BIOs handled with one ATA TRIM request.

r256843:
Merge CAM locking changes from the projects/camlock branch to radically
reduce lock congestion and improve SMP scalability of the SCSI/ATA stack,
preparing the ground for the coming next GEOM direct dispatch support.

r256888:
Unconditionally acquire periph reference on CCB allocation failure.

r256895:
Fix memory and references leak due to unfreed path.

r256960:
Move CAM_UNQUEUED_INDEX setting to the last moment and under the periph lock.
This fixes race condition with cam_periph_ccbwait(), causing use-after-free.

r256975:
Minor (mostly cosmetical) addition to r256960.

r257054:
Some microoptimizations for da and ada drivers:
- Replace ordered_tag_count counter with single flag;
- From da remove outstanding_cmds counter, duplicating pending_ccbs list;
- From da_softc remove unused links field.

r257482:
Fix lock recursion, triggered by `smartctl -a /dev/adaX`.

r257501:
Make getenv_*() functions and respectively TUNABLE_*_FETCH() macros not
allocate memory and so not require sleepable environment. getenv() has
already used on-stack temporary storage, so just use it more rationally.
getenv_string() receives buffer as argument, so don't need another one.

r257914:
Some CAM locks polishing:
- Fix LOR and possible lock recursion when handling high-power commands.
Introduce new lock to protect left power quota and list of frozen devices.
- Correct locking around xpt periph creation.
- Remove seems never used XPT_FLAG_OPEN xpt periph flag.

Again, Netflix assisted with testing the merge, but all of the credit goes
to Alexander and iX Systems.

Submitted by: mav
Sponsored by: iX Systems

MFC r256826:
Fix several target mode SIMs to not blindly clear ccb_h.flags field of
ATIO CCBs. Not all CCB flags there belong to them.

MFC r256705:
Optimize isp(4) to reduce CPU usage, especially in target mode:
- Remove two excessive and slow register reads from isp_intr(). Instead
of rereading value every time, assume that registers contain what we have
written there.
- Avoid sequential search through 4096 array elements when looking for
command tag. Use hash of lists to store active tags separately from free
ones and so greatly speedup the searches.