isp(4): Rework firmware handling/loading

Correctly identify the active firmware in flash on adapters with
primary and secondary firmware region in flash.
Correctly identify the active NVRAM on adapters with primary
and secondary NVRAM region in flash.

Loading ispfw(4) moved from isp_pci_attach() to isp_reset().
Drop the reference to ispfw(4) after using it so one can kldunload(8) it.
New isp_load_ram() function to load either ispfw(4) or flash firmware
into RISC's RAM.
New functions to read data from flash. The old ones will be removed later.
A bunch of new helper functions to identify and validate active flash
regions for firmware, auxiliary and NVRAM.
Overhaul ISP_FW_* macros and make use of it when comparing firmware
versions. We can handle firmware versions up to 255.255.255.

Firmware load priority slightly changed:
For 27xx and newer adapters:
- load ispfw(4) firmware
- request (active) flash firmware information
- compare version numbers of ispfw(4) and flash firmware
- load firmware with highest version into RISC's RAM
- if loading ispfw(4) is disabled or failed - load firmware from flash
- if everything else fails use MBOX_LOAD_FLASH_FIRMWARE as fallback

For 26xx and older adapters nothing changed:
- load ispfw(4) firmware and load it into RISC's RAM
- if loading ispfw(4) is disabled or failed use MBOX_EXEC_FIRMWARE
- for 26xx a preceding MBOX_LOAD_FLASH_FIRMWARE is used

New read only sysctl(8)'s:
dev.isp.N.fw_version_run: the firmware version actually running
dev.isp.N.fw_version_ispfw: the firmware version provided by ispfw(4)
dev.isp.N.fw_version_flash: the (active) firmware version in flash

While here:
- firmware attribute handling/parsing reworked
+ renamed defines from ISP2400_FW_ATTR_* to ISP_FW_ATTR_*
+ changed values to match new handling/parsing
+ added some more attributes
- enable FLT support on 26xx based adapters
- log level adjustments
- new function return status codes (some for now, some for later use)
- some minor style changes

Tested and approved to work on real hardware with:
- Qlogic ISP 2532 (QLogic QLE2560 8Gb FC Adapter)
- Qlogic ISP 2031 (QLogic QLE2662 16Gbit 2Port FC Adapter)
- Qlogic ISP 2722 (QLogic QLE2690 16Gb FC Adapter)
- Qlogic ISP 2812 (QLogic QLE2772 32Gbit 2Port FC Adapter)

PR: 273263
Reviewed by: mav
Pull Request: https://github.com/freebsd/freebsd-src/pull/877
MFC after: 1 month
Sponsored by: Technical University of Munich

sys: Remove $FreeBSD$: one-line .c comment pattern

Remove /^/[*/]\s*\$FreeBSD\$.*\n/

isp(4): Add support for QLogic 28xx devices

This covers the following HBAs:
ISP2812-based 64/32G Fibre Channel to PCIe Controller:
QLE2770 Single Port 32GFC PCIe Gen4 x8 Adapter
QLE2772 Dual Port 32GFC PCIe Gen4 x8 Adapter
QLE2870 Single Port 64GFC PCIe Gen4 x8 Adapter
QLE2872 Dual Port 64GFC PCIe Gen4 x8 Adapter

ISP2814-based 64/32G Fibre Channel to PCIe Controller:
QLE2774 Quad Port 32GFC PCIe Gen4 x16 Adapter
QLE2874 Quad Port 64GFC PCIe Gen4 x16 Adapter

While here, add required bits to support 64GB FC.

Default framesize is set to 2048 for ISP28xx based HBAs for now.

PR: 271062
Reviewed by: imp, mav
Sponsored by: Technical University of Munich
Pull Request: https://github.com/freebsd/freebsd-src/pull/726

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

ips(4): Remove a double word in a few source code comments

- s/for for/for/

MFC after: 3 days

Some code reorganization.

- Remove code duplication by adding two new functions to execute prepared
queue entry via either mbox or request queue and wait for result.
- Since the new function executing via request queue sleeps any way, make
it sleep also in case of overflows or handle shortages. It should make it
more reliable and less affecting other less flexible request queue users.
- Turn isp_target_put_entry() into not target-specific isp_send_entry().
- Make handling of responses with control handles more universal.
- Move RQSTYPE_RPT_ID_ACQ handling into new function.
- Inline isp_handle_other_response(), becoming trivial after above.
- Clean the list of IOCBs from pre-24xx ones.

More cleanup in response queue and reset code.

Some minor FCoE bits I had lying around.

Update RQSFLAG_* definitions.

Implement request queue overflow protection.

Before this change in case of request queue overflow driver just froze the
device queue for 100ms to retry after. It was pretty bad for performance.
This change introduces SIM queue freezing when free space on the request
queue drops below 255 entries (worst case of maximum I/O size S/G list),
checking for a chance to release it on I/O completion. If the queue still
get overflowed somehow, the old mechanism is still in place, just with
delay reduced to 10ms.

With the earlier queue length increase overflows should not happen often,
but it is still easily reachable on synthetic tests.

Remove remnants of execthrottle and maxalloc parameters.

The first was obsolete since 26xx, not used on 25xx and not needed on 24xx.
The second seems never worked on 24xx and up.

Remove parallel SCSI and 1/2Gb FC support from isp(4).

This removes 288KB (36%) of the driver code and zillions of hacks and
workarounds, making single driver uniformly support several different
generations of hardware interfaces, not counting minor card variations.
After years of the hopeless fight, I don't think it worth to continue
support for hardware obsolete for 15-20 years. Instead much cleaner
now code should allow to move forward toward better locking, multiple
queues and other cool features.

All the remaining Qlogic cards starting from 4Gb 24xx to 32Gb 27xx use
the same hardware/firmware interface with minor incremental improvements,
so it seems to be a good new starting point. Except one PCI-X model all
all of them are PCIe and so still usable in modern systems.

Discussed with: ken, scottl, jpaetzel, imp
Relnotes: yes

Fix CRN resets in the isp(4) driver in certain situations.

The Command Reference Number (CRN) is part of the FC-Tape features
that we enable when talking to tape drives. It starts at 1, and
goes to 255 and wraps around to 1. There are a number of reset
type conditions that result in the CRN getting reset to 1. These
are detailed in section 4.10 (table 8) of the FCP-4r02b specification.

One of the conditions is when a PRLI (Process Login) is sent by
the initiator, and the Establish Image Pair bit is set in Word 0
of the PRLI.

Previously, the isp(4) driver core sent a notification via
isp_async() that the target had changed or stayed in place, but
there was no indication of whether a PRLI was sent and whether the
Establish Image Pair bit was set.

The result of this was that in some situations, notably
switching back and forth between a direct connection and a switch
connection to a tape drive, the isp(4) driver would fail to reset
the CRN in situations that require it according to the spec. When
the CRN isn't reset in a situation that requires it, the tape drive
then rejects every subsequent command that is sent to the drive.
It is assuming that the commands are being sent out of order.

So, modify the isp(4) driver to include Word 0 of the PRLI command
when it sends isp_async() notifications of target changes. Look at
the Establish Image Pair bit, and reset the CRN if that bit is set.

With this change, I am able to switch a tape drive back and forth
between a direct connection and a switch connection, and the isp(4)
driver resets the CRN when it should.

sys/dev/isp_stds.h:
Add bit definitions for PRLI Word 0.

sys/dev/ispmbox.h:
Add PRLI Word 0 to the port database type, isp_pdb_t.

sys/dev/ispvar.h
Add PRLI Word 0 to fcportdb_t.

sys/dev/isp.c:
Populate the new prli_word0 parameter in the port database.

In isp_pdb_add_update(), add a check to see if the
Establish Image Pair bit is set in PRLI Word 0. If it is,
then that is an additional reason to create a change
notification.

sys/dev/isp_freebsd.c:
In isp_async(), if the device changed or stayed, look at
PRLI Word 0 to see if the Establish Image Pair bit is set.
If it is, reset the CRN if we haven't already.

MFC after: 1 week
Sponsored by: Spectra Logic
Differential Revision: https://reviews.freebsd.org/D19472

Add support for Enhanced Gen 5 (16Gb) and Gen 6 (32Gb) QLogic FC HBAs.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

sys/dev: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Switch fabric scans from GID_FT to GID_PT+GFF_ID/GFT_ID.

Instead of using GID_FT SNS request to get list of registered FCP ports,
use GID_PT to get list of all Nx_Ports, and then use GFF_ID and/or GFT_ID
requests to find whether they are FCP and target capable.

The problem with old approach is that GID_FT does not report ports without
FC-4 type registered. In particular it was impossible to boot OS from
FreeBSD FC target using QLogic FC BIOS, since one does not register FC-4
type even on new cards and so ignored by old code as incompatible.

As a side bonus this allows initiator to skip pointless logins to other
initiators by fetching that information from SNS instead.

In case some switches do not implement GFF_ID/GFT_ID correctly, add sysctls
to disable that functionality. I handled broken GFF_ID of my Brocade 200E,
but there may be other switches with different bugs.

Linux also uses GID_PT, but GFF_ID is disabled by default there, and GFT_ID
is not supported.

Sponsored by: iXsystems, Inc.

Slightly unify SNS requests for post- and pre-24xx.

Correct loop mode CRN resets to adhere to FCP-4 section 4.10

Prior to this change, the CRN (Command Reference Number) is reset on any
firmware LIP, LOOP DOWN, or LOOP RESET event in violation of FCP-4 which
specifies that the CRN should only be reset in response to a LIP Reset
(LIPyx) primitive. FCP-4 also indicates PLOGI/LOGO and PRLI/PRLO ELS
actions as conditions for resetting the CRN for the associated initiator
port.

These violations manifest themselves when the HBA is removed from the
loop, or a target device is removed (especially during an outstanding
command) without power cycling. If the HBA and and the target device
determine upon re-establishing the loop that no PLOGI or PRLI is
required, and the target does not issue a LIPxy to the initiator, the
CRN for the target will have been improperly reset by the isp driver. As
a result, the target port will silently ignore all FCP commands issued
during the device probe (which will time out) preventing the device from
attaching.

This change corrects thie CRN reset behavior in response to loop state
changes, also introduces CRN resets for the above mentioned ELS actions
as encountered through async PDB change events.

This change also adds cleanup of outstanding commands in isp_loop_dead()
that was previously missing.

sys/dev/isp/isp.c
Add the last login state to debug output when syncing the pdb

sys/dev/isp/isp_freebsd.c
Replace binary statement setting aborted ccb status in
isp_watchdog() with the XS_SETERR macro used elsewhere

In isp_loop_dead(), abort or complete pending commands as done
in isp_watchdog()

In isp_async(), segregate the ISPASYNC_LOOP_RESET action from
ISPASYNC_LIP, ISPASYNC_LOOP_DOWN, and ISPASYNC_LOOP_UP
fallthroughs, and only reset the CRN in the RESET case. Also add
checks to handle false LOOP RESET actions that do not have a
proper associated LIP primitive, and log the primitive in the
debug messages

In isp_async(), remove the goto from ISP_ASYNC_DEV_STAYED, and
only reset the CRN in the DEV_CHANGED action

In isp_async(), when processing an ISPASYNC_CHANGE_PDB status,
reset CRN(s) for the associated nphdl (or all ports) if the
change reason is some form of ELS login/logout. Also remove
assignment to fc since it is not used in the scope

sys/dev/isp/ispmbox.h
Add macro definition for the global N-Port handle, and correct a
macro typo 'PDB24XX_AE_PRLI_DONJE'

sys/dev/isp/ispvar.h
Add macros FCP_AL_DA_ALL, FCP_AL_PA, and FCP_IS_DEST_ALPD for
more legible code when determining if an AL_PD port matches the
portid for a given struct fcparam* by value or by virtue of the
AL_PD port being 0xFF

Submitted by: Reid Linnemann
Sponsored by: Spectra Logic
MFC after: 1 week

Add brackets to fix incorrect macro expansion.

Reported by: Andreas Hollmann / PVS-Studio
MFC after: 2 weeks

Add initial support for multiple MSI-X vectors.

For 24xx and above use 2 vectors (default and response queue).
For 26xx and above use 3 vectors (default, response and ATIO queues).
Due to global lock interrupt hardlers never run simultaneously now, but
at least this allows to save one regitster read per interrupt.

MFC after: 2 weeks

Register symbolic port/node names in FC name server.

This is cosmetics that simplifies identification of new ports on FC switch.

It would be good to use target name from CTL here instead of hostname, but
it is not passed here through CAM now.

MFC after: 2 weeks

Some polishing for command timeouts handling.

Add initial support for 16Gbps FC QLogic chips.

I still don't know how to read NVRAM there, so WWNs and other parameters
are incorrect, but other then that driver seems like attaching normally.

Rename ASYNC_LIP_F8 to ASYNC_LIP_NOS_OLS_RECV.

New name better repsents its meaning for modern chips.

Fix target mode support for Qlogic 2200 FC adapters.

Now target mode works for all supported FC adapters except ancient 2100,
which is not tested.

Add some more asynchronous event status codes.

Add mode mailbox command codes.

Another round of port scanner rewrite.

This change simplifies and unifies port adding/updating for loop and
fabric scanners. It also fixes problems with scanning restarts due to
concurrent port databases changes. It also fixes many cosmetic issues.

Register our FC4 Features in SNS.

Add real initial support for RQSTYPE_RPT_ID_ACQ.

Fix and improve error masking and reporting.

Improve/fix loop scanning routine.

For the most of chips (except anscient ones) port handlers have no relation
to port IDs. In such situation old code scanning first 125 handlers was
quite naive. Instead of doing that, send to chip single request to get full
list of port handlers available on specific virtual port and scan only them.

Old code had problems with case of several virtual ports enabled, when port
handlers allocated from global address space could easily go above 125.
This change was successfully tested on 23xx, 24xx and 25xx chips in loop
mode with 4 virtual initiator ports, each seing 50 virtual target ports.

Fix error code that looks like wrong.

Reimplement enable and implement disable of virtual ports.

Now on 24xx and above chips it is really possible to simulate several
virtual FC ports with single physical one. For example, it allows to
configure several targets in ctl.conf, assign each of them to separate
virtual port, and let user to control access to them with switch zoning.

I still doubt that all problems are solved there, but at now it passes
at least basic tests.

Try to keep Loop IDs persistent across chip reinits.

Add new field to Abort IOCB.

Minor additions to Status Type 0 IOCB.

Improve INOTs handling for 24xx and above chips.

Some more defines and polishing for INIT_FIRMWARE.

Zero mbox[0] for INIT_FIRMWARE to fix version 7.3 firmware.

While there, add new fields to isp_icb_2400_t structure.

Decode more firmware attributes.

Use the C99 flexible array construct to denote a variable amount of
data rather than the old-school [1] construct. We have required c99
compilers for some time.

-----------
MISC CHANGES

Add a new async event- ISP_TARGET_NOTIFY_ACK, that will guarantee
eventual delivery of a NOTIFY ACK. This is tons better than just
ignoring the return from isp_notify_ack and hoping for the best.

Clean up the lower level lun enable code to be a bit more sensible.

Fix a botch in isp_endcmd which was messing up the sense data.

Fix notify ack for SRR to use a sensible error code in the case
of a reject.

Clean up and make clear what kind of firmware we've loaded and
what capabilities it has.
-----------
FULL (252 byte) SENSE DATA

In CTIOs for the ISP, there's only a limimted amount of space
to load SENSE DATA for associated CHECK CONDITIONS (24 or 26
bytes). This makes it difficult to send full SENSE DATA that can
be up to 252 bytes.

Implement MODE 2 responses which have us build the FCP Response
in system memory which the ISP will put onto the wire directly.

On the initiator side, the same problem occurs in that a command
status response only has a limited amount of space for SENSE DATA.
This data is supplemented by status continuation responses that
the ISP pushes onto the response queue after the status response.
We now pull them all together so that full sense data can be
returned to the periph driver.

This is supported on 23XX, 24XX and 25XX cards.

This is also preparation for doing >16 byte CDBs.

-----------
FC TAPE

Implement full FC-TAPE on both initiator and target mode side. This
capability is driven by firmware loaded, board type, board NVRAM
settings, or hint configuration options to enable or disable. This
is supported for 23XX, 24XX and 25XX cards.

On the initiator side, we pretty much just have to generate a command
reference number for each command we send out. This is FCP-4 compliant
in that we do this per ITL nexus to generate the allowed 1 thru 255
CRN.

In order to support the target side of FC-TAPE, we now pay attention
to more of the PRLI word 3 parameters which will tell us whether
an initiator wants confirmed responses. While we're at it, we'll
pay attention to the initiator view too and report it.

On sending back CTIOs, we will notice whether the initiator wants
confirmed responses and we'll set up flags to do so.

If a response or data frame is lost the initiator sends us an SRR
(Sequence Retransmit Request) ELS which shows up as an SRR notify
and all outstanding CTIOs are nuked with SRR Received status. The
SRR notify contains the offset that the initiator wants us to restart
the data transfer from or to retransmit the response frame.

If the ISP driver still has the CCB around for which the data segment
or response applies, it will retransmit.

However, we typically don't know about a lost data frame until we
send the FCP Response and the initiator totes up counters for data
moved and notices missing segments. In this case we've already
completed the data CCBs already and sent themn back up to the periph
driver. Because there's no really clean mechanism yet in CAM to
handle this, a hack has been put into place to complete the CTIO
CCB with the CAM_MESSAGE_RECV status which will have a MODIFY DATA
POINTER extended message in it. The internal ISP target groks this
and ctl(8) will be modified to deal with this as well.

At any rate, the data is retransmitted and an an FCP response is
sent. The whole point here is to successfully complete a command
so that you don't have to depend on ULP (SCSI) to have to recover,
which in the case of tape is not really possible (hence the name
FC-TAPE).

Sponsored by: Spectralogic
MFC after: 1 month

Clean up multi-id mode so it's driven by the f/w loaded,
not by some hint setting. Do more preparations for FC-Tape.
Clean up resource counting for 24XX or later chipsets so
we find out after EXEC_FIRMWARE what is actually supported.
Set target mode exchange count based upon whether or not
we are supporting simultaneous target/initiator mode. Clean
up some old (pre-24XX) xfwoption and zfwoption issues.

Sponsored by: Spectralogic
MFC after: 3 days

Prepare for FC-Tape support. This involved doing a lot of little cleanups
and crosschecks against firmware documentation. We now check and report
FC firmware attributes and at least are now prepared for the upper 48 bits
of f/w attributes (which are probably for the 8100 or later cards). This
involed changing how inbits and outbits are calculated for varios commands,
hopefully clearer and cleaner. This also caused me to clean up the actual
mailbox register usage. Finally, we are now unconditionally using a CRN
for initiator mode.

A longstanding issue with the 2400/2500 is that they do *not* support
a "Prefer PTP followed by loop", which explains why enabling that
caused the f/w to crash.

A slightly more invasive change is to let the firmware load entirely
drive whether multi_id support is enabled or not.

Sponsored by: Spectralogic
MFC after: 1 week

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

MFC of 204397: fix problems with fast posting handles

Revamp the pieces of some of the stuff I forgot to do when shifting to
32 bit handles. The RIO (reduced interrupt operation) and fast posting
for the parallel SCSI cards were all 16 bit handles. Furthermore,
target mode parallel SCSI only can have 16 bit handles.

Use part of a supplied patch to switch over to using 32 bit handles.
Be a bit more conservative here and only do this for parallel SCSI
for the 12160 (Ultra3) cards. There were a lot of marginal Ultra2
cards, and, frankly, few are findable now for testing.

Fix the target handle routine to only do 16 bit handles for parallel
SCSI cards. This is okay because the upper sixteen bits of the new
32 bit handles is a sequence number to help protect against duplicate
completions. This would be very unlikely to happen with parallel
SCSI target mode, and wasn't present before, so we're no worse off
than we used to be.

While we're at it, finally split the async mailbox completion handlers
into FC and parallel SCSI functions. This makes it much cleaner and
easier to figure out what is or isn't a legal async mailbox completion
code for different card classes.

PR: kern/144250
Submitted partially by: Charles D
MFC after: 1 week

MFC a number of changes from head for ISP (203478,203463,203444,202418,201758,
201408,201325,200089,198822,197373,197372,197214,196162). Since one of those
changes was a semicolon cleanup from somebody else, this touches a lot more.

Redo how commands handles are created and managed and implement sequence
numbers and handle types in rational way. This will better protect from
(unwittingly) dealing with stale handles/commands.

Fix the watchdog timeout code to better protect itself from mistakes.

If we run an abort on a putatively timed out command, the command
may in fact get completed, so check to make sure the command we're
timing it out is still around. If the abort succeeds, btw, the command
should get returned via a different path.

(semiforced commit to add comment missed in last delta)
Add a maximum response length for FCP RSPNS IUs.

Clarify some of the FC option words for setting parameters
and try and disable automatic PRLI when in target mode- this
should correct some cases of N-port topologies with 23XX cards
where we put out an illegal PRLI (in target mode only we're
not supposed to put out a PRLI).

Remove file unused in freebsd.

Add 8Gb support (isp_2500). Fix a fair number of configuration and
firmware loading bugs.

Target mode support has received some serious attention to make it
more usable and stable.

Some backward compatible additions to CAM have been made that make
target mode async events easier to deal with have also been put
into place.

Further refinement and better support for NP-IV (N-port Virtualization)
is now in place.

Code for release prior to RELENG_7 has been stripped away for code clarity.

Sponsored by: Copan Systems

Reviewed by: scottl, ken, jung-uk kim
Approved by: re

- Import the HEAD csup code which is the basis for the cvsmode work.

Fix some stupid copyright mistakes that have been there for quite some time.

Don't attempt to load illegal hard loop addresses into
an ICB. This shows up on card restarts, and usually for
2200-2300 cards. What happens is that we start up,
attempting to acquire a hard address. We end up instead
being an F-port topology, which reports out a loop id
of 0xff (or 0xffff for 2K Login f/w). Then, if we restart,
we end up telling the card to go off an acquire this loop
address, which the card then rejects. Bah.

Compilation fixes from Solaris port.

Try an experiment with using DMA to load firmware into a 2200- VERIFY
CHECKSUM fails. Oh well, but keep a couple of the changes.

Avoid overflow in usec counters when waiting for mailbox completion.

clarify a comment slightly

Make the SAN login/logout stuff more common between different chipsets
and provied an isp_control entry point so that the outer layers can
do PLOGI/LOGO explicitly. Add MS IOCB support. This completes the cycle
for base support for SMI-S.

Increase the timeout for some SAN commands.

Only complain about FC Reponse errors if they're nonzero.

Shorten some PortID printouts for local loop.

Add an internal isp_xcmd_t data structure which we'll use for some
CT-Passthru support as part of adding SMI-S.

Add 4Gb (24XX) support and lay the foundation for a lot of new stuff.

Do various fixes to support firmware loading for the 2322
(and by extension, the 2422).

One peculiar thing I've found with the 2322 is that if you
don't force it to do Hard LoopID acquisition, the firmware
crashes. This took a while to figure out.

While we're at it, fix various bugs having to do with NVRAM
reading and option setting with respect to pieces of NVRAM.

Some more gratuitous format and name changes.

Pull in some target mode changes from a private branch.
Pull in some more RELENG_4 compilation changes.

A lot of lines changed, but not much content change yet.

a) clean up some declaration stuff (i.e., make more modern with respect
to getting rid u_int for uint and so on).

b) Turn back on 64 bit DAC support. Cheeze it a bit in that we have two
DMA callback functions- one when we have bus_addr_t > 4 bits in width and
the other which should be normal. Even Cheezier in that we turn off setting
up DMA maps to be BUS_SPACE_MAXADDR if we're in ISP_TARGET_MODE. More work
on this in a week or so.

c) Tested under amd64 and 1MB DFLTPHYS, sparc64, i386 (PAE, but insufficient
memory to really test > 4GB). LINT check under amd64.

MFC after: 1 month

First of several commits as this driver is dusted off and maybe brought
up to date. Principle changes for this reelase is to support 2K Port Login
firmware. This allows us to support the 2322 (and 2422 4Gb) cards which only
come with the 2K Port Login firmware. The 2322 should now work- but we don't
have firmware sets for it in ispfw (as the change to load 2K Port Login f/w
hasn't been made- that f/w is so big it has to be loaded in more than one
chunk).

Other changes are the beginnings of cleaning up some long standing target
mode issues. The next changes here will incorporate a lot of bug fixes
from others.

Finally, some copyright cleanup and attempts to make the parts of the
driver that are FreeBSD specific start conforming more to FreeBSD style.

MFC after: 1 month

Add an ioctl framework for doing FC task management functions from
a user space tool- useful for doing FC target mode certification.

Start each of the license/copyright comments with /*-, minor shuffle of lines

add MAKE_WWN_FROM_NODE_NAME macro

Add an explanatory comment about what operational modes in xfwopt are.

Add MBOX_INIT_REQ_QUEUE_A64/MBOX_INIT_RES_QUEUE_A64 definitions.
Define ispreq64_t to be the same as ispreqt3_t.

Add ICBXOPT_ZIO definition.

Add MBOX_DRIVER_HEARTBEAT/MBOX_FW_HEARTBEAT/FC4_FC_SVC defines.

MFC after: 1 week

Scale back # of luns supported for SCC to 16384- oops- top 3 bits are a
lun address modifier of sorts. Only an HP XP-512 seems to have cared.

Fix a few misplaced pointers for the new fabric goop, which has been
demonstrated to work on newer Brocades and McData switches now.
Put in commented out code which would run GFF_ID if the QLogic f/w
allowed it.

Don't whine about not being able to find a handle for a command if it
was a command aborted (by us).

Fix bus dma segment count to be based off of MAXPHYS, not BUS_SPACE_MAXSIZE.
Grumble. I've seen better documented architectures out of Redmond.

Redo fabric evaluation to not use GET ALL NEXT (GA_NXT). Switches seem
to be trying to wriggle out of supporting this well. Instead, use
GID_FT to get a list of Port IDs and then use GPN_ID/GNN_ID to find the
port and node wwn. This should make working on fabrics a bit cleaner and
more stable.

This also caused some cleanup of SNS subcommand canonicalization so that
we can actually check for FS_ACC and FS_RJT, and if we get an FS_RJT,
print out the reason and explanation codes.

We'll keep the old GA_NXT method around if people want to uncomment a
controlling definition in ispvar.h.

This also had us clean up ISPASYNC_FABRICDEV to use a local lportdb argument
and to have the caller explicitly say that a device is at the end of the
fabric list.

MFC after: 1 week

Support for f/w crash dumps (2200 && 23XX).

If you want QLogic to look at a potential f/w problem for FC cards, you really
have to provide them info in the format they expect. This involves dumping
a lot of hardware registers (> 300 16 bit registers) and a lot of SRAM
(> 128KB minimum). Thus all of this code is #ifdef protected which will
become an option so that the memory allocation of where to dump the crash
image is pretty expensive. It's worth it if you have a reproducible problem
because they have some tools that can tell them, given the f/w version,
the precise state of everything.

MFC after: 1 week

+ A variety of 23XX changes:
disable MWI on 2300

based on function code, set an 'isp_port' for the 2312- it's a
separate instance, but the NVRAM is shared, and the second port's
NVRAM is at offset 256.

+ Enable RIO operation for LVD SCSI cards. This makes a *big* difference
as even under reasonable load we get batched completions of about 30
commands at a time on, say, an ISP1080.

+ Do 'continuation' mailbox commands- this allows us to specify a work
area within the softc and 'continue' repeated mailbox commands. This is
more or less on an ad hoc basis and is currently only used for firmware
loading (which f/w now loads substantially faster becuase the calling
thread is only woken when all the f/w words are loaded- not for each
one of the 40000 f/w words that gets loaded).

+ If we're about to return from isp_intr with a 'bogus interrupt' indication,
and we're not a 23XX card, check to see whether the semaphore register is
currently *2* (not *1* as it should be) and whether there's an async completion
sitting in outgoing mailbox0. This seems to capture cases of lost fast posting
and RIO interrupts that the 12160 && 1080 have been known to pump out under
extreme load (extreme, as in > 250 active commands).

+ FC_SCRATCH_ACQUIRE/FC_SCRATCH_RELEASE macros.

+ Endian correct swizzle/unswizzle of an ATIO2 that has a WWPN in it.

MFC after: 1 week

Implement REDUCED INTERRUPT OPERATION usage form FC cards- this allows the
firmware to delay completion of commands so that it can attempt to batch
a bunch of completions at once- either returning 16 bit handles in mailbox
registers, or in a resposne queue entry that has a whole wad of 16 bit handles.

Distinguish between 2300 and 2312 chipsets- if only because the revisions
on the chips have different meanings.

Add more instrumentation plus ISP_GET_STATS and ISP_CLR_STATS ioctls.
Run up the maximum number of response queue entities we'll look at
per interrupt.

If we haven't set HBA role yet, always return success from isp_fc_runstate.

MFC after: 2 weeks

Major restructuring for swizzling to the request queue and unswizzling from
the response queue. Instead of the ad hoc ISP_SWIZZLE_REQUEST, we now have
a complete set of inline functions in isp_inline.h. Each platform is
responsible for providing just one of a set of ISP_IOX_{GET,PUT}{8,16,32}
macros.

The reason this needs to be done is that we need to have a single set of
functions that will work correctly on multiple architectures for both little
and big endian machines. It also needs to work correctly in the case that
we have the request or response queues in memory that has to be treated
specially (e.g., have ddi_dma_sync called on it for Solaris after we update
it or before we read from it). It also has to handle the SBus cards (for
platforms that have them) which, while on a Big Endian machine, do *not*
require *most* of the request/response queue entry fields to be swizzled
or unswizzled.

One thing that falls out of this is that we no longer build requests in the
request queue itself. Instead, we build the request locally (e.g., on the
stack) and then as part of the swizzling operation, copy it to the request
queue entry we've allocated. I thought long and hard about whether this was
too expensive a change to make as it in a lot of cases requires an extra
copy. On balance, the flexbility is worth it. With any luck, the entry that
we build locally stays in a processor writeback cache (after all, it's only
64 bytes) so that the cost of actually flushing it to the memory area that is
the shared queue with the PCI device is not all that expensive. We may examine
this again and try to get clever in the future to try and avoid copies.

Another change that falls out of this is that MEMORYBARRIER should be taken
a lot more seriously. The macro ISP_ADD_REQUEST does a MEMORYBARRIER on the
entry being added. But there had been many other places this had been missing.
It's now very important that it be done.

Additional changes:

Fix a longstanding buglet of sorts. When we get an entry via isp_getrqentry,
the iptr value that gets returned is the value we intend to eventually plug
into the ISP registers as the entry *one past* the last one we've written-
*not* the current entry we're updating. All along we've been calling sync
functions on the wrong index value. Argh. The 'fix' here is to rename all
'iptr' variables as 'nxti' to remember that this is the 'next' pointer-
not the current pointer.

Devote a single bit to mboxbsy- and set aside bits for output mbox registers
that we need to pick up- we can have at least one command which does not
have any defined output registers (MBOX_EXECUTE_FIRMWARE).

MFC after: 2 weeks

Misunderstanding documentation caused me to try and set 1Gbps/2Gps/Auto
connection speed for the 2300 in the wrong offset in the ICB. Oops.

Respect some QLogic errat wrt PCI errors on certain shared host/RISC registers.

Implement a call to get the actual link data rate (if 23XX) so we can
set whether it's a 2Gps or 1Gps link.

MFC after: 1 week

Add 2 Gigabit Fibre Channel support (2300 && 2312 cards). This required
some reworking (and consequent cleanup) of the interrupt service code.

Also begin to start a cleanup of target mode support that will (eventually)
not require more inforamtion routed with the ATIO to come back with the
CTIO other than tag.

MFC after: 4 weeks

Add MBOX_GET_PCI_PARAMS alias.

MFC after: 2 weeks

Add more MBOX and ASYNC event defines.

MFC after: 2 weeks

Add a bunch of additional defines for completion codes. Define
some of the RIO (reduced interrupt operation) stuff. Add 64 bit
data list (DSD type 1) and arbitrary data list (DSD type 2)
data structure defines.

Add macros that parameterize usage of the Request/Response in/out
queue pointers. When we finish 2300 support, different registers
will be accessed for the 2300.

Add some target mode definitions and firmware (FC only) attribute definitions.

Fix a longstanding bug- we had the sense of what bit 14
for the ICB firmware options meant- *I* had taken it to
mean that if you set it, Node Name would be ignored and
derived from Port Name. Actually, it meant the opposite.
As a consequence- change ICBOPT_USE_PORTNAME to the
define ICBOPT_BOTH_WWNS- makes more sense.

Fix wrong input bitmap for MBOX_DUMP_RAM command. Call
ISP_DUMPREGS if we get a f/w crash. Add ISPCTL_RUN_MBOXCMD
control command (so outer layers can run a mailbox command
directly) and add a ISPASYNC_UNHANDLED_RESPONSE hook so
outer layers can understand response queue entries we
might not know about.

Add structure defining FC-AL position maps. The only tool that I know of
that really uses this is luxadm(8) under Solaris.

Add some SNS "Register FC4 type" subcommand defines. Add some defines
that are pertinetnt for state flags on Qlogic 2X00 status completion
entries.

some copyright cleanups

various fixes

Add in macros && masks so that mailbox command errors can be
selectively printed/supressed in isp_mboxcmd.

add MBOX_GET_RESOURCE_COUNT command

Conrrect a macro with parenthesis.

Add in mailbox return codes for failed fabric logins (port_id_used,
loop_id_used, etc...)

Do a more precise structure for Get All Next name server responses.

Approved: jkh

Add in new async mbox defines for 2200- point to point stuff.
Add in definitions for the extended initialization control block
(2200 only again).

Approved: jkh@freebsd.org

add clarifying tag define for FC

Add missing target mode flag and fix the RQSTYPE_CTIO0
to be CTIO as it should be.

Add in missing ENABLE TARGET MODE opcode.

Remove target mode definitions (they'll come back later in
a different file).

$Id$ -> $FreeBSD$

add in new ICB structure variant (for 2200)

add 2200 f/w; fix botched define

Add in SNS and Fabric login/logout commands. Clean up ICBOPT defines.
Remove INVALID_PDB_OPTIONS defines. Define generic SNS request and
response structures and the bare minimum GAN and GP3 subcommands.

add a couple of missing commands

Remove incorrect BIG_ENDIAN defines and substitute in the SBus only macros
that will SBusify an isp header or the lun/target portions of a request IOCB-
and have these only valid iff __sparc__ (no non-sparc SBus machine that *I*
know about).

roll internal tag level

A wad of changes- prepping for 1080/1240 support (which caused a massive
thwank in register layout goop). A different mboxcmd approach. Some PDB change
infrastructure. Some better management of loopdown/loopup events (keep them
distinct from resource starvation for simq freeze/unfreeze actions).

roll internal release tag

Implement and use Fast Posting for both parallel && fibre. Redo a bit of
the startup code. Implement a call to outer framework function so that
asynchronous events can be handled (e.g., speed negotiation, target mode).

Roll internal release tags.

clarify headers;move uninit to outer layer;remove watchdog

do a bit of cleanup on some target mode structures and clarify a couple other minro things

Update QLogic ISP support for CAM. Add preliminary target mode support.

Submitted by: Matthew Jacob <mjacob@feral.com>

Add support for the Qlogic ISP SCSI && FC/AL Adapters