Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC 266281:
Clear the data buffer length field when freeing a command structure so that
it doesn't leak through when the command structure is reused for a user
command without a data buffer.

PR: 189668
Approved by: re (delphij)

MFC: r260044

Free the MSI again on detach if allocated. Arguably, this code would be
better off living in aac_pci.c, but it doesn't seem worth creating a
aac_pci_detach() and it's also not the first PCI-specific bit in aac.c

MFC: r251116

Allow unmapped I/O via aacd(4). It shouldn't be too hard to add the
same support for aacp(4), I'm lacking the necessary hardware for
testing, though.

MFC: r251115

- Remove pointless returns.
- Make cm_data a void pointer and cm_flags unsigned as appropriate.

MFC: r247570, r247591

- Make tables, device ID strings etc const. This includes #ifdef'ing 0
aac_command_status_table, which is actually unused since r111532.
While at it, make aac_if a pointer to the now const interface tables
instead of copying them over to the softc (this alone already reduces the
size of aac.ko on amd64 by ~1 KiB).
- Remove redundant softc members.
- Use DEVMETHOD_END.
- Use NULL instead of 0 for pointers.
- Remove redundant bzero(9)'ing of the softc.
- Use pci_enable_busmaster(9) instead of duplicating it.
- Remove redundant checking for PCIM_CMD_MEMEN (resource allocation will
just fail).
- Canonicalize the error messages in case of resource allocation failures.
- Add support for using MSI instead of INTx, controllable via the tunable
hw.aac.enable_msi (defaulting to on).

MFC r227293 (by ed):
Mark MALLOC_DEFINEs static that have no corresponding MALLOC_DECLAREs.

This means that their use is restricted to a single C file.

MFC: r227309 (partial)

Mark all SYSCTL_NODEs static that have no corresponding SYSCTL_DECLs.

The SYSCTL_NODE macro defines a list that stores all child-elements of
that node. If there's no SYSCTL_DECL macro anywhere else, there's no
reason why it shouldn't be static.

MFC r231589: Add a sysctl to report the firmware build number.

Some older firmware versions have issues that can be worked around by
avoiding certain operations. Add a sysctl dev.aac.#.firmware_build to
make it easy for scripts or userland tools to detect the firmware
version.

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

- Fix races on detach handling of AAC_IFFLAGS_* mask
- Fix races on setting AAC_AIFFLAGS_ALLOCFIBS
- Remove some unused AAC_IFFLAGS_* bits.
Please note that the kthread still makes a difference between the
total mask and AAC_AIFFLAGS_ALLOCFIBS because more flags may be
added in the future to aifflags.

Sponsored by: Sandvine Incorporated
Reported and reviewed by: emaste
MFC after: 2 weeks

Revert part of r173264. Both aac_ioctl_sendfib and aac_ioctl_send_raw_srb
make use of the aac_ioctl_event callback, if aac_alloc_command fails. This
can end up in an infinite loop in the while loop in aac_release_command.

Further investigation into the issue mentioned by Scott Long [1] will be
necessary.

[1] http://lists.freebsd.org/pipermail/freebsd-current/2007-October/078740.html

Previously, the aac driver did not handle enclosure management AIFs,
which were raised during hot-swap events. Now such events trigger cam
rescans, as is done in the mps driver.

Submitted by: Mark Johnston <mjohnston at sandvine dot com>

Use device_printf where possible, and otherwise at least include the
driver name in printf strings.

Reported by: Mark Johnston

Implement device unbusying via a cdevpriv destructor.

Suggested by: jhb
Tested by: Mark Johnston <mjohnston at sandvine dot com>
Reviewed by: emaste, jhb
MFC after: 10 days
X-MFC: r212661

Fix bogus busying mechanism from cdevsw callbacks:
- D_TRACKCLOSE may be used there as d_close() are expected to match up
d_open() calls
- Replace the hand-crafted counter and flag with the
device_busy()/device_unbusy() proper usage.

Sponsored by: Sandvine Incorporated
Reported by: Mark Johnston <mjohnston at sandvine dot com>
Tested by: Mark Johnston
Reviewed by: emaste

MFC after: 10 days

Avoid repeatedly spamming the console while a timed out command is waiting
to complete. Instead, print one message after the timeout period expires,
and one more when (if) the command eventually completes.

MFC after: 1 month

Sync some minor items with the upstream driver. Should have no functional
change.

Whitespace cleanup, in advance of next sync with Adaptec's driver. No
functional change.

Make the code more readable and compiling on 64-bits arch different
than amd64.

Sponsored by: Sandvine Incorporated
Submitted by: emaste
MFC: 2 weeks
X-MFC: r205160

Checkin a facility for specifying a passthrough FIB from userland.
arcconf tool by Adaptec already seems to use for identifying the
Serial Number of the devices.
Some simple things (like FIB setup and bound checks) are retrieved
from the Adaptec's driver, but this implementation is quite different
because it does use the normal buffer dmat area for loading segments
and not a special one (like the Adaptec's one does).

Sponsored by: Sandvine Incorporated
Discussed with: emaste, scottl
Reviewed by: emaste, scottl
MFC: 2 weeks

Minor diff reduction with Adaptec's driver: in aac_release_command() set
cm_queue to AAC_ADAP_NORM_CMD_QUEUE by default. In every place it was set,
it was set to AAC_ADAP_NORM_CMD_QUEUE anyhow.

Include command type in COMMAND TIMEOUT messages to aid in debugging.

Diff reduction with Adaptec's vendor driver.

Driver version 2.1.9 chosen as that Adaptec version roughly corresponds
with the current feature set merged to the in-tree driver.

Garbage collect Falcon/PPC support that has not been used in released
products, based on discussion with Adaptec.

Remove extraneous semicolons, no functional changes.

Submitted by: Marc Balmer <marc@msys.ch>
MFC after: 1 week

- Try pre-allocating all FIBs upfront. Previously we tried pre-allocating
128 FIBs first and allocated more later if necessary. Remove now unused
definitions from the header file[1].
- Force sequential bus scanning. It seems parallel scanning is in fact
slower and causes more harm than good[1]. Adjust a comment to reflect that.

PR: kern/141269
Submitted by: Alexander Sack (asack at niksun dot com)[1]
Reviewed by: scottl

Rename aac_fast_intr to aac_filter to reflect its current use. Eliminate
the fallback of using the filter as an interrupt handler, as it is no
longer needed.

Discussed with: scottl, jhb

Do first controller time sync after 1 minute, as in Adaptec's vendor
driver.

Whitespace fixup: 8 spaces -> tab

Whitespace tweak.

Submitted by: phk, via accidental commit in r196969.

Revert previous commit and add myself to the list of people who should
know better than to commit with a cat in the area.

Add necessary include.

Temporarily revert the new-bus locking for 8.0 release. It will be
reintroduced after HEAD is reopened for commits by re@.

Approved by: re (kib), attilio

Make the newbus subsystem Giant free by adding the new newbus sxlock.
The newbus lock is responsible for protecting newbus internIal structures,
device states and devclass flags. It is necessary to hold it when all
such datas are accessed. For the other operations, softc locking should
ensure enough protection to avoid races.

Newbus lock is automatically held when virtual operations on the device
and bus are invoked when loading the driver or when the suspend/resume
take place. For other 'spourious' operations trying to access/modify
the newbus topology, newbus lock needs to be automatically acquired and
dropped.

For the moment Giant is also acquired in some key point (modules subsystem)
in order to avoid problems before the 8.0 release as module handlers could
make assumptions about it. This Giant locking should go just after
the release happens.

Please keep in mind that the public interface can be expanded in order
to provide more support, if there are really necessities at some point
and also some bugs could arise as long as the patch needs a bit of
further testing.

Bump __FreeBSD_version in order to reflect the newbus lock introduction.

Reviewed by: ed, hps, jhb, imp, mav, scottl
No answer by: ariff, thompsa, yongari
Tested by: pho,
G. Trematerra <giovanni dot trematerra at gmail dot com>,
Brandon Gooch <jamesbrandongooch at gmail dot com>
Sponsored by: Yahoo! Incorporated
Approved by: re (ksmith)

We no longer need to use d_thread_t, migrate to struct thread *.

SVN rev 188743 modified aac_rx_get_fwstatus to use the AAC_RX_OMR0
register instead of AAC_RX_FWSTATUS, as that is the way it's done in
Adaptec's vendor driver and in the Linux drivers. (The same applies
to aac_rkt_get_fwstatus as well.)

However, a concern has been raised about the compatibility of this
change and old hardware / firmware versions. In the absense of
specific information, revert to the original behaviour if the firmware
does not support the "New comm." interface. Users of old cards or
firmware haven't reported the problems that are potentially solved by
switching to OMR0.

Sync with the official Adaptec vendor driver:

[1] Add the support for the NARK controller which seems a variant of
the i960Rx.
[2] Split up memory regions and other resources in 2 different parts
as long as NARK uses them separately (it is not clear to me
why though as long as there are no more informations available
on this controller). Please note that in all the other cases,
the regions overlaps leaving the default behaviour for all the
other controllers.
[3] Implement a clock daemon responsible for maintain updated the
wall clock time of the controller (run any 30 minutes)*.

Submitted by: Adaptec (driver build 15317 [1, 2] and 15727 [3])
Reviewed by: emaste
Tested by: emaste
Sponsored by: Sandvine Incorporated

* Please note that originally, in the Adaptec driver, the clock daemon
is not implemented with callouts as in our in-tree driver.

Use outbound message register 0 instead of mailbox 7 in
aac_{rx,rkt}_get_fwstatus, as done in Adaptec's vendor driver as well as
the Linux drivers.

Submitted by: jkim, from Adaptec's driver

Fix test for waiting AIFs in aac_poll(). This seems to solve the
problem where Adaptec's arcconf monitoring tool hangs after producing
its expected output.

Submitted by: Adaptec, via driver ver 15317
MFC after: 1 week

Calling RequestSupplementAdapterInfo before RequestAdapterInfo appears
to trip a bug causing the latter to return a zeroed struct
aac_adapter_info. This causes two issues. One is cosmetic only --
a verbose boot prints information about the controller, and shows all
zero:

aac0: Unknown processor 0MHz, 0MB memory (0MB cache, 0MB execution),
unknown battery platform

The second problem is that the firmware version information is stored
away for aac_rev_check, for userland tools (like aaccli) to query via
the FSACTL_MINIPORT_REV_CHECK and FSACTL_LNX_MINIPORT_REV_CHECK ioctls.
When aaccli encounters this issue it prints

Command Error: <The current AFAAPI.DLL is too old to work with the
current controller software.>

Move the RequestSupplementAdapterInfo call after RequestAdapterInfo,
which seems to fix both problems.

Implement FSACTL_LNX_GET_FEATURES and FSACTL_GET_FEATURES ioctls. RAID
tools (e.g. arcconf) need this to be able to create arrays larger than 2TB.

Submitted by: Adaptec, via driver build 15317

Add \n to the end of a printf string and remove it from panic strings.

Add 64-bit array support for RAIDs > 2TB. This corresponds to ~ Adaptec
driver build 15317.

Tested on:
Adaptec 2230S, Firmware 4.2-0 (8205)
ICP ICP5085BL, Firmware 5.2-0 (12814)

Submitted by: Adaptec

Correct data direction flags in aac_bio_command() in the
!AAC_FLAGS_RAW_IO && AAC_FLAGS_SG_64BIT case.

Submitted by: Adaptec

Diff reduction to Adaptec's driver (around build 15317): catch up with a
change in debugging routines.

The fwprintf macro in the AAC_DEBUG case (mapping to printf) isn't from the
Adaptec driver.

Diff reduction to Adaptec driver build 15317 (refactoring and code shuffling):

- Resource allocation in aac_alloc (moved from from aac_init)
- Interrupt setup in aac_setup_intr (from aac_attach)
- Container probing in aac_get_container_info (from aac_startup and
aac_handle_aif)
- Firmware status check moved to aac_check_firmware from aac_init

Restore creation of passthrough devices with newer controller firmware by
putting the correct size in the fib header. Presumably the older firmware
silently ignored a bad size field.

(This change tested with a 3805 controller. Passthrough devices were
created when running firmware build 12814, but not 15323 or later. With
this change they're created for both old and new firmware versions.)

Submitted by: Adaptec

Add ioctls FSACTL_SEND_LARGE_FIB, FSACTL_SEND_RAW_SRB,
FSACTL_LNX_SEND_LARGE_FIB, and FSACTL_LNX_SEND_RAW_SRB, and correct size
checks on FIBs passed in from userspace. Both changes were obtained from
Adaptec's driver build 15317. Adaptec's commandline RAID tool arcconf uses
these ioctls when creating a RAID-10 array (and probably other operations
too).

Change spelling and eliminate a typo in comments to reduce diffs with
Adaptec's vendor driver. I have some fixes to bring in and this makes
ongoing review of the FreeBSD-Adaptec driver diffs easier.

Avoid holding the aac_io_lock over copyout.

Submitted by: Achim Leubner @ Adaptec.

Do not recursively acquire aac_io_lock in aac_ioctl_event.

MFC After: 3 days

Add family support to allow the driver to attach to new devices that
share the same interface.

Submitted by: Achim Leubner at Adaptec

Allow simultaneous opens of the device for issuing commands to the
controller. This is merged from Adaptec driver build 11669.

Rework aac locking for MPSAFE CAM. This fixes a Giant mutex assertion
reported on freebsd-current [1].

Also dequeue all events in aac_release_command (instead of just one)
so that there's no risk of them getting stranded.

Reported by: Steven Brown [1]
Submitted by: scottl@

[1] http://lists.freebsd.org/pipermail/freebsd-current/2007-October/077928.html

Rename the kthread_xxx (e.g. kthread_create()) calls
to kproc_xxx as they actually make whole processes.
Thos makes way for us to add REAL kthread_create() and friends
that actually make theads. it turns out that most of these
calls actually end up being moved back to the thread version
when it's added. but we need to make this cosmetic change first.

I'd LOVE to do this rename in 7.0 so that we can eventually MFC the
new kthread_xxx() calls.

Correct calculation of aac_sg_tablesize.

Obtained from: Adaptec, via driver b11669

o break newbus api: add a new argument of type driver_filter_t to
bus_setup_intr()

o add an int return code to all fast handlers

o retire INTR_FAST/IH_FAST

For more info: http://docs.freebsd.org/cgi/getmsg.cgi?fetch=465712+0+current/freebsd-current

Reviewed by: many
Approved by: re@

The return value of aac_alloc_command() was misinterpreted in aac_ioctl_event().
Once triggered this would leak away all available commands and starve the rest
of the driver.

Reviewed by: scottl

Rearrange locking in the alloc_commands and ioctl paths to avoid problems.

Initialize the event tailq.

Submitted by: Frank Mayhar
PR: kern/90882
MFC After: 1 day

Remove the unused AAC_DRIVER_BUILD_DATE macro.
Nuke whitespace at EOL while I'm here.

Approved by: scottl (MAINTAINER)

Pickup I/O lock in aac_get_bus_info, as this code will call
aac_alloc_sync_fib(). aac_alloc_sync_fib() will assert that the I/O locks
are held. This fixes a panic on system boot up when the aac(4) device's
bus_generic_attach() routine is called.

Reviewed by: scottl

Fix a regression that prevented PERC3 hardware from working. Apparently they
do not support the GETINFO immediate command, unlike just about every other
variant of the hardware. Also document some magic values and fix some minor
nearby whitespace.

MFC After: 3 days

Fix a missing set of lock operations.

Submitted by: green
PR: 87191

Ue a better msleep identifier. Fix some whitespace.

Mega Update to the aac driver to support a whole new family of cards and
the modified interface that they use. Changes include:

- Register a different interrupt handler for the new interface. This one is
INTR_MPSAFE, not INTR_FAST, and directly processes completions and AIFs.
- Add an event registration and callback mechanism for the ioctl and CAM
modules can know when a resource shortage clears. This condition was
previously fatal in CAM due to programming oversights.
- Fix locking to play better with newbus.
- Provide access methods for talking to cards with the NEWCOMM interface.
- Fix up the CAM module to be better suited for dealing with newer firmware
on the PERC Si/Di series that requires talking to plain SCSI via aac.
- Add a whole slew of new PCI Id's.

Thanks to Adaptec for providing an initial version of this work and for
answering countless questions about it. There are still some rough edges in
this, but it works well enough to commit and test for now.

Obtained from: Adaptec, Inc.

Use time_uptime for timing commands instead of time_second. This is a bit
risky because the "current time" is supposed to be fed to the card during
initialization, and the current time is supposed to be put into each command
that is sent to the card. Hopefully either the card doesn't actually care
about the timestamps, or it doesn't care about the absolute values so long
and the relative values are consistent. Not an MFC candidate until more
thorough testing can be done.

Add a text description for the Intel IOP302/303 processors. Be slightly
more verbose about the allocation of RAM on the controller.

Sbumitted by: Jeremy Chadwick
PR: kern/81259
MFC-After: 3 days

Remove bus_{mem,p}io.h and related code for a micro-optimization on i386
and amd64. The optimization is a trivial on recent machines.

Reviewed by: -arch (imp, marcel, dfr)

purge dead code

Noticed by: Coverity Prevent analysis tool

Handle failures better in the passthrough bus creation code.

Submitted by: Coverity Prevent analysis tool

Complete the repo-copy of aac_ioctl.h from sys/dev/aac/to sys/sys.

Don't force busdma to pre-allocate bounce pages for static allocations.

If the timeout handler runs and notices that commands are timed out, check
the firmware status register on the card to see if the firmware is still
running. There is no way to recover from this, but at least it can give
a hint as whether the car has crashed (which happens all too often).

MFC after: 3 days

Put some of the probe messages under bootverbose so to lessen the noise.

Add support for the Adaptec RAID-On-Chip architecture. This in turn
provides support for the Adaptec 2130S adapter. Thanks to Adaptec for
providing hardware for this.

Destroy mutexes on detach.

Change FREE() to free()

Remove the AAC_LOCK macros. They no longer abstract anything and only
obfuscate the code. No functional differences.

Release the sync fib after the controller has been shut down. This also
releases the I/O lock instead of just leaking it.

Avoid casts as lvalues.

Do the dreaded s/dev_t/struct cdev */
Bump __FreeBSD_version accordingly.

Collapse sync fib locking into normal i/o locking. The former didn't
protect the registers so it was trivially possible for a sync command and
i/o command to fight each other and confuse the controller. Make the
sync fib alloc/release functions inline and remove the somewhat worthless
AAC_SYNC_LOCK_FORCE flag. Thanks to Adil Katchi for helping me to track
this down in RELENG_4.

Commit the correct version of the patch from last night. This fixes an
immediate panic when doing any i/o, and it closes a completion race.

Collapse aac_map_command() into aac_startio(). Check the AAC_QUEUE_FRZN in
every iteration of aac_startio(). This ensures that a command that is
deferred for lack of resources doesn't immediately get retried in the
aac_startio() loop. This avoids an almost certain livelock.

Remove the 'timeout' argument from aac_wait_command() as it isn't used and
never will be. Update the XXX comment for this function to accurately reflect
why things are the way they are.

kthread_exit() no longer requires Giant, so don't force callers to acquire
Giant just to call kthread_exit().

Requested by: many

Revert the last commit. I don't know what I was thinking, but this change
definitely doesn't help any thing.

Device megapatch 4/6:

Introduce d_version field in struct cdevsw, this must always be
initialized to D_VERSION.

Flip sense of D_NOGIANT flag to D_NEEDGIANT, this involves removing
four D_NOGIANT flags and adding 145 D_NEEDGIANT flags.

Fix a major brain-o. If the command needs to be put on the deferred queue,
take it off of the busy queue first. This should fix the 'command is on
another queue' panic that showed up recently.

Change the disk(9) API in order to make device removal more robust.

Previously the "struct disk" were owned by the device driver and this
gave us problems when the device disappared and the users of that device
were not immediately disappearing.

Now the struct disk is allocate with a new call, disk_alloc() and owned
by geom_disk and just abandonned by the device driver when disk_create()
is called.

Unfortunately, this results in a ton of "s/\./->/" changes to device
drivers.

Since I'm doing the sweep anyway, a couple of other API improvements
have been carried out at the same time:

The Giant awareness flag has been flipped from DISKFLAG_NOGIANT to
DISKFLAG_NEEDSGIANT

A version number have been added to disk_create() so that we can detect,
report and ignore binary drivers with old ABI in the future.

Manual page update to follow shortly.

Do some small cleanups to comments and remove AACQ_COMPLETE definitions since
the completion queue is long-gone.

If a command has to be deferred because there are no more resources for it
on the card, unmap it first. This allows it to be picked up properly when
the queue gets kicked again. This was the root problem for the lost command
(i.e. stuck in getblk/vinvalb) problem. While here, panic if commands don't
map correctly instead of just silently ignoring the problem and dropping
command. Also slow down the dynamic allocation of new commands.

It should be safe to go back into the aac waters. Thanks to everyone who
suffered through this and provided good feedback.

- Broaden the scope of locking in aac_command_thread() again to catch some
edge cases in the loop.

- Try to grab a command before dequeueing the bio from the bioq. The old
behaviour of requeuing deferred bios to the end of the bioq is arguably
wrong. This should be fixed in the future to check the bioq head without
automatically dequeueing the bio.

Take the plunge and make this driver be INTR_FAST. This re-arranges the
interrupt handler so that no locks are needed, and schedules the
command completion routine with a taskqueue_fast. This also corrects the
locking in the command thread and removes the need for operation flags.

Simple load tests show that this is now considerably faster than FreeBSD 4.x
in the SMP case when multiple i/o tasks are running.

- Implement selwakeuppri() which allows raising the priority of a
thread being waken up. The thread waken up can run at a priority as
high as after tsleep().

- Replace selwakeup()s with selwakeuppri()s and pass appropriate
priorities.

- Add cv_broadcastpri() which raises the priority of the broadcast
threads. Used by selwakeuppri() if collision occurs.

Not objected in: -arch, -current

Fix a couple of bugs with AIF handling:
- Correct the logic for the AIF array index pointers so that correct slot is
always looked at.
- Copy the full FIB payload size when copying AIF's, not just the first 64
bytes.

Thanks to Mirapoint, Inc, for pointing these problems out and offering a
solution.

Eliminate the use of a statically assign major number for the aac device.

Correctly wrap the producer queue index when dequeuing commands. This wasn't
a problem for command responses since we rarely ever filled the queue.
However, adapter-initiated commands have a much smaller queue and could
tickle this bug. It's possible that this might fix the recently reported
problems with the aac-2120s, though I haven't been able to reproduce the
problem locally.

MFC-After: 1 day

Commands submitted through the management interface won't have scatter/
gather lists. Stop ignoring them and instead call the callback directly.
This unbreaks the management interface.

If ~ chars were pennies, I'd be pennyless. This should fix all of the
'command not in queue' panics. Also fix a nearby style problem.

Use __FBSDID().
Also some minor style cleanups.

Make aac(4) compile cleanly on 64-bit machines. The code was already 64-bit
safe, but some (unneeded and/or harmless) downcasts were generating warnings.
The driver still is not endian-clean.

Handle the EINPROGRESS case of bus_dmamap_load() for data buffers.

Fix a missing } that got dropped from the last commit.

Add a new quirk for cards that incorrectly interpret the amount of memory
in the system. This might also have a small performance gain.

Mega busdma API commit.

Add two new arguments to bus_dma_tag_create(): lockfunc and lockfuncarg.
Lockfunc allows a driver to provide a function for managing its locking
semantics while using busdma. At the moment, this is used for the
asynchronous busdma_swi and callback mechanism. Two lockfunc implementations
are provided: busdma_lock_mutex() performs standard mutex operations on the
mutex that is specified from lockfuncarg. dftl_lock() is a panic
implementation and is defaulted to when NULL, NULL are passed to
bus_dma_tag_create(). The only time that NULL, NULL should ever be used is
when the driver ensures that bus_dmamap_load() will not be deferred.
Drivers that do not provide their own locking can pass
busdma_lock_mutex,&Giant args in order to preserve the former behaviour.

sparc64 and powerpc do not provide real busdma_swi functions, so this is
largely a noop on those platforms. The busdma_swi on is64 is not properly
locked yet, so warnings will be emitted on this platform when busdma
callback deferrals happen.

If anyone gets panics or warnings from dflt_lock() being called, please
let me know right away.

Reviewed by: tmm, gibbs

When scanning for changed containers, don't assume that the controller
will respond in a sane manner.

Thanks to Petri Helenius <pete@he.iki.fi> for spotting this and pestering
me to fix it.

Don't pass pointers to kernel data structures through 32-bit fields
as 64-bit architectures won't like this. Use virtual array indexes
instead. This *should* allow the driver to work on 64-bit platforms,
though it's still not endian clean.

Use bioq_flush() to drain a bio queue with a specific error code.
Retain the mistake of not updating the devstat API for now.

Spell bioq_disksort() consistently with the remaining bioq_*().

#include <geom/geom_disk.h> where this is more appropriate.

Add the ability to send 64-bit scatter/gather elements to aac cards. This
is enabled when both the size of bus_addr_t > 4 and the card claims support.

Don't wake up the kthread to allocate more commands if we know that we've
already allocated the max number of commands.

Begin support for 64-bit address support and workarounds for newer cards:
- Add data structuress for doing 64-bit scatter/gather
- Move busdma tag creations around so that only the parent is
created in aac_pci.c.
- Retrieve the capabilities word from the firmware before setting
up command structures and tags. This allows the driver to decide
whether to do 64-bit commands, and if work-arounds are needed for
systems with >2GB of RAM.
- Only enable the SCSI passthrough if it's enabled in the capabilities
word in the firmware.

This should fix problems with the 2120S and 2200S cards in systems with more
than 2GB of RAM. Full 64-bit support is forthcoming.

MFC-After: 1 week

Now that bus_dmamem_alloc() handles its Giant mutex requirements itself,
don't bother doing the same in the code that calls it.

Centralize the devstat handling for all GEOM disk device drivers
in geom_disk.c.

As a side effect this makes a lot of #include <sys/devicestat.h>
lines not needed and some biofinish() calls can be reduced to
biodone() again.

Gigacommit to improve device-driver source compatibility between
branches:

Initialize struct cdevsw using C99 sparse initializtion and remove
all initializations to default values.

This patch is automatically generated and has been tested by compiling
LINT with all the fields in struct cdevsw in reverse order on alpha,
sparc64 and i386.

Approved by: re(scottl)

The aac driver has evolved enough over the last few months that it no
longer resembles the 4.x version very much. Garbage collect the legacy
bits.

Bring aac out from under Giant:
- the mutex aac_io_lock protects the main codepaths which handle queues and
hardware registers. Only one acquire/release is done in the top-half and
the taskqueue. This mutex also applies to the userland command path and
CAM data path.
- Move the taskqueue to the new Giant-free version.
- Register the disk device with DISKFLAG_NOGIANT so the top-half processing
runs without Giant.
- Move the dynamic command allocator to the worker thread to avoid locking
issues with bus_dmamem_alloc().

This gives about 20% improvement in most of my benchmarks.

Introduce a new taskqueue that runs completely free of Giant, and in
turns runs its tasks free of Giant too. It is intended that as drivers
become locked down, they will move out of the old, Giant-bound taskqueue
and into this new one. The old taskqueue has been renamed to
taskqueue_swi_giant, and the new one keeps the name taskqueue_swi.

Move to 'struct disk*' API

Submitted by: phk

On detach, don't remove the child from our list of children unless it has
successfully detached.

The completion queue is no longer used, so nuke its associated code
and data structures.

Fix a 64-bit bogon. The hardware command structure only has one 32 bit
field for holding driver-dependant data. Instead of putting the pointer
to the driver command struct in there, take advantage of these structs
being a (virtually) contiguous array and just put the array index in the
field.

Make the aac driver be INTR_MPSAFE. Once the interrupt handler determines
that a command completion happened, all further processing is deferred to
a taskqueue. The taskqueue itself runs implicetely under Giant, but we
already used a taskqueue for the biodone() processing, so this at least
saves the contesting of Giant in the interrupt handler.

o Move the cleanup of the fib maps into aac_free_commands() so as to
retain symetry with aac_alloc_commans(). Since aac_alloc_commands()
allocates fib maps and places them onto the fib lists, aac_free_commands()
should reverse those operations.

o Combine two ifs with the same body with an ||.

o Switch from uintptr_t to uint32_t for fib map load operations.
The target is a uint32_t so using this type for the map load call
avoids an extra cast. uintptr_t should only be used when you need
an "int sized the same as the machine's poitner size" which is not
the case here.

o Removed the commented out M_WAITOK flag in the allocation in
aac_alloc_commands(). The kernel will only block in the allocator
if it can grow the size of the kernel. This usually results in a
page-out which could involve this aac device. Thus, sleeping here
could deadlock the machine. Assuming this operation is occurring outside
of attach time, we have enough fibs to operate anyway, so waiting for
fibs to free up is okay if not optimal.

o In aac_alloc_commands(), if we cannot dmamem_alloc additional fib
space, free the fib map.

o In aac_alloc_commands(), if we cannot create per-command dmamaps, don't
lose track of the fib map that is mapping all of the commands that we
have already released into the free pool. Instead, just cut out of
the loop and modify aac_free_commands to not attempt to free maps that
have not been allocated.

o Don't use a magic number when pre-allocating fibs.

o Use PAGE_SIZE to allocate in page sized chunks instead of an
architecture specific constant.

Submitted by: gibbs

Implement a new dynamic command allocator. FIBs are allocated in 1 page
blocks now, which should eliminate problems with the driver failing to
attach due to insufficient contiguous RAM. Allow the FIB pool to grow
from the default of 128 to the max of 512 as demand grows. Also pad the
adapter init struct to work around the 2120/2200 DMA bug now that there
is no longer a FIB slab.

Teach the CAM module how to deregister itself so it can be unloaded.

Free the container and sim objects on detach. Convert several uses of
MALLOC to malloc.

Don't include aac_cam.h, since it was nuked in the last commit.
Deregister the shutdown eventhandler on unload.

Various cleanups:
- Move the command timeout check from a separate repeating timeout to the
kthread since the kthread is already running periodically.
- Move printing the hardware print buffer to the kthread.
- Properly shut down the kernel thread on detach.
- Detach the child array devices on detach.
- Don't issue a controller halt command on detach. Doing so requires a PCI
reset to wake the controller back up. The driver can now be unloaded as
long as CAM support is not enabled.

Hack around a bug in the 2200 and 2120 controllers that connot DMA
commands from below the first 8K of physical memory. A better fix
is to modify the busdma api to allow either inclusion ranges or
multiple exclusion ranges, but that debate is for another day.

MFC After: 2 days

Major bugfixes for large memory and fast systems.

aac.c:
Re-arrange the interrupt handler to optimize the common case of
the adapter interrupting us because one or more commands are complete,
and do a read across the pci bus to ensure that all posted status
writes are flushed. This should close a race that could cause command
completion interrupts to be lost.
Follow the spec a bit closer when filling out command structures.
Enable the Fast Response feature to eliminate the need for the card
to DMA successfull command completions back into host memory.
Tell the controller how much physical memory we have. Without this
there was a chance that our DMA regions would collide with the
memory window used by the cache on the controller. The result would
be massive data corruption. This seemed to mainly affect systems with
>2GB of memory.
Fix a few whitespace problems.

aac_debug.c:
Add an extra diagnostic when printing out commands.

aac_disk.c:
Add extra sanity checks.

aacreg.h:
Prepare for making this 64-bit clean by reducing the use of enumeration
types in structures.

Many thanks to Justin Gibbs for helping track these down.

Use OPERATOR instead of WHEEL for make_dev(), use 0640 instead of 0644
for disk devices to be consistent with other storage devices.

Submitted by: kris
Reviewed by: scottl

Fix mis-indentation.

Approved by: scottl

Some kernel threads try to do significant work, and the default KSTACK_PAGES
doesn't give them enough stack to do much before blowing away the pcb.
This adds MI and MD code to allow the allocation of an alternate kstack
who's size can be speficied when calling kthread_create. Passing the
value 0 prevents the alternate kstack from being created. Note that the
ia64 MD code is missing for now, and PowerPC was only partially written
due to the pmap.c being incomplete there.
Though this patch does not modify anything to make use of the alternate
kstack, acpi and usb are good candidates.

Reviewed by: jake, peter, jhb

The AAC_COMPAT_LINUX option was really annoying, since it made the
aac driver dependent on the linux emulation module. This was
especially bad for the release engineers who tried to move the
aac driver from the kernel onto the drivers floppy. The linux
compat bits for this driver are now in their own driver, aac_linux.
It can be loaded as a module or compiled into the kernel. For
the latter case, the AAC_COMPAT_LINUX option is needed, along with
the COMPAT_LINUX option.

I've tested this in every configuration I can think of. This is an
MFC candidate for 4.7.

Idea from: rwatson
MFC after: 3 days

The aac driver needs 256K of contiguous physical memory in order to
attach. If it can't get that (highly likely if loaded as a module on
a system that's been up for a while), give a more descriptive error
message.
Also clean up some nearby style nits.

MFC after: 2 days

In continuation of early fileop credential changes, modify fo_ioctl() to
accept an 'active_cred' argument reflecting the credential of the thread
initiating the ioctl operation.

- Change fo_ioctl() to accept active_cred; change consumers of the
fo_ioctl() interface to generally pass active_cred from td->td_ucred.
- In fifofs, initialize filetmp.f_cred to ap->a_cred so that the
invocations of soo_ioctl() are provided access to the calling f_cred.
Pass ap->a_td->td_ucred as the active_cred, but note that this is
required because we don't yet distinguish file_cred and active_cred
in invoking VOP's.
- Update kqueue_ioctl() for its new argument.
- Update pipe_ioctl() for its new argument, pass active_cred rather
than td_ucred to MAC for authorization.
- Update soo_ioctl() for its new argument.
- Update vn_ioctl() for its new argument, use active_cred rather than
td->td_ucred to authorize VOP_IOCTL() and the associated VOP_GETATTR().

Obtained from: TrustedBSD Project
Sponsored by: DARPA, NAI Labs

bzero out the sync command buffer when sending commands. This was causing
problems when enumerating multiple arrays.

This is an MFC candidate.

Add a CAM interface to the aac driver. This is useful in case you should
ever connect a SCSI Cdrom/Tape/Jukebox/Scanner/Printer/kitty-litter-scooper
to your high-end RAID controller. The interface to the arrays is still
via the block interface; this merely provides a way to circumvent the
RAID functionality and access the SCSI buses directly. Note that for
somewhat obvious reasons, hard drives are not exposed to the da driver
through this interface, though you can still talk to them via the pass
driver. Be the first on your block to low-level format unsuspecting
drives that are part of an array!

To enable this, add the 'aacp' device to your kernel config.

MFC after: 3 days

Prepare for a major update to the aac driver:
Update the aac driver with the new crashdump api.
Protect sync fibs with a mutex.
Align all DMA buffers on a PAGE_SIZE boundary.

MFC after: 3 days

Don't attach to Dell PERC2/QC cards that have a firmware rev of 1.x. This
check is complicated by the fact that the Adaptec 5400S cards claim to use
1.x firmware also. PERC2/QC 1.x firmware is not compatible with this driver
and will cause a system hang.

MFC after: 3 days

Replace ffind_* with fget calls.

Make fget MPsafe.

Make fgetvp and fgetsock use the fget subsystem to reduce code bloat.

Push giant down in fpathconf().

Include sys/_lock.h and sys/_mutex.h to reduce namespace pollution.

Requested by: jhb

SMP Lock struct file, filedesc and the global file list.

Seigo Tanimura (tanimura) posted the initial delta.

I've polished it quite a bit reducing the need for locking and
adapting it for KSE.

Locks:

1 mutex in each filedesc
protects all the fields.
protects "struct file" initialization, while a struct file
is being changed from &badfileops -> &pipeops or something
the filedesc should be locked.

1 mutex in each struct file
protects the refcount fields.
doesn't protect anything else.
the flags used for garbage collection have been moved to
f_gcflag which was the FILLER short, this doesn't need
locking because the garbage collection is a single threaded
container.
could likely be made to use a pool mutex.

1 sx lock for the global filelist.

struct file * fhold(struct file *fp);
/* increments reference count on a file */

struct file * fhold_locked(struct file *fp);
/* like fhold but expects file to locked */

struct file * ffind_hold(struct thread *, int fd);
/* finds the struct file in thread, adds one reference and
returns it unlocked */

struct file * ffind_lock(struct thread *, int fd);
/* ffind_hold, but returns file locked */

I still have to smp-safe the fget cruft, I'll get to that asap.

Staticise the aac devclass.

Use d_thread_t to reduce diffs between current and stable.
Spell ACQUIRE correctly.

Add functionality and fix bugs so the driver will work with soon-to-be
released management apps.

1. Implement poll(). This will check for queued aif's so that a
subsequent ioctl call to retrieve the next aif will not block.
2. Don't catch signals when sleeping on a fib sent from userland. This
causes a race and panic due to the pthread context switcher waking
up the tsleep at inopportune times.
3. Fix some whitespace nits.

MFC after: 3 days

- Change the taskqueue locking to protect the necessary parts of a task
while it is on a queue with the queue lock and remove the per-task locks.
- Remove TASK_DESTROY now that it is no longer needed.
- Go back to inlining TASK_INIT now that it is short again.

Inspired by: dfr

Add locking to taskqueues. There is one mutex per task, one mutex per
queue, and a mutex to protect the global list of taskqueues. The only
visible change is that a TASK_DESTROY() macro has been added to mirror
the TASK_INIT() macro to destroy a task before it is free'd.

Submitted by: Andrew Reiter <awr@watson.org>

KSE Milestone 2
Note ALL MODULES MUST BE RECOMPILED
make the kernel aware that there are smaller units of scheduling than the
process. (but only allow one thread per process at this time).
This is functionally equivalent to teh previousl -current except
that there is a thread associated with each process.

Sorry john! (your next MFC will be a doosie!)

Reviewed by: peter@freebsd.org, dillon@freebsd.org

X-MFC after: ha ha ha ha

Bring the aac driver *much* closer to style(9).

Reviewed by: ken

Sigh. There are actually two ioctls ranges that need to be watched on the
linux side. This will all be over soon...

Restrict the range of linux ioctls we will accept.

Doh! Fix a comma that disappeared along the way.

Pointy-hat by: peter

Mega update to the aac driver.
1. Correctly handle commands initiated by the adapter. These commands
are defered to a kthread responsible for their processing, then are
properly returned to the controller.
2. Add/remove disk devices when notified by the card that a container was
created/deleted/offline.
3. Implement crashdump functionality.
4. Support all ioctls needed for the management CLI to work. The linux
version of this app can be found at the Dell or HP website. A native
version will be forthcoming.

MFC-after: 4.4-RELEASE

Adding missing () in a KASSERT format string.

Submitted by: brandt@fokus.gmd.de

Put the linux compat ioctl interface on death row. Add FreeBSD
definitions of all of the ioctls, plus round out all ioctl definitions to
match what exists for linux. Allow ioctls to be called through either the
native or linux interface, though no apps exist (yet) that can take
advantage of native calling.

Once again throw out the calculation of si_iosize_max and set it to 64K.
Although it can go higher, it is not safe to so do on arrays with many
members. Compromise by adding a tunable, "hw.aac.iosize_max" that can be
set at boottime. Also document in the aac(4) manpage.

MFC after: 4 weeks

Bugfixes. Close a race and logic bug in the timeout handling, don't call the
interrupt handler from the upper half, etc. This fixes some serious stability
problems that we were seeing on our production server. These patches have
been tested for almost 6 months and are a highly recommended MFC candidate.

Reviewed by: gibbs, merry, msmith
MFC after: 4 days

Fix a couple of copyright issues and an 80 column nit that I missed before

Reformat for 80 columns. Sorry, but I had to do it.

Add /dev/hpn? as an alias to /dev/aac? so that the HP version of the CLI
utility will work, and document it in the manpage.

MFC after: 3 days

Send the remains (such as I have located) of "block major numbers" to
the bit-bucket.

Create /dev/afaN as a symlink to /dev/aacN. This allows the CLI tools from
Dell and HP to run.

Reviewed by: msmith

This is an MFC candidate.

Add the AAC_DEBUG option to enable debugging in the aac driver.

Correct a race condition in the interrupt handler where the
controller may queue a fib to a response queue after the driver
has serviced the queue but before the interrupt is cleared.
This could leave a completed fib stranded in the response queue
unless another I/O completed and generated another interrupt.

Reviewed by: msmith

Kill a warning by marking an unused function with __unused.

Remove a mismatched splbio() in aac_start (the called functions each protect
themselves correctly).

Submitted by: ps

Major bugfix and minor update. This should resolve the current issues
with the driver locking up under load.

- Restructure so that we use a static pool of commands/FIBs, rather than
allocating them in clusters. The cluster allocation just made things
more complicated, and allowed us to waste more memory in peak load
situations.
- Make queueing macros more like my other drivers. This adds queue stats
for free. Add some debugging to take advantage of this.
- Reimplement the periodic timeout scan. Kick the interrupt handler
and the start routine every scan as well, just to be safe. Track busy
commands properly.
- Bring resource cleanup into line with resource allocation. We should
now clean up correctly after a failed probe/unload/etc.
- Try to start new commands when old ones are completed. We weren't doing
this before, which could lead to deadlock when the controller was full.
- Don't try to build a new command if we have found a deferred command.
This could cause us to lose the deferred command.
- Use diskerr() to report I/O errors.
- Don't bail if the AdapterInfo structure is the wrong size. Some variation
seems to be normal. We need to improve our handing of 2.x firmware sets.
- Improve some comments in an attempt to try to make things clearer.
- Restructure to avoid some warnings.

Disable calling timeout() when doing bio. It was causing more prolems than
solving. This will be fixed for real soon.

Further use of M_ZERO.

Submitted by: josh@zipperup.org
Submitted by: Robert Drehmel <robd@gmx.net>
Approved by: msmith

Convert all users of fldoff() to offsetof(). fldoff() is bad
because it only takes a struct tag which makes it impossible to
use unions, typedefs etc.

Define __offsetof() in <machine/ansi.h>

Define offsetof() in terms of __offsetof() in <stddef.h> and <sys/types.h>

Remove myriad of local offsetof() definitions.

Remove includes of <stddef.h> in kernel code.

NB: Kernelcode should *never* include from /usr/include !

Make <sys/queue.h> include <machine/ansi.h> to avoid polluting the API.

Deprecate <struct.h> with a warning. The warning turns into an error on
01-12-2000 and the file gets removed entirely on 01-01-2001.

Paritials reviews by: various.
Significant brucifications by: bde

First attempt at timeouts for bio commands sent to the controller. No retry is
attempted right now; the failure is merely passed up. This may help with the
mysterious lockups seen by some with Pablano controllers.

Remove unneeded #include <machine/clock.h>

Fix compiling with AAC_DEBUG.

Include <sys/proc.h> so that we can dereference something in the proc
structure. Looks like someone removed a nested include somewhere else.

Submitted by: gallatin, dg

A new driver for PCI:SCSI RAID controllers based on the Adaptec FSA
design. This includes integrated Dell RAID controllers, the Dell
PERC 2/QC and the HP NetRAID-4M.