Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Add basic support for DDF, often found on Adaptec HostRAID controllers.
Spares and rebuilds are not supported, so this code should be considered
for entertainment purposes only.

Add experimental support for SATA Port Multipliers

Support is working on the Silicon Image SiI3124/3132.
Support is working on some AHCI chips but far from all.

Remember this is WIP, so test reports and (constructive) suggestions are welcome!

Update copyright headers.

Format mask lacks one bit.

Reported by: jkim

Add r/w support for JMicron ATA RAID metadata.
Acknowledgement should definitly go to JMicron Technology for providing full
docs on the metadata format as the only vendor so far, big thanks from here.

Whitespace cleanup.

Get rid of the advertising clause in the copyright.

Add support for writing VIA metadata.

Null out the metadata on disks when array is deleted.

Add RAID0+1 and RAID5 support to VIA RAID code.
Fix support for multiple arrays.

Correct calculation of RAID0 sizes on VIA RAID arrays.

Add support for writing Intel MatrixRAID arrays.
Do a little better on handling volumes as well, however we cant create
multiple volumes from FreeBSD yet.

HW sponsored by: Mullet Scandinavia AB

Update the ICH7 support so it deals better with chips without AHCI.

Update Intel MatrixRAID support to be able to pick up RAID0+1 (RAID10)
and RAID5 arrays without panic'ing.
This has the side effect of now also supporting multiple volumes on
MatrixRAID's now I have the metadata better understood..

HW sponsored by: Mullet Scandinavia AB

Fix SiS SATA support, the SATA registers was off.
Add support for SiS metadata.

HW donated by: obrien

Fix problem with finding the still working disk in a broken mirror on VIA.

Add support for nVidia's software RAID "MediaShield".

HW Sponsored by: Yahoo!

Go back to the old way of finding the Promise metadata, the new way was
too simple causing older controllers metadata to get lost.

Take newbusification one step further, ie use the device_t more consequently
all way through the code down the layers, instead of the mix'n'match that
resulted from the conversion done earlier.

Sponsored by: pair.com

Adjust the RAID type pickup code for the VIA, we dont actually care
if the array is bootable or not (yet).

Better use the right name for the VIA software RAID.

Add support for VIA Tech metadata as used on thier SATA parts.

This is the much rumoured ATA mkIII update that I've been working on.

o ATA is now fully newbus'd and split into modules.
This means that on a modern system you just load "atapci and ata"
to get the base support, and then one or more of the device
subdrivers "atadisk atapicd atapifd atapist ataraid".
All can be loaded/unloaded anytime, but for obvious reasons you
dont want to unload atadisk when you have mounted filesystems.

o The device identify part of the probe has been rewritten to fix
the problems with odd devices the old had, and to try to remove
so of the long delays some HW could provoke. Also probing is done
without the need for interrupts, making earlier probing possible.

o SATA devices can be hot inserted/removed and devices will be created/
removed in /dev accordingly.
NOTE: only supported on controllers that has this feature:
Promise and Silicon Image for now.
On other controllers the usual atacontrol detach/attach dance is
still needed.

o Support for "atomic" composite ATA requests used for RAID.

o ATA RAID support has been rewritten and and now supports these
metadata formats:
"Adaptec HostRAID"
"Highpoint V2 RocketRAID"
"Highpoint V3 RocketRAID"
"Intel MatrixRAID"
"Integrated Technology Express"
"LSILogic V2 MegaRAID"
"LSILogic V3 MegaRAID"
"Promise FastTrak"
"Silicon Image Medley"
"FreeBSD PseudoRAID"

o Update the ioctl API to match new RAID levels etc.

o Update atacontrol to know about the new RAID levels etc
NOTE: you need to recompile atacontrol with the new sys/ata.h,
make world will take care of that.
NOTE2: that rebuild is done differently from the old system as
the rebuild is now done piggybacked on read requests to the
array, so atacontrol simply starts a background "dd" to rebuild
the array.

o The reinit code has been worked over to be much more robust.

o The timeout code has been overhauled for races.

o Support of new chipsets.

o Lots of fixes for bugs found while doing the modulerization and
reviewing the old code.

Missing or changed features from current ATA:

o atapi-cd no longer has support for ATAPI changers. Todays its
much cheaper and alot faster to copy those CD images to disk
and serve them from there. Besides they dont seem to be made
anymore, maybe for that exact reason.

o ATA RAID can only read metadata from all the above metadata formats,
not write all of them (Promise and Highpoint V2 so far). This means
that arrays can be picked up from the BIOS, but they cannot be
created from FreeBSD. There is more to it than just the missing
write metadata support, those formats are not unique to a given
controller like Promise and Highpoint formats, instead they exist
for several types, and even worse, some controllers can have
different formats and its impossible to tell which one.
The outcome is that we cannot reliably create the metadata of those
formats and be sure the controller BIOS will understand it.
However write support is needed to update/fail/rebuild the arrays
properly so it sits fairly high on the TODO list.

o So far atapicam is not supported with these changes. When/if this
will change is up to the maintainer of atapi-cam so go there for
questions.

HW donated by: Webveveriet AS
HW donated by: Frode Nordahl
HW donated by: Yahoo!
HW donated by: Sentex
Patience by: Vife and my boys (and even the cats)

Add support for LSI type software RAID's.

Made possible by: John Cagle @ HP

Change the disk(9) API in order to make device removal more robust.

Previously the "struct disk" were owned by the device driver and this
gave us problems when the device disappared and the users of that device
were not immediately disappearing.

Now the struct disk is allocate with a new call, disk_alloc() and owned
by geom_disk and just abandonned by the device driver when disk_create()
is called.

Unfortunately, this results in a ton of "s/\./->/" changes to device
drivers.

Since I'm doing the sweep anyway, a couple of other API improvements
have been carried out at the same time:

The Giant awareness flag has been flipped from DISKFLAG_NOGIANT to
DISKFLAG_NEEDSGIANT

A version number have been added to disk_create() so that we can detect,
report and ignore binary drivers with old ABI in the future.

Manual page update to follow shortly.

This is a major rework of the ATA driver (ATAng)

Restructure the way ATA/ATAPI commands are processed, use a common
ata_request structure for both. This centralises the way requests
are handled so locking is much easier to handle.

The driver is now layered much more cleanly to seperate the lowlevel
HW access so it can be tailored to specific controllers without touching
the upper layers. This is needed to support some of the newer
semi-intelligent ATA controllers showing up.

The top level drivers (disk, ATAPI devices) are more or less still
the same with just corrections to use the new interface.

Pull ATA out from under Gaint now that locking can be done in a sane way.

Add support for a the National Geode SC1100. Thanks to Soekris engineering
for sponsoring a Soekris 4801 to make this support.

Fixed alot of small bugs in the chipset code for various chips now
we are around in that corner anyways.

Change the way loadbalancing works on RAID1's.
Based on code partially by me and by <Tor.Egge@cvsup.no.freebsd.org>.

Add ioctl to add a spare disk to a RAID array.
Fix the discovery of RAID's to not grap unused disks.
Change the probe printing of a RAID a bit.

Fix a long standing bug in handling the last part of a stripe
on "odd" size disks.

Add printout of the RAID structure on verbose boot.

Convert to new disk API.

Prodded by: phk

First round off updates/fixes to the ATA driver.

This moves all chipset specific code to a new file 'ata-chipset.c'.
Extensive use of tables and pointers to avoid having the same switch
on chipset type in several places, and to allow substituting various
functions for different HW arch needs.
Added PIO mode setup and all DMA modes.
Support for all known SiS chipsets. Thanks to Christoph Kukulies for
sponsoring a nice ASUS P4S8X SiS648 based board for this work!

Tested on: i386, PC98, alpha and sparc64

use __packed.

Only calculate Promise magic if a device is there for info.

Add get-status to the ATA RAID subsystem.

Add support for creating/deleting ATA RAID's.
This completes the ATA RAID support, since all functions to manipulate
the RAID are accessible from FreeBSD, the BIOS on the ATA RAID cards
are only nessesary for booting.

I decided to allow for creation of ATA RAID's on any ATA controller, but
please keep in mind the restrictions on that. Due to the BIOS not
knowing what to do you can only boot from a RAID1 or the first disk
in a SPAN, if its not located on a "real" ATA RAID controller like
the Promise or Highpoint controllers.

Sponsored by: Advanis

Fix 64bit arch problems.

Update to the RAID1 rebuild code.

Run rebuild as a background process.

Sponsored by: Advanis

Even more Highpoint RAID support.

Fix the 80pin cable detection system.

Support newer Highpoint BIOS's extended config.

Fix a couble of bugs in the rebuild code, return errors properly.

Major update of the ATA RAID code, part 3:

Add code to properly detach/attach disks that are part of a RAID.

Mark a disk that is attached on an ATA channel belonging to a
RAID as a spare disk that can be used for rebuilding failed RAID1's.

Add support for rebuilding failed RAID1's.

Several fixes to the detach/attach code.

For replacing a disk in a failed RAID1 do the following:

Find the controller channel# of the failed disk.

Exec 'atacontrol detach <channel#>' to free the disk from the system.

Replace the failed disk with a new one of at least the same size.
If your have your disks in drawers/enclosures this can be done with
the system still running.

Exec 'atacontrol attach <channel#>' to add the disk to the system and
mark it as a valid spare for rebuild.

Exec 'atacontrol rebuild <array#>'

The system will rebuild the array on the fly, the array can still
be used during this, although with slower performance.

Please let me know of any problems with this!

Sponsored by: Advanis Inc.

MFC after: 2 weeks

Major update of the ATA RAID code, part 2:

More cleanups of the RAID1 failure mode code.

Add functionality that writes the changed RAID config setup
back to the disks (in controller BIOS specific format), so
that a reboot will make the BIOS pick up the changed config.

Make LINT compile after fruitless attempts to get the authors
to fix their code.

ata stuff:
Change name of ar_attach to not colide with existing ar_attach in if_ar.c.
usb stuff:
Create a dummy function to satisfy a call to it when in DEBUG mode.

Major update of the ATA RAID code, part 1:

Overhaul of the attach/detach code and structures, there were some nasty
bugs in the old implementation. This made it possible to collapse the
ATA/ATAPI device control structures into one generic structure.

A note here, the kernel is NOT ready for detach of active devices,
it fails all over in random places, but for inactive devices it works.
However for ATA RAID this works, since the RAID abstration layer
insulates the buggy^H^H^H^H^H^Hfragile device subsystem from the
physical disks.

Proberly detect the RAID's from the BIOS, and mark critical RAID1
arrays as such, but continue if there is enough of the mirror left
to do so.

Properly fail arrays on a live system. For RAID0 that means return EIO,
and for RAID1 it means continue on the still working part of the mirror
if possible, else return EIO.
If the state changes, log this to the console.

Allow for Promise & Highpoint controllers/arrays to coexist on the
same machine. It is not possible to distribute arrays over different
makes of controllers though.

If Promise SuperSwap enclosures are used, signal disk state on the
status LED on the front.

Misc fixes that I had lying around for various minor bugs.

Sponsored by: Advanis Inc.

Update the promise raid structure with some of the info I've gathered
before I'm accused of "lending" it from somebody else.

Overhaul to minimize stack usage, in some places >2K was used
on the stack *blush*...

Add session argument to *close_disk, allowing to set session type on fixate.
Add support for different blank/erase types.

Update headers.

Update the config gathering code for both Promise & HPT

Clean up the raid code a bit, also allow disks on HPT controllers to
be swapped around and still be put in the correct order in a raid.

Add support for ATA "pseudo" RAID controllers as the Promise Fasttrak
and HighPoint HPT370 controllers.

Use by defining the RAID in the BIOS and the "ar driver will pick it up
automagically...