Move ccb access before ccb_done to avoid a potential use-after-free
if pages can be freed from interrupt context. ok dlg@

use 1U << 31 to avoid undefined behaviour
ok miod@

Remove redundant DIAGNOSTIC wrappers around KASSERT macros.
From Crystal Kolipe.

atascsi: constify method tables

ok miod@

Access adapter softc via link->bus->sb_adapter_softc.

In sparc64 autoconf access 'luns' via sb_luns and 'adapter_buswidth'
via sb_adapter_buswidth.

Removes last post-config uses of the copies of bus related
information in scsi_link.

ahci_get_pmp_ccb() returns a non-NULL pointer, fails a
KASSERT(), or has already blown up by dereferencing the
pointer.

Two of the four invocations of ahci_get_pmp_ccb() check for
NULL. Save a couple of bytes by not bothering. Add comments
to the invocations to docuement the assumption.

Trailing whitespace, long lines, incorrect function names in
printf()'s.

Remove some "set but not used" variables in ahci(4)'s hibernate code.

ok jmatthew

Turn off the code which waits for AHCI_PREG_CMD_CR to be set by the
HBA after ahci_default_port_start() sets AHCI_PREG_CMD_ST. The AHCI
spec. rev. 1.3 only requires the inverse to be true, i. e. that a
HBA clears AHCI_PREG_CMD_CR when AHCI_PREG_CMD_ST gets cleared by
software/driver. In fact, some HBAs will not raise AHCI_PREG_CMD_CR
as an immediate consequence of AHCI_PREG_CMD_ST being set. Actually
neither the FreeBSD, Linux nor NetBSD counterpart of ahci(4) has an
analogous check. Disabling that wait fixes "failed to start command
DMA on port N, disabling" bails during attach.

From Marius Strobl
ok deraadt@ jmatthew@

Split up ahci_port_portreset into a few smaller bits, and also slightly
adjust port multiplier detection so it doesn't call ahci_port_portreset
again directly, but instead restarts the loop for the current call.

During attach, poll for device detection across all ports until either all
ports have detected a device, or one second has passed, rather than doing
them sequentially. Devices are still attached in order of port number,
so disk unit numbers won't change.

ok visa@

improve AHCI hibernate writeout performance by doing smaller delay()s
between each I/O.

ok kettenis@, jmatthew@

Remove comments about implementing FIS-based switching with port multipliers,
which we can safely assume isn't going to happen now. No code changes.

When we're unable to determine which NCQ command failed, fail all active
commands rather than proceeding to an assertion failure. This matches
how we handle the same condition with non-queued commands, and the
behaviour of every other ahci driver I can find. Add a printf there so
we might get some idea of how often this condition arises and what happens
afterwards.

ok dlg@

Some HABs reset parts of AHCI_PREG_CMD when AHCI_PREG_SCTL_DET_INIT
gets set. Therefore, ahci_port_softreset() restores the sate of the
former register once the device detection sequence is finished. The
device detection code in ahci_pmp_port_portreset() does not restore
AHCI_PREG_CMD afterwards, so let it catch up. Apparently, this part
was an oversight as ahci_pmp_port_portreset() did not otherwise use
"cmd".

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Some HBAs report NCQ capability despite only supporting one command
slot. Thus, extend the check whether NCQ actually should be enabled
accordingly.

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Given that ahci_port_alloc() grabs one CCB for use during NCQ error
recovery from the CCB pool sized based on the NCS capability, i. e.
number of command slots reported by the controller, it is necessary
to pretend at least 2 slots in sc->sc_ncmds for devices without NCQ
support. That way, also at least 1 available slot is made available
for atascsi(4). Otherwise, controllers having only a single command
slot will trigger "no free xfers on a new port" in atascsi(4).
Note that pretending 2 command slots is also fine with the abuse of
the NCQ error recovery CCB in ahci_port_softreset().

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Contrary to e. g. the xHCI specification, the AHCI rev. 1.3 spec
does not say anything about which type (snoopable/uncached etc.) of
bus transactions are issued by HBAs on accesses to the descriptors.
Thus, the right assumption would be "classical" concurrent accesses
by both CPU/driver as well as hardware to the descriptors and their
DMA memory backings respectively, which is also confirmed by actual
testing. Consequently, switch to BUS_DMA_COHERENT mappings for said
DMA memory as otherwise corruption of descriptors is seen.

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Enforce some naming sanity. Stop using 'sc_link' to mean two different
things by renaming the field 'SLIST_HEAD(, scsi_link) sc_link' to
'sc_link_list' in struct scsibus_softc. Use 'sb' as the short name
for scsibus_softc variables.

Impetus from & ok bluhm@

Fix memory leak in error path.
From Benjamin Baier, found by llvm/scan-build.

sizes for free()

Print speeds consistently.

ok brynet@, miod@

Print the current link speed for SATA devices attached on ahci(4).

The port numbers correspond to scsibus(4) targets:

ahci0: port 0, at 6Gbps
ahci0: port 2, at 3Gbps
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 ..
sd1 at scsibus1 targ 2 ..

ok deraadt@, tested by bmercer@

include extended capabilities in ahci debug output
ok dlg@

recognise ahci 1.3.1
ok dlg@

Turn ahci port start into a callback so sxiahci can do some magic to
setup dma. Modified version of a patch from Edwin Amsler which was
based on changes made in Bitrig by Artturi Alm.

ok dlg@

Some reallocarray() use; review Jean-Philippe Ouellet, patrick keshishian
ok tedu

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Work around compatibility problems with Intel ahci and Intel SSDs by retrying
device detection if the port is still busy after our normal timeout.

based on a diff by chris@, based on a change in dragonflybsd
tested by Peter J Philipp
ok dlg@

move the hardware structures into ahcireg.h.

move the last htoleXX things to htolemXX where possible.

ok jmatthew@

ahci dva things are 64bit and surprisingly aligned nicely to 64bit too.

ok jmatthew@

replace bzero with memset

massage the prdt (what an sgl is on every other chip) loading with
better ops and alignment.

ok jmatthew@ who was cruel and made me test hibernate before he oked it.

Reduce uvm include madness. Use <uvm/uvm_extern.h> instead of
<uvm/uvm.h> if possible and remove double inclusions.

ok beck@, mlarkin@, deraadt@

fix previous errorneous constants (compiled against the old header)

Recognize SATA-III ports in dmesg.

ok millert

Add a DVACT_WAKEUP op to the *_activate() API. This is called after the
kernel resumes normal (non-cold, able to run processes, etc) operation.
Previously we were relying on specific DVACT_RESUME op's in drivers
creating callback/threads themselves, but that has become too common,
indicating the need for a built-in mechanism.
ok dlg kettenis, tested by a sufficient amount of people

teach the side-effect free drivers about the partition they are dealing
by passing a start/length in the HIB_INIT op. Then rebase all
hibernate-time block offsets to be relative to the start of that partition.
This simplifies things a lot.
ok mlarkin

pay attention to the #ifdef

move AHCI_DEBUG stuff into ahci.c so we can build with it enabled again

ok dlg@

Split AHCI in order to support devices which are not attached via pci.
Tested on a few machines, including JMicron and Intel AHCIs.

ok dlg@ jmatthew@

use 1U << 31 to avoid undefined behaviour
ok miod@

Remove redundant DIAGNOSTIC wrappers around KASSERT macros.
From Crystal Kolipe.

atascsi: constify method tables

ok miod@

Access adapter softc via link->bus->sb_adapter_softc.

In sparc64 autoconf access 'luns' via sb_luns and 'adapter_buswidth'
via sb_adapter_buswidth.

Removes last post-config uses of the copies of bus related
information in scsi_link.

ahci_get_pmp_ccb() returns a non-NULL pointer, fails a
KASSERT(), or has already blown up by dereferencing the
pointer.

Two of the four invocations of ahci_get_pmp_ccb() check for
NULL. Save a couple of bytes by not bothering. Add comments
to the invocations to docuement the assumption.

Trailing whitespace, long lines, incorrect function names in
printf()'s.

Remove some "set but not used" variables in ahci(4)'s hibernate code.

ok jmatthew

Turn off the code which waits for AHCI_PREG_CMD_CR to be set by the
HBA after ahci_default_port_start() sets AHCI_PREG_CMD_ST. The AHCI
spec. rev. 1.3 only requires the inverse to be true, i. e. that a
HBA clears AHCI_PREG_CMD_CR when AHCI_PREG_CMD_ST gets cleared by
software/driver. In fact, some HBAs will not raise AHCI_PREG_CMD_CR
as an immediate consequence of AHCI_PREG_CMD_ST being set. Actually
neither the FreeBSD, Linux nor NetBSD counterpart of ahci(4) has an
analogous check. Disabling that wait fixes "failed to start command
DMA on port N, disabling" bails during attach.

From Marius Strobl
ok deraadt@ jmatthew@

Split up ahci_port_portreset into a few smaller bits, and also slightly
adjust port multiplier detection so it doesn't call ahci_port_portreset
again directly, but instead restarts the loop for the current call.

During attach, poll for device detection across all ports until either all
ports have detected a device, or one second has passed, rather than doing
them sequentially. Devices are still attached in order of port number,
so disk unit numbers won't change.

ok visa@

improve AHCI hibernate writeout performance by doing smaller delay()s
between each I/O.

ok kettenis@, jmatthew@

Remove comments about implementing FIS-based switching with port multipliers,
which we can safely assume isn't going to happen now. No code changes.

When we're unable to determine which NCQ command failed, fail all active
commands rather than proceeding to an assertion failure. This matches
how we handle the same condition with non-queued commands, and the
behaviour of every other ahci driver I can find. Add a printf there so
we might get some idea of how often this condition arises and what happens
afterwards.

ok dlg@

Some HABs reset parts of AHCI_PREG_CMD when AHCI_PREG_SCTL_DET_INIT
gets set. Therefore, ahci_port_softreset() restores the sate of the
former register once the device detection sequence is finished. The
device detection code in ahci_pmp_port_portreset() does not restore
AHCI_PREG_CMD afterwards, so let it catch up. Apparently, this part
was an oversight as ahci_pmp_port_portreset() did not otherwise use
"cmd".

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Some HBAs report NCQ capability despite only supporting one command
slot. Thus, extend the check whether NCQ actually should be enabled
accordingly.

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Given that ahci_port_alloc() grabs one CCB for use during NCQ error
recovery from the CCB pool sized based on the NCS capability, i. e.
number of command slots reported by the controller, it is necessary
to pretend at least 2 slots in sc->sc_ncmds for devices without NCQ
support. That way, also at least 1 available slot is made available
for atascsi(4). Otherwise, controllers having only a single command
slot will trigger "no free xfers on a new port" in atascsi(4).
Note that pretending 2 command slots is also fine with the abuse of
the NCQ error recovery CCB in ahci_port_softreset().

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Contrary to e. g. the xHCI specification, the AHCI rev. 1.3 spec
does not say anything about which type (snoopable/uncached etc.) of
bus transactions are issued by HBAs on accesses to the descriptors.
Thus, the right assumption would be "classical" concurrent accesses
by both CPU/driver as well as hardware to the descriptors and their
DMA memory backings respectively, which is also confirmed by actual
testing. Consequently, switch to BUS_DMA_COHERENT mappings for said
DMA memory as otherwise corruption of descriptors is seen.

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Enforce some naming sanity. Stop using 'sc_link' to mean two different
things by renaming the field 'SLIST_HEAD(, scsi_link) sc_link' to
'sc_link_list' in struct scsibus_softc. Use 'sb' as the short name
for scsibus_softc variables.

Impetus from & ok bluhm@

Fix memory leak in error path.
From Benjamin Baier, found by llvm/scan-build.

sizes for free()

Print speeds consistently.

ok brynet@, miod@

Print the current link speed for SATA devices attached on ahci(4).

The port numbers correspond to scsibus(4) targets:

ahci0: port 0, at 6Gbps
ahci0: port 2, at 3Gbps
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 ..
sd1 at scsibus1 targ 2 ..

ok deraadt@, tested by bmercer@

include extended capabilities in ahci debug output
ok dlg@

recognise ahci 1.3.1
ok dlg@

Turn ahci port start into a callback so sxiahci can do some magic to
setup dma. Modified version of a patch from Edwin Amsler which was
based on changes made in Bitrig by Artturi Alm.

ok dlg@

Some reallocarray() use; review Jean-Philippe Ouellet, patrick keshishian
ok tedu

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Work around compatibility problems with Intel ahci and Intel SSDs by retrying
device detection if the port is still busy after our normal timeout.

based on a diff by chris@, based on a change in dragonflybsd
tested by Peter J Philipp
ok dlg@

move the hardware structures into ahcireg.h.

move the last htoleXX things to htolemXX where possible.

ok jmatthew@

ahci dva things are 64bit and surprisingly aligned nicely to 64bit too.

ok jmatthew@

replace bzero with memset

massage the prdt (what an sgl is on every other chip) loading with
better ops and alignment.

ok jmatthew@ who was cruel and made me test hibernate before he oked it.

Reduce uvm include madness. Use <uvm/uvm_extern.h> instead of
<uvm/uvm.h> if possible and remove double inclusions.

ok beck@, mlarkin@, deraadt@

fix previous errorneous constants (compiled against the old header)

Recognize SATA-III ports in dmesg.

ok millert

Add a DVACT_WAKEUP op to the *_activate() API. This is called after the
kernel resumes normal (non-cold, able to run processes, etc) operation.
Previously we were relying on specific DVACT_RESUME op's in drivers
creating callback/threads themselves, but that has become too common,
indicating the need for a built-in mechanism.
ok dlg kettenis, tested by a sufficient amount of people

teach the side-effect free drivers about the partition they are dealing
by passing a start/length in the HIB_INIT op. Then rebase all
hibernate-time block offsets to be relative to the start of that partition.
This simplifies things a lot.
ok mlarkin

pay attention to the #ifdef

move AHCI_DEBUG stuff into ahci.c so we can build with it enabled again

ok dlg@

Split AHCI in order to support devices which are not attached via pci.
Tested on a few machines, including JMicron and Intel AHCIs.

ok dlg@ jmatthew@

Remove redundant DIAGNOSTIC wrappers around KASSERT macros.
From Crystal Kolipe.

atascsi: constify method tables

ok miod@

Access adapter softc via link->bus->sb_adapter_softc.

In sparc64 autoconf access 'luns' via sb_luns and 'adapter_buswidth'
via sb_adapter_buswidth.

Removes last post-config uses of the copies of bus related
information in scsi_link.

ahci_get_pmp_ccb() returns a non-NULL pointer, fails a
KASSERT(), or has already blown up by dereferencing the
pointer.

Two of the four invocations of ahci_get_pmp_ccb() check for
NULL. Save a couple of bytes by not bothering. Add comments
to the invocations to docuement the assumption.

Trailing whitespace, long lines, incorrect function names in
printf()'s.

Remove some "set but not used" variables in ahci(4)'s hibernate code.

ok jmatthew

Turn off the code which waits for AHCI_PREG_CMD_CR to be set by the
HBA after ahci_default_port_start() sets AHCI_PREG_CMD_ST. The AHCI
spec. rev. 1.3 only requires the inverse to be true, i. e. that a
HBA clears AHCI_PREG_CMD_CR when AHCI_PREG_CMD_ST gets cleared by
software/driver. In fact, some HBAs will not raise AHCI_PREG_CMD_CR
as an immediate consequence of AHCI_PREG_CMD_ST being set. Actually
neither the FreeBSD, Linux nor NetBSD counterpart of ahci(4) has an
analogous check. Disabling that wait fixes "failed to start command
DMA on port N, disabling" bails during attach.

From Marius Strobl
ok deraadt@ jmatthew@

Split up ahci_port_portreset into a few smaller bits, and also slightly
adjust port multiplier detection so it doesn't call ahci_port_portreset
again directly, but instead restarts the loop for the current call.

During attach, poll for device detection across all ports until either all
ports have detected a device, or one second has passed, rather than doing
them sequentially. Devices are still attached in order of port number,
so disk unit numbers won't change.

ok visa@

improve AHCI hibernate writeout performance by doing smaller delay()s
between each I/O.

ok kettenis@, jmatthew@

Remove comments about implementing FIS-based switching with port multipliers,
which we can safely assume isn't going to happen now. No code changes.

When we're unable to determine which NCQ command failed, fail all active
commands rather than proceeding to an assertion failure. This matches
how we handle the same condition with non-queued commands, and the
behaviour of every other ahci driver I can find. Add a printf there so
we might get some idea of how often this condition arises and what happens
afterwards.

ok dlg@

Some HABs reset parts of AHCI_PREG_CMD when AHCI_PREG_SCTL_DET_INIT
gets set. Therefore, ahci_port_softreset() restores the sate of the
former register once the device detection sequence is finished. The
device detection code in ahci_pmp_port_portreset() does not restore
AHCI_PREG_CMD afterwards, so let it catch up. Apparently, this part
was an oversight as ahci_pmp_port_portreset() did not otherwise use
"cmd".

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Some HBAs report NCQ capability despite only supporting one command
slot. Thus, extend the check whether NCQ actually should be enabled
accordingly.

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Given that ahci_port_alloc() grabs one CCB for use during NCQ error
recovery from the CCB pool sized based on the NCS capability, i. e.
number of command slots reported by the controller, it is necessary
to pretend at least 2 slots in sc->sc_ncmds for devices without NCQ
support. That way, also at least 1 available slot is made available
for atascsi(4). Otherwise, controllers having only a single command
slot will trigger "no free xfers on a new port" in atascsi(4).
Note that pretending 2 command slots is also fine with the abuse of
the NCQ error recovery CCB in ahci_port_softreset().

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Contrary to e. g. the xHCI specification, the AHCI rev. 1.3 spec
does not say anything about which type (snoopable/uncached etc.) of
bus transactions are issued by HBAs on accesses to the descriptors.
Thus, the right assumption would be "classical" concurrent accesses
by both CPU/driver as well as hardware to the descriptors and their
DMA memory backings respectively, which is also confirmed by actual
testing. Consequently, switch to BUS_DMA_COHERENT mappings for said
DMA memory as otherwise corruption of descriptors is seen.

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Enforce some naming sanity. Stop using 'sc_link' to mean two different
things by renaming the field 'SLIST_HEAD(, scsi_link) sc_link' to
'sc_link_list' in struct scsibus_softc. Use 'sb' as the short name
for scsibus_softc variables.

Impetus from & ok bluhm@

Fix memory leak in error path.
From Benjamin Baier, found by llvm/scan-build.

sizes for free()

Print speeds consistently.

ok brynet@, miod@

Print the current link speed for SATA devices attached on ahci(4).

The port numbers correspond to scsibus(4) targets:

ahci0: port 0, at 6Gbps
ahci0: port 2, at 3Gbps
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 ..
sd1 at scsibus1 targ 2 ..

ok deraadt@, tested by bmercer@

include extended capabilities in ahci debug output
ok dlg@

recognise ahci 1.3.1
ok dlg@

Turn ahci port start into a callback so sxiahci can do some magic to
setup dma. Modified version of a patch from Edwin Amsler which was
based on changes made in Bitrig by Artturi Alm.

ok dlg@

Some reallocarray() use; review Jean-Philippe Ouellet, patrick keshishian
ok tedu

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Work around compatibility problems with Intel ahci and Intel SSDs by retrying
device detection if the port is still busy after our normal timeout.

based on a diff by chris@, based on a change in dragonflybsd
tested by Peter J Philipp
ok dlg@

move the hardware structures into ahcireg.h.

move the last htoleXX things to htolemXX where possible.

ok jmatthew@

ahci dva things are 64bit and surprisingly aligned nicely to 64bit too.

ok jmatthew@

replace bzero with memset

massage the prdt (what an sgl is on every other chip) loading with
better ops and alignment.

ok jmatthew@ who was cruel and made me test hibernate before he oked it.

Reduce uvm include madness. Use <uvm/uvm_extern.h> instead of
<uvm/uvm.h> if possible and remove double inclusions.

ok beck@, mlarkin@, deraadt@

fix previous errorneous constants (compiled against the old header)

Recognize SATA-III ports in dmesg.

ok millert

Add a DVACT_WAKEUP op to the *_activate() API. This is called after the
kernel resumes normal (non-cold, able to run processes, etc) operation.
Previously we were relying on specific DVACT_RESUME op's in drivers
creating callback/threads themselves, but that has become too common,
indicating the need for a built-in mechanism.
ok dlg kettenis, tested by a sufficient amount of people

teach the side-effect free drivers about the partition they are dealing
by passing a start/length in the HIB_INIT op. Then rebase all
hibernate-time block offsets to be relative to the start of that partition.
This simplifies things a lot.
ok mlarkin

pay attention to the #ifdef

move AHCI_DEBUG stuff into ahci.c so we can build with it enabled again

ok dlg@

Split AHCI in order to support devices which are not attached via pci.
Tested on a few machines, including JMicron and Intel AHCIs.

ok dlg@ jmatthew@

atascsi: constify method tables

ok miod@

Access adapter softc via link->bus->sb_adapter_softc.

In sparc64 autoconf access 'luns' via sb_luns and 'adapter_buswidth'
via sb_adapter_buswidth.

Removes last post-config uses of the copies of bus related
information in scsi_link.

ahci_get_pmp_ccb() returns a non-NULL pointer, fails a
KASSERT(), or has already blown up by dereferencing the
pointer.

Two of the four invocations of ahci_get_pmp_ccb() check for
NULL. Save a couple of bytes by not bothering. Add comments
to the invocations to docuement the assumption.

Trailing whitespace, long lines, incorrect function names in
printf()'s.

Remove some "set but not used" variables in ahci(4)'s hibernate code.

ok jmatthew

Turn off the code which waits for AHCI_PREG_CMD_CR to be set by the
HBA after ahci_default_port_start() sets AHCI_PREG_CMD_ST. The AHCI
spec. rev. 1.3 only requires the inverse to be true, i. e. that a
HBA clears AHCI_PREG_CMD_CR when AHCI_PREG_CMD_ST gets cleared by
software/driver. In fact, some HBAs will not raise AHCI_PREG_CMD_CR
as an immediate consequence of AHCI_PREG_CMD_ST being set. Actually
neither the FreeBSD, Linux nor NetBSD counterpart of ahci(4) has an
analogous check. Disabling that wait fixes "failed to start command
DMA on port N, disabling" bails during attach.

From Marius Strobl
ok deraadt@ jmatthew@

Split up ahci_port_portreset into a few smaller bits, and also slightly
adjust port multiplier detection so it doesn't call ahci_port_portreset
again directly, but instead restarts the loop for the current call.

During attach, poll for device detection across all ports until either all
ports have detected a device, or one second has passed, rather than doing
them sequentially. Devices are still attached in order of port number,
so disk unit numbers won't change.

ok visa@

improve AHCI hibernate writeout performance by doing smaller delay()s
between each I/O.

ok kettenis@, jmatthew@

Remove comments about implementing FIS-based switching with port multipliers,
which we can safely assume isn't going to happen now. No code changes.

When we're unable to determine which NCQ command failed, fail all active
commands rather than proceeding to an assertion failure. This matches
how we handle the same condition with non-queued commands, and the
behaviour of every other ahci driver I can find. Add a printf there so
we might get some idea of how often this condition arises and what happens
afterwards.

ok dlg@

Some HABs reset parts of AHCI_PREG_CMD when AHCI_PREG_SCTL_DET_INIT
gets set. Therefore, ahci_port_softreset() restores the sate of the
former register once the device detection sequence is finished. The
device detection code in ahci_pmp_port_portreset() does not restore
AHCI_PREG_CMD afterwards, so let it catch up. Apparently, this part
was an oversight as ahci_pmp_port_portreset() did not otherwise use
"cmd".

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Some HBAs report NCQ capability despite only supporting one command
slot. Thus, extend the check whether NCQ actually should be enabled
accordingly.

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Given that ahci_port_alloc() grabs one CCB for use during NCQ error
recovery from the CCB pool sized based on the NCS capability, i. e.
number of command slots reported by the controller, it is necessary
to pretend at least 2 slots in sc->sc_ncmds for devices without NCQ
support. That way, also at least 1 available slot is made available
for atascsi(4). Otherwise, controllers having only a single command
slot will trigger "no free xfers on a new port" in atascsi(4).
Note that pretending 2 command slots is also fine with the abuse of
the NCQ error recovery CCB in ahci_port_softreset().

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Contrary to e. g. the xHCI specification, the AHCI rev. 1.3 spec
does not say anything about which type (snoopable/uncached etc.) of
bus transactions are issued by HBAs on accesses to the descriptors.
Thus, the right assumption would be "classical" concurrent accesses
by both CPU/driver as well as hardware to the descriptors and their
DMA memory backings respectively, which is also confirmed by actual
testing. Consequently, switch to BUS_DMA_COHERENT mappings for said
DMA memory as otherwise corruption of descriptors is seen.

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Enforce some naming sanity. Stop using 'sc_link' to mean two different
things by renaming the field 'SLIST_HEAD(, scsi_link) sc_link' to
'sc_link_list' in struct scsibus_softc. Use 'sb' as the short name
for scsibus_softc variables.

Impetus from & ok bluhm@

Fix memory leak in error path.
From Benjamin Baier, found by llvm/scan-build.

sizes for free()

Print speeds consistently.

ok brynet@, miod@

Print the current link speed for SATA devices attached on ahci(4).

The port numbers correspond to scsibus(4) targets:

ahci0: port 0, at 6Gbps
ahci0: port 2, at 3Gbps
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 ..
sd1 at scsibus1 targ 2 ..

ok deraadt@, tested by bmercer@

include extended capabilities in ahci debug output
ok dlg@

recognise ahci 1.3.1
ok dlg@

Turn ahci port start into a callback so sxiahci can do some magic to
setup dma. Modified version of a patch from Edwin Amsler which was
based on changes made in Bitrig by Artturi Alm.

ok dlg@

Some reallocarray() use; review Jean-Philippe Ouellet, patrick keshishian
ok tedu

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Work around compatibility problems with Intel ahci and Intel SSDs by retrying
device detection if the port is still busy after our normal timeout.

based on a diff by chris@, based on a change in dragonflybsd
tested by Peter J Philipp
ok dlg@

move the hardware structures into ahcireg.h.

move the last htoleXX things to htolemXX where possible.

ok jmatthew@

ahci dva things are 64bit and surprisingly aligned nicely to 64bit too.

ok jmatthew@

replace bzero with memset

massage the prdt (what an sgl is on every other chip) loading with
better ops and alignment.

ok jmatthew@ who was cruel and made me test hibernate before he oked it.

Reduce uvm include madness. Use <uvm/uvm_extern.h> instead of
<uvm/uvm.h> if possible and remove double inclusions.

ok beck@, mlarkin@, deraadt@

fix previous errorneous constants (compiled against the old header)

Recognize SATA-III ports in dmesg.

ok millert

Add a DVACT_WAKEUP op to the *_activate() API. This is called after the
kernel resumes normal (non-cold, able to run processes, etc) operation.
Previously we were relying on specific DVACT_RESUME op's in drivers
creating callback/threads themselves, but that has become too common,
indicating the need for a built-in mechanism.
ok dlg kettenis, tested by a sufficient amount of people

teach the side-effect free drivers about the partition they are dealing
by passing a start/length in the HIB_INIT op. Then rebase all
hibernate-time block offsets to be relative to the start of that partition.
This simplifies things a lot.
ok mlarkin

pay attention to the #ifdef

move AHCI_DEBUG stuff into ahci.c so we can build with it enabled again

ok dlg@

Split AHCI in order to support devices which are not attached via pci.
Tested on a few machines, including JMicron and Intel AHCIs.

ok dlg@ jmatthew@

Access adapter softc via link->bus->sb_adapter_softc.

In sparc64 autoconf access 'luns' via sb_luns and 'adapter_buswidth'
via sb_adapter_buswidth.

Removes last post-config uses of the copies of bus related
information in scsi_link.

ahci_get_pmp_ccb() returns a non-NULL pointer, fails a
KASSERT(), or has already blown up by dereferencing the
pointer.

Two of the four invocations of ahci_get_pmp_ccb() check for
NULL. Save a couple of bytes by not bothering. Add comments
to the invocations to docuement the assumption.

Trailing whitespace, long lines, incorrect function names in
printf()'s.

Remove some "set but not used" variables in ahci(4)'s hibernate code.

ok jmatthew

Turn off the code which waits for AHCI_PREG_CMD_CR to be set by the
HBA after ahci_default_port_start() sets AHCI_PREG_CMD_ST. The AHCI
spec. rev. 1.3 only requires the inverse to be true, i. e. that a
HBA clears AHCI_PREG_CMD_CR when AHCI_PREG_CMD_ST gets cleared by
software/driver. In fact, some HBAs will not raise AHCI_PREG_CMD_CR
as an immediate consequence of AHCI_PREG_CMD_ST being set. Actually
neither the FreeBSD, Linux nor NetBSD counterpart of ahci(4) has an
analogous check. Disabling that wait fixes "failed to start command
DMA on port N, disabling" bails during attach.

From Marius Strobl
ok deraadt@ jmatthew@

Split up ahci_port_portreset into a few smaller bits, and also slightly
adjust port multiplier detection so it doesn't call ahci_port_portreset
again directly, but instead restarts the loop for the current call.

During attach, poll for device detection across all ports until either all
ports have detected a device, or one second has passed, rather than doing
them sequentially. Devices are still attached in order of port number,
so disk unit numbers won't change.

ok visa@

improve AHCI hibernate writeout performance by doing smaller delay()s
between each I/O.

ok kettenis@, jmatthew@

Remove comments about implementing FIS-based switching with port multipliers,
which we can safely assume isn't going to happen now. No code changes.

When we're unable to determine which NCQ command failed, fail all active
commands rather than proceeding to an assertion failure. This matches
how we handle the same condition with non-queued commands, and the
behaviour of every other ahci driver I can find. Add a printf there so
we might get some idea of how often this condition arises and what happens
afterwards.

ok dlg@

Some HABs reset parts of AHCI_PREG_CMD when AHCI_PREG_SCTL_DET_INIT
gets set. Therefore, ahci_port_softreset() restores the sate of the
former register once the device detection sequence is finished. The
device detection code in ahci_pmp_port_portreset() does not restore
AHCI_PREG_CMD afterwards, so let it catch up. Apparently, this part
was an oversight as ahci_pmp_port_portreset() did not otherwise use
"cmd".

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Some HBAs report NCQ capability despite only supporting one command
slot. Thus, extend the check whether NCQ actually should be enabled
accordingly.

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Given that ahci_port_alloc() grabs one CCB for use during NCQ error
recovery from the CCB pool sized based on the NCS capability, i. e.
number of command slots reported by the controller, it is necessary
to pretend at least 2 slots in sc->sc_ncmds for devices without NCQ
support. That way, also at least 1 available slot is made available
for atascsi(4). Otherwise, controllers having only a single command
slot will trigger "no free xfers on a new port" in atascsi(4).
Note that pretending 2 command slots is also fine with the abuse of
the NCQ error recovery CCB in ahci_port_softreset().

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Contrary to e. g. the xHCI specification, the AHCI rev. 1.3 spec
does not say anything about which type (snoopable/uncached etc.) of
bus transactions are issued by HBAs on accesses to the descriptors.
Thus, the right assumption would be "classical" concurrent accesses
by both CPU/driver as well as hardware to the descriptors and their
DMA memory backings respectively, which is also confirmed by actual
testing. Consequently, switch to BUS_DMA_COHERENT mappings for said
DMA memory as otherwise corruption of descriptors is seen.

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Enforce some naming sanity. Stop using 'sc_link' to mean two different
things by renaming the field 'SLIST_HEAD(, scsi_link) sc_link' to
'sc_link_list' in struct scsibus_softc. Use 'sb' as the short name
for scsibus_softc variables.

Impetus from & ok bluhm@

Fix memory leak in error path.
From Benjamin Baier, found by llvm/scan-build.

sizes for free()

Print speeds consistently.

ok brynet@, miod@

Print the current link speed for SATA devices attached on ahci(4).

The port numbers correspond to scsibus(4) targets:

ahci0: port 0, at 6Gbps
ahci0: port 2, at 3Gbps
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 ..
sd1 at scsibus1 targ 2 ..

ok deraadt@, tested by bmercer@

include extended capabilities in ahci debug output
ok dlg@

recognise ahci 1.3.1
ok dlg@

Turn ahci port start into a callback so sxiahci can do some magic to
setup dma. Modified version of a patch from Edwin Amsler which was
based on changes made in Bitrig by Artturi Alm.

ok dlg@

Some reallocarray() use; review Jean-Philippe Ouellet, patrick keshishian
ok tedu

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Work around compatibility problems with Intel ahci and Intel SSDs by retrying
device detection if the port is still busy after our normal timeout.

based on a diff by chris@, based on a change in dragonflybsd
tested by Peter J Philipp
ok dlg@

move the hardware structures into ahcireg.h.

move the last htoleXX things to htolemXX where possible.

ok jmatthew@

ahci dva things are 64bit and surprisingly aligned nicely to 64bit too.

ok jmatthew@

replace bzero with memset

massage the prdt (what an sgl is on every other chip) loading with
better ops and alignment.

ok jmatthew@ who was cruel and made me test hibernate before he oked it.

Reduce uvm include madness. Use <uvm/uvm_extern.h> instead of
<uvm/uvm.h> if possible and remove double inclusions.

ok beck@, mlarkin@, deraadt@

fix previous errorneous constants (compiled against the old header)

Recognize SATA-III ports in dmesg.

ok millert

Add a DVACT_WAKEUP op to the *_activate() API. This is called after the
kernel resumes normal (non-cold, able to run processes, etc) operation.
Previously we were relying on specific DVACT_RESUME op's in drivers
creating callback/threads themselves, but that has become too common,
indicating the need for a built-in mechanism.
ok dlg kettenis, tested by a sufficient amount of people

teach the side-effect free drivers about the partition they are dealing
by passing a start/length in the HIB_INIT op. Then rebase all
hibernate-time block offsets to be relative to the start of that partition.
This simplifies things a lot.
ok mlarkin

pay attention to the #ifdef

move AHCI_DEBUG stuff into ahci.c so we can build with it enabled again

ok dlg@

Split AHCI in order to support devices which are not attached via pci.
Tested on a few machines, including JMicron and Intel AHCIs.

ok dlg@ jmatthew@

ahci_get_pmp_ccb() returns a non-NULL pointer, fails a
KASSERT(), or has already blown up by dereferencing the
pointer.

Two of the four invocations of ahci_get_pmp_ccb() check for
NULL. Save a couple of bytes by not bothering. Add comments
to the invocations to docuement the assumption.

Trailing whitespace, long lines, incorrect function names in
printf()'s.

Remove some "set but not used" variables in ahci(4)'s hibernate code.

ok jmatthew

Turn off the code which waits for AHCI_PREG_CMD_CR to be set by the
HBA after ahci_default_port_start() sets AHCI_PREG_CMD_ST. The AHCI
spec. rev. 1.3 only requires the inverse to be true, i. e. that a
HBA clears AHCI_PREG_CMD_CR when AHCI_PREG_CMD_ST gets cleared by
software/driver. In fact, some HBAs will not raise AHCI_PREG_CMD_CR
as an immediate consequence of AHCI_PREG_CMD_ST being set. Actually
neither the FreeBSD, Linux nor NetBSD counterpart of ahci(4) has an
analogous check. Disabling that wait fixes "failed to start command
DMA on port N, disabling" bails during attach.

From Marius Strobl
ok deraadt@ jmatthew@

Split up ahci_port_portreset into a few smaller bits, and also slightly
adjust port multiplier detection so it doesn't call ahci_port_portreset
again directly, but instead restarts the loop for the current call.

During attach, poll for device detection across all ports until either all
ports have detected a device, or one second has passed, rather than doing
them sequentially. Devices are still attached in order of port number,
so disk unit numbers won't change.

ok visa@

improve AHCI hibernate writeout performance by doing smaller delay()s
between each I/O.

ok kettenis@, jmatthew@

Remove comments about implementing FIS-based switching with port multipliers,
which we can safely assume isn't going to happen now. No code changes.

When we're unable to determine which NCQ command failed, fail all active
commands rather than proceeding to an assertion failure. This matches
how we handle the same condition with non-queued commands, and the
behaviour of every other ahci driver I can find. Add a printf there so
we might get some idea of how often this condition arises and what happens
afterwards.

ok dlg@

Some HABs reset parts of AHCI_PREG_CMD when AHCI_PREG_SCTL_DET_INIT
gets set. Therefore, ahci_port_softreset() restores the sate of the
former register once the device detection sequence is finished. The
device detection code in ahci_pmp_port_portreset() does not restore
AHCI_PREG_CMD afterwards, so let it catch up. Apparently, this part
was an oversight as ahci_pmp_port_portreset() did not otherwise use
"cmd".

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Some HBAs report NCQ capability despite only supporting one command
slot. Thus, extend the check whether NCQ actually should be enabled
accordingly.

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Given that ahci_port_alloc() grabs one CCB for use during NCQ error
recovery from the CCB pool sized based on the NCS capability, i. e.
number of command slots reported by the controller, it is necessary
to pretend at least 2 slots in sc->sc_ncmds for devices without NCQ
support. That way, also at least 1 available slot is made available
for atascsi(4). Otherwise, controllers having only a single command
slot will trigger "no free xfers on a new port" in atascsi(4).
Note that pretending 2 command slots is also fine with the abuse of
the NCQ error recovery CCB in ahci_port_softreset().

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Contrary to e. g. the xHCI specification, the AHCI rev. 1.3 spec
does not say anything about which type (snoopable/uncached etc.) of
bus transactions are issued by HBAs on accesses to the descriptors.
Thus, the right assumption would be "classical" concurrent accesses
by both CPU/driver as well as hardware to the descriptors and their
DMA memory backings respectively, which is also confirmed by actual
testing. Consequently, switch to BUS_DMA_COHERENT mappings for said
DMA memory as otherwise corruption of descriptors is seen.

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Enforce some naming sanity. Stop using 'sc_link' to mean two different
things by renaming the field 'SLIST_HEAD(, scsi_link) sc_link' to
'sc_link_list' in struct scsibus_softc. Use 'sb' as the short name
for scsibus_softc variables.

Impetus from & ok bluhm@

Fix memory leak in error path.
From Benjamin Baier, found by llvm/scan-build.

sizes for free()

Print speeds consistently.

ok brynet@, miod@

Print the current link speed for SATA devices attached on ahci(4).

The port numbers correspond to scsibus(4) targets:

ahci0: port 0, at 6Gbps
ahci0: port 2, at 3Gbps
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 ..
sd1 at scsibus1 targ 2 ..

ok deraadt@, tested by bmercer@

include extended capabilities in ahci debug output
ok dlg@

recognise ahci 1.3.1
ok dlg@

Turn ahci port start into a callback so sxiahci can do some magic to
setup dma. Modified version of a patch from Edwin Amsler which was
based on changes made in Bitrig by Artturi Alm.

ok dlg@

Some reallocarray() use; review Jean-Philippe Ouellet, patrick keshishian
ok tedu

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Work around compatibility problems with Intel ahci and Intel SSDs by retrying
device detection if the port is still busy after our normal timeout.

based on a diff by chris@, based on a change in dragonflybsd
tested by Peter J Philipp
ok dlg@

move the hardware structures into ahcireg.h.

move the last htoleXX things to htolemXX where possible.

ok jmatthew@

ahci dva things are 64bit and surprisingly aligned nicely to 64bit too.

ok jmatthew@

replace bzero with memset

massage the prdt (what an sgl is on every other chip) loading with
better ops and alignment.

ok jmatthew@ who was cruel and made me test hibernate before he oked it.

Reduce uvm include madness. Use <uvm/uvm_extern.h> instead of
<uvm/uvm.h> if possible and remove double inclusions.

ok beck@, mlarkin@, deraadt@

fix previous errorneous constants (compiled against the old header)

Recognize SATA-III ports in dmesg.

ok millert

Add a DVACT_WAKEUP op to the *_activate() API. This is called after the
kernel resumes normal (non-cold, able to run processes, etc) operation.
Previously we were relying on specific DVACT_RESUME op's in drivers
creating callback/threads themselves, but that has become too common,
indicating the need for a built-in mechanism.
ok dlg kettenis, tested by a sufficient amount of people

teach the side-effect free drivers about the partition they are dealing
by passing a start/length in the HIB_INIT op. Then rebase all
hibernate-time block offsets to be relative to the start of that partition.
This simplifies things a lot.
ok mlarkin

pay attention to the #ifdef

move AHCI_DEBUG stuff into ahci.c so we can build with it enabled again

ok dlg@

Split AHCI in order to support devices which are not attached via pci.
Tested on a few machines, including JMicron and Intel AHCIs.

ok dlg@ jmatthew@

Remove some "set but not used" variables in ahci(4)'s hibernate code.

ok jmatthew

Turn off the code which waits for AHCI_PREG_CMD_CR to be set by the
HBA after ahci_default_port_start() sets AHCI_PREG_CMD_ST. The AHCI
spec. rev. 1.3 only requires the inverse to be true, i. e. that a
HBA clears AHCI_PREG_CMD_CR when AHCI_PREG_CMD_ST gets cleared by
software/driver. In fact, some HBAs will not raise AHCI_PREG_CMD_CR
as an immediate consequence of AHCI_PREG_CMD_ST being set. Actually
neither the FreeBSD, Linux nor NetBSD counterpart of ahci(4) has an
analogous check. Disabling that wait fixes "failed to start command
DMA on port N, disabling" bails during attach.

From Marius Strobl
ok deraadt@ jmatthew@

Split up ahci_port_portreset into a few smaller bits, and also slightly
adjust port multiplier detection so it doesn't call ahci_port_portreset
again directly, but instead restarts the loop for the current call.

During attach, poll for device detection across all ports until either all
ports have detected a device, or one second has passed, rather than doing
them sequentially. Devices are still attached in order of port number,
so disk unit numbers won't change.

ok visa@

improve AHCI hibernate writeout performance by doing smaller delay()s
between each I/O.

ok kettenis@, jmatthew@

Remove comments about implementing FIS-based switching with port multipliers,
which we can safely assume isn't going to happen now. No code changes.

When we're unable to determine which NCQ command failed, fail all active
commands rather than proceeding to an assertion failure. This matches
how we handle the same condition with non-queued commands, and the
behaviour of every other ahci driver I can find. Add a printf there so
we might get some idea of how often this condition arises and what happens
afterwards.

ok dlg@

Some HABs reset parts of AHCI_PREG_CMD when AHCI_PREG_SCTL_DET_INIT
gets set. Therefore, ahci_port_softreset() restores the sate of the
former register once the device detection sequence is finished. The
device detection code in ahci_pmp_port_portreset() does not restore
AHCI_PREG_CMD afterwards, so let it catch up. Apparently, this part
was an oversight as ahci_pmp_port_portreset() did not otherwise use
"cmd".

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Some HBAs report NCQ capability despite only supporting one command
slot. Thus, extend the check whether NCQ actually should be enabled
accordingly.

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Given that ahci_port_alloc() grabs one CCB for use during NCQ error
recovery from the CCB pool sized based on the NCS capability, i. e.
number of command slots reported by the controller, it is necessary
to pretend at least 2 slots in sc->sc_ncmds for devices without NCQ
support. That way, also at least 1 available slot is made available
for atascsi(4). Otherwise, controllers having only a single command
slot will trigger "no free xfers on a new port" in atascsi(4).
Note that pretending 2 command slots is also fine with the abuse of
the NCQ error recovery CCB in ahci_port_softreset().

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Contrary to e. g. the xHCI specification, the AHCI rev. 1.3 spec
does not say anything about which type (snoopable/uncached etc.) of
bus transactions are issued by HBAs on accesses to the descriptors.
Thus, the right assumption would be "classical" concurrent accesses
by both CPU/driver as well as hardware to the descriptors and their
DMA memory backings respectively, which is also confirmed by actual
testing. Consequently, switch to BUS_DMA_COHERENT mappings for said
DMA memory as otherwise corruption of descriptors is seen.

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Enforce some naming sanity. Stop using 'sc_link' to mean two different
things by renaming the field 'SLIST_HEAD(, scsi_link) sc_link' to
'sc_link_list' in struct scsibus_softc. Use 'sb' as the short name
for scsibus_softc variables.

Impetus from & ok bluhm@

Fix memory leak in error path.
From Benjamin Baier, found by llvm/scan-build.

sizes for free()

Print speeds consistently.

ok brynet@, miod@

Print the current link speed for SATA devices attached on ahci(4).

The port numbers correspond to scsibus(4) targets:

ahci0: port 0, at 6Gbps
ahci0: port 2, at 3Gbps
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 ..
sd1 at scsibus1 targ 2 ..

ok deraadt@, tested by bmercer@

include extended capabilities in ahci debug output
ok dlg@

recognise ahci 1.3.1
ok dlg@

Turn ahci port start into a callback so sxiahci can do some magic to
setup dma. Modified version of a patch from Edwin Amsler which was
based on changes made in Bitrig by Artturi Alm.

ok dlg@

Some reallocarray() use; review Jean-Philippe Ouellet, patrick keshishian
ok tedu

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Work around compatibility problems with Intel ahci and Intel SSDs by retrying
device detection if the port is still busy after our normal timeout.

based on a diff by chris@, based on a change in dragonflybsd
tested by Peter J Philipp
ok dlg@

move the hardware structures into ahcireg.h.

move the last htoleXX things to htolemXX where possible.

ok jmatthew@

ahci dva things are 64bit and surprisingly aligned nicely to 64bit too.

ok jmatthew@

replace bzero with memset

massage the prdt (what an sgl is on every other chip) loading with
better ops and alignment.

ok jmatthew@ who was cruel and made me test hibernate before he oked it.

Reduce uvm include madness. Use <uvm/uvm_extern.h> instead of
<uvm/uvm.h> if possible and remove double inclusions.

ok beck@, mlarkin@, deraadt@

fix previous errorneous constants (compiled against the old header)

Recognize SATA-III ports in dmesg.

ok millert

Add a DVACT_WAKEUP op to the *_activate() API. This is called after the
kernel resumes normal (non-cold, able to run processes, etc) operation.
Previously we were relying on specific DVACT_RESUME op's in drivers
creating callback/threads themselves, but that has become too common,
indicating the need for a built-in mechanism.
ok dlg kettenis, tested by a sufficient amount of people

teach the side-effect free drivers about the partition they are dealing
by passing a start/length in the HIB_INIT op. Then rebase all
hibernate-time block offsets to be relative to the start of that partition.
This simplifies things a lot.
ok mlarkin

pay attention to the #ifdef

move AHCI_DEBUG stuff into ahci.c so we can build with it enabled again

ok dlg@

Split AHCI in order to support devices which are not attached via pci.
Tested on a few machines, including JMicron and Intel AHCIs.

ok dlg@ jmatthew@

Turn off the code which waits for AHCI_PREG_CMD_CR to be set by the
HBA after ahci_default_port_start() sets AHCI_PREG_CMD_ST. The AHCI
spec. rev. 1.3 only requires the inverse to be true, i. e. that a
HBA clears AHCI_PREG_CMD_CR when AHCI_PREG_CMD_ST gets cleared by
software/driver. In fact, some HBAs will not raise AHCI_PREG_CMD_CR
as an immediate consequence of AHCI_PREG_CMD_ST being set. Actually
neither the FreeBSD, Linux nor NetBSD counterpart of ahci(4) has an
analogous check. Disabling that wait fixes "failed to start command
DMA on port N, disabling" bails during attach.

From Marius Strobl
ok deraadt@ jmatthew@

Split up ahci_port_portreset into a few smaller bits, and also slightly
adjust port multiplier detection so it doesn't call ahci_port_portreset
again directly, but instead restarts the loop for the current call.

During attach, poll for device detection across all ports until either all
ports have detected a device, or one second has passed, rather than doing
them sequentially. Devices are still attached in order of port number,
so disk unit numbers won't change.

ok visa@

improve AHCI hibernate writeout performance by doing smaller delay()s
between each I/O.

ok kettenis@, jmatthew@

Remove comments about implementing FIS-based switching with port multipliers,
which we can safely assume isn't going to happen now. No code changes.

When we're unable to determine which NCQ command failed, fail all active
commands rather than proceeding to an assertion failure. This matches
how we handle the same condition with non-queued commands, and the
behaviour of every other ahci driver I can find. Add a printf there so
we might get some idea of how often this condition arises and what happens
afterwards.

ok dlg@

Some HABs reset parts of AHCI_PREG_CMD when AHCI_PREG_SCTL_DET_INIT
gets set. Therefore, ahci_port_softreset() restores the sate of the
former register once the device detection sequence is finished. The
device detection code in ahci_pmp_port_portreset() does not restore
AHCI_PREG_CMD afterwards, so let it catch up. Apparently, this part
was an oversight as ahci_pmp_port_portreset() did not otherwise use
"cmd".

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Some HBAs report NCQ capability despite only supporting one command
slot. Thus, extend the check whether NCQ actually should be enabled
accordingly.

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Given that ahci_port_alloc() grabs one CCB for use during NCQ error
recovery from the CCB pool sized based on the NCS capability, i. e.
number of command slots reported by the controller, it is necessary
to pretend at least 2 slots in sc->sc_ncmds for devices without NCQ
support. That way, also at least 1 available slot is made available
for atascsi(4). Otherwise, controllers having only a single command
slot will trigger "no free xfers on a new port" in atascsi(4).
Note that pretending 2 command slots is also fine with the abuse of
the NCQ error recovery CCB in ahci_port_softreset().

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Contrary to e. g. the xHCI specification, the AHCI rev. 1.3 spec
does not say anything about which type (snoopable/uncached etc.) of
bus transactions are issued by HBAs on accesses to the descriptors.
Thus, the right assumption would be "classical" concurrent accesses
by both CPU/driver as well as hardware to the descriptors and their
DMA memory backings respectively, which is also confirmed by actual
testing. Consequently, switch to BUS_DMA_COHERENT mappings for said
DMA memory as otherwise corruption of descriptors is seen.

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Enforce some naming sanity. Stop using 'sc_link' to mean two different
things by renaming the field 'SLIST_HEAD(, scsi_link) sc_link' to
'sc_link_list' in struct scsibus_softc. Use 'sb' as the short name
for scsibus_softc variables.

Impetus from & ok bluhm@

Fix memory leak in error path.
From Benjamin Baier, found by llvm/scan-build.

sizes for free()

Print speeds consistently.

ok brynet@, miod@

Print the current link speed for SATA devices attached on ahci(4).

The port numbers correspond to scsibus(4) targets:

ahci0: port 0, at 6Gbps
ahci0: port 2, at 3Gbps
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 ..
sd1 at scsibus1 targ 2 ..

ok deraadt@, tested by bmercer@

include extended capabilities in ahci debug output
ok dlg@

recognise ahci 1.3.1
ok dlg@

Turn ahci port start into a callback so sxiahci can do some magic to
setup dma. Modified version of a patch from Edwin Amsler which was
based on changes made in Bitrig by Artturi Alm.

ok dlg@

Some reallocarray() use; review Jean-Philippe Ouellet, patrick keshishian
ok tedu

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Work around compatibility problems with Intel ahci and Intel SSDs by retrying
device detection if the port is still busy after our normal timeout.

based on a diff by chris@, based on a change in dragonflybsd
tested by Peter J Philipp
ok dlg@

move the hardware structures into ahcireg.h.

move the last htoleXX things to htolemXX where possible.

ok jmatthew@

ahci dva things are 64bit and surprisingly aligned nicely to 64bit too.

ok jmatthew@

replace bzero with memset

massage the prdt (what an sgl is on every other chip) loading with
better ops and alignment.

ok jmatthew@ who was cruel and made me test hibernate before he oked it.

Reduce uvm include madness. Use <uvm/uvm_extern.h> instead of
<uvm/uvm.h> if possible and remove double inclusions.

ok beck@, mlarkin@, deraadt@

fix previous errorneous constants (compiled against the old header)

Recognize SATA-III ports in dmesg.

ok millert

Add a DVACT_WAKEUP op to the *_activate() API. This is called after the
kernel resumes normal (non-cold, able to run processes, etc) operation.
Previously we were relying on specific DVACT_RESUME op's in drivers
creating callback/threads themselves, but that has become too common,
indicating the need for a built-in mechanism.
ok dlg kettenis, tested by a sufficient amount of people

teach the side-effect free drivers about the partition they are dealing
by passing a start/length in the HIB_INIT op. Then rebase all
hibernate-time block offsets to be relative to the start of that partition.
This simplifies things a lot.
ok mlarkin

pay attention to the #ifdef

move AHCI_DEBUG stuff into ahci.c so we can build with it enabled again

ok dlg@

Split AHCI in order to support devices which are not attached via pci.
Tested on a few machines, including JMicron and Intel AHCIs.

ok dlg@ jmatthew@

Split up ahci_port_portreset into a few smaller bits, and also slightly
adjust port multiplier detection so it doesn't call ahci_port_portreset
again directly, but instead restarts the loop for the current call.

During attach, poll for device detection across all ports until either all
ports have detected a device, or one second has passed, rather than doing
them sequentially. Devices are still attached in order of port number,
so disk unit numbers won't change.

ok visa@

improve AHCI hibernate writeout performance by doing smaller delay()s
between each I/O.

ok kettenis@, jmatthew@

Remove comments about implementing FIS-based switching with port multipliers,
which we can safely assume isn't going to happen now. No code changes.

When we're unable to determine which NCQ command failed, fail all active
commands rather than proceeding to an assertion failure. This matches
how we handle the same condition with non-queued commands, and the
behaviour of every other ahci driver I can find. Add a printf there so
we might get some idea of how often this condition arises and what happens
afterwards.

ok dlg@

Some HABs reset parts of AHCI_PREG_CMD when AHCI_PREG_SCTL_DET_INIT
gets set. Therefore, ahci_port_softreset() restores the sate of the
former register once the device detection sequence is finished. The
device detection code in ahci_pmp_port_portreset() does not restore
AHCI_PREG_CMD afterwards, so let it catch up. Apparently, this part
was an oversight as ahci_pmp_port_portreset() did not otherwise use
"cmd".

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Some HBAs report NCQ capability despite only supporting one command
slot. Thus, extend the check whether NCQ actually should be enabled
accordingly.

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Given that ahci_port_alloc() grabs one CCB for use during NCQ error
recovery from the CCB pool sized based on the NCS capability, i. e.
number of command slots reported by the controller, it is necessary
to pretend at least 2 slots in sc->sc_ncmds for devices without NCQ
support. That way, also at least 1 available slot is made available
for atascsi(4). Otherwise, controllers having only a single command
slot will trigger "no free xfers on a new port" in atascsi(4).
Note that pretending 2 command slots is also fine with the abuse of
the NCQ error recovery CCB in ahci_port_softreset().

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Contrary to e. g. the xHCI specification, the AHCI rev. 1.3 spec
does not say anything about which type (snoopable/uncached etc.) of
bus transactions are issued by HBAs on accesses to the descriptors.
Thus, the right assumption would be "classical" concurrent accesses
by both CPU/driver as well as hardware to the descriptors and their
DMA memory backings respectively, which is also confirmed by actual
testing. Consequently, switch to BUS_DMA_COHERENT mappings for said
DMA memory as otherwise corruption of descriptors is seen.

From Marius Strobl

tested by awolk (amd64), bluhm (amd64, i386), myself (amd64, armv7)
ok jmatthew@

Enforce some naming sanity. Stop using 'sc_link' to mean two different
things by renaming the field 'SLIST_HEAD(, scsi_link) sc_link' to
'sc_link_list' in struct scsibus_softc. Use 'sb' as the short name
for scsibus_softc variables.

Impetus from & ok bluhm@

Fix memory leak in error path.
From Benjamin Baier, found by llvm/scan-build.

sizes for free()

Print speeds consistently.

ok brynet@, miod@

Print the current link speed for SATA devices attached on ahci(4).

The port numbers correspond to scsibus(4) targets:

ahci0: port 0, at 6Gbps
ahci0: port 2, at 3Gbps
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 ..
sd1 at scsibus1 targ 2 ..

ok deraadt@, tested by bmercer@

include extended capabilities in ahci debug output
ok dlg@

recognise ahci 1.3.1
ok dlg@

Turn ahci port start into a callback so sxiahci can do some magic to
setup dma. Modified version of a patch from Edwin Amsler which was
based on changes made in Bitrig by Artturi Alm.

ok dlg@

Some reallocarray() use; review Jean-Philippe Ouellet, patrick keshishian
ok tedu

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Work around compatibility problems with Intel ahci and Intel SSDs by retrying
device detection if the port is still busy after our normal timeout.

based on a diff by chris@, based on a change in dragonflybsd
tested by Peter J Philipp
ok dlg@

move the hardware structures into ahcireg.h.

move the last htoleXX things to htolemXX where possible.

ok jmatthew@

ahci dva things are 64bit and surprisingly aligned nicely to 64bit too.

ok jmatthew@

replace bzero with memset

massage the prdt (what an sgl is on every other chip) loading with
better ops and alignment.

ok jmatthew@ who was cruel and made me test hibernate before he oked it.

Reduce uvm include madness. Use <uvm/uvm_extern.h> instead of
<uvm/uvm.h> if possible and remove double inclusions.

ok beck@, mlarkin@, deraadt@

fix previous errorneous constants (compiled against the old header)

Recognize SATA-III ports in dmesg.

ok millert

Add a DVACT_WAKEUP op to the *_activate() API. This is called after the
kernel resumes normal (non-cold, able to run processes, etc) operation.
Previously we were relying on specific DVACT_RESUME op's in drivers
creating callback/threads themselves, but that has become too common,
indicating the need for a built-in mechanism.
ok dlg kettenis, tested by a sufficient amount of people

teach the side-effect free drivers about the partition they are dealing
by passing a start/length in the HIB_INIT op. Then rebase all
hibernate-time block offsets to be relative to the start of that partition.
This simplifies things a lot.
ok mlarkin

pay attention to the #ifdef

move AHCI_DEBUG stuff into ahci.c so we can build with it enabled again

ok dlg@

Split AHCI in order to support devices which are not attached via pci.
Tested on a few machines, including JMicron and Intel AHCIs.

ok dlg@ jmatthew@