Remove svn:mergeinfo carried over from stable/9.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Fix the outstanding mergeinfo part of r249374.

MFC r250963, r251013, r251056: Driver 'aacraid' added to support Adaptec by PMC RAID controller families Series 6, 7, 8 and upcoming products.

Approved by: emaste (co-mentor)

MFC r248933:

Use a shared lock for VOP_GETEXTATTR, as it is a read-like operation.

Improve error handling when unwrapping received data.

Submitted by: Rick Macklem

MFC of 246876 and 246877

MFC: 246876:

Add barrier write capability to the VFS buffer interface. A barrier
write is a disk write request that tells the disk that the buffer
being written must be committed to the media along with any writes
that preceeded it before any future blocks may be written to the drive.

Barrier writes are provided by adding the functions bbarrierwrite
(bwrite with barrier) and babarrierwrite (bawrite with barrier).

Following a bbarrierwrite the client knows that the requested buffer
is on the media. It does not ensure that buffers written before that
buffer are on the media. It only ensure that buffers written before
that buffer will get to the media before any buffers written after
that buffer. A flush command must be sent to the disk to ensure that
all earlier written buffers are on the media.

Reviewed by: kib
Tested by: Peter Holm

MFC 246877:

The UFS2 filesystem allocates new blocks of inodes as they are needed.
When a cylinder group runs short of inodes, a new block for inodes is
allocated, zero'ed, and written to the disk. The zero'ed inodes must
be on the disk before the cylinder group can be updated to claim them.
If the cylinder group claiming the new inodes were written before the
zero'ed block of inodes, the system could crash with the filesystem in
an unrecoverable state.

Rather than adding a soft updates dependency to ensure that the new
inode block is written before it is claimed by the cylinder group
map, we just do a barrier write of the zero'ed inode block to ensure
that it will get written before the updated cylinder group map can
be written. This change should only slow down bulk loading of newly
created filesystems since that is the primary time that new inode
blocks need to be created.

Reported by: Robert Watson
Reviewed by: kib
Tested by: Peter Holm

MFS of 246289:

For UFS2 i_blocks is unsigned. The current "sanity" check that it
has gone below zero after the blocks in its inode are freed is a
no-op which the compiler fails to warn about because of the use of
the DIP macro. Change the sanity check to compare the number of
blocks being freed against the value i_blocks. If the number of
blocks being freed exceeds i_blocks, just set i_blocks to zero.

Reported by: Pedro Giffuni (pfg@)

MFC r230590 (by ken) except parts changing ABI:
Add CAM infrastructure to allow reporting when a drive's long read capacity
data changes.

MFC r244750:
Add net.link.stf.permit_rfc1918 sysctl variable. It can be used to allow
the use of private IPv4 addresses with stf(4).

MFC r244752:
Add an ability to set net.link.stf.permit_rfc1918 from the loader.

MFC: 234930

Fix so that ,usr and ,os work correctly with fixed function (IAF)
counters.

MFC: 238366, 240924

Initial commit of an I/O provider for DTrace on FreeBSD.

These probes are most useful when looking into the structures
they provide, which are listed in io.d. For example:

dtrace -n 'io:kernel::start { printf("%d\n", args[0]->bio_bcount); }'

Note that the I/O systems in FreeBSD and Solaris/Illumos are sufficiently
different that there is not a 1:1 mapping from scripts that work
with one to the other.

This commit includes the fix so that our probes use "kernel"
instead of the Solaris specific "genunix"

MFC: r239864

- Unlike cache invalidation and TLB demapping IPIs, reading registers from
other CPUs doesn't require locking so get rid of it. As the latter is used
for the timecounter on certain machine models, using a spin lock in this
case can lead to a deadlock with the upcoming callout(9) rework.
- Merge r134227/r167250 from x86:
Avoid cross-IPI SMP deadlock by using the smp_ipi_mtx spin lock not only
for smp_rendezvous_cpus() but also for the MD cache invalidation and TLB
demapping IPIs.
- Mark some unused function arguments as such.

MFC r239257:

Reserve room for the terminating NUL when setting or getting kernel
environment variables. KENV_MNAMELEN and KENV_MVALLEN doesn't include
space for the terminating NUL.

MFC: r240720
Modify the NFSv4 client so that it can handle owner
and owner_group strings that consist entirely of
digits, interpreting them as the uid/gid number.
This change was needed since new (>= 3.3) Linux
servers reply with these strings by default.
This change is mandated by the rfc3530bis draft.
Reported on freebsd-stable@ under the Subject
heading "Problem with Linux >= 3.3 as NFSv4 server"
by Norbert Aschendorff on Aug. 20, 2012.

MFC 239779:
Shorten the name of the fast SWI taskqueue to "fast taskq" so that
it fits.

MFC 239008,239020:
Improve the handling of static DMA buffers that use non-default memory
attributes (currently just BUS_DMA_NOCACHE):
- Don't call pmap_change_attr() on the returned address, instead use
kmem_alloc_contig() to ask the VM system for memory with the requested
attribute.
- As a result, always use kmem_alloc_contig() for non-default memory
attributes, even for sub-page allocations. This requires adjusting
bus_dmamem_free()'s logic for determining which free routine to use.
- For x86, add a new dummy bus_dmamap that is used for static DMA
buffers allocated via kmem_alloc_contig(). bus_dmamem_free() can then
use the map pointer to determine which free routine to use.
- For powerpc, add a new flag to the allocated map (bus_dmamem_alloc()
always creates a real map on powerpc) to indicate which free routine
should be used.

Note that the BUS_DMA_NOCACHE handling in powerpc is currently #ifdef'd out.
I have left it disabled but updated it to match x86.

MFC 239771: Fix misspelled "Infiniband".

MFC: r240289
Add a simple printf() based debug facility to the new nfs client.
Use it for a printf() that can be harmlessly generated for mmap()'d
files. It will be used extensively for the NFSv4.1 client.
Debugging printf()s are enabled by setting vfs.nfs.debuglevel to
a non-zero value. The higher the value, the more debugging printf()s.

Merge rev. 239673 from head to stable/9:
(g_multipath_rotate): Fix algorithm so that it does rotate over all good
providers, not just the last two.

MFC: merge rev. 239012 from head to stable/9
New command "gmultipath prefer" to force selection of a specified
provider in an Active/Passive configuration.

MFC 239128:
Don't try to stop the IPMI watchdog timer if it is not running.
Starting or stopping the IPMI watchdog is rather expensive with the
current implementation as all IPMI requests are bounced via thread.
This is not viable during shutdown or dumps, and this avoids headache
in the common case that the watchdog is not enabled. The IPMI watchdog
should probably be reworked to not use a separate thread to fix this
in the case when the watchdog timer is enabled.

MFC 239103:
Explicitly enable busmastering on PCI-PCI bridges. Transactions initiated
on the secondary side of a bridge will not be propagated to the primary
bus unless this is enabled. Busmastering is not enabled by default (we
have relied on firmware to set this bit to date). The OS needs to set it
for any bridges not configured by system firmware.

MFC 238424:
Make the interval timings for EVFILT_TIMER more accurate. tvtohz() always
adds an extra tick to account for the current partial clock tick. However,
that is not appropriate for a repeating timer when the exact tvtohz() value
should be used for subsequent intervals. Fix repeating callouts for
EVFILT_TIMER by subtracting 1 tick from the tvtohz() result similar to the
fix used in realitexpire() for interval timers.

While here, update a few comments to note that if the EVFILT_TIMER code
were to move out of kern_event.c, it should move to kern_time.c (where the
interval timer code it mimics lives) rather than kern_timeout.c.

MFC 238311:
Add a clts() wrapper around the 'clts' instruction to <machine/cpufunc.h>
on x86 and use that to implement stop_emulating() in the fpu/npx code.
Reimplement start_emulating() in the non-XEN case by using load_cr0() and
rcr0() instead of the 'lmsw' and 'smsw' instructions. Intel explicitly
discourages the use of 'lmsw' and 'smsw' on 80386 and later processors in
the description of these instructions in Volume 2 of the ADM.

MFC 238142,238179:
Now that our assembler supports the xsave family of instructions, use them
natively rather than hand-assembled versions. For xgetbv/xsetbv, add a
wrapper API to deal with xcr* registers: rxcr() and load_xcr().

MFC 238310:
Partially revert r217515 so that the mem_range_softc variable is always
present on x86 kernels. This fixes the build of kernels that include
'device acpi' but do not include 'device mem'.

MFC 238109,238166:
Several fixes to the amd64 disassembler:
- Decode the 'xsave', 'xrstor', 'xsaveopt', 'xgetbv', 'xsetbv', and
'rdtscp' instructions.
- Add generic support for opcodes that are escape bytes used for
multi-byte opcodes (such as the 0x0f prefix). Use this to replace
the hard-coded 0x0f special case and add support for three-byte
opcodes that use the 0x0f38 prefix.
- Decode all Intel VMX instructions. invept and invvpid in particular are
three-byte opcodes that use the 0x0f38 escape prefix.
- Rework how the special 'SDEP' size flag works such that the default
instruction name (i_name) is the instruction when the data size
prefix (0x66) is not specified, and the alternate name in i_extra is
used when the prefix is included.
- Add a new 'ADEP' size flag similar to 'SDEP' except that it chooses
between i_name and i_extra based on the address size prefix (0x67).
Use this to fix the decoding for jrcxz vs jecxz which is determined
by the address size prefix, not the operand size prefix. Also, jcxz
is not possible in 64-bit mode, but jrcxz is the default instruction
for that opcode.
- Add support for handling instructions that have a mandatory 'rep'
prefix (this means not outputting the 'repe ' prefix until determining
if it is used as part of an opcode). Make 'pause' less of a special
case this way.
- Decode 'cmpxchg16b' and 'cdqe' which are variants of other instructions
but with a REX.W prefix.

MFC 238000,239584:
Honor db_pager_quit in 'show malloc', 'show uma', and 'show witness'.

MFC 238077:
Fix panics triggered by older mfiutil binaries run on the new mfi(4) driver.
The new driver changed the size of the mfi_dcmd_frame structure in such a
way that a MFI_IOC_PASSTHRU ioctl from an old amd64 binary is treated as an
MFI_IOC_PASSTHRU32 ioctl in the new driver. As a result, the user pointer
is treated as the buffer length. mfi_user_command() doesn't have a bounds
check on the buffer length, so it passes a really big value to malloc()
which panics when it tries to exhaust the kmem_map. Fix this two ways:
- Only honor MFI_IOC_PASSTHRU32 if the binary has the SV_ILP32 flag set,
otherwise treat it as an unknown ioctl.
- Add a bounds check on the buffer length passed by the user. For now
it fails any user attempts to use a buffer larger than 1MB.

While here, fix a few other nits:
- Remove an unnecessary check for a NULL return from malloc(M_WAITOK).
- Use the ENOTTY errno for invalid ioctl commands instead of ENOENT.

MFC 237338:
Don't return an error if a kld does not contain any modules (e.g. a
kld that only contained a sysctl). The kernel linker allows such
modules, so the boot loader should not reject them.

MFC 230782,237274:
Refine the implementation of POSIX_FADV_NOREUSE to perform
POSIX_FADV_DONTNEED requests on the currently accessed portion of the
file on each read(2) or write(2) rather than using direct I/O. This
gives much better performance including read-ahead and write clustering
similar to normal read(2) and write(2) calls.

If subsequent read(2) and write(2) calls are sequential, then the
POSIX_FADV_DONTNEED requests will cover the entire sequentially-accessed
range.

MFC 233040,233198,233870,234183:
Add OFED and the associated options and drivers to x86 LINT builds:
- Fix build with INET6 disabled.
- Fix build of OFED bits with debugging options enabled.
- Fix build on i386.
- Mark 'sdp' as requiring 'inet'.
- Always include "opt_inet.h" and "opt_inet6.h" and modify the IB
driver Makefiles to honor WITH/WITHOUT_INET/INET6/_SUPPORT options
to determine what should be enabled during a module build.
- Fix the mlxen(4) driver and the core IB code to compile without
if INET is disabled (including when both INET and INET6 are disabled).

MFC 233872:
Add descriptions after the 'device' line for several NICs to match the
existing style.

MFC r238502:

Fix a bug with memguard(9) on 32-bit architectures without a
VM_KMEM_MAX_SIZE.

The code was not taking into account the size of the kernel_map, which
the kmem_map is allocated from, so it could produce a sub-map size too
large to fit. The simplest solution is to ignore VM_KMEM_MAX entirely
and base the memguard map's size off the kernel_map's size, since this
is always relevant and always smaller.

Found by: Justin Hibbits

MFC: r239079

Merge r236494 from x86:

Isolate the global TTE list lock from data and other locks to prevent false
sharing within the cache.

MFC 238164:
Add another PS/2 keyboard PNP ID. This ID is listed as
"Reserved by Microsoft" in the standard PNP ID table, but has been seen
in the wild on at least one laptop.

Approved by: re (kib)

Pull the tier-2 card and change the sparc64 ZFS loader to no longer probe
all diskN aliases for providers (which more or less corresponds to how the
x86 version behaves) but instead probe only those listed in the boot-device
OFW environment variable. This has the following advantages:
- avoids otherwise unavoidable OFW warnings about failures to open disks
for which aliases exist but no actual hardware is connected
- avoids issues due to different diskN naming schemes
- aligns us with Solaris

Approved by: re (kib)

MFC: r238621

Revert the use of BUS_DMA_ALLOCNOW when creating the DMA tag for user
data introduced in r236061 (MFC'ed to stable/9 in r237186). Using that
flag doesn't make that much sense on this case as the DMA maps using
it are also created during sym_pci_attach(). Moreover, due to the
maxsegsz parameter used, doing so may exhaust the bounce pages pool
on architectures requiring bounce pages. [1]
While at it, use a slightly more appropriate maxsegsz parameter.

PR: 169526
Submitted by: Mike Watters [1]
Approved by: re (kib)

MFC r237852

Add the same check as vlan(4) where we ignore the ifnet departure event if the
interface is just being renamed.

PR: kern/169557

MFC: r237547

Fix size of the bcopy when extracting ethernet address

Obtained from: DragonFly

MFC: r237546

Correct sizeof usage

Obtained from: DragonFly

MFC: r237842

Switch back to the 4BSD scheduler for now. There is some more or less
recent regression with ULE, causing processes to get stuck in getblk
as well as interrupt handler execution delays to rise above the command
timeout of mpt(4).

Merge r237736 from HEAD:
Kick the current-state report timer when a V1 group report would
be triggered.

Submitted by: rpaulo@

Merge r237735 from HEAD:
Fix a typo in MLD query exponent processing.

Submitted by: rpaulo@

Merge r237734 from HEAD:
In MLDv2 general query processing, do not enforce the strict check
on query origins.

Submitted by: Gu Yong

MFC r236503: free wdog_kern_pat calls in post-panic paths from under
SW_WATCHDOG

MFC 235024,235029,235556,235834,235845:
Use MADT to match ACPI Processor objects to CPUs. MADT and DSDT/SSDTs may
list CPUs in different orders, especially for disabled logical cores. Now
we match ACPI IDs from the MADT with Processor objects, strictly order CPUs
accordingly, and ignore disabled cores. This prevents us from executing
methods for other CPUs, e. g., _PSS for disabled logical core, which may not
exist. Unfortunately, it is known that there are a few systems with buggy
BIOSes that do not have unique ACPI IDs for MADT and Processor objects. To
work around these problems, 'debug.acpi.cpu_unordered' tunable is added.
Set this to a non-zero value to restore the old behavior.

MFC: r236581

The loaddev environment variable is not modifiable once set, so it is not
update for ZFS. It seems that this does not really affect anything except
the help command. Nevertheless, rearrange things so loaddev is set only
once in all cases in order to get it right.
Pointed out by: avg

MFC 233191:
Fix madvise(MADV_WILLNEED) to properly handle individual mappings larger
than 4GB. Specifically, the inlined version of 'ptoa' of the the 'int'
count of pages overflowed on 64-bit platforms. While here, change
vm_object_madvise() to accept two vm_pindex_t parameters (start and end)
rather than a (start, count) tuple to match other VM APIs as suggested
by alc@.

MFC 237334:
Move the per-thread deferred user map entries list into a private list
in vm_map_process_deferred() which is then iterated to release map entries.
This avoids having a nested vm map unlock operation called from the loop
body attempt to recuse into vm_map_process_deferred(). This can happen if
the vm_map_remove() triggers the OOM killer.

MFC 237008,237271,237272,237673:
- Fix a couple of bugs that prevented windows in PCI-PCI bridges from
growing "downward" (moving the start address down). First, an off by
one error caused the end address to be moved down an extra alignment
chunk unnecessarily. Second, when aligning the new candidate starting
address, the wrong bits were masked off.
- Add a 'wmask' variable to hold the expression '(1ul << w->step) - 1' in
pcib_grow_window().
- For subtractively decoding bridges, don't try to grow windows but pass
the request up the tree in order to be on the safe side. Growing windows
in this case would mean to switch resources to positive decoding and
it's unclear how to correctly handle this. At least with ALi/ULi M5249
PCI-PCI bridges, this also just doesn't work out of the box.

MFC r235390: zfs boot code: mark spa_t arguments as const where they are
used as such

MFC r235392,235394,235395: fixes and cleanups for zfs boot MFC

MFC r235330: zfs boot: try to set vfs.root.mountfrom from currdev as a
fallback

MFC r235329,235343,235361,235364: zfsboot/zfsloader: support accessing
filesystems within a pool

MFC r235264: MFi386: improve argument passing via btxldr

MFC r235219: cdboot, pxeldr: make use of bootargs.h instead of
redefining flag constants

MFC r235156: i386 zfsloader: rename LIBZFS to LIBZFSBOOT

MFC r235155: i386 boot: consolidate MAXBDDEV definition

MFC r235153: sys/boot: add common CTASSERT definition

MFC 228161,230774,230822,236415:
Add a new -e flag to pciconf(8)'s list mode to display PCI error details.
Currently this dumps the status of any error bits in the PCI status register
and PCI-express device status register. It also lists any errors indicated
by version 1 of PCI-express Advanced Error Reporting (AER).

MFC 236405:
Remove unnecessary initializations. The BSS of boot2 is in fact
zero'd when boot2 begins execution by the _start() routine in btxcsu.S.

MFC 236404:
Extend VERBOSE_SYSINIT to also print out the name of variables passed
to SYSINIT routines if they can be resolved via symbol look up in DDB.
To avoid false positives, only honor a name if the symbol resolves
exactly to the pointer value (no offset).

MFC 234494:
Include the associated wait channel message for context switch ktrace
records. kdump supports both the old and new messages.

MFC r236178

if_lagg: allow to invoke SIOCSLAGGPORT multiple times in a row

MFC r236062

Turn LACP debugging from a compile time option to a sysctl, it is very handy to
be able to turn it on when negotiation to a switch misbehaves.

MFC 233925,236357:
Add new ktrace records for the start and end of VM faults. This gives
a pair of records similar to syscall entry and return that a user can
use to determine how long page faults take. The new ktrace records are
enabled via the 'p' trace type, but are not enabled in the default set of
trace points.

MFC: r237244
Fix the NFSv4 client for the case where mmap'd files are
written, but not msync'd by a process. A VOP_PUTPAGES()
called when VOP_RECLAIM() happens will usually fail, since
the NFSv4 Open has already been closed by VOP_INACTIVE().
Add a vm_object_page_clean() call to the NFSv4 client's
VOP_INACTIVE(), so that the write happens before the NFSv4
Open is closed. kib@ suggested using vgone() instead and
I will explore this, but this patch fixes things in the
meantime. For some reason, the VOP_PUTPAGES() is still
attaempted in VOP_RECLAIM(), but having this fail doesn't
cause any problems except a "stateid0 in write" being logged.

MFC: r237200
Move the nfsrpc_close() call in ncl_reclaim() for the NFSv4 client
to below the vnode_destroy_vobject() call, since that is where
writes are flushed.

MFC: r231616, r232497, r234337

Add __aeabi_read_tp function required for thread-local storage.

MFC 230063
Clean up a switch statement for uncore events on Westmere processors.

Submitted by: Davide Italiano
Reviewed by: gnn

MFC: r231617

Add ARM relocations types used for thread-local storage

Reviewed by: cognet

MFC: r235348, r236191, r236497

Add glue/support for the SAM9XE512-based Ethernut 5 boards. Currently,
all integrated and on-board peripherals except NAND Flash (missing NAND
framework/integration) are working.

MFC: r236496

- Loop up to 3 seconds when waiting for a device to get ready. [1]
- Make the device description match the driver name.
- Identify the chip variant based on the JEDEC and use that information
to use the proper values for page count, offset and size instead of
hardcoding a AT45DB642x with 2^N byte page support disabled.
- Take advantage of bioq_takefirst().
- Given that CONTINUOUS_ARRAY_READ_HF (0x0b) command isn't even mentioned
in Atmel's DataFlash Application Note, as suggested by the previous
comment may not work on all all devices and actually doesn't properly
on at least AT45DB321D (JEDEC 0x1f2701), rewrite at45d_task() to use
CONTINUOUS_ARRAY_READ (0xe8) for reading instead. This rewrite is laid
out in a way allowing to easily add support for BIO_DELETE later on.
- Add support for reads and writes not starting on a page boundary.
- Verify the flash content after writing.
- Let at45d_task() gracefully handle errors on SPI transfers and the
device not becoming ready afterwards again. [1]
- Use DEVMETHOD_END. [1]
- Use NULL instead of 0 for pointers. [1]

Additional testing by: Ian Lepore

Submitted by: Ian Lepore [1]

MFC: r237239

Revert the part of r236495 (MFC'ed to stable/9 in r237095) that
introduced checking of SPI_SR_TXEMPTY for TX transfer completion as
for reasons unknown this occasionally causes SPI_SR_RXBUFF and
SPI_SR_ENDRX to not rise.
In any case, once the RX part of the transfer is done it's obvious
that the preceding TX part had finished and checking of SPI_SR_TXEMPTY
was introduced to rule out a possible cause for the data corruption
mentioned in r236495 but which didn't turn out to be the problem
anyway.

MFC: r230242, r237102, r237236

- Add support for the FT2232 based egnite Turtelizer 2 JTAG/RS232 Adapter.
This includes adding support for skipping FTDI interfaces used for JTAG
leaving them for userland and just attaching to the RS232 half, similarly
to how the corresponding Linux drivers handles these kind of adapters.
While at it, sort uftdi_devs and return BUS_PROBE_SPECIFIC (because
uftdi_probe() alters the instance variables for better or worse as do
other probe routines of USB drivers) instead of 0.
- Remove duplicated entries for BeagleBone.
- Use DEVMETHOD_END.
- Use NULL instead of 0 for pointers.
- Remove some stray lines.

MFC of 236937

In softdep_setup_inomapdep() we may have to allocate both inodedep
and bmsafemap dependency structures in inodedep_lookup() and
bmsafemap_lookup() respectively. The setup of these structures must
be done while holding the soft-dependency mutex. If the inodedep is
allocated first, it may be freed in the I/O completion callback when
the mutex is released to allocate the bmsafemap. If the bmsafemap is
allocated first, it may be freed in the I/O completion callback when
the mutex is released to allocate the inodedep.

To resolve this problem, bmsafemap_lookup has had a parameter added
that allows a pre-malloc'ed bmsafemap to be passed in so that it does
not need to release the mutex to create a new bmsafemap. The
softdep_setup_inomapdep() routine pre-malloc's a bmsafemap dependency
before acquiring the mutex and starting to build the inodedep with a
call to inodedep_lookup(). The subsequent call to bmsafemap_lookup()
is passed this pre-allocated bmsafemap entry so that it need not
release the mutex if it needs to create a new one.

Reported by: Peter Holm
Tested by: Peter Holm

MFC of 236825

When synchronously syncing a device (MNT_WAIT), wait for buffers
to become available. Otherwise we may excessively spin and fail
with ``fsync: giving up on dirty''.

Reviewed by: kib
Tested by: Peter Holm

MFC r236246

Add device ids for the Winbond 83627DHG-P chip and set the registers to
trigger the keyboard reset line on timeout.

MFC r236245

Fix a typo in wbwd so that CRF5 is actually written to the data register
rather than the index register.

MFC: r237107

- As a baind-aid, disable ATAPI DMA when using ATA_CAM for these controllers
as well as it causes the kernel to hang during boot.
Reported and tested by: Kevin Oberman
- Use NULL instead of 0 for a pointer.

MFC: r230179

BeagleBone uses an FTDI chip with
an altered Product ID.

MFC: r237101

Fix a braino in r236469 (MFC'ed to stable/9 in r236468); the number of
DMA tags required for handling MAXPHYS should be based on PAGE_SIZE rather
than SYM_CONF_DMA_BOUNDARY.
While at it, reuse the SYM_CONF_MAX_SG macro for specifying the maximum
number of DMA tags so sym(4) itself doesn't size memory beyond what's
required for handling MAXPHYS.

PR: 168928

MFC r236916

Fix a panic I introduced in r234487, the bridge softc pointer is set to null
early in the detach so rearrange things not to explode.

Reported by: David Roffiaen, Gustau Perez Querol

MFC: r236495

- Prepend the device description with "AT91" to reflect its nature. [1]
- Move DMA tag and map creature to at91_spi_activate() where the other
resource allocation also lives. [1]
- Flesh out at91_spi_deactivate(). [1]
- Work around the "Software Reset must be Written Twice" erratum.
- For now, run the bus at the slowest speed possible in order to work
around data corruption on transit even seen with 9 MHz on ETHERNUT5
(15 MHz maximum) and AT45DB321D (20 MHz maximum). This also serves as
a poor man's work-around for the "NPCSx rises if no data data is to be
transmitted" erratum of RM9200. Being able to use the appropriate bus
speed would require:
1) Adding a proper work-around for the RM9200 bug consisting of taking
the chip select control away from the SPI peripheral and managing it
directly as a GPIO line.
2) Taking the maximum frequencies supported by the actual board and the
slave devices into account and basing the whole thing on the master
clock instead of hardcoding a divisor as previously done.
3) Fixing the above mentioned data corruption.
- KASSERT that TX/RX command and data sizes match on transfers.
- Introduce a mutex ensuring that only one child device is running a SPI
transfer at a time. [1]
- Add preliminary, #ifdef'ed out support for setting the chip select. [1]
- Use the RX instead of the TX commando size when setting up the RX side
of a transfer.
- For controllers having SPI_SR_TXEMPTY, i.e. !RM9200, also wait for the
completion of the TX part of transfers before stopping the whole thing
again.
- Use DEVMETHOD_END. [1]
- Use NULL instead of 0 for pointers. [1, partially]

Additional testing by: Ian Lepore

Submitted by: Ian Lepore [1]

MFC: r225882

Remove pointless semicolons after label

MFC: r236579

The workaround added in r151650 for handling firmwares that don't allow
a single device to be opened multiple times concurrently unfortunately
isn't sufficient with ZFS. This is due to the fact, that ZFS may open
different partitions of a single device simultaneously. So the best we
can do in this case is to cache the lastly used device path and close
and open devices in ofwd_strategy() as needed.

PR: 165025
Submitted by: Gavin Mu

MFC: r235381
Fix two cases in the new NFS server where a tsleep() is
used, when the code should actually protect the tested
variable with a mutex. Since the tsleep()s had a 10sec
timeout, the race would have only delayed the allocation
of a new clientid for a client. The sleeps will also
rarely occur, since having a callback in progress when
a client acquires a new clientid, is unlikely.
in practice, since having a callback in progress when
a fresh clientid is being acquired by a client is unlikely.

MFC r236371:
Remove unnecessary device_printfs.

MFC 233760:
Export some more useful info about shared memory objects to userland
via procstat(1) and fstat(1):
- Change shm file descriptors to track the pathname they are associated
with and add a shm_path() method to copy the path out to a caller-supplied
buffer.
- Use the fo_stat() method of shared memory objects and shm_path() to
export the path, mode, and size of a shared memory object via
struct kinfo_file.
- Add a struct shmstat to the libprocstat(3) interface along with a
procstat_get_shm_info() to export the mode and size of a shared memory
object.
- Change procstat to always print out the path for a given object if it
is valid.
- Teach fstat about shared memory objects and to display their path,
mode, and size.

MFC 228509,228620,228533:
Add a helper API to allow in-kernel code to map portions of shared memory
objects created by shm_open(2) into the kernel's address space. This
provides a convenient way for creating shared memory buffers between
userland and the kernel without requiring custom character devices.

MFC: r236491

Add missing prototypes. While at it, sort them alphabetically.

MFC: r236488

Take advantage of nitems().

MFC: r236486

Add nitems(), a macro for determining the number of elements in a
statically-allocated array.

Obtained from: OpenBSD (in principle)

MFC: r236489

MFC: r233666

Fix build after changes to trap headers.

MFC: r233635

Allow multiple inclusion of trap.h. This has always been broken, but
until recently never caused problems.

MFC: r236328

Try to finally get the point in time at which bge_add_sysctls() is called
right; it needs to be called before bge_can_use_msi() but in turn requires
bge_flags to be properly set.

Submitted by: yongari

MFC r235829: vm_pager_object_lookup: small performance optimization

MFC: r236156

- Fix some typos in mmc_acquire_bus() and mmc_send_csd().
- Fix some math errors in mmc_decode_csd_sd().
- Fix incorrect arguments to mmc_send_app_op_cond() in mmc_go_discovery().
- Add reporting of CSD for debug purposes.
- Add detection (and skipping) of password-locked cards.
- Add setting of block length on card if necessary.

Submitted by: Patrick Kelsey

MFC: r236070

Consistently use USB_PAGE_SIZE. Currently, this is cosmetic.

MFC: r236069, r236073

Make the VIA workaround actually do its intended job.

MFC: r236063

Remove extraneous empty lines.

Revert changes accidentally committed as part of r236468.

MFC: r236061

- When creating the DMA tag for user data, don't ask for more segments
than required for handling MAXPHYS and report the resulting maximum
I/O size to CAM instead of implicitly limiting it to DFLTPHYS.
- Move the variables of sym_action2() out of nested scope as required
by style(9) and remove extraneous curly braces.
- Replace a magic value for PCIR_COMMAND with the appropriate macro.
- Use DEVMETHOD_END.
- Use NULL instead of 0 for pointers.

Tested with a HBA donated by wilko.

MFC r235391: i386 bootinfo: re-arrange EFI fields for natural alignment
and packing

MFC 236212:

Update mxge(4) firmware to the latest version available from
Myricom (1.4.55).

Sponored by: Myricom, Inc.

MFC 235833:
Only check to see if a memory resource is a PCI ROM BAR when activating
and deactivating PCI resources. Previously, if a device had more than
48 MSI interrupts, then activating message 48 (which has a rid == PCIR_BIOS)
would incorrectly try to enable the PCI ROM BAR.

MFC 234501:
The amr(4) firmware contains a rather dubious "feature" where it
assumes for small buffers (< 64k) that the OS driver is actually using
a buffer rounded up to the next power of 2. It also assumes that the
buffer is at least 4k in size. Furthermore, there is at least one
known instance of megarc sending a request with a 12k buffer where the
firmware writes out a 24k-ish reply.

To workaround the data corruption triggered by this "feature", ensure
that buffers for user commands use a minimum size of 32k, and that
buffers between 32k and 64k use a 64k buffer.

MFC 234099:
Properly parse 40G media types from newer Mellanox adapters that are
40G capable. For now, map all 40G links to 40GBase-CR4.

MFC r235821:
Don't force max payload size to 128. Root complex and Endpoint will
negotiate with each other on the TLP payload size so blindly
forcing the size to 128 can cause a completion error which in turn
will stop device.

Reported by: Geans Pin < geanspin <> broadcom dot com >

MFC r235816:
Make IPMI work in the bce driver even when the interface is
configured down. Formerly, IPMI communication was lost whenever the
interface was not up. The reason was that the BCE_EMAC_MODE
register was not configured with the correct media settings. There
are two parts to the fix.

First, resetting the chip in bce_reset() causes the BCE_EMAC_MODE
register to be initialized to a default value that does not
necessarily correspond to the actual media settings. The fix
implemented here is a bit of a hack. Ideally, at the end of
bce_reset() we would poll the PHY to determine the negotiated media,
and then we would set the BCE_EMAC_MODE register accordingly. That
is difficult, since the PHY is abstracted behind the MII layer and is
not supposed to be queried directly from the MAC driver. Instead,
we read the BCE_EMAC_MODE register at the beginning of bce_reset()
and then restore its media bits to their original values before
returning. If IPMI is up and running, then the link is already
established and the BCE_EMAC_MODE register is already set appropriately
when bce_reset() is called. If IPMI is not running, no harm is
done by preserving the BCE_EMAC_MODE settings. The driver will set
the register properly once the interface is configured up and link
is established.

Second, bce_miibus_statchg() is sometimes called when the link is
down. In that case, the reported media settings are invalid.
Formerly, the driver used them anyway to setup the BCE_EMAC_MODE
register. We now avoid changing any MAC registers unless link is
active and the reported media settings are valid.

Submitted by: jdp
Tested by: jdp

MFC of 235610

Add missing `continue' statement at end of case.

Found by: Kevin Lo (kevlo@)

MFC: r234740
Fix a leak of namei lookup path buffers that occurs when a
ZFS volume is exported via the new NFS server. The leak occurred
because the new NFS server code didn't handle the case where
a file system sets the SAVENAME flag in its VOP_LOOKUP() and
ZFS does this for the DELETE case.

MFC: r235332
PR# 165923 reported intermittent write failures for dirty
memory mapped pages being written back on an NFS mount.
Since any thread can call VOP_PUTPAGES() to write back a
dirty page, the credentials of that thread may not have
write access to the file on an NFS server. (Often the uid
is 0, which may be mapped to "nobody" in the NFS server.)
Although there is no completely correct fix for this
(NFS servers check access on every write RPC instead of at
open/mmap time), this patch avoids the common cases by
holding onto a credential that recently opened the file
for writing and uses that credential for the write RPCs
being done by VOP_PUTPAGES() for both NFS clients.

MFC: r234524

o Fixes:
- When switching to 4-bit operation, send a SET_CLR_CARD_DETECT command
to disconnect the card-detect pull-up resistor from the DAT3 line before
sending the SET_BUS_WIDTH command.
- Add the missing "reserved" zero entry to the mantissa table used to
decode various CSD fields. This was causing SD cards to report that they
could run at 30 MHz instead of the maximum 25 MHz mandated in the spec.
o Enhancements:
- At the MMC layer, format various info from the CID into a string that
uniquely identifies the card instance (manufacturer number, serial
number, product name and revision, etc). Export it as an instance
variable.
- At the MMCSD layer, display the formatted card ID string, and also
report the clock speed of the hardware (not the card's max speed), and
the number of bits and number of blocks per transfer. It comes out like
this now:
mmcsd0: 968MB <SD SD01G 8.0 SN 276886905 MFG 08/2008 by 3 SD> at mmc0
22.5MHz/4bit/128-block
o Use DEVMETHOD_END.
o Use NULL instead of 0 for pointers.

PR: 156496
Submitted by: Ian Lepore

MFC: r234901

- Add missing locking in at91_usart_getc().
- Align the RX buffers on the cache line size, otherwise the requirement
of partial cache line flushes on every are pretty much guaranteed. [1]
- Make the code setting the RX timeout match its comment (apparently,
start and stop bits were missed in the previous calculation). [1]
- Cover the busdma operations in at91_usart_bus_{ipend,transmit}() with
the hardware mutex, too, so these don't race against each other.
- In at91_usart_bus_ipend(), reduce duplication in the code dealing with
TX interrupts.
- In at91_usart_bus_ipend(), turn the code dealing with RX interrupts
into an else-if cascade in order reduce its complexity and to improve
its run-time behavior.
- In at91_usart_bus_ipend(), add missing BUS_DMASYNC_PREREAD calls on
the RX buffer map before handing things over to the hardware again. [1]
- In at91_usart_bus_getsig(), used a variable of sufficient width for
storing the contents of USART_CSR.
- Use KOBJMETHOD_END.
- Remove an unused header.

Submitted by: Ian Lepore [1]
Reviewed by: Ian Lepore

MFC: r234561

Interrupts must be disabled while handling a partial cache line flush,
as otherwise the interrupt handling code may modify data in the non-DMA
part of the cache line while we have it stashed away in the temporary
stack buffer, then we end up restoring a stale value.

PR: 160431
Submitted by: Ian Lepore

MFC: r234560

- Add support for MCI1 revision 2xx controllers and a work-around for their
"Data Write Operation and number of bytes" erratum.
- Use DEVMETHOD_END.
- Use NULL instead of 0 for pointers.

MFC: r234293

Generate an obviously missing STOP when having finished transmitting data.
This fixes communication with PCF8563.

MFC: r234291, r234292

Add support for the Atmel SAM9XE family of microcontrollers, which
consist of a ARM926EJ-S processor core with up to 512 Kbytes of on-chip
flash. Tested with SAM9XE512.

MFC: r234281

- Try to bring these files closer to style(9).
- Use DEVMETHOD_END.
- Use NULL instead of 0 for pointers.

MFC: r234248

Add a driver for the NXP (Philips) PCF8563 RTC.

Obtained from: NetBSD (pcf8563reg.h)

MFC: r234898, r235207

Add initial support for booting from ZFS on sparc64. At least on Sun Fire
V100, the firmware is known to be broken and not allowing to simultaneously
open disk devices, causing attempts to boot from a mirror or RAIDZ to cause
a crash. This will be worked around later. The firmwares of newer sun4u models
don't seem to exhibit this problem though.

PR: 165025
Submitted by: Gavin Mu

MFC r234936 (emaste)

Relax restriction on direct tx to child ports

Lagg(4) restricts the type of packet that may be sent directly to a child
port, to avoid undesired output from accidental misconfiguration.
Previously only ETHERTYPE_PAE was permitted.

BPF writes to a lagg(4) child port are presumably intentional, so just
allow them, while still blocking other packets that should take the
aggregation path.

PR: kern/138620

MFC r231130 (pjd)

Allow to set if_bridge(4) sysctls from /boot/loader.conf.

MFC r235147

Do not reinitialise the interface if it is already running, this prevents the
bootp+nfs code from working as it calls init on each dhcp send and rx fails to
start in time.

MFC r235144

The DEVICE_POLLING dereference of sc->tsec_ifp needs to be checked for null
first or this will panic. Condense three blocks that check sc->tsec_ifp into
one while I am here.

MFC r234488

Move the interface media check to a taskqueue, some interfaces (usb) sleep
during SIOCGIFMEDIA and we were holding locks.

MFC r234487

Add linkstate to bridge(4), set the link to up when at least one underlying
interface is up, otherwise the link is down.

This, among other things, allows carp to work on a bridge.

MFC r234163

Set the proto to LAGG_PROTO_NONE before calling the detach routine so packets
are discarded, this is an issue because lacp drops the lock which may allow
network threads to access freed memory. Expand the lock coverage so the
detach/attach happen atomically.

Submitted by: Andrew Boyer (earlier version)

MFC r232629,r232640

Add the ability to set which packet layers are used for the load balance hash
calculation.

MFC r232118

Only look for a usable MAC address for the bridge ID from ports within our
bridge, this allows us to have more than one independent bridge in the same
STP domain.

PR: kern/164369
Submitted by: Nikos Vassiliadis (earlier version)

MFC r232014,r232030,r232070

- bstp_input() always consumes the packet so remove the mbuf handling dance
around it.
- Now that network interfaces advertise if they support linkstate notifications
we do not need to perform a media ioctl every 15 seconds.
- Indicate this function decrements the timer as well as testing for expiry.

MFC: r235681

Rewrite nd6_sysctl_{d,p}rlist() to avoid misaligned accesses to char arrays
casted to structs by getting rid of these buffers entirely. In r169832, it
was tried to paper over this issue by 32-bit aligning the buffers. Depending
on compiler optimizations that still was insufficient for 64-bit architectures
with strong alignment requirements though.
While at it, add comments regarding the total lack of locking in this area.

Tested by: bz
Reviewed by: bz (slightly earlier version), yongari (earlier version)

MFC: r234897

Add a command for showing the heap usage.

PR: 165025
Submitted by: Gavin Mu

MFC: r234789

Add multiple inclusion protection.

PR: 165025
Submitted by: Gavin Mu

MFC: r234348

Turn on PREEMPTION by default. After fixing several bugs over time, the
last show-stopper keeping PREEMPTION from being usable on sparc64 should
have been dealt with in r230662 (MFC'ed to stable/9 in r230662).
At least on 2-way systems, PREEMPTION causes a little bit of a degradation
in worldstone performance. However, FreeBSD seems to have started building
up regressions in !PREEMPTION cases so sparc64 better should not be an
oddball in this regard.

MFC: r235568
A problem with the NFSv4 server was reported by Andrew Leonard
to freebsd-fs@, where the setfacl of an NFSv4 acl would fail.
This was caused by the VOP_ACLCHECK() call for ZFS replying
EOPNOTSUPP. After discussion with rwatson@, it was determined
that a call to VOP_ACLCHECK() before doing VOP_SETACL() is not
required. This patch fixes the problem by deleting the
VOP_ACLCHECK() call.

MFC r235151:
Implement basic remote PHY support. Remote PHY allows the
controller to perform MDIO type accesses to a remote transceiver
using message pages defined through MRBE(multirate backplane
ethernet). It's used in blade systems(e.g Dell Blade m610) which
are connected to pass-through blades rather than traditional
switches.
This change directly manipulates firmware's mailboxes to control
remote PHY such that it does not use mii(4). Alternatively, as
David said, it could be implemented in brgphy(4) by creating a fake
PHY and let brgphy(4) do necessary mii accesses and bce(4) can
implement mailbox accesses based on the type of brgphy(4)'s mii
accesses. Personally, I think it would make brgphy(4) hard to
maintain since it would have to access many bce(4) registers in
brgphy(4). Given that there are users who are suffering from lack
of remote PHY support, it would be better to get working system
rather than waiting for complete/perfect implementation.

MFC 234098:
Add media types for 40G media that might be used with FreeBSD.

Repair mergeinfo from merges done into a sparse checkout.

MFC 234190,234196,234280:
- Extend the KDB interface to add a per-debugger callback to print a
backtrace for an arbitrary thread (rather than the calling thread).
A kdb_backtrace_thread() wrapper function uses the configured debugger
if possible, otherwise it falls back to using stack(9) if that is
available.
- Replace a direct call to db_trace_thread() in propagate_priority()
with a call to kdb_backtrace_thread() instead.

Move mergeinfo from sys/kern/subr_witness.c up to sys/.

MFC of 234386, 234400, 234441, 234443, 234482, 234483, 235052, 235241,
235246, and 235619

MFC: 234386

Replace the MNT_VNODE_FOREACH interface with MNT_VNODE_FOREACH_ALL.
The primary changes are that the user of the interface no longer
needs to manage the mount-mutex locking and that the vnode that
is returned has its mutex locked (thus avoiding the need to check
to see if its is DOOMED or other possible end of life senarios).

To minimize compatibility issues for third-party developers, the
old MNT_VNODE_FOREACH interface will remain available so that this
change can be MFC'ed to 9. Following the MFC to 9, MNT_VNODE_FOREACH
will be removed in head.

The reason for this update is to prepare for the addition of the
MNT_VNODE_FOREACH_ACTIVE interface that will loop over just the
active vnodes associated with a mount point (typically less than
1% of the vnodes associated with the mount point).

Reviewed by: kib
Tested by: Peter Holm
MFC after: 2 weeks

MFC: 234400

Drop export of vdestroy() function from kern/vfs_subr.c as it is
used only as a helper function in that file. Replace sole call to
vbusy() with inline code in vholdl(). Replace sole calls to vfree()
and vdestroy() with inline code in vdropl().

The Clang compiler already inlines these functions, so they do not
show up in a kernel backtrace which is confusing. Also you cannot
set their frame in kgdb which means that it is impossible to view
their local variables. So, while the produced code is unchanged,
the debugging should be easier.

Discussed with: kib
MFC after: 2 weeks

MFC: 234441

Fix a memory leak of M_VNODE_MARKER introduced in 234386.

Found by: Peter Holm

MFC: 234443

Delete a no longer useful VNASSERT missed during changes in 234400.

Suggested by: kib

MFC: 234482

This change creates a new list of active vnodes associated with
a mount point. Active vnodes are those with a non-zero use or hold
count, e.g., those vnodes that are not on the free list. Note that
this list is in addition to the list of all the vnodes associated
with a mount point.

To avoid adding another set of linkage pointers to the vnode
structure, the active list uses the existing linkage pointers
used by the free list (previously named v_freelist, now renamed
v_actfreelist).

This update adds the MNT_VNODE_FOREACH_ACTIVE interface that loops
over just the active vnodes associated with a mount point (typically
less than 1% of the vnodes associated with the mount point).

Reviewed by: kib
Tested by: Peter Holm
MFC after: 2 weeks

MFC: 234483

This update uses the MNT_VNODE_FOREACH_ACTIVE interface that loops
over just the active vnodes associated with a mount point to replace
MNT_VNODE_FOREACH_ALL in the vfs_msync, ffs_sync_lazy, and qsync
routines.

The vfs_msync routine is run every 30 seconds for every writably
mounted filesystem. It ensures that any files mmap'ed from the
filesystem with modified pages have those pages queued to be
written back to the file from which they are mapped.

The ffs_lazy_sync and qsync routines are run every 30 seconds for
every writably mounted UFS/FFS filesystem. The ffs_lazy_sync routine
ensures that any files that have been accessed in the previous
30 seconds have had their access times queued for updating in the
filesystem. The qsync routine ensures that any files with modified
quotas have those quotas queued to be written back to their
associated quota file.

In a system configured with 250,000 vnodes, less than 1000 are
typically active at any point in time. Prior to this change all
250,000 vnodes would be locked and inspected twice every minute
by the syncer. For UFS/FFS filesystems they would be locked and
inspected six times every minute (twice by each of these three
routines since each of these routines does its own pass over the
vnodes associated with a mount point). With this change the syncer
now locks and inspects only the tiny set of vnodes that are active.

Reviewed by: kib
Tested by: Peter Holm
MFC after: 2 weeks

MFC: 235052 (by pluknet)

Fix mount mutex handling missed in r234386.

MFC: 235241 (by pluknet)

Fix mount interlock oversights from the previous change in r234386.

Reported by: dougb
Submitted by: Mateusz Guzik <mjguzik at gmail com>
Reviewed by: Kirk McKusick
Tested by: pho

MFC: 235246

Fix mount mutex handling missed in r234386.

MFC: 235619

Update comment to document that the vnode free-list mutex needs to be
held when updating mnt_activevnodelist and mnt_activevnodelistsize.

MFC 234186
If a linker file contains at least one module, but all of the modules
fail to load (the MOD_LOAD event fails) during a kldload(2), unload the
linker file and fail the kldload(2) with ENOEXEC.

MFC 234182:
Don't update if_obytes when transmitting packets. That is already done
in IFQ_HANDOFF() when the packet is passed to the start routine, so doing
it here resulted in double counting.

MFC 235563:
Don't expose i386-only ptrace constants on amd64. This broke gdb with
libthread_db on amd64.

MFC 234152:
Allow device_busy() and device_unbusy() to be invoked while a device is
being attached. This is implemented by adding a new DS_ATTACHING state
while a device's DEVICE_ATTACH() method is being invoked. A driver is
required to not fail an attach of a busy device. The device's state will
be promoted to DS_BUSY rather than DS_ACTIVE() if the device was marked
busy during DEVICE_ATTACH()

MFC 233709,233781,233793:
- Don't malloc() new MCA records for machine checks logged due to a
CMCI or MC# exception. Instead, use a pre-allocated pool of records.
When a CMCI or MC# exception fires, schedule a task to refill the pool.
The pool is sized to hold at least one record per available machine
bank, and one record per CPU. This should handle the case of all CPUs
triggering a single bank at once as well as the case a single CPU
triggering all of its banks. The periodic scans still use malloc()
since they are run from a safe context.
- Make machine check exception logging more readable. On newer Intel systems,
an uncorrected ECC error tends to fire on all CPUs in a package
simultaneously and the current printf hacks are not sufficient to make
the messages legible. Instead, use the existing mca_lock spinlock to
serialize calls to mca_log() and change the machine check code to panic
directly when an unrecoverable error is encoutered rather than falling
back to a trap_fatal() call in trap() (which adds nearly a screen-full of
logging messages that aren't useful for machine checks).

MFC r230643: stop_scheduler -> td_stopsched

MFC r234489:

The value of flags matching VNOVAL can't be supported. Return EOPNOTSUPP
from setfflags() in this case. This fixes the return value of
chflags(path, -1).

MFC r235119:
Restore jumbo frame configuration which was broken in r218423.

Submitted by: Andrey Zonov <andrey <> zonov dot org > (initial version)
Tested by: Andrey Zonov <andrey <> zonov dot org >

MFC: r235255

- Change the module order of these MAC drivers to be last so they are
deterministically handled after the corresponding PHY drivers when
loaded as modules. Otherwise, when these MAC/PHY driver pairs are
compiled into a single module probing the PHY driver may fail. This
makes r151438 and r226154 actually work. [1]
Reported and tested by: yongari (fxp(4))
- Use DEVMETHOD_END.
- Use NULL instead of 0 for pointers.

Submitted by: jhb [1]

MFC r229854: enable stop_scheduler_on_panic by default

MFC r228765: ukbd: adjust for SCHEDULER_STOPPED() and overhaul locking code

MFC r228760: adapt usb transfer code for SCHEDULER_STOPPED

MFC r228767: sc_cngrab: switch to console vty when possible

MFC r228644: syscons: provide a first iteration of cngrab/cnungrab
implementation

MFC r228633,228634,228638,228642,228643: introduce cngets,
a method for kernel to read a string from console

MFC r228632: introduce cngrab/cnungrab stub calls in some places where they
make sense

MFC r228631: kern cons: introduce infrastructure for console grabbing by
kernel

MFC r228424,228448: panic: add a switch and infrastructure for stopping
other CPUs in SMP case

MFC: r234742
It was reported via email that some non-FreeBSD NFS servers
do not include file attributes in the reply to an NFS create RPC
under certain circumstances.
This resulted in a vnode of type VNON that was not usable.
This patch adds an NFS getattr RPC to nfs_create() for this case,
to fix the problem. It was tested by the person that reported
the problem and confirmed to fix this case for their server.

MFC r233961,234074,234105,234564,234723,234989,235231-235232:
- Fix a bug in x86 and sparc64 where the intr cpu mask does only
enables the BSP.
- Remove the SMP constraints for intr_add_cpu

MFC r234104:

Apply changes from r233787 to ext2fs:

- Use more natural ip->i_flags instead of vap->va_flags in the final
flags check.
- Style improvements.

No functional change intended.

MFC r234139:

Restore the blank line incorrectly removed in r234104.

MFC r233875: Add a check for unsupported file flags to ufs_setattr().

MFC r233787:

- Use more natural ip->i_flags instead of vap->va_flags in the final
flags check.
- Add a comment for the immutable/append check done after handling of
the flags.
- Style improvements.

No functional change intended.

MFC r234421:

The part about exec atime no longer applies in the comment.

MFC r234038
If a page belonging a reservation is cached, then mark the reservation so
that it will be freed to the cache pool rather than the default pool.
Otherwise, the cached pages within the reservation may be recycled sooner
than necessary.

MFC r234039
Fix mincore(2) so that it reports PG_CACHED pages as resident.

MFC r233097
With the changes over the past year to how accesses to the page's dirty
field are synchronized, there is no need for pmap_protect() to acquire
the page queues lock unless it is going to access the pv lists.

MFC r234554
Updates to the vm_page's flags no longer require the page queues lock.

MFC r232166
Simplify vm_mmap()'s control flow.

Add a comment describing what vm_mmap_to_errno() does.

MFC r234339: zfsboot: honor -q if it's present in boot.config

MFC r234338: intpm: add ATI IXP400 pci id

PR: kern/136762

MFC r234208: add actual interrupt counters to back ipi_invlcache_counts

MFC r234207: bump INTRCNT_COUNT values to reflect actual numbers of IPI counters

MFC r234121:
Back out r228476.
r228476 fixed superfluous link UP/DOWN messages but broke IPMI
access during boot. It's not clear why r228476 breaks IPMI and
should be revisited.

Reported by: Paul Guyot <paulguyot <> ieee dot org >

MFC r227730:

Initial version of cesa(4) driver for Marvell crypto engine and security
accelerator.

The following algorithms and schemes are supported:
- 3DES, AES, DES
- MD5, SHA1

Obtained from: Semihalf
Written by: Piotr Ziecik

MFC r233230, r233323:

Improve device tree blob (DTB) handling in loader(8).

Enable using the statically embedded blob from the kernel, if present. The
KLD loaded DTB takes precedence, but they are both recognized and handled in
the same way.

Improve FDT handling in loader(8) and make it more robust.

o Fix buffer overflows when using a long property body in node paths.
o Fix loop end condition when iterating through the symbol table.
o Better error handling during node modification, better problem reporting.
o Eliminate build time warnings.

Submitted by: Lukasz Wojcik
Obtained from: Semihalf

MFC of r227310:

Don astbestos garment and remove the warning about TMPFS being experimental
-- highly experimental even. So far the closest to a bug in TMPFS that people
have gotten to relates to how ZFS can take away from the memory that TMPFS
needs. One can argue that such is not a bug in TMPFS. Irrespective, even if
there is a bug here and there in TMPFS, it's not in our own advantage to
scare people away from using TMPFS. I for one have been using it, even with
ZFS, very successfully.

Reviewed by: marcel

MFC: r234247

Merge from x86:

r233961:

Fix interrupt load balancing regression, introduced in revision
222813, that left all un-pinned interrupts assigned to CPU 0.
In intr_shuffle_irqs(), remove CPU_SETOF() call that initialized
the "intr_cpus" cpuset to only contain CPU0.

This initialization is too late and nullifies the results of calls
to the intr_add_cpu() that occur much earlier in the boot process.

r234074 (partial):

The BSP is not added to the mask of valid target CPUs for interrupts.
Fix this by adding the BSP as an interrupt target directly in

r234105:

Fix !SMP build after r234074.

MFC of 234158:

Export vinactive() from kern/vfs_subr.c (e.g., make it no longer
static and declare its prototype in sys/vnode.h) so that it can be
called from process_deferred_inactive() (in ufs/ffs/ffs_snapshot.c)
instead of the body of vinactive() being cut and pasted into
process_deferred_inactive().

Reviewed by: kib
MFC after: 2 weeks

MFC of 234157:

Whitespace cleanup.

MFC of 233817:

A file cannot be deallocated until its last name has been removed
and it is no longer referenced by a user process. The inode for a
file whose name has been removed, but is still referenced at the
time of a crash will still be allocated in the filesystem, but will
have no references (e.g., they will have no names referencing them
from any directory).

With traditional soft updates these unreferenced inodes will be
found and reclaimed when the background fsck is run. When using
journaled soft updates, the kernel must keep track of these inodes
so that it can find and reclaim them during the cleanup process.
Their existence cannot be stored in the journal as the journal only
handles short-term events, and they may persist for days. So, they
are tracked by keeping them in a linked list whose head pointer is
stored in the superblock. The journal tracks them only until their
linked list pointers have been commited to disk. Part of the cleanup
process involves traversing the list of unreferenced inodes and
reclaiming them.

This bug was triggered when confusion arose in the commit steps
of keeping the unreferenced-inode linked list coherent on disk.
Notably, a race between the link() system call adding a link-count
to a file and the unlink() system call removing a link-count to
the file. Here if the unlink() ran after link() had looked up
the file but before link() had incremented the link-count of the
file, the file's link-count would drop to zero before the link()
incremented it back up to one. If the file was referenced by a
user process, the first transition through zero made it appear
that it should be added to the unreferenced-inode list when in
fact it should not have been added. If the new name created by
link() was deleted within a few seconds (with the file still
referenced by a user process) it would legitimately be a candidate
for addition to the unreferenced-inode list. The result was that
there were two attempts to add the same inode to the unreferenced-inode
list which scrambled the unreferenced-inode list's pointers leading
to a panic. The fix is to detect and avoid the false attempt at
adding it to the unreferenced-inode list by having the link()
system call check to see if the link count is zero before it
increments it. If it is, the link() fails with ENOENT (showing that
it has failed the link()/unlink() race).

While tracking down this bug, we have added additional assertions
to detect the problem sooner and also simplified some of the code.

Reported by: Kirk Russell
Fix submitted by: Jeff Roberson
Tested by: Peter Holm
PR: kern/159971

MFC of 233627, 234024, 234025, and 234026

Restore per mount-point counts of synchronous and asynchronous
reads and writes associated with UFS and MSDOS filesystems.

MFS 233627:
Keep track of the mount point associated with a special device
to enable the collection of counts of synchronous and asynchronous
reads and writes for its associated filesystem. The counts are
displayed using `mount -v'.

Ensure that buffers used for paging indicate the vnode from
which they are operating so that counts of paging I/O operations
from the filesystem are collected.

This checkin only adds the setting of the mount point for the
UFS/FFS filesystem, but it would be trivial to add the setting
and clearing of the mount point at filesystem mount/unmount
time for other filesystems too.

Reviewed by: kib

MFC 234024:
Drop an unnecessary setting of si_mountpt when updating a UFS mount point.
Clearly it must have been set when the mount was done.

Reviewed by: kib

MFC 234025:
Add I/O accounting to msdos filesystem.

Suggested and reviewed by: kib

MFC 234026:
Expand locking around identification of filesystem mount point when
accounting for I/O counts at completion of I/O operation. Also switch
from using global devmtx to vnode mutex to reduce contention.

Suggested and reviewed by: kib

MFC 234059:
Recognize the RDRAND instruction feature.

MFC r232799:
- add comments to syscalls.master and linux(32)_dummy about which linux
kernel version introduced the sysctl (based upon a linux man-page)
- add comments to syscalls.master regarding some names of sysctls which are
different than the linux-names (based upon the linux unistd.h)
- add some dummy sysctls
- name an unimplemented sysctl

MFC: r233888

- Const'ify the device lookup-table.
- Use DEVMETHOD_END.
- Use NULL instead of 0 for pointers.
- Enable support for flow control.
Tested by: yongari

MFC r234043: intpm: return only SMB bus error codes from SMB methods

MFC r234040: intpm: reflect the fact that SB800 and later AMD chipsets
are not supported

MFC r233918: zfs_ioctl: no need for ddi_copyin/out here because
sys_ioctl handles that

MFC r231558: newvers.sh: unbreak git/git-svn support

MFC 233547:
Use VM_MEMATTR_UNCACHEABLE instead of VM_MEMATTR_UNCACHED for UC mappings.
VM_MEMATTR_UNCACHED is actually the x86-specific UC- mode (where a WC
MTRR can override the PAT setting).

MFC 233670,233671:
- Use VM_MEMATTR_UNCACHEABLE for the constant for UC memory rather than
VM_MEMATTR_UNCACHED on mips.
- Rename VM_MEMATTR_UNCACHED to VM_MEMATTR_WEAK_UNCACHEABLE on x86 to
be less ambiguous and more clearly identify what it means. An alias
from VM_MEMATTR_WEAK_UNCACHEABLE to VM_MEMATTR_WEAK_UNCACHED remains
on x86 to preserve the KPI.
- Remove the VM_MEMATTR_UNCACHED alias from powerpc.

MFC 233676:
Use a more proper fix for enabling HT MSI mapping windows on Host-PCI
bridges. Rather than blindly enabling the windows on all of them, only
enable the window when an MSI interrupt is enabled for a device behind
the bridge, similar to what already happens for HT PCI-PCI bridges.

MFC 233893:
Disable INET6 support in modules when building the LINT-NOINET6 kernel.

MFC 233305,233623:
- Mark the 'lapics' and 'ioapics' arrays here static since they are
private to this file. The 'lapics' array was actually shadowing a
completely different 'lapics' array that is private to local_apic.c.
- Allocate the ioapics[] array dynamically since it is only needed for the
duration of madt_setup_io(). This avoids having the array take up
permanent space in the BSS.

MFC 232228,233613:
Move the DTrace return IDT vector back up from 0x20 to 0x92. The 0x20
vector is currently dedicated to servicing IRQ 0 from the 8259A's, so
it shouldn't be overloaded for DTrace.

MFC 232744,232747,233031:
- Allow a native i386 kernel to be built with 'nodevice atpic'. Just as on
amd64, if 'device isa' is present quiesce the 8259A's during boot and
resume from suspend.
- Move i386's intr_machdep.c to the x86 tree and share it with amd64.
- Merge r232744 changes to pc98.
(Allow a kernel to be built with 'nodevice atpic'.)
- Move ICU related defines from x86/isa/atpic.c to x86/isa/icu.h and
use them in x86/x86/intr_machdep.c.

Note, I normally would have merged 232747 separately, but 233031 assumed
232747 was already merged and 232744 needs to be merged with 233031.

MFC 232742:
MFamd64:
- Return failure for a suspend attempt if we have no wake address.
- Use intr_disable()/intr_restore() instead of ACPI_DISABLE_IRQS().
- Invoke intr_suspend() earlier and call intr_resume() if suspend
fails.
- Use pause in the loop waiting for CPU to suspend.
- Restore PAT MSR, switchtime, switchticks, and MTRRs on resume.

MFC r233688-233689:
r233688:
Remove task queue based link state change handler. Driver no longer
needs to defer link state handling.
While I'm here, mark IFF_DRV_RUNNING before changing media. If
link is established without any delay, that link state change
handling could be lost.

r233689:
Do not report current link status if driver is not running.
This change also workarounds dhclient's link state handling bug by
not giving current link status.

Unlike other controllers, ale(4)'s PHY hibernation perfectly works
such that driver does not see a valid link if the controller is not
brought up. If dhclient(8) runs on ale(4) it will blindly waits
until link UP and then gives up after 10 seconds. Because
dhclient(8) still thinks interface got a valid link when IFM_AVALID
is not set for selected media, this change makes dhclient initiate
DHCP without waiting for link UP.

MFC r233585-233587:
r233585:
Partially revert r223608 and selectively allow microcode loading
for 82550C. For 82550 controllers this change restores CPUSaver
microcode loading. Due to silicon bug on 82550 and 82550C with
server extension, these controllers seem to require CPUSaver
microcode to receive fragmented UDP datagrams. However the
microcode shouldn't be used on client featured 82550C as it locks
up the controller. In addition, client featured 82550C does not
have the silicon bug. Also clear temporary memory used for
microcode loading since the same memory area is used for other
commands.
While I'm here use 82550C in probe message instead of generic
82550.

Reported by: Andreas Longwitz <longwitz <> incore de>
Tested by: Andreas Longwitz <longwitz <> incore de>

r233586:
Load entire EEPROM contents in device attach time and verify
whether the checksum of EEPROM is valid or not. Because driver
heavily relies on EEPROM information when it selectively enables
features/workarounds, it would be helpful to know whether driver
sees valid EEPROM.
While I'm here remove all other EEPROM accesses since the entire
EEPROM is loaded at device attach time.

r233587:
Remove unnecessary #if as the software workaround for PCI protocol
violation should be activated unless the system is cold-booted
after updating EEPROM.
The PCI protocol violation happens only when established link is
10Mbps so the workaround should be updated whenever link state
change is detected. Previously the workaround was activated only
when user checks current media status with ifconfig(8).

MFC: r233827

Fix probing of SAS1068E with a device ID of 0x0059 after r232411 (MFC'ed
to stable/9 in r232562).
Reported by: infofarmer

MFC: r233886

Refine r233827; as it turns out, controllers with a device ID of 0x0059
can be upgraded to MegaRAID mode, in which case mfi(4) should attach to
these based on the sub-vendor and -device ID instead (not currently done).
Therefore, let mpt_pci_probe() return BUS_PROBE_LOW_PRIORITY.
While it, let mpt_pci_probe() return BUS_PROBE_DEFAULT instead of 0 in
the default case.

MFC 233675:
Restore proper use of bounce buffers for ISA DMA. When locking was
added, the call to pmap_kextract() was moved up, and as a result the
code never updated the physical address to use for DMA if a bounce
buffer was used. Restore the earlier location of pmap_kextract() so
it takes bounce buffers into account.

MFC: r233747, r233748

- Fix panic on kernel traps having a mapping in trap_sig b0rked in r206086.
Reported by: David E. Cross
- Remove checks that are redundant due to tf_type being unsigned.

MFC 233236: Add sys/ofed to the 'make cscope' target.

MFC 232700:
Add a new sched_clear_name() method to the scheduler interface to clear
the cached name used for KTR_SCHED traces when a thread's name changes.
This way KTR_SCHED traces (and thus schedgraph) will notice when a thread's
name changes, most commonly via execve().

MFC: r233701

- Remove erroneous trailing semicolon. [1]
- Correctly determine the maximum payload size for setting the TX link
frequent NACK latency and replay timer thresholds.

Submitted by: stefanf [1]

MFC: 232612

Properly mask off bits that are not supported in the IAP counters.
This fixes a bug where users would see massively large counts, near
to 2**64 -1, due to the bits not being cleared.

MFC r233256
Eliminate vm.pmap.shpgperproc and vm.pmap.pv_entry_max because they no
longer serve any purpose.

MFC: r233427

Add cas(4), gem(4) and hme(4) to x86 GENERICs as suggested by netchild@ in
<20120222095239.Horde.0hpYHJjmRSRPRKzXsoFRbYk@webmail.leidinger.net>.
According to some private emails received, it apparently is not unpopular
to use at least Quad GigaSwift cards driven by cas(4) in x86 machines.

MFC: r233425

Consistently update to the MPI header set version 01.05.20 after r224761.
Requested by: mjacob

MFC: r233423

Initialize the mutexes used for the NVM and the swflag as MTX_DUPOK in
order to avoid otherwise harmless witness warnings when these are acquired
at the same time and due to both using MTX_NETWORK_LOCK as their type.
The right fix actually would be to use different, descriptive types for
these. However, the latter would require undesirable changes to the shared
code base. Another approach would be to just supply NULL as the type, which
was deemed as less desirable though as it would cause the unique but cryptic
name also to be used for the type and to diverge from the type used by other
network device drivers.

MFC: r233421

Given that this is a host-PCI-Express bridge driver, create the parent
DMA tag with a 4 GB boundary as required by PCI-Express. With r232403
(MFC'ed to stable/9 in r233393) in place this actually is redundant.
However, the host-PCI-Express bridge driver is the more appropriate
place for implementing this restriction.

MFC: r233403, r233404

- Use the PCI ID macros from mpi_cnfg.h rather than duplicating them here.
Note that this driver additionally probes some device IDs for the most
part not know to other MPT drivers, if at all. So rename the macros not
present in mpi_cnfg.h to match the naming scheme in the latter and but
suffix them with a _FB in order to not cause conflicts.
- Like mpt_set_config_regs(), comment out mpt_read_config_regs() as the
content of the registers read isn't actually used and both functions
aren't exactly up to date regarding the possible layouts of the BARs
(these function might be helpful for debugging though, so don't remove
them completely).
- Use DEVMETHOD_END.
- Use NULL rather than 0 for pointers.
- Remove an unusual check for the softc being NULL.
- Remove redundant zeroing of the softc.
- Remove an overly banal and actually partly incorrect as well as partly
outdated comment regarding the allocation of the memory resource.

MFC: r233287

Use the common/shared CRC-32 implementation instead of duplicating it.

MFC: r233282

- First pass at const'ifying ata(4) as appropriate.
- Use DEVMETHOD_END.
- Use NULL instead of 0 for pointers.

MFC: r233274

Remove remnants of ATA_LOCKING uses in the ATA_CAM case and wrap it
along with functions, SYSCTLs and tunables that are not used with
ATA_CAM in #ifndef ATA_CAM, similar to the existing #ifdef'ed ATA_CAM
code for the other way around. This makes it easier to understand
which parts of ata(4) actually are used in the new world order and
to later on remove the !ATA_CAM bits. It also makes it obvious that
there is something fishy with the C-bus front-end as well as in the
ATP850 support, as these used ATA_LOCKING which is defunct in the
ATA_CAM case. When fixing the former, ATA_LOCKING probably needs to
be brought back in some form or other.

Reviewed by: mav

MFC: r233273

Exclude devices which are mutually exclusive with ATA_CAM. For better
or worse, the former are still built as modules as part of the LINT
builds.

Reviewed by: mav

MFC 232669:
Use bus_get_dma_tag() to inherit the 4G boundary restriction from the
parent PCI bus and remove the home-grown version in this driver.

MFC r233096:

Hide a few declarations from userland (including `struct inpcbgroup'). This
removes the dependency on <machine/param.h> which was introduced with SVN
rev 222748 (due to CACHE_LINE_SIZE).

Reviewed by: bde
MFC after: 10 days

MFC r233291
Handle spurious page faults that may occur in no-fault sections of the
kernel.

MFC of 232351, 233438, and 233629

MFC reviewed by: kib

MFC 232351:

This change avoids a kernel deadlock on "snaplk" when using
snapshots on UFS filesystems running with journaled soft updates.
This is the first of several bugs that need to be fixed before
removing the restriction added in -r230250 to prevent the use
of snapshots on filesystems running with journaled soft updates.

The deadlock occurs when holding the snapshot lock (snaplk)
and then trying to flush an inode via ffs_update(). We become
blocked by another process trying to flush a different inode
contained in the same inode block that we need. It holds the
inode block for which we are waiting locked. When it tries to
write the inode block, it gets blocked waiting for the our
snaplk when it calls ffs_copyonwrite() to see if the inode
block needs to be copied in our snapshot.

The most obvious place that this deadlock arises is in the
ffs_copyonwrite() routine when it updates critical metadata
in a snapshot and tries to write it out before proceeding.
The fix here is to write the data and indirect block pointer
for the snapshot, but to skip the call to ffs_update() to
write the snapshot inode. To ensure that we will never have
to update a pointer in the inode itself, the ffs_snapshot()
routine that creates the snapshot has to ensure that all the
direct blocks are allocated as part of the creation of the
snapshot.

A less obvious place that this deadlock occurs is when we hold
the snaplk because we are deleting a snapshot. In the course of
doing the deletion, we need to allocate various soft update
dependency structures and allocate some journal space. If we
hit a resource limit while doing this we decrease the resources
in use by flushing out an existing dirty file to get it to give
up the soft dependency resources that it holds. The flush can
cause an ffs_update() to be done on the inode for the file that
we have selected to flush resulting in the same deadlock as
described above when the inode that we have chosen to flush
resides in the same inode block as the snapshot inode that we hold.
The fix is to defer cleaning up any time that the inode on which
we are operating is a snapshot.

Help and review by: Jeff Roberson
Tested by: Peter Holm

MFC 233438:

Add a third flags argument to ffs_syncvnode to avoid a possible conflict
with MNT_WAIT flags that passed in its second argument.

Discussed with: kib

MFC 233629:

A refinement of change 232351 to avoid a race with a forcible unmount.
While we have a snapshot vnode unlocked to avoid a deadlock with another
inode in the same inode block being updated, the filesystem containing
it may be forcibly unmounted. When that happens the snapshot vnode is
revoked. We need to check for that condition and fail appropriately.

Spotted by: kib
Reviewed by: kib

MFC r232517:
Change SYSINIT priorities so that ip_mroute_modevent() is executed
before vnet_mroute_init(), since vnet_mroute_init() depends on mfchashsize
tunable to be set, and that is done in in ip_mroute_modevent().
Apparently I broke that ordering with r208744 almost 2 years ago...

PR: kern/162201
Submitted by: Stevan Markovic (mcafee.com)
MFC after: 3 days

MFC: 232487
Properly restore curvnet context when returning early from
ether_input_internal().

This change only affects options VIMAGE kernel builds.

PR: kern/165643
Submitted by: Vijay Singh
MFC after: 3 days

MFC r233126:

Cast wallclock.tv_sec to uint64_t to avoid overflow in the calculation.

PR: kern/161552

MFC r232951,232953,233158:
r232951:
fxp(4) does not handle deferred dma map loading. Tell
bus_dmamap_load(9) that it should return immediately with error
when there are insufficient mapping resources.

r232953:
Fix white space nits.

r233158:
Do not change current media when driver is already running. If
driver is running driver would have already completed flow control
configuration. This change removes unnecessary media changes in
controller reconfiguration cases such that it does not trigger link
reestablishment for configuration change requests like promiscuous
mode change.

Reported by: Many
Tested by: Mike Tancsa <mike <> sentex dot net>

MFC r232849-232850:
r232849:
Show PCI bus speed and width as well as running mode of PCI-X
device in device attach. This would help to narrow down issue to a
specific controller and operating mode of the controller.
While I'm here rename BGE_MISCCFG_BOARD_ID with
BGE_MISCCFG_BOARD_ID_MASK.

r232850:
Make if_ierrors updated whenever any of the following counters are
updated.
o Number of times NIC ran out of RX buffer descriptors
o Number of inbound packet errors
o Number of inbound packets that were chosen to be discarded
Previously only the discarded packet counter was used to update
if_ierrors. This change fixes wrong if_ierrors counter on
BCM570[0-4] controllers. For BCM5705 and later controllers bge(4)
already correctly counted it.

Reported by: Eugene Grosbein <egrosbein <> rdtc dot ru>

MFC r232848:
Add workaround for PCI-X BCM5704 controller that live behind
AMD-8131 PCI-X bridge. The bridge seems to reorder write access to
mailbox registers such that it caused watchdog timeouts by
out-of-order TX completions.

Tested by: Michael L. Squires <mikes <> siralan dot org >

MFC r232246:
Prefer RL_GMEDIASTAT register to RGEPHY_MII_SSR register to
extract a link status of PHY when parent driver is re(4).
RGEPHY_MII_SSR register does not seem to report correct PHY status
on some integrated PHYs used with re(4).
Unfortunately, RealTek PHYs have no additional information to
differentiate integrated PHYs from external ones so relying on PHY
model number is not enough to know that. However, it seems
RGEPHY_MII_SSR register exists for external RealTek PHYs so
checking parent driver would be good indication to know which PHY
was used. In other words, for non-re(4) controllers, the PHY is
external one and its revision number is greater than or equal to 2.
This change fixes intermittent link UP/DOWN messages reported on
RTL8169 controller.

Also, mii_attach(9) is tried after setting interface name since
rgephy(4) have to know parent driver name.

PR: kern/165509

MFC r232145:
Use correct Config registers for RTL8139 family. Unlike RTL8168 and
RTL810x family , RTL8139 has different register map for Config
registers.

While here, follow the lead of re(4) in WOL configuration.
- Disable WOL_UCAST and WOL_MCAST capabilities by default.
- Config5 register write does not need to unlock EEPROM access
on RTL8139 family but unlocking EEPROM access does not affect
its operation and make it consistent with re(4).

Reported by: Matt Renzelmann mjr <> cs dot wisc dot edu

MFC r232019,232021,232025,232027,232029,232031,232040:
r232019:
Give hardware chance to drain active DMA cycles.

r232021:
If there are not enough RX buffers, release partially allocated RX
buffers.

r232025:
Introduce sf_ifmedia_upd_locked() and have driver reset PHY before
switching to selected media. While here, set if_drv_flags before
switching to selected media.

r232027:
No need to reprogram hardware RX filter when driver is not running.

r232029:
Remove taskqueue based MII stat change handler.
Driver does not need deferred link state change processing.
While I'm here, do not report current link status if interface is
not UP.

r232031:
With r232015, sf(4) gets correct speed/duplex of established link.
Add more strict speed check in sf_miibus_statchg() and do not touch
MAC config registers when driver lost a link.

r232040:
Add check for IFF_DRV_RUNNING flag after serving an interrupt and
don't give RX path more priority than TX path.
Also remove infinite loop in interrupt handler and limit number of
iteration to 32. This change addresses system load fluctuations
under high network load.

MFC r232015-232016:
r232015:
Add Seeq Technology 80220 PHY support to smcphy(4). This PHY is
found on Adaptec AIC-6915 Starfire ethernet controller.
While here, use status register to know resolved speed/duplex.
With this change, sf(4) correctly reports speed/duplex of
established link.

r232016:
Connect smcphy(4) to mii module build.

MFC: r233105

Declare some variables static in order to reduce the object size and
redo r232822 (MFC'ed to stable/9 in r232962) in a less hackish way.
The latter now no longer breaks compiling the x86 boot2 with clang.

MFC 232470:
Use pci_find_cap() instead of pci_find_extcap() to locate PCI
find capabilities as the latter API is deprecated for this purpose.

MFC 232403,232667:
- Add a bus_dma tag to each PCI bus that is a child of a Host-PCI bridge.
The tag enforces a single restriction that all DMA transactions must not
cross a 4GB boundary. Note that while this restriction technically only
applies to PCI-express, this change applies it to all PCI devices as it
is simpler to implement that way and errs on the side of caution.
- Add a softc structure for PCI bus devices to hold the bus_dma tag and
a new pci_attach_common() routine that performs actions common to the
attach phase of all PCI bus drivers. Right now this only consists of
a bootverbose printf and the allocate of a bus_dma tag if necessary.
- Adjust all PCI bus drivers to allocate a PCI bus softc and to call
pci_attach_common() from their attach routines.

MFC 232401:
Similar to the fixes in 226967 and 226987, purge any name cache entries
associated with the previous vnode (if any) associated with the target of
a rename(). Otherwise, a lookup of the target pathname concurrent with a
rename() could re-add a name cache entry after the namei(RENAME) lookup
in kern_renameat() had purged the target pathname.

MFC 232367:
Properly handle failures in igb_setup_msix() by returning 0 if MSI or MSI-X
allocation fails.

MFC 232360:
Add pci_save_state() and pci_restore_state() wrappers around
pci_cfg_save() and pci_cfg_restore() for device drivers to use when
saving and restoring state (e.g. to handle device-specific resets).

Record mergeinfo for 233374.

MFC 232218:
Clear the a device's description string anytime it's driver changes.
Descriptions are specific to drivers and we don't change drivers on attached
devices. This fixes a few places where we were not clearing the description
when detaching a driver (e.g. if device_attach() failed). While here, fix
a few other nits:
- Remove spurious call to remove a device's driver from
devclass_driver_deleted(). device_detach() removes it already.
- Fix a typo.

MFC 232265:
Properly clear a device's devclass if DEVICE_ATTACH() fails if the device
does not have a fixed devclass.

MFC r231949:
Fix found places where uio_resid is truncated to int.

Add the sysctl debug.iosize_max_clamp, enabled by default. Setting the
sysctl to zero allows to perform the SSIZE_MAX-sized i/o requests from
the usermode.

MFC r232493:
Remove unneeded cast to u_int. The values as small enough to fit into
int, beside the use of MIN macro which performs type promotions.

MFC r232494:
Instead of incomplete handling of read(2)/write(2) return values that
does not fit into registers, declare that we do not support this case
using CTASSERT(), and remove endianess-unsafe code to split return value
into td_retval.

While there, change the style of the sysctl debug.iosize_max_clamp
definition.

MFC r232495:
pipe_read(): change the type of size to int, and remove signed clamp.
pipe_write(): change the type of desiredsize back to int, its value fits.

MFC 230547:
Add a timeout on positive name cache entries in the NFS client. That is,
we will only trust a positive name cache entry for a specified amount of
time before falling back to a LOOKUP RPC, even if the ctime for the file
handle matches the cached copy in the name cache entry. The timeout is
configured via a new 'nametimeo' mount option and defaults to 60 seconds.
It may be set to zero to disable positive name caching entirely.

MFC 232116:
Adjust the nfs_skip_wcc_data_onerr setting so that it does not block
post-op attributes for ENOENT errors now that the name caching logic
depends on working post-op attributes.

MFC 230394,230441,230489,230552,232420:
Close a race in NFS lookup processing that could result in stale name cache
entries on one client when a directory was renamed on another client. The
root cause for the stale entry being trusted is that each per-vnode nfsnode
structure has a single 'n_ctime' timestamp used to validate positive name
cache entries. However, if there are multiple entries for a single vnode,
they all share a single timestamp. To fix this, extend the name cache
to allow filesystems to optionally store a timestamp value in each name
cache entry. The NFS clients now fetch the timestamp associated with
each name cache entry and use that to validate cache hits instead of the
timestamps previously stored in the nfsnode. Another part of the fix is
that the NFS clients now use timestamps from the post-op attributes of
RPCs when adding name cache entries rather than pulling the timestamps out
of the file's attribute cache. The latter is subject to races with other
lookups updating the attribute cache concurrently.

MFC r232975: Add an assert for proctree_lock to proc_to_reap().

Merge 232685 from head:
Merge from OpenBSD:

revision 1.146
date: 2010/05/12 08:11:11; author: claudio; state: Exp; lines: +2 -3
bzero() the full compressed update struct before setting the values.
This is needed because pf_state_peer_hton() skips some fields in certain
situations which could result in garbage beeing sent to the other peer.
This seems to fix the pfsync storms seen by stephan@ and so dlg owes me
a whiskey.

I didn't see any storms, but this definitely fixes a useless memory
allocation on the receiving side, due to non zero scrub_flags field
in a pfsync_state_peer structure.

Merge 228321 from head:
Fix double free.

PR: kern/163089
Submitted by: Herbie Robinson <Herbie.Robinson stratus.com>

Merge from head:
r227785:
- Reduce severity for all ARP events, that can be triggered from remote
machine to LOG_NOTICE. Exception left to "using my IP address".
- Fix multicast ARP warning: add newline and also log the bad MAC address.

Tested by: Alexander Wittig <wittigal msu.edu>

r227790:
Be more informative for "unknown hardware address format" message.

Submitted by: Andrzej Tobola <ato iem.pw.edu.pl>

Merge 227308 from head/:
In icmp6_redirect_input:

- Assert that we got a valid mbuf with rcvif pointer. [1]
- Use __func__ in logging.

Submitted by: prabhakar lakhera <prabhakar.lakhera gmail.com> [1]
Submitted by: Kristof Provost <kristof sigsegv.be> [1]

MFC 229621:
Convert all users of IF_ADDR_LOCK to use new locking macros that specify
either a read lock or write lock.

MFC: r232315

Use a more appropriate default for the maximum number of addresses in the
bridge forwarding table.

MFC r227430:
On i386, fbt probes are implemented by writing an invalid opcode over
certain instructions in a function prologue or epilogue. DTrace has a
hook into the invalid opcode fault handler that checks whether the fault
was due to an probe and if so, runs the DTrace magic.

Upon returning from an invalid opcode fault caused by a probe, DTrace must
emulate the instruction that was replaced with the invalid opcode and then
return control to the instruction following the invalid opcode.

There were a pair of related bugs in the emulation for the leave
instruction. The leave instruction is used to pop off a stack frame prior
to returning from a function. The emulation for this instruction must
move the trap frame for the invalid opcode fault down the stack to the
bottom of the stack frame that is being removed, and then execute an iret.

At two points in this process, the emulation code was storing values above
the current value of the stack pointer. This opened up a window in which
if we were two take an interrupt, the trap frame for the interrupt would
overwrite the values stored on the stack, causing the system to panic
later.

The first bug was that at one point the emulation code saves the new value
for $esp above the current stack pointer value. The fix is to save this
value instead inside of the original trap frame. At this point we do
not need the original trap frame so this is safe.

The second bug is that when the emulate code loads $esp from the stack, it
points part-way through the new trap frame instead of at its beginning.
The emulation code adjusts the stack pointer to the correct value
immediately afterwards, but this still leaves a one instruction window in
which an interrupt would corrupt this trap frame. Fix this by adjusting
the stack frame value before loading it into $esp.

This fixes panics in invop_leave on i386 when using fbt return probes.

Ok'd by: rstone

MFC 226340,226340:
Use queue(3) macros instead of home-rolled versions in several places in
the INET6 code. This includes retiring the 'ndpr_next' and 'pfr_next'
macros.

MFC r232518:

Respect phy-handle property in Ethernet nodes of the device tree.

This lets specify whereabouts of the parent PHY for a given MAC node
(and get rid of ugly kludges in mge(4) and tsec(4)).

Obtained from: Semihalf

MFC r232512:

Remove unused #defines. All this is now retrieved from the device tree.

MFC: r232822

Fix a bug introduced in r223938; on big-endian machines coping a 32-bit
quantum bytewise to the address of a 64-bit variable results in writing
to the "wrong" 32-bit half so adjust the address accordingly. This fix
is implemented in a hackish way for two reasons:
o in order to be able to get it into 8.3 with zero impact on the little-
endian architectures where this bug has no effect and
o to avoid blowing the x86 boot2 out of the water again when compiling
it with clang, which all sane versions of this fix tested do.
This change fixes booting from UFS1 file systems on big-endian machines.

Merge 231025 from head:
Add new socket options: TCP_KEEPINIT, TCP_KEEPIDLE, TCP_KEEPINTVL and
TCP_KEEPCNT, that allow to control initial timeout, idle time, idle
re-send interval and idle send count on a per-socket basis.

Reviewed by: andre, bz, lstewart

MFC: r2323467
The name caching changes of r230394 exposed an intermittent bug
in the new NFS server for NFSv4, where it would report ENOENT
when the file actually existed on the server. This turned out
to be caused by not initializing ni_topdir before calling lookup()
and there was a rare case where the value on the stack location
assigned to ni_topdir happened to be a pointer to a ".." entry,
such that "dp == ndp->ni_topdir" succeeded in lookup().
This patch initializes ni_topdir to fix the problem.

MFC: r232327
Fix the NFS clients so that they use copyin() instead of bcopy(),
when doing direct I/O. This direct I/O code is not enabled by default.

MFC r230865:

Adjust mvs(4) to handle interrupt cause reg depending on the actual number of
channels available

- current code treats bits 4:7 in 'SATAHC interrupt mask' and 'SATAHC
interrupt cause' as flags for SATA channels 2 and 3

- for embedded SATA controllers (SoC) these bits have been marked as reserved
in datasheets so far, but for some new and upcoming chips they are used for
purposes other than SATA

Submitted by: Lukasz Plachno
Reviewed by: mav
Obtained from: Semihalf

MFC r228504, r228530.

r228504:
Make *intr{cnt,names} on ARM reside in data section, similar to other arches.

sintrnames and sintrcnt are initialized with non-zero values, which were
discarded by the .bss directive, so consumers like "vmstat -i" were not
getting correct data.

Submitted by: Lukasz Plachno
Obtained from: Semihalf

r228530:
ARM pmap fixes:

- Write Buffers have to be drained after write to Page Table even if caches
are in write-through mode.

- Make sure to sync PTE in pmap_zero_page_generic().

Submitted by: Michal Mazur
Reviewed by: cognet
Obtained from: Semihalf

MFC r231852,232127:

Merge multi-FIB IPv6 support.

Extend the so far IPv4-only support for multiple routing tables (FIBs)
introduced in r178888 to IPv6 providing feature parity.

This includes an extended rtalloc(9) KPI for IPv6, the necessary
adjustments to the network stack, and user land support as in netstat.

Sponsored by: Cisco Systems, Inc.

MFC 230857:

If multiple threads call kevent() to get AIO events on same kqueue fd,
it is possible that a single AIO event will be reported to multiple
threads, it is not threading friendly, and the existing API can not
control this behavior.
Allocate a kevent flags field sigev_notify_kevent_flags for AIO event
notification in sigevent, and allow user to pass EV_CLEAR, EV_DISPATCH
or EV_ONESHOT to AIO kernel code, user can control whether the event
should be cleared once it is retrieved by a thread. This change should
be comptaible with existing application, because the field should have
already been zero-filled, and no additional action will be taken by
kernel.

PR: kern/156567

MFC 231006:

Add 32-bit compat code for AIO kevent flags introduced in revision 230857.

MFC 231724:

Add notes about sigev_notify_kevent_flags introduced in revision 230857
which enables thread-friendly polling on same fd for AIO events.

Reviewed by: delphij

MFC 231777:

Bump .Dd date for previous revision.

MFC r232008,232010,232080,232089

Using the flowid in the mbuf assumes the network card is giving a good hash for
the traffic flow, this may not be the case giving poor traffic distribution.
Add a sysctl which allows us to fall back to our own flow hash code.

PR: kern/164901

MFC: r232050
hrs@ reported a panic to freebsd-stable@ under the subject line
"panic in 8.3-PRERELEASE" on Feb. 22, 2012. This panic was caused
by use of a mix of tsleep() and msleep() calls on the same event
in the new NFS server DRC code. It did "mtx_unlock(); tsleep();"
in two places, which kib@ noted introduced a slight risk that the
wakeup() would occur before the tsleep(), resulting in a 10sec
delay before waking up. This patch fixes the problem by replacing
"mtx_unlock(); tsleep();" with mtx_sleep(..PDROP..). It also
changes a nfsmsleep() call to mtx_sleep() so that the code uses
mtx_sleep() consistently within the file.

MFC: r231985

Fix memset sizeof

MFC: r231914

Probe the National DP83849, which is a dual-port version of the PHYTER.

Obtained from: NetBSD

MFC: r231913

- Probe BCM57780.
- In case the parent is bge(4), don't set the Jumbo frame settings unless
the MAC actually is Jumbo capable as otherwise the PHY might not have the
corresponding registers implemented. This is also in line with what the
Linux tg3 driver does.

PR: 165032
Submitted by: Alexander Milanov
Obtained from: OpenBSD

MFC r231904
Close a race due to dropping of the map lock between creating a map entry
for a shared mapping and marking the entry for inheritance.

MFC: r231621

- As it turns out, MSI-X is broken for at least LSI SAS1068E when passed
through by VMware so blacklist their PCI-PCI bridge for MSI/MSI-X here.
Note that besides currently there not being a quirk type that disables
MSI-X only and there's no evidence that MSI doesn't work with the VMware
pass-through, it's really questionable whether MSI generally works in
that setup as VMware only mention three know working devices [1, p. 4].
Also not that this quirk entry currently doesn't affect the devices
emulated by VMware in any way as these don't claim support MSI/MSI-X to
begin with. [2]
While at it, make the PCI quirk table const and static.
- Remove some duplicated empty lines.
- Use DEVMETHOD_END.

PR: 163812, http://forums.freebsd.org/showthread.php?t=27899 [2]
Reviewed by: jhb

[1]: http://www.vmware.com/pdf/vsp_4_vmdirectpath_host.pdf

MFC 230183: Make XENHVM work on i386 by fixing the definition of __ffs.

MFC 230184, 230185: Add XENHVM config file to i386.

MFC r231622:
For RTL8168/8111D controller, make sure to wake PHY from power down
mode. Otherwise, PHY access times out under certain conditions.

MFC: r231805
Delete a couple of out of date comments that are no longer true in
the new NFS client.

MFC r230972,r230980,r231559,r231565: Add MK_SOURCELESS build option, and
associated fixes.

Approved by: kib (mentor)

MFC r229363
Don't pass VM_ALLOC_ZERO to vm_page_grab() in tmpfs_mappedwrite() and
tmpfs_nocacheread(). It is both unnecessary and a pessimization. It
results in either the page being zeroed twice or zeroed first and then
overwritten by an I/O operation.

MFC r230286,230337-230338,231159:
r230286:
Introduce a tunable that disables use of MSI.
Non-zero value will use INTx.

r230337-230338:
Rename dev.bge.%d.msi_disable to dev.bge.%d.msi which matches
enable/disable and default it to on.

r231159:
Call bge_add_sysctls() early and especially before bge_can_use_msi() so
r230337 actually has a chance of working and doesn't always unconditionally
disable the use of MSIs.

MFC r230336:
Fix a logic error which resulted in putting PHY into sleep when WOL
is active. If WOL is active driver should not put PHY into sleep.
This change makes WOL work on RTL8168E.

MFC r230575-230576:
r230275:
Use a RX DMA tag to free loaded RX DMA maps.
Previously it used a TX DMA tag.

r230276:
Free allocated jumbo buffers when controller is stopped.

MFC r229940:
style. No functional changes.

MFC: r230803
When a "mount -u" switches an NFS mount point from TCP to UDP,
any thread doing an I/O RPC with a transfer size greater than
NFS_UDPMAXDATA will be hung indefinitely, retrying the RPC.
After a discussion on freebsd-fs@, I decided to add a warning
message for this case, as suggested by Jeremy Chadwick.

MFC: r230801
jwd@ reported a problem via email to freebsd-fs@ on Aug 25, 2011
under the subject "F_RDLCK lock to FreeBSD NFS fails to R/O target file".
This occurred because the server side NLM always checked for VWRITE
access, irrespective of the type of lock request. This patch
replaces VOP_ACCESS(..VWRITE..) with one appropriate to
the lock operation. It allows unlock and lock cancellation
to be done without a check of VOP_ACCESS(), so that files
can't be left locked indefinitely after the file permissions
have been changed.

MFC: r231228

Remove extra newlines from panic messages.

MFC: r231518

Remove extra newlines from panic messages.

MFC 230340:
Properly return success once a matching VPD entry is found in
pci_get_vpd_readonly_method(). Previously the loop was always running
to completion and falling through to failing with ENXIO.

MFC: r231133
r228827 fixed a problem where copying of NFSv4 open credentials into
a credential structure would corrupt it. This happened when the
p argument was != NULL. However, I now realize that the copying of
open credentials should only happen for p == NULL, since that indicates
that it is a read-ahead or write-behind. This patch fixes this.
After this commit, r228827 could be reverted, but I think the code is
clearer and safer with the patch, so I am going to leave it in.
Without this patch, it was possible that a NFSv4 VOP_SETATTR() could have
changed the credentials of the caller. This would have happened if
the process doing the VOP_SETATTR() did not have the file open, but
some other process running as a different uid had the file open for writing
at the same time.

MFC r230157: dadump: don't leak the periph lock on i/o error

MFC: r230605
A problem with respect to data read through the buffer cache for both
NFS clients was reported to freebsd-fs@ under the subject "NFS
corruption in recent HEAD" on Nov. 26, 2011. This problem occurred when
a TCP mounted root fs was changed to using UDP. I believe that this
problem was caused by the change in mnt_stat.f_iosize that occurred
because rsize was decreased to the maximum supported by UDP. This
patch fixes the problem by using v_bufobj.bo_bsize instead of f_iosize,
since the latter is set to f_iosize when the vnode is allocated, but
does not change for a given vnode when f_iosize changes.

MFC r231177:
Rev. 228360 moved the call to cpu_set_upcall() to happen before
td_proc gets initialized in td (=newtd). Use td0 instead.

MFC 229465: Use correct locking when traversing interface address list.

MFC 229614:
Add new variants of the IF_ADDR_*LOCK*() macros used for protecting
interface address lists that distinguish read locks from write locks.
To preserve the KPI, the previous operations are mapped to the write
lock macros. The lock is still kept as a mutex for now.

MFC 229613:
Update recently added drivers to use the if_*addr_r*lock() wrapper
functions instead of using the IF_ADDR_LOCK directly. The wrapper
functions are the supported interface for device drivers.

MFC 229598:
Remove use of explicit bus space tags and handles and use methods that
operate on resource objects instead.

Hoist mergeinfo for modules/sound/driver/* up to sys/.

MFC 228962:
Use curthread rather than PCPU_GET(curthread). 'curthread' uses
special-case optimizations on several platforms and is preferred.

MFC 228957:
Implement linux_fadvise64() and linux_fadvise64_64() using
kern_posix_fadvise().

MFC r230247:

Revert r212360 now that PowerPC can handle large sparse arguments to
pmap_remove() (changed in r228412).

Pick a constant high IRQ value for the PS3 IPI, which lets PS3 devices be
usefully loaded and unloaded as modules.

Submitted by: geoffrey dot levand at mail dot ru

MFC r230398:

Prevent an error resulting from signed/unsigned comparison on systems
that do not comply with the OF spec.

Submitted by: Anders Gavare

MFC: r230345
Martin Cracauer reported a problem to freebsd-current@ under the
subject "Data corruption over NFS in -current". During investigation
of this, I came across an ugly bogusity in the new NFS client where
it replaced the cr_uid with the one used for the mount. This was
done so that "system operations" like the NFSv4 Renew would be
performed as the user that did the mount. However, if any other
thread shares the credential with the one doing this operation,
it could do an RPC (or just about anything else) as the wrong cr_uid.
This patch fixes the above, by using the mount credentials instead of
the one provided as an argument for this case. It appears
to have fixed Martin's problem.
This patch is needed for NFSv4 mounts and NFSv3 mounts against
some non-FreeBSD servers that do not put post operation attributes
in the NFSv3 Statfs RPC reply.

Tested by: cracauer at cons.org, dim

MFC 230592:

Bring in the LSI-supported version of the mps(4) driver.

This involves significant changes to the mps(4) driver, but is not a
complete rewrite.

Some of the changes in this version of the driver:
- Integrated RAID (IR) support.
- Support for WarpDrive controllers.
- Support for SCSI protection information (EEDP).
- Support for TLR (Transport Level Retries), needed for tape drives.
- Improved error recovery code.
- ioctl interface compatible with LSI utilities.

mps.4: Update the mps(4) driver man page somewhat for the driver
changes. The list of supported hardware still needs to be
updated to reflect the full list of supported cards.

conf/files: Add the new driver files.

mps/mpi/*: Updated version of the MPI header files, with a BSD style
copyright.

mps/*: See above for a description of the new driver features.

modules/mps/Makefile:
Add the new mps(4) driver files.

Submitted by: Kashyap Desai <Kashyap.Desai@lsi.com>
Sponsored by: LSI, Spectra Logic
Reviewed by: ken

MFC: r227980

Move to SCHED_ULE by default. Since r226057 SCHED_ULE and sparc64 are
compatible with each other and since r227539 the last issue seen when
using SCHED_ULE is fixed (MFC'ed to stable/9 in r230691 and r227714
respectively). At least on UP and 2-way machines SCHED_4BSD still
performs better than SCHED_ULE, however, the optimizations done in
r225889 (MFC'ed to stable/9 in r230673) pretty much compensate that so
there's at least no net regression.
Thanks go to Peter Jeremy for extensive testing.

MFC: r230664

As it turns out r227960 (MFC'ed to stable/9 in r228126) may still be
insufficient with PREEMPTION so try harder to get the CDMA sync
interrupt delivered and also in a more efficient way:
- wrap the whole process of sending and receiving the CDMA sync
interrupt in a critical section so we don't get preempted,
- send the CDMA sync interrupt to the CPU that is actually waiting
for it to happen so we don't take a detour via another CPU,
- instead of waiting for up to 15 seconds for the interrupt to
trigger try the whole process for up to 15 times using a one
second timeout (the code was also changed to just ignore belated
interrupts of a previous tries should they appear).

According to testing done by Peter Jeremy with the debugging also
added as part of this commit the first two changes apparently are
sufficient to now properly get the CDMA sync interrupts delivered
at the first try though.

MFC: r230662

Fully disable interrupts while we fiddle with the FP context in the
VIS-based block copy/zero implementations. While with 4BSD it's
sufficient to just disable the tick interrupts, with ULE+PREEMPTION
it's otherwise also possible that these are preempted via IPIs.

MFC: r230633, r230634

Now that we have a working OF_printf() since r230631 and a OF_panic()
helper since r230632 (MFC'ed to stable/9 in r230884 and r230886
respectively), use these for output and panicing during the early
cycles and move cninit() until after the static per-CPU data has
been set up. This solves a couple of issue regarding the non-
availability of the static per-CPU data:
- panic() not working and only making things worse when called,
- having to supply a special DELAY() implementation to the low-level
console drivers,
- curthread accesses of mutex(9) usage in low-level console drivers
that aren't conditional due to compiler optimizations (basically,
this is the problem described in r227537 but in this case for
keyboards attached via uart(4)). [1]

PR: 164123 [1]

MFC: r230628

Mark cpu_{halt,reset}() as __dead2 as appropriate.

MFC: r230632

- Now that we have a working OF_printf() since r230631 (MFC'ed to stable/9
in r230884), use it for implementing a simple OF_panic() that may be used
during the early cycles when panic() isn't available, yet.
- Mark cpu_{exit,shutdown}() as __dead2 as appropriate.

MFC: r230631

Implement OF_printf() using kvprintf() directly, avoiding to use a
buffer and allowing to handle newlines properly.

MFC: r230630

For machines where the kernel address space is unrestricted increase
VM_KMEM_SIZE_SCALE to 2, awaiting more insight from alc@. As it turns
out, the VM apparently has problems with machines that have large holes
in the physical address space, causing the kmem_suballoc() call in
kmeminit() to fail with a VM_KMEM_SIZE_SCALE of 1. Using a value of 2
allows these, namely Blade 1500 with 2GB of RAM, to boot.

PR: 164227

MFC: r230627

Using ATA_CAM along with ATAPI DMA causes data corruption with ALI_NEW
and CMD controllers for reasons unknown so disable it.

PR: 164226

MFC r228820, r228851:
Merge to da driver quirks hinting 4K physical sector sizes for SATA disks
connected via SAS or USB. Unluckily I've found that SAS (mps) and USB-SATA
I have translate models in different ways, requiring twice more quirks.
Unluckily for Hitachi, their model names are trimmed on SAS, making
impossible to identify 4K sector drives that way.

MFC r230545: Clarify and improve the boot menu with some small changes.

PR: misc/160818
Submitted by: Warren Block <wblock wonkity com>

MFC: 230000, 230544

Fix a race condition in CAM peripheral free handling, locking
in the CAM XPT bus traversal code, and a number of other periph level
issues.

r230544 | ken | 2012-01-25 10:58:47 -0700 (Wed, 25 Jan 2012) | 9 lines

Fix a bug introduced in r230000. We were eliminating all LUNs on a target
in response to CAM_DEV_NOT_THERE, instead of just the LUN in question.

This will now just eliminate the specified LUN in response to
CAM_DEV_NOT_THERE.

Reported by: Richard Todd <rmtodd@servalan.servalan.com>

r230000 | ken | 2012-01-11 17:41:48 -0700 (Wed, 11 Jan 2012) | 72 lines

Fix a race condition in CAM peripheral free handling, locking
in the CAM XPT bus traversal code, and a number of other periph level
issues.

cam_periph.h,
cam_periph.c: Modify cam_periph_acquire() to test the CAM_PERIPH_INVALID
flag prior to allowing a reference count to be gained
on a peripheral. Callers of this function will receive
CAM_REQ_CMP_ERR status in the situation of attempting to
reference an invalidated periph. This guarantees that
a peripheral scheduled for a deferred free will not
be accessed during its wait for destruction.

Panic during attempts to drop a reference count on
a peripheral that already has a zero reference count.

In cam_periph_list(), use a local sbuf with SBUF_FIXEDLEN
set so that mallocs do not occur while the xpt topology
lock is held, regardless of the allocation policy of the
passed in sbuf.

Add a new routine, cam_periph_release_locked_buses(),
that can be called when the caller already holds
the CAM topology lock.

Add some extra debugging for duplicate peripheral
allocations in cam_periph_alloc().

Treat CAM_DEV_NOT_THERE much the same as a selection
timeout (AC_LOST_DEVICE is emitted), but forgo retries.

cam_xpt.c: Revamp the way the EDT traversal code does locking
and reference counting. This was broken, since it
assumed that the EDT would not change during
traversal, but that assumption is no longer valid.

So, to prevent devices from going away while we
traverse the EDT, make sure we properly lock
everything and hold references on devices that
we are using.

The two peripheral driver traversal routines should
be examined. xptpdperiphtraverse() holds the
topology lock for the entire time it runs.
xptperiphtraverse() is now locked properly, but
only holds the topology lock while it is traversing
the list, and not while the traversal function is
running.

The bus locking code in xptbustraverse() should
also be revisited at a later time, since it is
complex and should probably be simplified.

scsi_da.c: Pay attention to the return value from cam_periph_acquire().

Return 0 always from daclose() even if the disk is now gone.

Add some rudimentary error injection support.

scsi_sg.c: Fix reference counting in the sg(4) driver.

The sg driver was calling cam_periph_release() on close,
but never called cam_periph_acquire() (which increments
the reference count) on open.

The periph code correctly complained that the sg(4)
driver was trying to decrement the refcount when it
was already 0.

Sponsored by: Spectra Logic

MFC r227758,227759,227788:
Introduce macro stubs in the mutex and sxlock implementation that will
be always defined and will allow consumers, willing to provide options,
file and line to locking requests, to not worry about options
redefining the interfaces.
This is typically useful when there is the need to build another
locking interface on top of the mutex one.

Requested by: kib

MFC r228270,228277:

Add a devd notification for closing/opening the lid on PowerBooks and iBooks.

Approved by: nwhitehorn (mentor)

MFC r230531: Remove unused variable.

MFC r230101:

Convert FFS mount error messages from kernel printf's to using the
vfs_mount_error error message facility provided by the nmount
interface.

Clean up formatting of mount warnings which still need to use
kernel printf's since they do not return errors.

Requested by: Craig Rodrigues <rodrigc@crodrigues.org>

MFC r226449:

Add support for special keys (volume/brightness/eject) on Apple laptops with
ADB keyboards.

Approved by: nwhitehorn (mentor)

MFC: r228858

Update a comment to reflect reality and explain why we're using the
medany code model.

MFC: r228211

It doesn't make much sense to check whether child is NULL after already
having dereferenced it. We either should generally check the device_t's
supplied to bus functions before using them (which we seem to virtually
never do) or just assume that they are not NULL.
While at it make this code fit 78 columns.

Found with: Coverity Prevent(tm)
CID: 4230

MFC: r228209

- In device_probe_child(9) check the return value of device_set_driver(9)
when actually setting a driver as especially ENOMEM is fatal in these
cases.
- Annotate other calls to device_set_devclass(9) and device_set_driver(9)
without the return value being checked and that are okay to fail.

Reviewed by: yongari (slightly earlier version)

MFC: r228027

Move the scsi_da_bios_params() prototype from pc98_machdep.h to md_var.h
where the prototype for pc98_ata_disk_firmware_geom_adjust() also lives
in order to avoid an #ifdef'ed include in cam(4).

MFC: r228024

Update comment.

MFC: r228005

Change another instance of amd(4) to esp(4) missed in r227006 (MFC'ed to
stable/9 in r227305).

Submitted by: Garrett Cooper

MFC r230249:

Make sure all intermediate variables holding mount flags (mnt_flag)
and that all internal kernel calls passing mount flags are declared
as uint64_t so that flags in the top 32-bits are not lost.

MFC r230250:

There are several bugs/hangs when trying to take a snapshot on a UFS/FFS
filesystem running with journaled soft updates. Until these problems
have been tracked down, return ENOTSUPP when an attempt is made to
take a snapshot on a filesystem running with journaled soft updates.

MFC: r227912

- Just use cam_calc_geometry(9) on newer version of FreeBSD rather than
duplicating it.
- In hptmv(4) and hptrr(4) use __FBSDID and DEVMETHOD_END.

MFC: r227907, r22791 (for diff reduction)

Add BCM5785 but wrap it in #ifdef notyet for now. According to yongari@ there
are issues probably needing workarounds in bge(4) when brgphy(4) handles this
PHY. Letting ukphy(4) handle it instead results in a working configuration,
although likely with performance penalties.

MFC: r227906

Sync with NetBSD rev. 1.104

Obtained from: NetBSD

MFC: r227688

There's no need export the device interface methods of miibus(4).

MFC: r227687, r228290

- Add a hint.miibus.X.phymask hint, allowing do individually exclude PHY
addresses from being probed and attaching something including ukphy(4)
to it. This is mainly necessarily for PHY switches that create duplicate
or fake PHYs on the bus that can corrupt the PHY state when accessed or
simply cause problems when ukphy(4) isolates the additional instances.
- Change miibus(4) to be a hinted bus, allowing to add child devices via
hints and to set their attach arguments (including for automatically
probed PHYs). This is mainly needed for PHY switches that violate IEEE
802.3 and don't even implement the basic register set so we can't probe
them automatically. However, the ability to alter the attach arguments
for automatically probed PHYs is also useful as for example it allows
to test (or tell a user to test) new variant of a PHY with a specific
driver by letting an existing driver attach to it via manipulating the
IDs without the need to touch the source code or to limit a Gigabit
Ethernet PHY to only announce up to Fast Ethernet in order to save
energy by limiting the capability mask. Generally, a driver has to
be hinted via hint.phydrv.X.at="miibusY" and hint.phydrv.X.phyno="Z"
(which already is sufficient to add phydrvX at miibusY at PHY address
Z). Then optionally the following attach arguments additionally can
be configured:
hint.phydrv.X.id1
hint.phydrv.X.id2
hint.phydrv.X.capmask
- Some minor cleanup.

Reviewed by: adrian, ray

MFC: r227686

There's no need to read DC_10BTSTAT twice in dcphy_status().

MFC: r227685

- There's no need to ignore the return value of mii_attach(9) when attaching
dcphy(4) (CID 9283).
- In dc_detach(), check whether ifp is NULL as dc_attach() may call the
former without ifp being allocated (CID 4288).

Found with: Coverity Prevent(tm)

MFC: r227043

Sprinkle some const.

MFC: r226950

Add multiple inclusion protection.

MFC: r226948

Remove unnecessary DMA constraints.

MFC: r226270

- Remove unused remnants of MII bitbang'ing.
- Sprinkle const.

MFC: r226175

In device_get_children() avoid malloc(0) in order to increase portability
to other operating systems.

PR: 154287

MFC: r226171

Sprinkle const.

MFC: r226057

- Currently, sched_balance_pair() may cause a CPU to send an IPI_PREEMPT to
itself, which sparc64 hardware doesn't support. One way to solve this
would be to directly call sched_preempt() instead of issuing a self-IPI.
However, quoting jhb@:
"On the other hand, you can probably just skip the IPI entirely if we are
going to send it to the current CPU. Presumably, once this routine
finishes, the current CPU will exit softlock (or will do so "soon") and
will then pick the next thread to run based on the adjustments made in
this routine, so there's no need to IPI the CPU running this routine
anyway. I think this is the better solution. Right now what is probably
happening on other platforms is as soon as this routine finishes the CPU
processes its self-IPI and causes mi_switch() which will just switch back
to the softclock thread it is already running."
- With r226054 (MFC'ed to stable/9 in r230690) and the the above change in
place, sparc64 now no longer is incompatible with ULE and vice versa.
However, powerpc/E500 still is.

Submitted by: jhb
Reviewed by: jeff

MFC: r226054

- Use atomic operations rather than sched_lock for safely assigning pm_active
and pc_pmap for SMP. This is key to allowing adding support for SCHED_ULE.
Thanks go to Peter Jeremy for additional testing.
- Add support for SCHED_ULE to cpu_switch().

MFC: r225931, r225932, r227000

Make sparc64 compatible with NEW_PCIB and enable it:
- Implement bus_adjust_resource() methods as far as necessary and in non-PCI
bridge drivers as far as feasible without rototilling them.
- As NEW_PCIB does a layering violation by activating resources at layers
above pci(4) without previously bubbling up their allocation there, move
the assignment of bus tags and handles from the bus_alloc_resource() to
the bus_activate_resource() methods like at least the other NEW_PCIB
enabled architectures do. This is somewhat unfortunate as previously
sparc64 (ab)used resource activation to indicate whether SYS_RES_MEMORY
resources should be mapped into KVA, which is only necessary if their
going to be accessed via the pointer returned from rman_get_virtual() but
not for bus_space(9) as the later always uses physical access on sparc64.
Besides wasting KVA if we always map in SYS_RES_MEMORY resources, a driver
also may deliberately not map them in if the firmware already has done so,
possibly in a special way. So in order to still allow a driver to decide
whether a SYS_RES_MEMORY resource should be mapped into KVA we let it
indicate that by calling bus_space_map(9) with BUS_SPACE_MAP_LINEAR as
actually documented in the bus_space(9) page. This is implemented by
allocating a separate bus tag per SYS_RES_MEMORY resource and passing the
resource via the previously unused bus tag cookie so we later on can call
rman_set_virtual() in sparc64_bus_mem_map(). As a side effect this now
also allows to actually indicate that a SYS_RES_MEMORY resource should be
mapped in as cacheable and/or read-only via BUS_SPACE_MAP_CACHEABLE and
BUS_SPACE_MAP_READONLY respectively.
- Do some minor cleanup like taking advantage of rman_init_from_resource(),
factor out the common part of bus tag allocation into a newly added
sparc64_alloc_bus_tag(), hook up some missing newbus methods and replace
some homegrown versions with the generic counterparts etc.
- While at it, let apb_attach() (which can't use the generic NEW_PCIB code
as APB bridges just don't have the base and limit registers implemented)
regarding the config space registers cached in pcib_softc and the SYSCTL
reporting nodes set up.

MFC: r225901

Remove obsolete macros.

MFC: r225900

Nuke SUN4U #ifdef's which with the demise of sun4v no longer serve any
purpose.

MFC: r225899

Also allocate space for the PIL counters. Given that no machine actually
uses IV_MAX interrupt vectors this wasn't a problem in practice though.

MFC: r225891

Re-reading the Schizo errata suggests that it's actually tolerable to
also use the streaming buffer of pre version 5/revision 2.3 hardware as
long as we stay away from context flushes (which iommu(4) so far doesn't
take advantage of). OpenSolaris does the same.

MFC: r225890

- Add protective parentheses to macros as far as possible.
- Move {r,w,}mb() to the top of this file where they live on most of the
other architectures.

MFC: r225889, r228222

In total store which we use for running the kernel and all of the userland
atomic operations behave as if they were followed by a CPU memory barrier
so there's no need to include ones in the acquire variants of atomic(9) and
it's sufficient to just use include compiler memory barriers to satisfy
the requirements of atomic(9). Removing the CPU memory barriers results in
a small performance improvement, specifically this is sufficient to
compensate the performance loss seen in the worldstone benchmark seen when
using SCHED_ULE instead of SCHED_4BSD.
This change is inspired by Linux even more radically doing the equivalent
thing some time ago.
Thanks go to Peter Jeremy for additional testing.

MFC: r225888

Add a comment about why contrary to what once would think running all of
userland with total store order actually is appropriate.

MFC: r225887

Use the extended integer condition code when comparing 64-bit values. Given
that ATOMIC_INC_LONG currently is unused this happened to not be fatal.

MFC: r225886

- Right-justify backslashes as suggested by style(9).
- Rename ATOMIC_INC_ULONG to ATOMIC_INC_LONG in order to be consistent with
the names of the other macros in this file an adjust accordingly.

MFC: r228022, r228026

For sparc64 also adjust the geometry of da(4) driven disks to not overflow
the 16-bit cylinders field of the VTOC8 disk label (at around 502GB). The
geometry chosen for disks above that limit allows to use disks up to 2TB,
which is the limit of the extended VTOC8 format. The geometry used for
disks smaller than the 16-bit cylinders limit stays the same as used by
cam_calc_geometry(9) for extended translation.
Thanks to Hans-Joerg Sirtl for providing hardware for testing this change.

MFC: r230100
Tai Horgan reported via email that there were two places in
the new NFSv4 server where the code follows the wrong list.
Fortunately, for these fairly rare cases, the lc_stateid[]
lists are normally empty. This patch fixes the code to
follow the correct list.

MFC r230496: Remove the stale XXX rt_newaddrmsg comment.

MFC 227460

A default route learned from the RAs could be deleted manually
after its installation. This removal may be accidental and can
prevent the default route from being installed in the future if
the associated default router has the best preference. The cause
is the lack of status update in the default router on the state
of its route installation in the kernel FIB. This patch fixes
the described problem.

Reviewed by: hrs, discussed with hrs

MFC: r229956
jwd@ reported via email that the "CacheSize" field reported by "nfsstat -e -s"
would go negative after using the "-z" option to zero out the stats.
This patch fixes that by not zeroing out the srvcache_size field
for "-z", since it is the size of the cache and not a counter

MFC r230256:
Fix the "lock &zrl->zr_mtx already initialized" assertion by initializing
the allocated memory before calling mtx_init(9) on mtx pointing to it.
Otherwize, random contents of uninitialized memory might occasionally
trigger the assertion.

Reported by: Pavel Polyakov <bsd kobyla org>
Reviewed by: pjd

MFC r229694:

r222004 changed sbuf_finish() to not clear the buffer error status. As a
consequence sbuf_len() will return -1 for buffers which had the error
status set prior to sbuf_finish() call. This causes a problem in
pfs_read() which purposely uses a fixed size sbuf to discard bytes which
are not needed to fulfill the read request.

Work around the problem by using the full buffer length when
sbuf_finish() indicates an overflow. An overflowed sbuf with fixed size
is always full.

PR: kern/163076

MFC r229692:

Check the return value of sbuf_finish() in pfs_readlink() and return
ENAMETOOLONG if the buffer overflowed.

MFC: r229802
opt_inet6.h was missing from some files in the new NFS subsystem.
The effect of this was, for clients mounted via inet6 addresses,
that the DRC cache would never have a hit in the server. It also
broke NFSv4 callbacks when an inet6 address was the only one available
in the client. This patch fixes the above, plus deletes opt_inet6.h
from a couple of files it is not needed for.

MFC r228923, r228935, and r229007
Eliminate many of the unnecessary differences between the native and
paravirtualized pmap implementations for i386.

Fix a bug in the Xen pmap's implementation of
pmap_extract_and_hold(): If the page lock acquisition is retried,
then the underlying thread is not unpinned.

Wrap nearby lines that exceed 80 columns.

Merge r216333 and r216555 from the native pmap
When r207410 eliminated the acquisition and release of the page
queues lock from pmap_extract_and_hold(), it didn't take into
account that pmap_pte_quick() sometimes requires the page queues
lock to be held. This change reimplements pmap_extract_and_hold()
such that it no longer uses pmap_pte_quick(), and thus never
requires the page queues lock.

Merge r177525 from the native pmap
Prevent the overflow in the calculation of the next page
directory. The overflow causes the wraparound with consequent
corruption of the (almost) whole address space mapping.

Strictly speaking, r177525 is not required by the Xen pmap because
the hypervisor steals the uppermost region of the normal kernel
address space. I am nonetheless merging it in order to reduce the
number of unnecessary differences between the native and Xen pmap
implementations.

MFC r228746
The Xen pmap doesn't support superpages. So, there is no point in it
initializing structures, like the pv table, that are only used to
implement superpages. In fact, some of the unnecessary code in
pmap_init() was actually doing harm. It was preventing the kernel from
booting on virtual machines with more than 768 MB of memory.

MFC r227827
Define __FreeBSD_kernel__ macro in sys/param.h.

__FreeBSD_kernel__ indicates that this system uses the kernel of FreeBSD,
which by definition is always true on FreeBSD. This macro is also defined
on other systems that use the kernel of FreeBSD, such as GNU/kFreeBSD.

It is tempting to use this macro in userland code when we want to enable
kernel-specific routines, and in fact it's fine to do this in code that
is part of FreeBSD itself. However, be aware that as presence of this
macro is still not widespread (e.g. older FreeBSD versions, 3rd party
compilers, etc), it is STRONGLY DISCOURAGED to check for this macro in
external applications without also checking for __FreeBSD__ as an
alternative.

Approved by: kib (mentor)

MFC r228398
Avoid the possibility of integer overflow in the calculation of
VM_KMEM_SIZE_MAX. Specifically, if the user/kernel address space split
was changed such that the kernel address space was greater than or equal
to 2 GB, then overflow would occur.

PR: 161721

MFC r226163, r228317, and r228324
Fix the handling of an empty kmem map by sysctl_kmem_map_free().

Eliminate the possibility of 32-bit arithmetic overflow in the
calculation of vm_kmem_size that may occur if the system
administrator has specified a vm.vm_kmem_size tunable value that
exceeds the hard cap.

Eliminate stale numbers from a comment.

PR: 162741

MFC: 229965

Fix for PR 138526.

Add the ability for /dev/null and /dev/zero to accept
being set into non blocking mode via fcntl(). This
brings the code into compliance with IEEE Std 1003.1-2001
as referenced in another PR, 94729.

Reviewed by: jhb

MFC r228869,r228874:

Implement hwpmc counting PMC support for PowerPC G4+ (MPC745x/MPC744x).
Sampling is in progress.

Approved by: nwhitehorn (mentor)

MFC r230032, r230050, r230090, r230091 and r228493.
- Various XHCI and USB 3.0 related issues.
- USB 3.0 HUBs should work after this change.

MFC r230035:

Add PWM monitoring sysctl to G4 MDD (Windtunnel) fan driver. While there, clean
up some style nits.

Merge r229085 from head:

Default to not performing the early-boot memory tests when we detect we
are booting inside a VM. There are three reasons to disable this:

o It causes the VM host to believe that all the tested pages or RAM are
in use. This in turn may force the host to page out pages of RAM
belonging to other VMs, or otherwise cause problems with fair resource
sharing on the VM cluster.
o It adds significant time to the boot process (around 1 second/Gig in
testing)
o It is unnecessary - the host should have already verified that the
memory is functional etc.

Note that this simply changes the default when in a VM - it can still be
overridden using the hw.memtest.tests tunable.

Early MFC requested by: bz

MFC r228985,229881:

Unset the environment variables associated with individual menu items
before invoking the kernel.

Get rid of a spurious warning on the console when booting the kernel
from the interactive loader(8) prompt and beastie_disable="YES" is set
in loader.conf(5).

Submitted by: Devin Teske <devin dott teske fisglobal.com>

MFC r228265: critical_exit: ignore td_owepreempt if kdb_active is set

MFC r228718: ule: ensure that batch timeshare threads are scheduled
fairly

MFC r228710: opensolaris compat: fix vcmn_err so that panic(9) produces
a proper message

MFC r228426: syscons: make sc_puts static as it is used only privately

MFC r228430,228433: put sys/systm.h at its proper place or add it if missing

MFC r228267: zfsboot: print boot.config contents before parsing it

MFC r228266: zfs boot: allow file vdevs to be used in testing (e.g. with zfsboottest)

MFC 229665,229672,229700:
Remove the assertion from tcp_input() that rcv_nxt is always greater
than or equal to rcv_adv and fix tcp_twstart() to handle this case by
assuming the last window was zero rather than a negative value.

The code in tcp_input() already safely handled this case. It can happen
due to delayed ACKs along with a remote sender that sends data beyond
the window we previously advertised. If we have room in our socket buffer
for the extra data beyond the advertised window, we will accept it.
However, if the ACK for that segment is delayed, then we will not
effectively fixup rcv_adv to account for that extra data until the
next segment arrives and forces out an ACK. When that next segment
arrives, rcv_nxt will be beyond rcv_adv.

MFC 228961:
Further relax the strictness of enforcing allocations to only come from
decoded ranges. Pass any request for a specific range that fails because
it is not in a decoded range for an ACPI Host-PCI bridge up to the parent
to see if it can still be allocated. This is based on the assumption that
many BIOSes are inconsistent/broken and that settings programmed into BARs
or resources assigned to other built-in components are more trustworthy than
the list of decoded resource ranges in _CRS. This effectively limits the
decoded ranges to only being used for "wildcard" ranges when allocating
fresh resources for a BAR, etc. At some point I would like to only be
this permissive during an early scan of firmware-assigned resources during
boot and to be strict about all later allocations, but that isn't viable
currently.

MFC 228960:
Cap the priority calculated from the current thread's running tick count
at SCHED_PRI_RANGE to prevent overflows in the priority value. This can
happen due to irregularities with clock interrupts under certain
virtualization environments.

MFC 229519:
Fix 'make clean' for this module so it cleans up the generated gssd.h.

MFC 229429:
Some small fixes to CPU accounting for threads:
- Only initialize the per-cpu switchticks and switchtime in sched_throw()
for the very first context switch on APs during boot. This avoids a
small gap between the middle of thread_exit() and sched_throw() where
time is not accounted to any thread.
- In thread_exit(), update the timestamp bookkeeping to track the changes
to mi_switch() introduced by td_rux so that the code once again matches
the comment claiming it is mimicing mi_switch(). Specifically, only
update the per-thread stats directly and depend on ruxagg() to update
p_rux rather than adjusting p_rux directly. While here, move the
timestamp bookkeeping as late in the function as possible.