Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Merge the second part of the unmapped I/O changes. This enables the
infrastructure in the block layer and UFS filesystem as well as a few
drivers. The list of MFC revisions is long, so I won't quote changelogs.

r248508,248510,248511,248512,248514,248515,248516,248517,248518,
248519,248520,248521,248550,248568,248789,248790,249032,250936

Submitted by: kib
Approved by: kib
Obtained from: Netflix

MFC r250966:
Fix the data corruption on the swap-backed md. Assign the rv variable a
success code if the pager was not asked for the page.

MFC r243373:

Print correct unit number when attaching preloaded memory disks.
Retire now unused mdunits variable.

MFC r243372:

Disallow attaching preloaded memory disks via ioctl.

- The feature is dangerous because the kernel code didn't check
validity of the memory address provided from user space.
- It seems that mdconfig(8) never really supported attaching preloaded
memory disks.
- Preloaded memory disks are automatically attached during md(4)
initialization. Thus there shouldn't be much use for the feature.

PR: kern/169683

MFC r242744:
Zero the newly allocated md(4) swap-backed page to prevent random
kernel memory leakage to userspace.

MFC r238991:

Disallow sectorsize larger than MAXPHYS and mediasize smaller than
sectorsize.

PR: 169947

MFC r239065:
Stop including vm_param.h into vm_page.h. Include vm_param.h
explicitely for the kernel code which needs it.

MFC r239038:
Remove verbose unused commented out debugging printf.

Toss bogus mergeinfo.

MFC r235634

Fix and update battery status bits according to linux driver

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Include sys/sbuf.h directly.

Move the ZERO_REGION_SIZE to a machine-dependent file, as on many
architectures (i386, for example) the virtual memory space may be
constrained enough that 2MB is a large chunk. Use 64K for arches
other than amd64 and ia64, with special handling for sparc64 due to
differing hardware.

Also commit the comment changes to kmem_init_zero_region() that I
missed due to not saving the file. (Darn the unfamiliar development
environment).

Arch maintainers, please feel free to adjust ZERO_REGION_SIZE as you
see fit.

Requested by: alc
MFC after: 1 week
MFC with: r221853

Usa a globally visible region of zeros for both /dev/zero and the md
device. There are likely other kernel uses of "blob of zeros" than can
be converted.

Reviewed by: alc
MFC after: 1 week

Implement BIO_DELETE for vnode devices by simply overwriting the deleted
sectors with all-zeroes.

The zeroes come from a static buffer; null(4) uses a dynamic buffer for
the same purpose (for /dev/zero). It might be a good idea to have a
static, shared, read-only all-zeroes page somewhere in the kernel that
md(4), null(4) and any other code that needs zeroes could use.

Reviewed by: kib
MFC after: 3 weeks

Use the preload_fetch_addr() and preload_fetch_size() convenience
functions and only create the MD device when we have a non-zero
pointer and size.

Sponsored by: Juniper Networks

Add support for BIO_DELETE on swap-backed md(4). In the case of BIO_DELETE
covering the whole page, free the page. Otherwise, clear the region and
mark it clean. Not marking the page dirty could reinstantiate cleared
data, but it is allowed by BIO_DELETE specification and saves unneeded
write to swap.

Reviewed by: alc
Tested by: pho
MFC after: 2 weeks

Bio shall not be accessed after g_io_deliver(9).

Reported and tested by: pho
Reviewed by: ae, phk
MFC after: 1 week

Add missed ().

Noted by: alc
MFC after: 3 days

There is no point in calling vm_object_set_writeable_dirty() on an object
that is definitively known to be swap backed since its only effects are on
vnode-backed objects.

Reviewed by: kib

Add reporting of GEOM::candelete BIO_GETATTR for md(4) and geom_disk(4).
Non-zero value of attribute means that device supports BIO_DELETE.

Suggested and reviewed by: pjd
Tested by: pho
MFC after: 1 week

Add sysctl vm.md_malloc_wait, non-zero value of which switches malloc-backed
md(4) to using M_WAITOK malloc calls.

M_NOWAITOK allocations may fail when enough memory could be freed, but not
immediately. E.g. SU UFS becomes quite unhappy when metadata write return
error, that would happen for failed malloc() call.

Reported and tested by: pho
MFC after: 1 week

Allow the MDIOCATTACH ioctl operation to originate from within the kernel.
To protect against malicious software, we demand that the file name is at
a particular location (i.e. appended to the mdio structure) for it to be
treated as in-kernel.

- Remove some extra white space.
- Wrap g_md_dumpconf() prototype to 80 columns.

Convert md(4) to use alloc_unr(9) and alloc_unr_specific(9) for unit
number allocation. The old approach had some problems such as it allowed
an overflow to occur in the unit number calculation.

PR: kern/122288

Calculate nshift only once.

Also noted by: avg
MFC after: 1 week

Eliminate unnecessary page queues locking.

Lock the page around vm_page_activate() and vm_page_deactivate() calls
where it was missed. The wrapped fragments now protect wire_count with
page lock.

Reviewed by: alc

Fix panic on invalid 'mdconfig -at preload' usage.

PR: kern/80136

(S)LIST_HEAD_INITIALIZER takes a (S)LIST_HEAD as an argument.
Fix some wrong usages.
Note: this does not affect generated binaries as this argument is not used.

PR: 137213
Submitted by: Eygene Ryabinkin (initial version)
MFC after: 1 month

Implement global and per-uid accounting of the anonymous memory. Add
rlimit RLIMIT_SWAP that limits the amount of swap that may be reserved
for the uid.

The accounting information (charge) is associated with either map entry,
or vm object backing the entry, assuming the object is the first one
in the shadow chain and entry does not require COW. Charge is moved
from entry to object on allocation of the object, e.g. during the mmap,
assuming the object is allocated, or on the first page fault on the
entry. It moves back to the entry on forks due to COW setup.

The per-entry granularity of accounting makes the charge process fair
for processes that change uid during lifetime, and decrements charge
for proper uid when region is unmapped.

The interface of vm_pager_allocate(9) is extended by adding struct ucred *,
that is used to charge appropriate uid when allocation if performed by
kernel, e.g. md(4).

Several syscalls, among them is fork(2), may now return ENOMEM when
global or per-uid limits are enforced.

In collaboration with: pho
Reviewed by: alc
Approved by: re (kensmith)

Add cpu_flush_dcache() for use after non-DMA based I/O so that a
possible future I-cache coherency operation can succeed. On ARM
for example the L1 cache can be (is) virtually mapped, which
means that any I/O that uses temporary mappings will not see the
I-cache made coherent. On ia64 a similar behaviour has been
observed. By flushing the D-cache, execution of binaries backed
by md(4) and/or NFS work reliably.
For Book-E (powerpc), execution over NFS exhibits SIGILL once in
a while as well, though cpu_flush_dcache() hasn't been implemented
yet.

Doing an explicit D-cache flush as part of the non-DMA based I/O
read operation eliminates the need to do it as part of the
I-cache coherency operation itself and as such avoids pessimizing
the DMA-based I/O read operations for which D-cache are already
flushed/invalidated. It also allows future optimizations whereby
the bcopy() followed by the D-cache flush can be integrated in a
single operation, which could be implemented using on-chips DMA
engines, by-passing the D-cache altogether.

Add a new internal mount flag (MNTK_EXTENDED_SHARED) to indicate that a
filesystem supports additional operations using shared vnode locks.
Currently this is used to enable shared locks for open() and close() of
read-only file descriptors.
- When an ISOPEN namei() request is performed with LOCKSHARED, use a
shared vnode lock for the leaf vnode only if the mount point has the
extended shared flag set.
- Set LOCKSHARED in vn_open_cred() for requests that specify O_RDONLY but
not O_CREAT.
- Use a shared vnode lock around VOP_CLOSE() if the file was opened with
O_RDONLY and the mountpoint has the extended shared flag set.
- Adjust md(4) to upgrade the vnode lock on the vnode it gets back from
vn_open() since it now may only have a shared vnode lock.
- Don't enable shared vnode locks on FIFO vnodes in ZFS and UFS since
FIFO's require exclusive vnode locks for their open() and close()
routines. (My recent MPSAFE patches for UDF and cd9660 already included
this change.)
- Enable extended shared operations on UFS, cd9660, and UDF.

Submitted by: ups
Reviewed by: pjd (ZFS bits)
MFC after: 1 month

Remove unnecessary page queues locking around vm_page_wakeup(). (This
change is applicable to RELENG_7 but not RELENG_6.)

MFC after: 1 week

Add the possibility to specify "-o force" with "mdconfig -du".

Reviewed by: scottl
Approved by: rwatson (mentor)
Sponsored by: FreeBSD Foundation

Fix forced mdconfig -du. E.g. the following would previously
result in panic:

mdconfig -af blah.img -o force
mount /dev/md0 /mnt
mdconfig -du 0

Reviewed by: scottl
Approved by: rwatson (mentor)
Sponsored by: FreeBSD Foundation

Decontextualize the couplet VOP_GETATTR / VOP_SETATTR as the passed thread
was always curthread and totally unuseful.

Tested by: Giovanni Trematerra <giovanni dot trematerra at gmail dot com>

Remove the distinction between device minor and unit numbers.

Even though we got rid of device major numbers some time ago, device
drivers still need to provide unique device minor numbers to make_dev().
These numbers are only used inside the kernel. They are not related to
device major and minor numbers which are visible in devfs. These are
actually based on the inode number of the device.

It would eventually be nice to remove minor numbers entirely, but we
don't want to be too agressive here.

Because the 8-15 bits of the device number field (si_drv0) are still
reserved for the major number, there is no 1:1 mapping of the device
minor and unit numbers. Because this is now unused, remove the
restrictions on these numbers.

The MAXMAJOR definition was actually used for two purposes. It was used
to convert both the userspace and kernelspace device numbers to their
major/minor pair, which is why it is now named UMINORMASK.

minor2unit() and unit2minor() have now become useless. Both minor() and
dev2unit() now serve the same purpose. We should eventually remove some
of them, at least turning them into macro's. If devfs would become
completely minor number unaware, we could consider using si_drv0 directly,
just like si_drv1 and si_drv2.

Approved by: philip (mentor)

Zero sc->vnode if mdsetcred() fails.

This fixes the panic which happens when mdcreate_vnode() calls vn_close()
and mddestroy() calls it again further down the error handling path.

Reviewed by: kris, kib
MFC after: 3 days

VOP_LOCK1() (and so VOP_LOCK()) and VOP_UNLOCK() are only used in
conjuction with 'thread' argument passing which is always curthread.
Remove the unuseful extra-argument and pass explicitly curthread to lower
layer functions, when necessary.

KPI results broken by this change, which should affect several ports, so
version bumping and manpage update will be further committed.

Tested by: kris, pho, Diego Sardina <siarodx at gmail dot com>

vn_lock() is currently only used with the 'curthread' passed as argument.
Remove this argument and pass curthread directly to underlying
VOP_LOCK1() VFS method. This modify makes the code cleaner and in
particular remove an annoying dependence helping next lockmgr() cleanup.
KPI results, obviously, changed.

Manpage and FreeBSD_version will be updated through further commits.

As a side note, would be valuable to say that next commits will address
a similar cleanup about VFS methods, in particular vop_lock1 and
vop_unlock.

Tested by: Diego Sardina <siarodx at gmail dot com>,
Andrea Di Pasquale <whyx dot it at gmail dot com>

Put back devstat support that was lost during GEOM transition. Initially,
I've tried to move md(4) to use geom_disk class, like real disks do, but
this requires major rework of some of the existing features such as
configuration dumping for example. Therefore just putting devstat support
directly into md(4) seems to be optimal solution.

Now you can see md(4) stats in `systat -vm' again.

MFC after: 2 weeks

Rename the kthread_xxx (e.g. kthread_create()) calls
to kproc_xxx as they actually make whole processes.
Thos makes way for us to add REAL kthread_create() and friends
that actually make theads. it turns out that most of these
calls actually end up being moved back to the thread version
when it's added. but we need to make this cosmetic change first.

I'd LOVE to do this rename in 7.0 so that we can eventually MFC the
new kthread_xxx() calls.

Commit 14/14 of sched_lock decomposition.
- Use thread_lock() rather than sched_lock for per-thread scheduling
sychronization.
- Use the per-process spinlock rather than the sched_lock for per-process
scheduling synchronization.

Tested by: kris, current@
Tested on: i386, amd64, ULE, 4BSD, libthr, libkse, PREEMPTION, etc.
Discussed with: kris, attilio, kmacy, jhb, julian, bde (small parts each)

Revert UF_OPENING workaround for CURRENT.
Change the VOP_OPEN(), vn_open() vnode operation and d_fdopen() cdev operation
argument from being file descriptor index into the pointer to struct file.

Proposed and reviewed by: jhb
Reviewed by: daichi (unionfs)
Approved by: re (kensmith)

Resolve two deadlocks that could be caused by busy md device backed
by vnode. Allow for md thread and the thread that owns lock on vnode
backing the md device to do the write even when runningbufspace is
exhausted.

Tested by: Peter Holm
Reviewed by: tegge
MFC after: 2 weeks

Style nits.

Fix md(4) panic which occurs when I/O request different than
BIO_READ/BIO_WRITE is sent to vnode-backed provider (BIO_DELETE or
BIO_FLUSH).

Reported by: ceri

Add support for BIO_FLUSH to vnode-backed md(4) devices based on
VOP_FSYNC().

- Conditionally acquire Giant in mdstart_vnode(), mdcreate_vnode(), and
mddestroy() only if the file is from a non-MPSAFE VFS.
- No longer unconditionally hold Giant in the md kthread for vnode-backed
kthreads.
- Improve the handling of the thread exit race when destroying an md
device.

Teach md(4) and mdconfig(8) how to understand XML. Right now there won't be
a problem with listing large number of md(4) devices. Either 'list' or
'query' mode uses XML.

Additionally, new functionality was introduced. It's possible to pass
multiple devices to -u:

# ./mdconfig -l -u md0,md1

Approved by: cognet (mentor)

make sure that the start and end preloaded MFS markers are
in contiguous strings, and that the compiler does not optimize them
away because it thinks they are unused.

Call NDFREE() only when vn_open() succeeded.

MFC after: 3 days

o Fix typos in the comments.

Submitted by: Wojciech A. Koszek

Normalize a significant number of kernel malloc type names:

- Prefer '_' to ' ', as it results in more easily parsed results in
memory monitoring tools such as vmstat.

- Remove punctuation that is incompatible with using memory type names
as file names, such as '/' characters.

- Disambiguate some collisions by adding subsystem prefixes to some
memory types.

- Generally prefer lower case to upper case.

- If the same type is defined in multiple architecture directories,
attempt to use the same name in additional cases.

Not all instances were caught in this change, so more work is required to
finish this conversion. Similar changes are required for UMA zone names.

Make sure that the worker thread knows the type early enough to
grab Giant for vnode backing.

Found by: pho & tegge

Fix configuration locking in MD.

Remove md_mtx.

Remove GIANT from the mdctl device driver and avoid DROP_GIANT,
PICKUP_GIANT and geom events since we can call into GEOM directly
now.

Pick up Giant around vn_close().

Apply an exclusive sx around mdctls ioctl and preloading to protect
lists etc..

Don't initialize our lock (md_mtx or md_sx) from a
SYSINIT when there is a perfectly good pair of _fini/_init
functions to do it from.

Prune any final fractional sector from the mediasize to
keep GEOM happy.

Cleanups:

Unify MDIOVERSION check in (x)mdctlioctl()

Add pointer to start() routine to softc to eliminate a switch{}

Inline guts of mddetach().

Always pass error pointer to mdnew(), simplify implementation.

Do not destroy the queue mutex until the thread is done with it.

- Add md_mtx lock to protect ID number and list of devices.
- Always check mdnew() return value, as even in !autounit case
kthread_create() can fail.

Those two changes fix serval panics provked by simple stress test.

Tested by: Kris The BugMagnet
MFC after: 3 days

Ensure that file flags such as schg, sappnd (and others) are honored
by md(4). Before this change, it was possible to by-pass these flags
by creating memory disks which used a file as a backing store and
writing to the device.

This was discussed by the security team, and although this is problematic,
it was decided that it was not critical as we never guarantee that root will
be restricted.

This change implements the following behavior changes:

-If the user specifies the readonly flag, unset write operations before
opening the file. If the FWRITE mask is unset, the device will be
created with the MD_READONLY mask set. (readonly)
-Add a check in g_md_access which checks to see if the MD_READONLY mask
is set, if so return EROFS
-Do not gracefully downgrade access modes without telling the user. Instead
make the user specify their intentions for the device (assuming the file is
read only). This seems like the more correct way to handle things.

This is a RELENG_6 candidate.

PR: kern/84635
Reviewed by: phk

Request a CPU private mapping from sf_buf_alloc(). If the swap-backed
memory disk is larger than the number of available sf_bufs, this improves
performance on SMPs by eliminating interprocessor TLB shootdowns. For
example, with 6656 sf_bufs, the default on my test machine, and a 256MB
swap-backed memory disk, I see the command
"dd if=/dev/md0 of=/dev/null bs=64k" achieve ~489MB/sec with the default,
shared mappings, and ~587MB/sec with CPU private mappings.

Use MAXMINOR

- Don't destroy UMA zone on error in mdcreate_malloc(), because we need it
in mddestroy() to properly free already allocated memory.
This fixes a panic when we want to create too big memory backed device
with preallocate memory (-o reserve).
- Remove redundant { }.

MFC after: 1 week

Add a couple of mtx_asserts() to try to narrow down the window on
a bug repeatedly reported.

Start each of the license/copyright comments with /*-, minor shuffle of lines

Add needed synchronization to the error handling code that was introduced
in revision 1.141.

Lock assertion failures reported by: Kris Kennaway

Stop explicitly touching td_base_pri outside of the scheduler and simply
set a thread's priority via sched_prio() when that is the desired action.
The schedulers will start managing td_base_pri internally shortly.

Rewrite piece of code which I committed some time ago that allows to
show file name for 'mdconfig -l -u <x>' command.
This allows to preserve API/ABI compatibility with version 0 (that's why
I changed version number back to 0) and will allow to merge this change
to RELENG_5.

MFC after: 5 days

Fix the MDIOCDETACH ioctl() for md(4). Now that the md_file field in
the mdio structure is an array and not a pointer, we cannot test for
it to be NULL. It never is. Instead, test for md_file[0] to be '\0'.

Be consistent and use 'if (error != 0)' instead of 'if (error)' everywhere.

For file backed md(4) devices output their source file via
'mdconfig -l -u <unit>'.
Bump version number, as this change breaks ABI/API.

Don't explicitly call g_waitidle(), it happens automagically now.

Account for failure in vm_pager_allocate() or vm_pager_get_pages() in
md(8). The former is generally not going to fail, but the latter can
fail when the underlying swap device returns an error.

There are still plenty of other places where vm_pager_get_pages() failing
will lead directly to crashes, so it's a good idea to put your swap on
RAID if you care enough to put any of your disks on RAID....

Actually this order (unlock, wakeup) in this case is race-safe and can
save us 2 context switches.

Explained by: njl

- Make md(4) 64-bit clean.
After this change it should be possible to use very big md(4) devices.
- Clean up and simplify the code a bit.
- Use humanize_number(3) to print size of md(4) devices.
- Add 't' suffix which stands for terabyte.
- Make '-S' to really work with all types of devices.
- Other minor changes.

There is no need to keep 'npage' value inside our softc structure,
it is only used in one function. While doing so, change its type to
vm_ooffset_t.
We are still limited for swap-backed devices to 16TB on 32-bit architectures
where PAGE_SIZE is 4096 bytes.

- Do not use bio_pblkno as it is going away anyway.
- Prefer bio_length than bio_bcount.

First wakeup, then unlock.

Type 'int' is too small for 'i' and 'lastp' variables. Use proper type,
which is vm_pindex_t (unsigned 64bit on i386).

Deallocate VM object on failure.

One more missing NDFREE(9).

- Don't forget about NDFREE() in case of vn_open() failure.
- Don't forget about vn_close() in case of failure.

Fix UMA zone leak.

Use bioq_takefirst()

Don't g_waitidle() when initializing a preloaded md. This fixes a
deadlock which otherwise occurs during the boot process.

Reported by: kensmith
MFC after: 3 days
(assuming that re@ approves)

When creating a new md, wait for geom's event queue to become empty
before returning. Device nodes are created via the "taste" mechanism,
so this is necessary in order to make sure that devfs entries are
created before mdconfig(8) returns.

This may be a MFC candidate for 5.3.

Suggested by: phk

Tag all geom classes in the tree with a version number.

Use a ->fini() from the geom class to destroy the control device.
Use default initialization of geom methods.

Do a pass over all modules in the kernel and make them return EOPNOTSUPP
for unknown events.

A number of modules return EINVAL in this instance, and I have left
those alone for now and instead taught MOD_QUIESCE to accept this
as "didn't do anything".

Do the dreaded s/dev_t/struct cdev */
Bump __FreeBSD_version accordingly.

Fix panic which occurs when given sector size for memory-backed device
is less than DEV_BSIZE (512) bytes.

Reported by: Mike Bristow <mike@urgle.com>
Approved by: phk

Ooops, removed this acknowledgement bogusly.

Eagle Eyes: bde

Remove advertising clause from University of California Regent's
license, per letter dated July 22, 1999 and email from Peter Wemm,
Alan Cox and Robert Watson.

Approved by: core, peter, alc, rwatson

In some cases, sf_buf_alloc() should sleep with pri PCATCH; in others, it
should not. Add a new parameter so that the caller can specify which is
the case.

Reported by: dillon

Fix a bug with preloaded image -- for some reason [that i don't
completely understand], md_takeroot() runs before md_preloaded(),
rendering both useless.
As a fix, move the body (effectively one line!) of md_takeroot()
into md_preloaded(), and get rid of the stuff that has become useless.

Bug and fix reported 10 days ago on -current, no reply.

- Remove some unused #includes.
- Apply some style fixes to mdstart_swap().

Utilize sf_buf_alloc() and sf_buf_free() to implement the ephemeral
mappings required by mdstart_swap(). On i386, if the ephemeral mapping
is already in the sf_buf mapping cache, a swap-backed md performs
similarly to a malloc-backed md. Even if the ephemeral mapping is not
cached, this implementation is still faster. On 64-bit platforms, this
change has the effect of using the direct virtual-to-physical mapping,
avoiding ephemeral mapping overheads, such as TLB shootdowns on SMPs.

On a 2.4GHz, 400MHz FSB P4 Xeon configured with 64K sf_bufs and
"mdmfs -S -o async -s 128m md /mnt"

before:
dd if=/dev/md0 of=/dev/null bs=64k
134217728 bytes transferred in 0.430923 secs (311465697 bytes/sec)

after with cold sf_buf cache:
dd if=/dev/md0 of=/dev/null bs=64k
134217728 bytes transferred in 0.367948 secs (364773576 bytes/sec)

after with warm sf_buf cache:
dd if=/dev/md0 of=/dev/null bs=64k
134217728 bytes transferred in 0.252826 secs (530870010 bytes/sec)

malloc-backed md:
dd if=/dev/md0 of=/dev/null bs=64k
134217728 bytes transferred in 0.253126 secs (530240978 bytes/sec)

Allow swap-backed devices to run without Giant.

Fix a long-standing deadlock issue with vnode backed md(4) devices:

On vnode backed md(4) devices over a certain, currently undetermined
size relative to the buffer cache our "lemming-syncer" can provoke
a buffer starvation which puts the md thread to sleep on wdrain.

This generally tends to grind the entire system to a stop because the
event that is supposed to wake up the thread will not happen until a fair
bit of the piled up I/O requests in the system finish, and since a lot
of those are on a md(4) vnode backed device which is currently waiting
on wdrain until a fair amount of the piled up ... you get the picture.

The cure is to issue all VOP_WRITES on the vnode backing the device
with IO_SYNC.

In addition to more closely emulating a real disk device with a
non-lying write-cache, this makes the writes exempt from rate-limited
(there to avoid starving the buffer cache) and consequently prevents
the deadlock.

Unfortunately performance takes a hit.

Add "async" option to give people who know what they are doing the
old behaviour.

kthread_exit() no longer requires Giant, so don't force callers to acquire
Giant just to call kthread_exit().

Requested by: many

Make swapbacked md(4) devices respect the -x and -y emulation arguments.

Use DEV_BSIZE byte sectors instead of PAGE_SIZE byte sectors for
swap-backed memory disks. This reduces filesystem allocation overhead
and makes swap-backed memory disks compatible with broken code (dd,
for example) which expects to see 512 byte sectors. The size of a
swap-backed memory disk must still be a multiple of the page size.

When performing page-aligned operations, this change has zero
performance impact.

Reviewed by: phk
Approved by: rwatson (mentor)

Device megapatch 4/6:

Introduce d_version field in struct cdevsw, this must always be
initialized to D_VERSION.

Flip sense of D_NOGIANT flag to D_NEEDGIANT, this involves removing
four D_NOGIANT flags and adding 145 D_NEEDGIANT flags.

Allow specification of a geometry for vnode backed devices as well as
for malloc backed devices.

Fix a locking problem with MD_ROOT_SIZE.

Retire md(4)'s static major number.

Use the class->init() to hitch up preload devices, rather than rely on
the "old" SYSINIT. This makes sure things happen in the right order.

XXX: md(4) needs to be fully geom-ified and in particluar /dev/md.ctl
should be abandonded for the GEOM OaM api.

Approved by: re@

Don't initialize unused bio_blkno field.

The present defaults for the open and close for device drivers which
provide no methods does not make any sense, and is not used by any
driver.

It is a pretty hard to come up with even a theoretical concept of
a device driver which would always fail open and close with ENODEV.

Change the defaults to be nullopen() and nullclose() which simply
does nothing.

Remove explicit initializations to these from the drivers which
already used them.

Consistently use the BSD u_int and u_short instead of the SYSV uint and
ushort. In most of these files, there was a mixture of both styles and
this change just makes them self-consistent.

Requested by: bde (kern_ktrace.c)

Change the implementation of swap backing to use the VM system in normal
ways, and drop the need for vm_pager_strategy().

Add fdidx argument to vn_open() and vn_open_cred() and pass -1 throughout.

Remove 256 unit limit, there is no evil minor number encoding to
deal with any more.

Spotted by: "Darren Freestone" <df@cops.org>

Remove the G_CLASS_INITIALIZER, we do not need it anymore.

The IO_NOWDRAIN and B_NOWDRAIN hacks are no longer needed to prevent
deadlocks with vnode backed md(4) devices because md now uses a
kthread to run the bio requests instead of doing it directly from
the bio down path.

Use vm_object_deallocate(), not vm_pager_deallocate(), to destroy a
vm object. (vm_pager_deallocate() does not, in fact, destroy a vm object.)

Approved by: re (scottl)
Reviewed by: phk

Call g_wither_geom(), instead of just setting the flag.

Add a couple of undocumented test options to MD(4) to aid in regression
testting of GEOM.

Remove all references to BIO_SETATTR. We will not be using it.

Use bioq_flush() to drain a bio queue with a specific error code.
Retain the mistake of not updating the devstat API for now.

Spell bioq_disksort() consistently with the remaining bioq_*().

#include <geom/geom_disk.h> where this is more appropriate.

Don't include <sys/disk.h>.

remove a blank line.

Allocate the toplevel indir with M_WAITOK to avoid complicating things
needlessly.

Detected by: rwatsons EvilMalloc(9)

Change g_class initialization to sparse format.

Including <sys/stdint.h> is (almost?) universally only to be able to use
%j in printfs, so put a newsted include in <sys/systm.h> where the printf
prototype lives and save everybody else the trouble.

Centralize the devstat handling for all GEOM disk device drivers
in geom_disk.c.

As a side effect this makes a lot of #include <sys/devicestat.h>
lines not needed and some biofinish() calls can be reduced to
biodone() again.

Add a "-S sectorsize" option to enable Kirk to find a bug :-)

Gigacommit to improve device-driver source compatibility between
branches:

Initialize struct cdevsw using C99 sparse initializtion and remove
all initializations to default values.

This patch is automatically generated and has been tested by compiling
LINT with all the fields in struct cdevsw in reverse order on alpha,
sparc64 and i386.

Approved by: re(scottl)

Back out M_* changes, per decision of the TRB.

Approved by: trb

Mark our provider with G_PF_CANDELETE in the cases where this is actually
the case.

NO_GEOM cleanup: unifdef

Implement MDIOCLIST which returns the unit numbers of configured md(4)
devices.

We use the md_pad[] array and if there are more units than its size the
last returned unit number will be -1, but the number of units returned
is correct.

Remove M_TRYWAIT/M_WAITOK/M_WAIT. Callers should use 0.
Merge M_NOWAIT/M_DONTWAIT into a single flag M_NOWAIT.

OK Ok, so I didn't check the NO_GEOM case for the final version...

Stumbled on by: bde

Enable the new h0h0magic code which on GEOM kernels make the md(4)
driver a _real_ GEOM driver.

Add a mutex around the per unit bioqueue.

Only grab giant in the per unit kthread for SWAP and VNODE backed devices.

Initialize the bioq before the kthread gets a chance to study it.

Don't lock Giant in mddone_swap, we shouldn't need it.

Remove the printf which announces the creation of malloc disks: it is
inconsistent when we do not do it for swap or vnode.

We still printf for preloaded disks because of the weak debugging
options people have in embedded/tiny environments where this is
usually used.

Add code to make md(4) a GEOM device driver instead of relying in
the disk mini-layer.

This is currently not enabled.

Shift things around a bit in preparation for future evilness.

Move the check for the MD_SHUTDOWN flag to before the tsleep() call
in the per-device kthread. This ensures that synchronisation with
mddestroy() succeeds even if the kthread was not waiting in tsleep()
at the time of the wakeup(). Among other things, this fixes the
problem of mdconfig getting stuck when an attempt is made to use a
zero-length file as a vnode-type backing store.

Approved by: re

We want /dev/md0 for ramdisk roots, not /dev/md0c.

Sponsored by: DARPA & NAI Labs

Use ENOSPC error return, not ENOMEM.

Use %jd rather than %lld.

MODINFO_SIZE metadata has type size_t, not unsigned. This makes preloaded
md root work on sparc64.

Some kernel threads try to do significant work, and the default KSTACK_PAGES
doesn't give them enough stack to do much before blowing away the pcb.
This adds MI and MD code to allow the allocation of an alternate kstack
who's size can be speficied when calling kthread_create. Passing the
value 0 prevents the alternate kstack from being created. Note that the
ia64 MD code is missing for now, and PowerPC was only partially written
due to the pmap.c being incomplete there.
Though this patch does not modify anything to make use of the alternate
kstack, acpi and usb are good candidates.

Reviewed by: jake, peter, jhb

Put the casts on the right hand side of =.

Initialize fwsectors/fwheads to allow the DIOCGFWSECTORS and
DIOCGFWHEADS ioctls to return meaningful values to disklabel/newfs

Approved by: phk

(This commit touches about 15 disk device drivers in a very consistent
and predictable way, and I apologize if I have gotten it wrong anywhere,
getting prior review on a patch like this is not feasible, considering
the number of people involved and hardware availability etc.)

If struct disklabel is the messenger: kill the messenger.

Inside struct disk we had a struct disklabel which disk drivers used to
communicate certain metrics to the disklayer above (GEOM or the disk
mini-layer). This commit changes this communication to use four
explicit fields instead.

Amongst the benefits is that the fields do not get overwritten by
wrong or bogus on-disk disklabels.

Once that is clear, <sys/disk.h> which is included in the drivers
no longer need to pull <sys/disklabel.h> and <sys/diskslice.h> in,
the few places that needs them, have gotten explicit #includes for
them.

The disklabel inside struct disk is now only for internal use in
the disk mini-layer, so instead of embedding it, we malloc it as
we need it.

This concludes (modulus any mistakes) the series of disklabel related
commits.

I belive it all amounts to a NOP for all the rest of you :-)

Sponsored by: DARPA & NAI Labs.

Replace (ab)uses of "NULL" where "0" is really meant.

Yet another warning fix for 64 bits platforms.

Reviewed by: phk

mdcreate_vnode() isn't correctly clearing things out of the linked
list if the file is of 0 size or mdsetcred() fails.

Submitted by: Martin Faxer <gmh003532@brfmasthugget.se>

- Whitespace only: use return statement consistentlt (return (foo), not
return(foo)), kill extra blank names between function names;
- fix format string in printf(): devtoname() returns string, not pointer.

Use a per-device worker thread to avoid blocking in mdstrategy()
until the I/O completes. This fixes some easily reproducable deadlocks
that occur when using md(4) with GEOM.

Reviewed by: phk

Mis-edit in last commit.

Be a bit smarter about rewriting data so we don't loose too much performance.

Sponsored by: DARPA & NAI Labs.

Use an umazone per unit for allocating the sectors for malloc backing.

Clean up things properly when we unconfigure malloc backed units.

Sponsored by: DARPA & NAI Labs.

Give the "malloc" backing of md(4) an adaptive multilevel index tree to
remove the need for a contiguous array with pointers to all the sectors.

Try to make failure to malloc(9) memory a non-hang situation.

Eventually this will allow us to test the 64bit cleanness of the disk
I/O patch, but more work is outstanding here and elsewhere.

Sponsored by: DARPA & NAI Labs.

Fix a memory-leak when configuring a vnode backed md(4) device fails.

Submitted by: Martin Faxér <gmh003532@brfmasthugget.se>
MFC after: 4 weeks

Remove unused include.

The previous commit missed fixing 2 old printf format errors and
introduced a format printf error.

Fix printf warning caused by recent changes in bio_pblkno's type.

Introduce the new 64-bit size disk block, daddr64_t. Change
the bio and buffer structures to have daddr64_t bio_pblkno,
b_blkno, and b_lblkno fields which allows access to disks
larger than a Terabyte in size. This change also requires
that the VOP_BMAP vnode operation accept and return daddr64_t
blocks. This delta should not affect system operation in
any way. It merely sets up the necessary interfaces to allow
the development of disk drivers that work with these larger
disk block addresses. It also allows for the development of
UFS2 which will use 64-bit block addresses.

Simple p_ucred -> td_ucred changes to start using the per-thread ucred
reference.

Staticize the malloc definitions.

Obtained from: ~bde/sys.dif.gz

Gah! last commit botched indentation, fix indentation and some other
white-space nits while at it.

Restructure slightly, eliminating some repetitive source lines and
making GEOM patches simpler and more readable at the same time.

Actually make use of the md_version field of 'struct mdio'. In order
not to needlessly break compatibility, decrement MDIOVERSION to 0.

Approved by: phk

Implement IO_NOWDRAIN and B_NOWDRAIN - prevents the buffer cache from blocking
in wdrain during a write. This flag needs to be used in devices whos
strategy routines turn-around and issue another high level I/O, such as
when MD turns around and issues a VOP_WRITE to vnode backing store, in order
to avoid deadlocking the dirty buffer draining code.

Remove a vprintf() warning from MD when the backing vnode is found to be
in-use. The syncer of buf_daemon could be flushing the backing vnode at
the time of an MD operation so the warning is not correct.

MFC after: 1 week

Change the kernel's ucred API as follows:
- crhold() returns a reference to the ucred whose refcount it bumps.
- crcopy() now simply copies the credentials from one credential to
another and has no return value.
- a new crshared() primitive is added which returns true if a ucred's
refcount is > 1 and false (0) otherwise.

Use crhold() instead of crdup(). The md(4) driver doesn't modify the ucred
that it uses, so it merely needs to bump its refcount to make it immutable
rather than obtain its own copy.

KSE Milestone 2
Note ALL MODULES MUST BE RECOMPILED
make the kernel aware that there are smaller units of scheduling than the
process. (but only allow one thread per process at this time).
This is functionally equivalent to teh previousl -current except
that there is a thread associated with each process.

Sorry john! (your next MFC will be a doosie!)

Reviewed by: peter@freebsd.org, dillon@freebsd.org

X-MFC after: ha ha ha ha

OOPS, remove local change that somehow slipped into a commit (I swear that
I already deleted it some time ago). This should fix problem people have with
unsefined reference to `MD_PRELOAD_COMPRESSED'.

Submitted by: Manfred Antar <null@pozo.com>

- On module unload try to detach all configured disks and let unload proceed
if all disks were detached sucessfully;
- use consistent style for return statements and fix several others style
inconsistencies.

Reviewed by: ru
Approved by: phk

There is no MD_OBJET disk type, it's actually MD_SWAP. I guess the
former was either a previous or proposed name that kind of snuck in.

Introduce a force option, MD_FORCE, that instructs the driver to
bypass some extra anti-foot-shooting measures. Currently, its only
effect is to allow detaching a device while it's still open (e.g.,
mounted). This is useful for testing how the system reacts to a disk
suddenly going away, which can happen with some removeable media.

At this point, the force option is only checked on detach, so it
would've been possible to allow the option to be passed with the
MDIOCDETACH operation. This was not done to allow the possibility of
having the force flag influence other tests in the future, which may
not necessarily deal with detaching the device.

Reviewed by: sobomax
Approved by: phk

- Deny detaching requests until device is still open, otherwise it is possible
to hang or panic kernel by detaching disk from which fs is mounted;
- replace "md" with MD_NAME in yet another place.

Reviewed by: phk
Approved by: phk

Make sure the total number of sectors is not 0 for a vnode-type md to
avoid a division by zero which would occur on open() in this case.

Reviewed by: phk

Use MD_NAME and MDCTL_NAME constants where appropriate.

With Alfred's permission, remove vm_mtx in favor of a fine-grained approach
(this commit is just the first stage). Also add various GIANT_ macros to
formalize the removal of Giant, making it easy to test in a more piecemeal
fashion. These macros will allow us to test fine-grained locks to a degree
before removing Giant, and also after, and to remove Giant in a piecemeal
fashion via sysctl's on those subsystems which the authors believe can
operate without Giant.

We don't need the vm lock to perform a few simple calculations on the
md device's softc.

Remove MFS compat bits.

Acquire vm_mtx before calling vm_pager_deallocate.

Reviewed by: phk

Sort includes.

Introduce a global lock for the vm subsystem (vm_mtx).

vm_mtx does not recurse and is required for most low level
vm operations.

faults can not be taken without holding Giant.

Memory subsystems can now call the base page allocators safely.

Almost all atomic ops were removed as they are covered under the
vm mutex.

Alpha and ia64 now need to catch up to i386's trap handlers.

FFS and NFS have been tested, other filesystems will need minor
changes (grabbing the vm lock when twiddling page properties).

Reviewed (partially) by: jake, jhb

Polish error handling code using biofinish()

Actually biofinish(struct bio *, struct devstat *, int error) is more general
than the bioerror().

Most of this patch is generated by scripts.

Fix a panic if MD devices were left half-created.

XXX: the real bug is that devstat isn't part of the disk minilayer.

PR: 27158
Submitted by: Anders Nordby <anders@fix.no>

Undo part of the tangle of having sys/lock.h and sys/mutex.h included in
other "system" header files.

Also help the deprecation of lockmgr.h by making it a sub-include of
sys/lock.h and removing sys/lockmgr.h form kernel .c files.

Sort sys/*.h includes where possible in affected files.

OK'ed by: bde (with reservations)

Fix a reference to the "vn" driver in a warning message.

Use a more BIOS friendly geometry.

Submitted by: joe

Make "md" and "mdctl" macroized parameters.

Implement "-l" option to mdconfig which can list one or all md devices.

Submitted by: Dima Dorfman <dima@unixfreak.org>

Make md/mdconfig do kld.

Submitted by: dcs

Remove devstat entries in mddelete()

Spotted: tegge

General cleanup.

Fix a maybe-not-so-harmless warning.

Either cvs(1) or I forgot this file in my last commit.

Please see commit log for rev 1.4 of src/sbin/mdconfig/mdconfig.c

This is the first snapshot of the new all-singing-and-dancing md(4).

Using the mdconfig(8) program you can now configure memory disks
on malloc(9), swap or a file/vnode. preloaded md disks also work
as usual.

Enforce disk unit numbers upper limit in cloning.

Convert more malloc+bzero to malloc+M_ZERO.

Submitted by: josh@zipperup.org
Submitted by: Robert Drehmel <robd@gmx.net>

Avoid the modules madness I inadvertently introduced by making the
cloning infrastructure standard in kern_conf. Modules are now
the same with or without devfs support.

If you need to detect if devfs is present, in modules or elsewhere,
check the integer variable "devfs_present".

This happily removes an ugly hack from kern/vfs_conf.c.

This forces a rename of the eventhandler and the standard clone
helper function.

Include <sys/eventhandler.h> in <sys/conf.h>: it's a helper #include
like <sys/queue.h>

Remove all #includes of opt_devfs.h they no longer matter.

* Completely rewrite the alpha busspace to hide the implementation from
the drivers.
* Remove legacy inx/outx support from chipset and replace with macros
which call busspace.
* Rework pci config accesses to route through the pcib device instead of
calling a MD function directly.

With these changes it is possible to cleanly support machines which have
more than one independantly numbered PCI busses. As a bonus, the new
busspace implementation should be measurably faster than the old one.

Remove all traces of Julians DEVFS (incl from kern/subr_diskslice.c)

Remove old DEVFS support fields from dev_t.

Make uid, gid & mode members of dev_t and set them in make_dev().

Use correct uid, gid & mode in make_dev in disk minilayer.

Add support for registering alias names for a dev_t using the
new function make_dev_alias(). These will show up as symlinks
in DEVFS.

Use makedev() rather than make_dev() for MFSs magic devices to prevent
DEVFS from noticing this abuse.

Add a field for DEVFS inode number in dev_t.

Add new DEVFS in fs/devfs.

Add devfs cloning to:
disk minilayer (ie: ad(4), sd(4), cd(4) etc etc)
md(4), tun(4), bpf(4), fd(4)

If DEVFS add -d flag to /sbin/inits args to make it mount devfs.

Add commented out DEVFS to GENERIC

Rename MDNSECT to MD_NSECT and declare it as something that isn't
default in NOTES.

Requested by: bde
Approved by: phk

Fix the "almost clone" semantics.

Separate the struct bio related stuff out of <sys/buf.h> into
<sys/bio.h>.

<sys/bio.h> is now a prerequisite for <sys/buf.h> but it shall
not be made a nested include according to bdes teachings on the
subject of nested includes.

Diskdrivers and similar stuff below specfs::strategy() should no
longer need to include <sys/buf.> unless they need caching of data.

Still a few bogus uses of struct buf to track down.

Repocopy by: peter

Complete the bio/buf divorce for all code below devfs::strategy

Exceptions:
Vinum untouched. This means that it cannot be compiled.
Greg Lehey is on the case.

CCD not converted yet, casts to struct buf (still safe)

atapi-cd casts to struct buf to examine B_PHYS

Remove B_READ, B_WRITE and B_FREEBUF and replace them with a new
field in struct buf: b_iocmd. The b_iocmd is enforced to have
exactly one bit set.

B_WRITE was bogusly defined as zero giving rise to obvious coding
mistakes.

Also eliminate the redundant struct buf flag B_CALL, it can just
as efficiently be done by comparing b_iodone to NULL.

Should you get a panic or drop into the debugger, complaining about
"b_iocmd", don't continue. It is likely to write on your disk
where it should have been reading.

This change is a step in the direction towards a stackable BIO capability.

A lot of this patch were machine generated (Thanks to style(9) compliance!)

Vinum users: Greg has not had time to test this yet, be careful.

Initialize type correctly.

Retire MFS_ROOT and MFS_ROOT_SIZE options from the MFS implementation.

Add MD_ROOT and MD_ROOT_SIZE options to the md driver.

Make the md driver handle MFS_ROOT and MFS_ROOT_SIZE options for compatibility.

Add md driver to GENERIC, PCCARD and LINT.

This is a cleanup which removes the need for some of the worse hacks in
MFS: We really want to have a rootvnode but MFS on a preloaded image
doesn't really have one. md is a true device, so it is less trouble.

This has been tested with make release, and if people remember to add
the "md" pseudo-device to their kernels, PicoBSD should be just fine
as well. If people have no other use for MFS, it can be removed from
the kernel.

Teach the md driver to use preloaded files of type "md_image".

remove unused #include

Remove five now unused fields from struct cdevsw. They should never
have been there in the first place. A GENERIC kernel shrinks almost 1k.

Add a slightly different safetybelt under nostop for tty drivers.

Add some missing FreeBSD tags

Fix includes (remove unnecessary ones, reorder necessary ones.)
Also, correct an %x to be %lx.

Reviewed by: phk

fix a buglet which jordan made me provoke :-)

Add an experimental Memory-Disk driver. This driver will allocate
memory with malloc(9) using a few tricks to save space on the way.