Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Fix clang warnings.

Approved by: philip (mentor)

The flags argument of mpool_get() is meaningful, document it.

MFC after: 2 weeks

Bump dates in dbopen(3) and cpuset_getaffinity(2) from r212441 and
r212438, repectively.

Approved by: keramida (mentor)
MFC after: 1 week

Note O_SYNC and O_NOFOLLOW flags in dbopen(3) since r190497.

PR: 150030
Submitted by: Janne Snabb snabb at epipe com
Patch by: Janne Snabb
Approved by: keramida (mentor)
MFC after: 1 week

libc/db/hash: cap auto-tuned block size with a value that actually works

This fix mostly matters after r206129 that made it possible for
st_blksize to be greater than 4K. For this reason, this change should
be MFC-ed before r206129.
Also, it seems that all FreeBSD uitlities that use db(3) hash databases
and create new databases in files, specify their own block size value
and thus do not depend on block size autotuning.

PR: bin/144446
Submitted by: Peter Jeremy <peterjeremy@acm.org>
MFC after: 5 days

hash.3: fix a factual mistake in the man page

PR: bin/144446
Submitted by: Peter Jeremy <peterjeremy@acm.org>
MFC after: 3 days

Consider flag == 0 as the same of flag == R_NEXT. This change will restore
a historical behavior that has been changed by revision 190491, and has seen
to break exim.

Update SCCS IDs for Berkeley DB 1.86 merge.

style: operators should appear at the line end if we have to wrap.

Add some casts to silence compiler warning about signedness.

Minor changes from Berkeley DB 1.86 and further improvements from OpenBSD.
This does not include the new hash routines since they will cause problems
when reading old hash files.

Since mpool(3) has been changed, provide a compatibility shim for older
binaries.

Obtained from: OpenBSD

Allow O_SYNC and O_NOFOLLOW flags in dbopen().

Obtained from: OpenBSD

Plug memory leaks and a potential NULL dereference.

Obtained from: OpenBSD

Simplify the logic when determining whether to zero out a db file to after
open(). The previous logic only initializes the database when O_CREAT is
set, but as long as we can open and write the database, and the database
is empty, we should initialize it anyway.

Obtained from: OpenBSD

- If (keysize+datasize)%(bsize=14)==0, insertion of a `big key' would cause
an invariant (actually, an ugly hack) to fail, and all Hell would break
loose.

When deleting a big key, the offset of an empty page should be bsize, not
bsize-1; otherwise an insertion into the empty page will cause the new key to
be elongated by 1 byte.

Make the packing more dense in a couple of cases.

- fix NULL dereference exposed on big bsize values;

Obtained from: NetBSD via OpenBSD

Avoid NULL deference which causes DEBUG1 to crash, not to print useful
information.

Obtained from: OpenBSD

- Avoid overwriting the cursor page when the cursor page becomes the
LRU page.
- Fix for sequential retrieval failure when using large key/data pairs.

Obtained from: OpenBSD

Fix a crash when iterating over a hash and removing its elements.

Obtained from: OpenBSD

Only squeeze a short key/value pair onto a page with other complete key/value
pairs, not onto a page containing the end of a big pair.

Obtained from: NetBSD via OpenBSD

Diff reduce against OpenBSD, no functional change.

Return meaningful errno in overflow case; print error message to stderr
in one more case.

Obtained from: NetBSD via OpenBSD

Use pread(2) and pwrite(2) instead of lseek(2) + read(2) / write(2).

Obtained from: NetBSD via OpenBSD

db/btree/bt_open.c: check return value of snprintf() and return value
if the result is truncated.

db/hash/hash_page.c: use the same way to create temporary file as
bt_open.c; check snprintf() return value.

Obtained from: OpenBSD

Several signed/unsigned warning fixes.

When allocating memory, zero out them if we don't intend to overwrite them
all; before freeing memory, zero out them before we release it as free
heap. This will eliminate some potential information leak issue.

While there, remove the PURIFY option. There is a slight difference between
the new behavior and the old -DPURIFY behavior, with the latter initializes
memory with 0xff's. The difference between old and new approach does not
generate observable difference.

Obtained from: OpenBSD (partly).

Save errno before calling _close(), which may clear it.

Obtained from: OpenBSD

use more proper format string.

Obtained from: NetBSD via OpenBSD

Our realloc(3) and reallocf(3) can handle NULL, which turns it into a
malloc(3) call, so don't test if a pointer is NULL.

Obtained from: OpenBSD (in spirit)

Explicitly specify bit width for on-disk data structure.

Obtained from: OpenBSD

Style changes (including additional casts to shut up warnings). This
commit does not affect MD5 of object file.

Rename variable 'index' to 'idx' to avoid name collision with index(3),
this commit does not affect any object code.

Obtained from: OpenBSD
Verified with: md5(1)

Diff reduction against OpenBSD: ANSI'fy prototypes.

(This is part of a larger changeset which is intended to reduce diff only,
thus some prototypes were left intact since they will be changed in the
future).

Verified with: md5(1)

- Remove superfluous comment

PR: docs/129400
Submitted by: Gavin Atkinson <gavin@freebsd.org>

getopt(3) returns -1, not EOF.

The usage of "info" in init_hash() is read-only, so constify
the internal interface instead of casting away the constant
constraint upon calling.

Use C comments since we now preprocess these files with CPP.

Per Regents of the University of Calfornia letter, remove advertising
clause.

# If I've done so improperly on a file, please let me know.

o dbm_close(3) returns void not zero or anything else. Touch .Dd.

PR: docs/80070
Submitted by: Aleksej Saushev
MFC after: 2 weeks

Add each directory's symbol map file to SYM_MAPS.

Add symbol maps and initial symbol version definitions to libc.

Reviewed by: davidxu

Fix parameter types of close and get members in DB.

Expand *n't contractions.

Fix a condition where the hole would be inserted in the wrong
place during a split.

Obtained from: NetBSD

I found "portsdb -u" dumps core with recent INDEX file, and this is
caused by refering broken (uninitialized?) pointer which is retrieved
from __bt_new() (and from mpool_new()).

I don't know why this linp[0] is read before stored because this
should be controlled by .lower and .upper member of PAGE structure
which are correctly initialized.

But this workaround fixes the problem on my environment and this
module has #ifdef PURIFY option which initializes new and reused
memory from mpool by memset(p, 0xff, size) like as I did.

Please feel free to fix the real bug instead of my workaround.

Fix format strings to unbreak with -DDEBUG option.

Remove the misleading description of the 'key' argument to
mpool_open(3) - it is *not* really used for synchronization; in fact,
it is not used at all.

PR: 70929
Submitted by: Martin Kammerhofer <dada@sbox.tugraz.at>
MFC after: 3 days

Mechanically kill hard sentence breaks.

Simplify mpool_get() and mpool_write() by using pread() and pwrite()
instead of lseek()/_read() and lseek()/_write().

PR: bin/54276
Submitted by: <dnelson@allantgroup.com>

mdoc(7): Use the new feature of the .In macro.

Fix a sizeof error in __bt_put: when writing they key and data sizes
to a buffer in the big key/data case, memmove() was used on pointers
to size_ts, but only sizeof(u_int32_t) bytes where copied. This broke
on big_endian architectures where sizeof(size_t) > sizeof(u_int32_t).

This bug broke portupgrade (by way of ruby_bdb1) on sparc64.

Approved by: re (rwatson)

Eliminate 61 warnings emitted at WARNS=2 (leaving 53 to go).
Only warnings that could be fixed without changing the generated object
code and without restructuring the source code have been handled.

Reviewed by: /sbin/md5

The .Fn function

Correct typos, mostly s/ a / an / where appropriate. Some whitespace cleanup,
especially in troff files.

mdoc(7) police: "The .Fa argument.".

mdoc(7) police: "The .Fn function".

libc_r wasn't so tied to libc for 22 months.

Correct the headers needed to use dbopen(3) and friends.

Assume __STDC__, remove non-__STDC__ code.

Submitted by: keramida

W/o __P, the internal declarations will all fit on one line.

Fix the style of the SCM ID's.
I believe have made all of libc .h's as consistent as possible.

Fix the style of the SCM ID's.
I believe have made all of libc .c's as consistent as possible.

Remove multi-line __P() usage.

Remove __P() usage.

Remove 'register' keyword.

mdoc(7) police: Use the new .In macro for #include statements.

Mark some functions as __printflike() and/or taking const char * arguments
instead of char *.

MFC after: 2 weeks

mdoc(7) police: replace `\*(Ba' with a simple `|', it's handled specially.

Don't clobber the default for CFLAGS.

mdoc(7) police: sort SEE ALSO xrefs (sort -b -f +2 -3 +1 -2).

Remove duplicate words.

Add new, from scratch implementation of hsearch() et al that actually works.

Obtained from: NetBSD
MFC after: 1 month

MAN[1-9] -> MAN.

mdoc(7) cleanup.

Add a man page for the dbm_* functions, and update the Makefile to link
it in.

Some review from -hackers (some time ago), and I think the best way to
get this improved (if it needs improving) or updating, is to bring it in.

PR: docs/12557
Submitted by: Tim Singletary <tsingle@triana.gsfc.nasa.gov>

Fix typo: compatability -> compatibility.

Compatability is not an existing english word.

Add $FreeBSD$.

Fixed C error(s) in synopsis. Many were hiding under mdoc errors.

Remove _THREAD_SAFE and make libc thread-safe by default by
adding (weak definitions to) stubs for some of the pthread
functions. If the threads library is linked in, the real
pthread functions will pulled in.

Use the following convention for system calls wrapped by the
threads library:
__sys_foo - actual system call
_foo - weak definition to __sys_foo
foo - weak definition to __sys_foo

Change all libc uses of system calls wrapped by the threads
library from foo to _foo. In order to define the prototypes
for _foo(), we introduce namespace.h and un-namespace.h
(suggested by bde). All files that need to reference these
system calls, should include namespace.h before any standard
includes, then include un-namespace.h after the standard
includes and before any local includes. <db.h> is an exception
and shouldn't be included in between namespace.h and
un-namespace.h namespace.h will define foo to _foo, and
un-namespace.h will undefine foo.

Try to eliminate some of the recursive calls to MT-safe
functions in libc/stdio in preparation for adding a mutex
to FILE. We have recursive mutexes, but would like to avoid
using them if possible.

Remove uneeded includes of <errno.h> from a few files.

Add $FreeBSD$ to a few files in order to pass commitprep.

Approved by: -arch

man(7) -> mdoc(7).

Fix SCCS id string abuse I introduced.

Fix a tailq conversion bug that resulted in, e.g., nvi crashing upon
quitting every time. The way to free a CIRCLEQ was to loop until
the current == current->head, but the way to free a TAILQ is to loop
until current->head == NULL.

In any case, the CORRECT way to do it is a loop of TAILQ_EMPTY() checks
and TAILQ_REMOVE()al of TAILQ_FIRST(). This bug wouldn't have happened
if the loop wasn't hard-coded...

There may be more bugs of this type from the conversion.

Use TAILQ instead of CIRCLEQ.

Use size_t rather than a 16-bit data type fo the length.

PR: 9350
Submitted by: Danny J. Zerkel <dzerkel@columbus.rr.com>

More secure temporary filename. This needs to be revisited to use
mkstemp().

If using a DB_RECNO, db::put should return the new key if R_IAFTER is
set, not the previous key.

Add $FreeBSD$, not taking this off the vendor branch because it's not on.

Prevent TMPDIR overflow.

cleanup the tsearch import.

remove (comment out) functions defined or depricated elsewhere:
bsearch, lfind, lsearch, insque, remque

change hcreate to take a size_t rather than uint (essentially the same)

since hcreate/hdestroy are now in <search.h>, remove private search.h
in lib/libc/db/hash/

add $FreeBSD tags to hsearch.c

Simplify sytem call renaming. Instead of _foo() <-- _libc_foo <-- foo(),
just use _foo() <-- foo(). In the case of a libpthread that doesn't do
call conversion (such as linuxthreads and our upcoming libpthread), this
is adequate. In the case of libc_r, we still need three names, which are
now _thread_sys_foo() <-- _foo() <-- foo().

Convert all internal libc usage of: aio_suspend(), close(), fsync(), msync(),
nanosleep(), open(), fcntl(), read(), and write() to _foo() instead of foo().

Remove all internal libc usage of: creat(), pause(), sleep(), system(),
tcdrain(), wait(), and waitpid().

Make thread cancellation fully POSIX-compliant.

Suggested by: deischen

Add three-tier symbol naming in support of POSIX thread cancellation
points. For library functions, the pattern is __sleep() <--
_libc_sleep() <-- sleep(). The arrows represent weak aliases. For
system calls, the pattern is _read() <-- _libc_read() <-- read().

$Header$ -> $FreeBSD$

$Id$ -> $FreeBSD$

Add $Id$, to make it simpler for members of the translation teams to
track.

The $Id$ line is normally at the bottom of the main comment block in the
man page, separated from the rest of the manpage by an empty comment,
like so;

.\" $Id$
.\"

If the immediately preceding comment is a @(#) format ID marker than the
the $Id$ will line up underneath it with no intervening blank lines.
Otherwise, an additional blank line is inserted.

Approved by: bde

Replace memory leaking instances of realloc with non-leaking reallocf.
In some cases replace if (a == null) a = malloc(x); else a =
realloc(a, x); with simple reallocf(a, x). Per ANSI-C, this is
guaranteed to be the same thing.

I've been running these on my system here w/o ill effects for some
time. However, the CTM-express is at part 6 of 34 for the CAM
changes, so I've not been able to do a build world with the CAM in the
tree with these changes. Shouldn't impact anything, but...

Change obsolete $@ to ${.TARGET}.

Don't trust TMPDIR if we're setuid root. This is used only for the
backing file for an anonymous (memory based) btree, and I don't think
that any setuid programs actually use it, but it is better to be safe
than sorry. This has been in my tree for a long time, maybe a year or
more...

Inspired by: Similar changes in OpenBSD, if memory serves (like nearly
a year ago)

Fix btree problems. This passes regressions tests.
PR: 7009
Obtained from: http://www.sleepycat.com/update/patch.185.html, patches 1.2,
1.3 and 1.4

Sorted lists.

Removed the subdirectory paths from the definitions of MAN[1-9]. They
were a workaround for limitations in bsd.man.mk that were fixed about
2 years ago.

Submitted by: Sin'ichiro MIYATANI / Phase One, Inc <siu@phaseone.co.jp>
Basic support for the Shift JIS encoding of japanese.
(and one tiny typo fixed in a comment)

Many places in the code NULL is used in integer context, where
plain 0 should be used. This happens to work because we #define
NULL to 0, but is stylistically wrong and can cause problems
for people trying to port bits of code to other environments.

PR: 2752
Submitted by: Arne Henrik Juul <arnej@imf.unit.no>

Changed all paths to be relative to src/lib instead of src/lib/libc
so that all these makefiles can be used to build libc_r too.

Added .if ${LIB} == "c" tests to restrict man page builds to libc
to avoid needlessly building them with libc_r too.

Split libc Makefile into Makefile and Makefile.inc to allow the
libc_r Makefile to include Makefile.inc too.

Revert $FreeBSD$ to $Id$

Reviewed by: Bruce Evans <bde@freebsd.org>

Guard against possible buffer overrun in filename passed.
Another candidate for 2.2.

Sweep through the tree fixing mmap() usage:

- Use MAP_FAILED instead of the constant -1 to indicate
failure (required by POSIX).
- Removed flag arguments of '0' (required by POSIX).
- Fixed code which expected an error return of 0.
- Fixed code which thought any address with the high bit set
was an error.
- Check for failure where no checks were present.

Discussed with: bde

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

make u_char * -> char * conversion explicit.
Found-By: insight's "insure++" tool.

Back out most of the last backout. :-) Guido removed the 1.1.1.1->1.1.1.2
changes from the vendor branch as well, backing the db-1.82 changes.

This file should now be the same as it was in rev 1.1.1.2.

Backout bzero patch.

Somehow, I also managed to get quite some other changes in this file at
the same time. All I did was checkout the file and made a single change.
If someone has an explanation how these PURIFFY defines got in...

When freeing buffers in the db routines, also zeroize them
This should solve the bug where a coredumping ftpd reveals
encrypted passwords.
Obtained from: OpenBSD

delete doubled words, e.g.: "the the" -> "the"

Bring in fixes to db 1.85 from NetBSD. These fixes have been documented as
being sent back to Bostic by the NetBSD crew.
Obtained from: NetBSD-current

General -Wall warning cleanup, part I.
Submitted-By: Kent Vander Velden <graphix@iastate.edu>

Close PR#218. Don't reference non-existant dbm(3) and ndbm(3) manual
pages.

Fix conflicts and merge into mainline

This commit was generated by cvs2svn to compensate for changes in r14272,
which included commits to RCS files with non-trunk default branches.

move stat behind open to cover corner case

If a .db file is 0 length, initialize it as if it did not exist.
Reviewed by: wollman

Reviewed by: julian and (hsu?)
Submitted by: John Birrel(L?)

changes for threadsafe operations

Remove trailing whitespace.

Added the mpool.3 manpage to the installed manpages. It is referred from the
other manpages and there is no conflict.

First crack at making libc work with the new make macros. It compiles on
my machine, and a simple static (genassym) and shared (sysctl) executable
both work. Still to be done: RPCand YP merge.

This commit was generated by cvs2svn to compensate for changes in r1573,
which included commits to RCS files with non-trunk default branches.