Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFH r236751: document sha256 / sha512 support
MFH r236892: remove mention of auth.conf from programs that don't use it
MFH r236963: remove dead code relating to auth.conf
MFH r236965 r236966 r236967 r237005 r237006 r237011: retire auth.conf

Approved by: re

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

- Commit work from libprocstat project. These patches add support for runtime
file and processes information retrieval from the running kernel via sysctl
in the form of new library, libprocstat. The library also supports KVM backend
for analyzing memory crash dumps. Both procstat(1) and fstat(1) utilities have
been modified to take advantage of the library (as the bonus point the fstat(1)
utility no longer need superuser privileges to operate), and the procstat(1)
utility is now able to display information from memory dumps as well.

The newly introduced fuser(1) utility also uses this library and able to operate
via sysctl and kvm backends.

The library is by no means complete (e.g. KVM backend is missing vnode name
resolution routines, and there're no manpages for the library itself) so I
plan to improve it further. I'm commiting it so it will get wider exposure
and review.

We won't be able to MFC this work as it relies on changes in HEAD, which
was introduced some time ago, that break kernel ABI. OTOH we may be able
to merge the library with KVM backend if we really need it there.

Discussed with: rwatson

Merger of the quota64 project into head.

This joint work of Dag-Erling Smørgrav and myself updates the
FFS quota system to support both traditional 32-bit and new 64-bit
quotas (for those of you who want to put 2+Tb quotas on your users).

By default quotas are not compiled into the kernel. To include them
in your kernel configuration you need to specify:

options QUOTA # Enable FFS quotas

If you are already running with the current 32-bit quotas, they
should continue to work just as they have in the past. If you
wish to convert to using 64-bit quotas, use `quotacheck -c 64';
if you wish to revert from 64-bit quotas back to 32-bit quotas,
use `quotacheck -c 32'.

There is a new library of functions to simplify the use of the
quota system, do `man quotafile' for details. If your application
is currently using the quotactl(2), it is highly recommended that
you convert your application to use the quotafile interface.
Note that existing binaries will continue to work.

Special thanks to John Kozubik of rsync.net for getting me
interested in pursuing 64-bit quota support and for funding
part of my development time on this project.

Remove login(3), logout(3) and logwtmp(3) from libutil.

These functions only apply to utmp(5). They cannot be kept intact when
moving towards utmpx. The login(3) function would break, because its
argument is an utmp structure. The logout(3) and logwtmp(3) functions
cannot be used, since they provide a functionality which partially
overlaps.

Increment SHLIB_MAJOR to 9 to indicate the removal.

Build lib/ with WARNS=6 by default.

Similar to libexec/, do the same with lib/. Make WARNS=6 the norm and
lower it when needed.

I'm setting WARNS?=0 for secure/. It seems secure/ includes the
Makefile.inc provided by lib/. I'm not going to touch that directory.
Most of the code there is contributed anyway.

Bump the version of all non-symbol-versioned shared libraries in
preparation for 8.0-RELEASE. Add the previous version of those
libraries to ObsoleteFiles.inc and bump __FreeBSD_Version.

Reviewed by: kib
Approved by: re (rwatson)

Add filler man pages for the kinfo functions I added recently.
While here, hook up the hexdump(3) man page which wasn't being installed.

Merge user/peter/kinfo branch as of r185547 into head.

This changes struct kinfo_filedesc and kinfo_vmentry such that they are
same on both 32 and 64 bit platforms like i386/amd64 and won't require
sysctl wrapping.

Two new OIDs are assigned. The old ones are available under
COMPAT_FREEBSD7 - but it isn't that simple. The superceded interface
was never actually released on 7.x.

The other main change is to pack the data passed to userland via the
sysctl. kf_structsize and kve_structsize are reduced for the copyout.
If you have a process with 100,000+ sockets open, the unpacked records
require a 132MB+ copyout. With packing, it is "only" ~35MB. (Still
seriously unpleasant, but not quite as devastating). A similar problem
exists for the vmentry structure - have lots and lots of shared libraries
and small mmaps and its copyout gets expensive too.

My immediate problem is valgrind. It traditionally achieves this
functionality by parsing procfs output, in a packed format. Secondly, when
tracing 32 bit binaries on amd64 under valgrind, it uses a cross compiled
32 bit binary which ran directly into the differing data structures in 32
vs 64 bit mode. (valgrind uses this to track file descriptor operations
and this therefore affected every single 32 bit binary)

I've added two utility functions to libutil to unpack the structures into
a fixed record length and to make it a little more convenient to use.

Fixed style issues with variable ordering and naming, spacing and
parentheses.

Fixed alignment issue in gr_dup() in its assignment of gr_mem using a
struct to force alignment without performing alignment mathematics. This
was noticed recently with libutil was built with WARNS=6 on platform such
as sparc64.

Added checks to gr_dup(), gr_equal() and gr_make() to prevent segfaults
when examining struct group's with the struct members pointing to NULL's.

With fix of alignment issue, restore WARNS?=6.

Reviewed by: des
MFC after: 1 week

Comment out WARNS. There are too many alignment issues in libutil.

Disconnect gr_util.c from the build. It isn't documented or used anywhere
in the tree, and due to unsafe pointer arithmetic, it will most likely crash
on architectures with strict alignment requirements.

libutil now builds at WARNS level 6.

MFC after: 3 weeks

Merge hexdump(9) to userland as hexdump(3) in libutil. I'm tired of doing
this by hand in userland utilities.

MFC after: 1 month

Add four utility functions related to struct grp processing modeled in-part
after similar calls related to struct pwd in libutil/pw_util.c:
- gr_equal()
Perform a deep comparison of two struct grp's. It does a thorough, yet
unoptimized comparison of all the members regardless of order.

- gr_make()
Create a string (see group(5)) from a struct grp.

- gr_dup()
Duplicate a struct grp. Returns a value that is a single contiguous
block of memory.

- gr_scan()
Create a struct grp from a string (as produced by gr_make()).

MFC after: 3 weeks

Implement expand_number(3), which is the opposite of humanize_number(3), ie.
a number in human-readable form is converted to int64_t, for example:
123b -> 123
10k -> 10240
16G -> 17179869184

First version submitted by: Eric Anderson <anderson@freebsd.org>
Approved by: re (bmah)

Bump library versions in preparation for 7.0.

Ok'd by: kan

I'm tired of seeing this done incorrectly and non-portably, so add a
flopen(3) function which reliably opens and locks a file.

MFC after: 3 weeks

style.Makefile(5) is good for our eyes.

Specify default path for SHLIBDIR before bsd.own.mk does.
This fix shared library installed correct place.

Don't build IPv6 support if we have choosen not to have it.

Bump library majro version for gethostbyaddr(3).

Add utility functions for checking if a given kernel module is loaded,
and loading it.

Add a family of functions for reliable pidfiles handling.

Idea from: jmg
Discussed on: arch@

Bump the shared library version number of all libraries that have not
been bumped since RELENG_5.

Reviewed by: ru
Approved by: re (not needed for commit check but in principle...)

Add humanize_number(3) to libutil for formating numbers into a human
readable form.

Obtained from: NetBSD

Back out the "clean_environment()" function from libutil.

Further contemplation has convinced me that this was
not going to really solve the problem of environment-poisoning
without raising serious administrative headaches. There
must be a better way...

Add a clean_environment call to libutil.

This function removes all environment variables except
the ones listed on a "whitelist."

The function accepts two whitelist arguments.
If the first is NULL, a built-in default list will be
used. This allows callers to get a variety of behaviors:
* Default screening: provide NULL for both lists
* Custom screening: provide a custom list for the first argument
* Modified default screening: provide NULL for first arg,
list of additional variables to preserve in the second arg

Idea from: Jacques Vidrine

MFC after: 2 weeks

Bump the major version on libtuil. libutil now relies on the mac_*
symbols exported by newer versions of libc, and so we want applications
depending on the newer library code to be required to link against the
newer libc.

Discussed with: scottl, kris, imp

Remove a GCC specifig CFLAG. We should be using WARNS=? for this.
WARNS=? is not added here at this point, because I've not tested
it on enough platforms, and I don't want to break builds.

style.Makefile(5)

Stage 3 of dynamic root support. Make all the libraries needed to run
binaries in /bin and /sbin installed in /lib. Only the versioned files
reside in /lib, the .so symlink continues to live /usr/lib so the
toolchain doesn't need to be modified.

Zap now-unused SHLIB_MINOR

Backout previous delta (addition of -I${.CURDIR}/../../sys).

Submitted by: bde

Add -I${.CURDIR}/../../sys into CFLAGS, which should fix the world broken
by RLIMIT_VMEM addition.

Add passwd manipulation code based on parts of vipw and chpass.

Sponsored by: DARPA, NAI Labs

o Reflect repo-copy of extattr.[c3] from libutil to libc, moving
extattr namespace routines to the libc/posix1e directory. While
the extattr calls are not strictly POSIX.1e, POSIX.1e wasn't
strictly ever approved, so I think that's OK.

Obtained from: TrustedBSD Project

remove emalloc,ecalloc,erealloc,estrdup

add ecalloc, emalloc, erealloc, estrdup - versions of the e-less
functions that exit instead of failing

Add a manual page for extattr_string_to_namespace and
extattr_namespace_to_string.

Reviewed by: rwatson

Removed -I${.CURDIR}/.../sys from CFLAGS.

MAN[1-9] -> MAN.

o To support new EA interface with explicit namespaces, introduce two
utility functions which convert between string namespace names and
numeric constants used by the interface. Right now, two namespaces
are supported, EXTATTR_NAMESPACE_SYSTEM ("system") and
EXTATTR_NAMESPACE_USER ("user"). These functions are used by
various userland EA utilities, rather than hard coding the routines
all over the place.

Obtained from: TrustedBSD Project

Move trimdomain() into it's own source file and tidy things up a bit.
Fix disorder in the Makefile.

Reviewed (mostly) by: bde

Fix problems people were having with large -O levels with GCC and
getting libutil/libcrypt to work properly. I've determined that GCC
thinks it can inline all functions, including weak-symboled ones, if
it feels like it.

Create a new stub.c and move any stubs there to prevent inlining.
Thanks to jdp and William S. Duncanson for helping me finally find the
problem.

Move setproctitle() from libutil to libc (after a repo-copy)
and bump __FreeBSD_version to 500012 to mark the occasion.

setproctitle() is prototyped in unistd.h as opposed to stdlib.h
where OpenBSD and NetBSD have it.

Reviewed by: peter

Stick login_setcryptfmt() in its own file to make pulling in of
-lcrypt only happen if truly necessary.

Add working and easy crypt(3)-switching. Yes, we need a whole new API
for crypt(3) by now. In any case:

Add crypt_set_format(3) + documentation to -lcrypt.
Add login_setcryptfmt(3) + documentation to -lutil.
Support for switching crypt formats in passwd(8).
Support for switching crypt formats in pw(8).

The simple synopsis is:
edit login.conf; add a passwd_format field set to "des" or "md5"; go nuts :)

Reviewed by: peter

Update major version.

Now libutil depends on libc.so.4, so needs to update the major version.
Without this, old binaries which use libutil and build with libc.so.3
will coredump on recent 4.0.

Solicited comment for cvs-committers and there seems to be no objection.

Approved by: jkh

Historically file flags (schg, uschg, etc) have been converted from
string to u_long and back using two functions, flags_to_string and
string_to_flags, which co-existed with 'ls'. As time has progressed
more and more other tools have used these private functions to
manipulate the file flags.

Recently I moved these functions from /usr/src/bin/ls to libutil,
but after some discussion with bde it's been decided that they
really ought to go in libc.

There are two already existing libc functions for manipulating file
modes: setmode and getmode. In keeping with these flags_to_string
has been renamed getflags and string_to_flags to setflags.

The manual page could probably be improved upon ;)

several tcp apps IPv6 update
-inetd
-rshd
-rlogind
-telnetd
-rsh
-rlogin

Reviewed by: freebsd-arch, cvs-committers
Obtained from: KAME project

Replace beforeinstall target with new variables used by .mk system.

Reviewed by: marcel, and make world

Moved flags_to_string and string_to_flags into libutil. It's used in
many places nowadays.

Connect fparseln(3) for mailwrapper(8)

Link manual page for login_getpwclass(3) to login_cap(3).

PR: docs/14673
Submitted by: Andrew <andrew@ugh.net.au>

Axe LOGIN_CAP_AUTH.

PR: 10115
Reported by: Gene Skonicki <gene@cif.rochester.edu>
Requested by: jdp

Fix commented out CFLAGS addition for LOGIN_CAP_AUTH, which was missing
a make -D option.

PR: 12591
Submitted by: Craig Leres <leres@ee.lbl.gov>

add MLINKS for two functions used from login_auth.c
comment out unused functions from login_auth.3

Tidy up trimdomain() and document it.
Don't truncate one byte short of the passed length.

Add realhostname() - a function to correctly lookup
a name by address and ensure that the name resolves
back to the original address.

Add some rudimentary documentation for my new functions.

Add a simple mechanism for reading property lists from files (which
I'll convert sysinstall to use shortly) and a simple call which uses
this mechanism to implement an /etc/auth.conf file. I'll let Mark Murray
handle the format and checkin of the sample auth.conf file.
Reviewed by: markm

Add uu_lock_txfr() to transfer ownership of a successful
uu_lock() to another process.

MF22: add login_auth.3 to man page list.

Remove login_progok()
Suggested by: guido

Add full support for determining if a user
is restricted from running a given program.

-I${DESTDIR}/sys -> -I${.CURDIR}/../../sys.

Add appropriate ${DESTDIR} in front of absolute paths.

Back out previous revision. Shlib version numbers are supposed to be
bumped only 0.1 or 1.0 between releases. (See handbook.)

Note that if you have built world in -current in the last 48 hours or
so, you should manually remove /usr/lib/libutil.so.2.3 before
rebuilding world to cleanse your system.

Bump shared lib version to 2.3.

Suggested by: bde

Summary of login.conf support changes:

o Incorporated BSDI code and enhancements, better logging for error
checking (which has been shown to be a problem, and is therefore
justified, imho); also some minor things we were missing, including
better quad_t math, which checks for under/overflows.

o setusercontext() now allows user resource limit overrides, but
does this AFTER dropping root privs, to restrict the user to
droping hard limits and set soft limits within the kernel's
allowed user limits.

o umask() only set once, and only if requested.

o add _secure_path(), and use in login.conf to guard against
symlinks etc. and non-root owned or non-user owned files being
used. Derived from BSDI contributed code.

o revamped authentication code to BSDI's latest api, which
includes deleting authenticate() and adding auth_check()
and a few other functions. This is still marked as depecated
in BSDI, but is included for completeness. No other source
in the tree uses this anyway, so it is now bracketed with
#ifdef LOGIN_CAP_AUTH which is by default not defined. Only
auth_checknologin() and auth_cat() are actually used in
module login_auth.c.

o AUTH_NONE definition removed (collided with other includes
in the tree). [bde]

o BSDI's login_getclass() now accepts a char *classname
parameter rather than struct passwd *pwd. We now do likewise,
but added login_getpwclass() for (sort of) backwards
compatiblity, namely because we handle root as a special
case for the default class. This will require quite a few
changes elsewhere in the source tree.

o We no longer pretend to support rlim_t as a long type.

o Revised code formatting to be more bsd-ish style.

Move login_cap.h from src/include for easier maintenance with
related files.

MLINK uu_lockerr(3).

Move uucplock into libutil and create a manual page.

Added -Wall to CFLAGS, cleaned up (all avoidable) warnings.

Bump libutil.so version (2.1 -> 2.2) since a whole heap of new functions
were added with the login class stuff. This is needed since libutil.so.2.1
is what is used in RELENG_2_2 and well into the release cycle. We only
bump once per release cycle as needed.

Library functions relating to the login class capabilities database,
including manpages.
See also login_cap.h.

Finally document the interfaces found in libutil. While being here,
also add the missing declaration of forkpty() to libutil.h.

Btw., the calling interface for login(3) is crude. Some better
abstraction is needed, perhaps similar to logwtmp(3).

2.2 candidate, but i'll wait for the spelling police first. :)

Bump libutil revision after recent addition of setproctitle().

Install (optional) libutil.h with prototypes for the functions and
document this in the man page.

minor cleanups to the various routines, include the prototype file, declare
return codes etc.

Bring in an initial version of setproctitle().. This is intended to
replace the dozen other various hacks in the code that do all sorts
of crude things including spamming the envrionment strings with the new
argv string.

This version is mainly inspired by the sendmail version, with a couple of
ideas taken from the NetBSD implementation as well.

This commit was generated by cvs2svn to compensate for changes in r1573,
which included commits to RCS files with non-trunk default branches.

BSD 4.4 Lite Lib Sources