MFC bectl(8)/libbe(3): r337663-337664,337667,337697-337699,337800,337805,
337915-337918,337921,337924,337947,337993-337995,338221-338222,338303,
338417,339047,339972,339994,340334,340507-340508,340592-340594,
340635-340636,340722-340723,340974,342466,342849,342903,342911,343335,
343543,343977,343993-343994,344034,344067,344084,345302,345769,
345845-345846,345848,346082

There are simply too many small changes to enumerate; in summary:

bectl(8)/libbe(3) has been introduced from current state in -CURRENT and
added to the stable/11 rescue build. bectl(8) is a tool for managing ZFS
boot environments, largely inspired by beadm. It includes features such as
being able to jail a boot environment or easily mount it for modification.

Relnotes: probably

Revert r342807, mistakenly including libbe(3)/bectl(8) MFC

MFC r342362-r342363: config(8) duplicate option handling

r342362:
config(8): Allow duplicate options to be specified

config(8)'s option handling has been written to allow duplicate options; if
the value changes, then the latest value is used and an informative message
is printed to stderr like so:

/usr/src/sys/amd64/conf/TEST: option "VERBOSE_SYSINIT" redefined from 0 to 1

Currently, this is only a possibility for cpu types, MAXUSERS, and
MACHINE_ARCH. Anything else duplicated in a config file will use the first
value set and error about duplicated options on subsequent appearances,
which is arguably unfriendly since one could specify:

include GENERIC
nooptions VERBOSE_SYSINIT
options VERBOSE_SYSINIT

to redefine the value later anyways.

Reported by: mmacy

r342363:
config(8): Remove all instances of an option when opting out

Quick follow-up to r342362: options can appear multiple times now, so
clean up all of them as needed. For non-OPTIONS options, this has no effect
since they're already de-duplicated.

MFC r303505, r303506, r303512, r303513, r303646, r320418, r323082,
r326169, r326563, r326649, r326716, r326764, r326765 and r329222:

RoCE/infiniband/iWarp upgrade to Linux 4.9 for kernel and userspace.
This commit merges projects/bsd_rdma_4_9 to 11-stable.

Compatibility wrappers have been made for existing 11-stable ibcore
APIs, including ib_reg_phys_mr().
Refer to "sys/ofed/include/rdma/ib_verbs_compat.h" for more information.

The iw_cxgb driver has not been updated and has been disconnected from
the build.

Sponsored by: Mellanox Technologies

MFC r326169 and r326563:
RoCE/infiniband upgrade to Linux v4.9 for kernel and userspace.

List of kernel sources used:
============================

1) kernel sources were cloned from git://github.com/torvalds/linux.git
Top commit 69973b830859bc6529a7a0468ba0d80ee5117826 - tag: v4.9, linux-4.9

2) krping was cloned from https://github.com/larrystevenwise/krping
Top commit 292a2f1abf0348285e678a82264740d52e4dcfe4

List of userspace sources used:
===============================

1) rdma-core was cloned from https://github.com/linux-rdma/rdma-core.git
Top commit d65138ef93af30b3ea249f3a84aa6a24ba7f8a75

2) OpenSM was cloned from git://git.openfabrics.org/~halr/opensm.git
Top commit 85f841cf209f791c89a075048a907020e924528d

3) libibmad was cloned from git://git.openfabrics.org/~iraweiny/libibmad.git
Tag 1.3.13 with some additional patches from Mellanox.

4) infiniband-diags was cloned from git://git.openfabrics.org/~iraweiny/infiniband-diags.git
Tag 1.6.7 with some additional patches from Mellanox.

NOTES:
======

1) The mthca driver has been removed from userspace.
2) All GPLv2 only sources have been removed and where applicable
rewritten from scratch under a BSD license.
3) List of fully supported drivers in userspace and kernel:
a) iw_cxgbe (Chelsio)
b) mlx4ib (Mellanox)
c) mlx5ib (Mellanox)
4) WITH_OFED=YES is still required by make in order to build
OFED userspace and kernel code.
5) Full support has been added for routable RoCE, RoCE v2.

MFC r326649:
Disconnect OFED after r326169 broke all DIRDEPS support for it.

MFC r326716:
Correctly define the unordered_map namespace in ofed/libibnetdisc .

This should fix ofed/libibnetdisc compilation with C-compilers
different from clang and GCC v4.2.1.

Submitted by: kib
Sponsored by: Mellanox Technologies

MFC r326764:
ofed: Remove duplicated symbols from the version file.

ld.bfd accepts multiple listing of the same symbol in the version script.
lld is stricter and errors out. Since arm64 and sometimes amd64 use lld,
we should correct this cosmetic issue.

Sponsored by: Mellanox Technologies
Reviewed by: hselasky
Differential revision: https://reviews.freebsd.org/D13329

MFC r326765:
ofed: Define barriers for mips and arm.

I used the strongest barriers available on the architectures, so if
the future analysis show that it is excessive, the barriers could be
relaxed. Still, it is unlikely that it is meaningful to run IB on 32bit
ARM or current MIPS machines, so the change is to make WITH_OFED to pass
tinderbox.

Sponsored by: Mellanox Technologies
Reviewed by: hselasky
Differential revision: https://reviews.freebsd.org/D13329

MFC r303505:
sdp: Use an mbufq for received control packets.

This is simpler than the hand-rolled queue, and fixes a use-after-free.

Sponsored by: EMC / Isilon Storage Division

MFC r303506:
sdp: Destroy the PCB lock before freeing to the zone.

Sponsored by: EMC / Isilon Storage Division

MFC r303512:
sdp: Use malloc(9) instead of the Linux compat layer.

SDP transmit and receive rings are always created in a sleepable context,
so we can use M_WAITOK and remove error checks.

Sponsored by: EMC / Isilon Storage Division

MFC r303513:
sdp: Destroy the RDMA ID after destroying the connection's queue pair.

This is the ordering documented by rdma_destroy_qp(). Also add a useful
KASSERT to sdp_pcbfree().

Sponsored by: EMC / Isilon Storage Division

MFC r303646:
ipoib: Bound the number of egress mbufs buffered during pathrec lookups.

In pathological situations where the master subnet manager becomes
unresponsive for an extended period, we may otherwise end up queuing all
of the system's mbufs while waiting for a response to a path record lookup.

This addresses the same issue as commit 1e85b806f9 in Linux.

Reviewed by: cem, ngie
Sponsored by: EMC / Isilon Storage Division

MFC r329222:
Import the mthca kernel side infiniband driver from Linux 4.9 and fix
compilation under FreeBSD. The mthca driver was temporarily removed as
part of the Linux 4.9 RoCE/infinband upgrade.

Top commit in Linux source tree:
69973b830859bc6529a7a0468ba0d80ee5117826

Sponsored by: Mellanox Technologies

MFC r320418. Note that the socket lock _is_ the same as so_rcv's lock
in 11 and this is a no-op in this branch.

Sponsored by: Chelsio Communications

MFC r323082:
cxgbe/iw_cxgbe: Set TCP_NODELAY before initiating connection so that
t4_tom picks it up right away. This is less work than waiting for
the connection to be established before applying the setting.

Sponsored by: Chelsio Communications

MFC r320872:
Create libdl.so.1 as a filter for libc.so.7 which exports public dl*
functions.

MFC Loader Fixes 2017q4p2: r324453, r324454

r324453: Create sys/boot/libsa and build libstand.a there

r324454: Disconnect libstand from the build.

MFC r320012,r320028,r320061,r320118:

r320012:
Fix LIBAMU location to fix 'stale .depend' rebuilds in usr.sbin/amd.
r320028:
Fix more incorrect library directories fix 'stale .depend' rebuilds.
r320061:
Fix Makefiles which override LIBDIR to not add incorrect dependencies into
.depend.
r320118:
Follow-up r320061: Need to respect make.conf/env LIBDIR overrides.

Approved by: re (gjb)

MFC efivar(8) (by imp):

List of revisions merged:
r307070
r307071
r307072
r307074
r307189
r307224
r307339
r307390
r307391
r309776
r314231
r314232
r314615
r314616
r314617
r314618
r314619
r314620
r314621
r314623
r314890
r314925
r314926
r314927
r314928
r315770
r315771

Discussed with: gjb (re), imp
Sponsored by: The FreeBSD Foundation

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Add basic blacklist build support

Reviewed by: rpaulo
Approved by: rpaulo
Relnotes: YES
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D5913

zfsd(8), the ZFS fault management daemon

Add zfsd, which deals with hard drive faults in ZFS pools. It manages
hotspares and replements in drive slots that publish physical paths.

cddl/usr.sbin/zfsd
Add zfsd(8) and its unit tests

cddl/usr.sbin/Makefile
Add zfsd to the build

lib/libdevdctl
A C++ library that helps devd clients process events

lib/Makefile
share/mk/bsd.libnames.mk
share/mk/src.libnames.mk
Add libdevdctl to the build. It's a private library, unusable by
out-of-tree software.

etc/defaults/rc.conf
By default, set zfsd_enable to NO

etc/mtree/BSD.include.dist
Add a directory for libdevdctl's include files

etc/mtree/BSD.tests.dist
Add a directory for zfsd's unit tests

etc/mtree/BSD.var.dist
Add /var/db/zfsd/cases, where zfsd stores case files while it's shut
down.

etc/rc.d/Makefile
etc/rc.d/zfsd
Add zfsd's rc script

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c
Fix the resource.fs.zfs.statechange message. It had a number of
problems:

It was only being emitted on a transition to the HEALTHY state.
That made it impossible for zfsd to take actions based on drives
getting sicker.

It compared the new state to vdev_prevstate, which is the state that
the vdev had the last time it was opened. That doesn't make sense,
because a vdev can change state multiple times without being
reopened.

vdev_set_state contains logic that will change the device's new
state based on various conditions. However, the statechange event
was being posted _before_ that logic took effect. Now it's being
posted after.

Submitted by: gibbs, asomers, mav, allanjude
Reviewed by: mav, delphij
Relnotes: yes
Sponsored by: Spectra Logic Corp, iX Systems
Differential Revision: https://reviews.freebsd.org/D6564

Move casper library entries to proper places.

Sponsored by: EMC / Isilon Storage Division

Convert casperd(8) daemon to the libcasper.
After calling the cap_init(3) function Casper will fork from it's original
process, using pdfork(2). Forking from a process has a lot of advantages:
1. We have the same cwd as the original process.
2. The same uid, gid and groups.
3. The same MAC labels.
4. The same descriptor table.
5. The same routing table.
6. The same umask.
7. The same cpuset(1).
From now services are also in form of libraries.
We also removed libcapsicum at all and converts existing program using Casper
to new architecture.

Discussed with: pjd, jonathan, ed, drysdale@google.com, emaste
Partially reviewed by: drysdale@google.com, bdrewery
Approved by: pjd (mentor)
Differential Revision: https://reviews.freebsd.org/D4277

Start on a new library (libsysdecode) that provides routines for decoding
system call information such as system call arguments. Initially this
will consist of pulling duplicated code out of truss and kdump though it
may prove useful for other utilities in the future.

This commit moves the shared utrace(2) record parser out of kdump into
the library and updates kdump and truss to use it. One difference from
the previous version is that the library version treats unknown events
that start with the "RTLD" signature as unknown events. This simplifies
the interface and allows the consumer to decide how to handle all
non-recognized events. Instead, this function only generates a string
description for known malloc() and RTLD records.

Reviewed by: bdrewery
Differential Revision: https://reviews.freebsd.org/D4537

Convert contrib/ofed libraries to LIBADD.

Sponsored by: EMC / Isilon Storage Division

Sort the LIB list.

Sponsored by: EMC / Isilon Storage Division

Add LIBNANDFS.

Sponsored by: EMC / Isilon Storage Division

Add assertions that capture invalid configurations for new libraries.

Fix current findings, which should fix cases of NO_SHARED not building
properly.

Given libfoo:
- Ensure that a LIBFOO is set. For INTERNALLIBS advise setting this in
src.libnames.mk, otherwise bsd.libnames.mk.
- Ensure that a LIBFOODIR is properly set.
- Ensure that _DP_foo is set and matches the LIBADD in the build of foo's own
Makefile

Sponsored by: EMC / Isilon Storage Division

Revert incomplete r291623.

Add assertions that capture invalid configurations for new libraries.

Fix current findings.

Given libfoo:
- Ensure that a LIBFOO is set. For INTERNALLIBS advise setting this in
src.libnames.mk, otherwise bsd.libnames.mk.
- Ensure that a LIBFOODIR is properly set.
- Ensure that _DP_foo is set and matches the LIBADD in the build of foo's own
Makefile

Sponsored by: EMC / Isilon Storage Division

Define a LIB*SRCDIR for all known _LIBRARIES.

This is a follow-up to r291327 which added a LIB*DIR for all known
_LIBRARIES.

Sponsored by: EMC / Isilon Storage Division

Add missing LIB80211 entry for DPADD needs.

Sponsored by: EMC / Isilon Storage Division

Define a LIB<NAME>DIR for every library that LIBADD provides.

This is going to be used to allow DIRDEPS to be bootstrapped off of
LIBADD/DPADD. It currently works for internal libraries which have a
DIR defined for them but also use the .a library from a src-mapped obj
directory. It can also be useful for using -L without a --sysroot per
LIBADD to use the OBJDIR version of the libraries.

I didn't review every LIBADD, so it is possible this is missing some.

Sponsored by: EMC / Isilon Storage Division

Significant upgrades to sa(4) and mt(1).

The primary focus of these changes is to modernize FreeBSD's
tape infrastructure so that we can take advantage of some of the
features of modern tape drives and allow support for LTFS.

Significant changes and new features include:

o sa(4) driver status and parameter information is now exported via an
XML structure. This will allow for changes and improvements later
on that will not break userland applications. The old MTIOCGET
status ioctl remains, so applications using the existing interface
will not break.

o 'mt status' now reports drive-reported tape position information
as well as the previously available calculated tape position
information. These numbers will be different at times, because
the drive-reported block numbers are relative to BOP (Beginning
of Partition), but the block numbers calculated previously via
sa(4) (and still provided) are relative to the last filemark.
Both numbers are now provided. 'mt status' now also shows the
drive INQUIRY information, serial number and any position flags
(BOP, EOT, etc.) provided with the tape position information.
'mt status -v' adds information on the maximum possible I/O size,
and the underlying values used to calculate it.

o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed.

The extra devices were originally added as place holders for
density-specific device nodes. Some OSes (NetBSD, NetApp's OnTap
and Solaris) have had device nodes that, when you write to them,
will automatically select a given density for particular tape drives.

This is a convenient way of switching densities, but it was never
implemented in FreeBSD. Only the device nodes were there, and that
sometimes confused users.

For modern tape devices, the density is generally not selectable
(e.g. with LTO) or defaults to the highest availble density when
the tape is rewritten from BOT (e.g. TS11X0). So, for most users,
density selection won't be necessary. If they do need to select
the density, it is easy enough to use 'mt density' to change it.

o Protection information is now supported. This is either a
Reed-Solomon CRC or CRC32 that is included at the end of each block
read and written. On write, the tape drive verifies the CRC, and
on read, the tape drive provides a CRC for the userland application
to verify.

o New, extensible tape driver parameter get/set interface.

o Density reporting information. For drives that support it,
'mt getdensity' will show detailed information on what formats the
tape drive supports, and what formats the tape drive supports.

o Some mt(1) functionality moved into a new mt(3) library so that
external applications can reuse the code.

o The new mt(3) library includes helper routines to aid in parsing
the XML output of the sa(4) driver, and build a tree of driver
metadata.

o Support for the MTLOAD (load a tape in the drive) and MTWEOFI
(write filemark immediate) ioctls needed by IBM's LTFS
implementation.

o Improve device departure behavior for the sa(4) driver. The previous
implementation led to hangs when the device was open.

o This has been tested on the following types of drives:
IBM TS1150
IBM TS1140
IBM LTO-6
IBM LTO-5
HP LTO-2
Seagate DDS-4
Quantum DLT-4000
Exabyte 8505
Sony DDS-2

contrib/groff/tmac/doc-syms,
share/mk/bsd.libnames.mk,
lib/Makefile,
Add libmt.

lib/libmt/Makefile,
lib/libmt/mt.3,
lib/libmt/mtlib.c,
lib/libmt/mtlib.h,
New mt(3) library that contains functions moved from mt(1) and
new functions needed to interact with the updated sa(4) driver.

This includes XML parser helper functions that application writers
can use when writing code to query tape parameters.

rescue/rescue/Makefile:
Add -lmt to CRUNCH_LIBS.

src/share/man/man4/mtio.4
Clarify this man page a bit, and since it contains what is
essentially the mtio.h header file, add new ioctls and structure
definitions from mtio.h.

src/share/man/man4/sa.4
Update BUGS and maintainer section.

sys/cam/scsi/scsi_all.c,
sys/cam/scsi/scsi_all.h:
Add SCSI SECURITY PROTOCOL IN/OUT CDB definitions and CDB building
functions.

sys/cam/scsi/scsi_sa.c
sys/cam/scsi/scsi_sa.h
Many tape driver changes, largely outlined above.

Increase the sa(4) driver read/write timeout from 4 to 32
minutes. This is based on the recommended values for IBM LTO
5/6 drives. This may also avoid timeouts for other tape
hardware that can take a long time to do retries and error
recovery. Longer term, a better way to handle this is to ask
the drive for recommended timeout values using the REPORT
SUPPORTED OPCODES command. Modern IBM and Oracle tape drives
at least support that command, and it would allow for more
accurate timeout values.

Add XML status generation. This is done with a series of
macros to eliminate as much duplicate code as possible. The
new XML-based status values are reported through the new
MTIOCEXTGET ioctl.

Add XML driver parameter reporting, using the new MTIOCPARAMGET
ioctl.

Add a new driver parameter setting interface, using the new
MTIOCPARAMSET and MTIOCSETLIST ioctls.

Add a new MTIOCRBLIM ioctl to get block limits information.

Add CCB/CDB building routines scsi_locate_16, scsi_locate_10,
and scsi_read_position_10().

scsi_locate_10 implements the LOCATE command, as does the
existing scsi_set_position() command. It just supports
additional arguments and features. If/when we figure out a
good way to provide backward compatibility for older
applications using the old function API, we can just revamp
scsi_set_position(). The same goes for
scsi_read_position_10() and the existing scsi_read_position()
function.

Revamp sasetpos() to take the new mtlocate structure as an
argument. It now will use either scsi_locate_10() or
scsi_locate_16(), depending upon the arguments the user
supplies. As before, once we change position we don't have a
clear idea of what the current logical position of the tape
drive is.

For tape drives that support long form position data, we
read the current position and store that for later reporting
after changing the position. This should help applications
like Bacula speed tape access under FreeBSD once they are
modified to support the new ioctls.

Add a new quirk, SA_QUIRK_NO_LONG_POS, that is set for all
drives that report SCSI-2 or older, as well as drives that
report an Illegal Request type error for READ POSITION with
the long format. So we should automatically detect drives
that don't support the long form and stop asking for it after
an initial try.

Add a partition number to the sa(4) softc.

Improve device departure handling. The previous implementation
led to hangs when the device was open.

If an application had the sa(4) driver open, and attempted to
close it after it went away, the cam_periph_release() call in
saclose() would cause the periph to get destroyed because that
was the last reference to it. Because destroy_dev() was
called from the sa(4) driver's cleanup routine (sacleanup()),
and would block waiting for the close to happen, a deadlock
would result.

So instead of calling destroy_dev() from the cleanup routine,
call destroy_dev_sched_cb() from saoninvalidate() and wait for
the callback.

Acquire a reference for devfs in saregister(), and release it
in the new sadevgonecb() routine when all devfs devices for
the particular sa(4) driver instance are gone.

Add a new function, sasetupdev(), to centralize setting
per-instance devfs device parameters instead of repeating the
code in saregister().

Add an open count to the softc, so we know how many
peripheral driver references are a result of open
sessions.

Add the D_TRACKCLOSE flag to the cdevsw flags so
that we get a 1:1 mapping of open to close calls
instead of a N:1 mapping.

This should be a no-op for everything except the
control device, since we don't allow more than one
open on non-control devices.

However, since we do allow multiple opens on the
control device, the combination of the open count
and the D_TRACKCLOSE flag should result in an
accurate peripheral driver reference count, and an
accurate open count.

The accurate open count allows us to release all
peripheral driver references that are the result
of open contexts once we get the callback from devfs.

sys/sys/mtio.h:
Add a number of new mt(4) ioctls and the requisite data
structures. None of the existing interfaces been removed
or changed.

This includes definitions for the following new ioctls:

MTIOCRBLIM /* get block limits */
MTIOCEXTLOCATE /* seek to position */
MTIOCEXTGET /* get tape status */
MTIOCPARAMGET /* get tape params */
MTIOCPARAMSET /* set tape params */
MTIOCSETLIST /* set N params */

usr.bin/mt/Makefile:
mt(1) now depends on libmt, libsbuf and libbsdxml.

usr.bin/mt/mt.1:
Document new mt(1) features and subcommands.

usr.bin/mt/mt.c:
Implement support for mt(1) subcommands that need to
use getopt(3) for their arguments.

Implement a new 'mt status' command to replace the old
'mt status' command. The old status command has been
renamed 'ostatus'.

The new status function uses the MTIOCEXTGET ioctl, and
therefore parses the XML data to determine drive status.
The -x argument to 'mt status' allows the user to dump out
the raw XML reported by the kernel.

The new status display is mostly the same as the old status
display, except that it doesn't print the redundant density
mode information, and it does print the current partition
number and position flags.

Add a new command, 'mt locate', that will supersede the
old 'mt setspos' and 'mt sethpos' commands. 'mt locate'
implements all of the functionality of the MTIOCEXTLOCATE
ioctl, and allows the user to change the logical position
of the tape drive in a number of ways. (Partition,
block number, file number, set mark number, end of data.)
The immediate bit and the explicit address bits are
implemented, but not documented in the man page.

Add a new 'mt weofi' command to use the new MTWEOFI ioctl.
This allows the user to ask the drive to write a filemark
without waiting around for the operation to complete.

Add a new 'mt getdensity' command that gets the XML-based
tape drive density report from the sa(4) driver and displays
it. This uses the SCSI REPORT DENSITY SUPPORT command
to get comprehensive information from the tape drive about
what formats it is able to read and write.

Add a new 'mt protect' command that allows getting and setting
tape drive protection information. The protection information
is a CRC tacked on to the end of every read/write from and to
the tape drive.

Sponsored by: Spectra Logic
MFC after: 1 month

Add LIBXO.

Add a new device control utility for new-bus devices called devctl. This
allows the user to request administrative changes to individual devices
such as attach or detaching drivers or disabling and re-enabling devices.
- Add a new /dev/devctl2 character device which uses ioctls for device
requests. The ioctls use a common 'struct devreq' which is somewhat
similar to 'struct ifreq'.
- The ioctls identify the device to operate on via a string. This
string can either by the device's name, or it can be a bus-specific
address. (For unattached devices, a bus address is the only way to
locate a device.) Bus drivers register an eventhandler to claim
unrecognized device names that the driver recognizes as a valid address.
Two buses currently support addresses: ACPI recognizes any device
in the ACPI namespace via its full path starting with "\" and
the PCI bus driver recognizes an address specification of
'pci[<domain>:]<bus>:<slot>:<func>' (identical to the PCI selector
strings supported by pciconf).
- To make it easier to cut and paste, change the PnP location string
in the PCI bus driver to output a full PCI selector string rather
than 'slot=<slot> function=<func>'.
- Add a devctl(3) interface in libdevctl which provides a wrapper around
the ioctls and is the preferred interface for other userland code.
- Add a devctl(8) program which is a simple wrapper around the requests
supported by devctl(3).
- Add a device_is_suspended() function to check DF_SUSPENDED.
- Add a resource_unset_value() function that can be used to remove a
hint from the kernel environment. This is used to clear a
hint.<driver>.<unit>.disabled hint when re-enabling a boot-time
disabled device.

Reviewed by: imp (parts)
Requested by: imp (changing PCI location string)
Relnotes: yes

Remove MINUSLPAM the LIBADD framework handles static dependencies just fine

Import libgpio.

This is a thin wrapper around the kernel interface which should make
it easier to write GPIO applications. gpioctl(8) will be converted to
use this library in a separate commit.

Differential Revision: https://reviews.freebsd.org/D1183
Reviewed by: adrian, loos
Discussed on: arm@, embedded@
Relnotes: yes

Enforce -lpthread and -lc to always be the 2 last components of the link list

Differential Revision: https://reviews.freebsd.org/D1118
Suggested by: kib

Add new libraries/utilities for data throughput visualization.
dpv(3): dialog progress view library
dpv(1): stream data from stdin or multiple paths with dialog progress view
figpar(3): configuration file parsing library

Reviews: D714
Reviewed by: jelischer, shurd
Discussed at: MeetBSD California 2014 Vendor/Dev Summit
Discussed on: -current
MFC after: 21 days
X-MFC-to: stable/10 stable/9

Introduce missing definition for LIBTERMCAPW

Some Makefiles expect this value to exist

Approved by: rpaulo (mentor)
MFC after: 5 days
Phabric: D675 (as part of a larger diff)
PR: 192762

Fix "make checkdpadd" for lib/libc when MK_SSP != no

Add LIBSSP_NONSHARED to bsd.libnames.mk and append LIBSSP_NONSHARED to DPADD in
lib/libc when MK_SSP != no

Approved by: rpaulo (mentor)
MFC after: 3 days
Phabric: D675 (as part of a larger diff)
PR: 192728

Rework privatelib/internallib

Make sure everything linking to a privatelib and/or an internallib does it directly
from the OBJDIR rather than DESTDIR.
Add src.libnames.mk so bsd.libnames.mk is not polluted by libraries not existsing
in final installation
Introduce the LD* variable which is what ld(1) is expecting (via LDADD) to link to
internal/privatelib
Directly link to the .so in case of private library to avoid having to complexify
LDFLAGS.

Phabric: https://phabric.freebsd.org/D553
Reviewed by: imp, emaste

The GNU readline library is now an INTERNALLIB - that is, it is
statically linked into consumers (GDB and variants) in the base
system, and the shared library is no longer installed.

That also allows ports to be able to use a modern version of readline

PR: 162948
Reviewed by: emaste

Change libatf-c and libatf-c++ to be private libraries.

We should not be leaking these interfaces to the outside world given
that it's much easier for third-party components to use the devel/atf
package from ports.

As a side-effect, we can also drop the ATF pkgconfig and aclocal files
from the base system. Nothing in the base system needs these, and it
was quite ugly to have to get them installed only so that a few ports
could build. The offending ports have been fixed to depend on
devel/atf explicitly.

Reviewed by: bapt

LIBUNBOUND can be unconditionally set.

Don't need to make these conditional on MK_LIBCPLUSPLUS. They can be
defined all the time without ill effect.

LIBMILTER can be defined always. No need to restrict to only when
we're building sendmail.

Always define LIBLDNS. We don't need to only define it when it is
enabled.

Remove IPX support.

IPX was a network transport protocol in Novell's NetWare network operating
system from late 80s and then 90s. The NetWare itself switched to TCP/IP
as default transport in 1998. Later, in this century the Novell Open
Enterprise Server became successor of Novell NetWare. The last release
that claimed to still support IPX was OES 2 in 2007. Routing equipment
vendors (e.g. Cisco) discontinued support for IPX in 2011.

Thus, IPX won't be supported in FreeBSD 11.0-RELEASE.

Remove libyaml, it has been replaced by libucl

Import libucl into head

UCL is heavily infused by nginx configuration as the example of a convenient
configuration system. However, UCL is fully compatible with JSON format and is
able to parse json files.

UCL is used by pkg(8) for its configuration file as well for the manifest format
in packages, it will be used in base for the pkg boostrap (signature checking
and configuration file parsing.)

libucl has been developped and is maintained by vsevolod@

Correct value of LIBCOMPILER_RT.

Caught by: Luca Bayer

Replace LIBGCC by LIBCOMPILER_RT.

We now use libcompiler_rt on all platforms now. Instead of referring
directly to -lgcc and LIBGCC, use -lcompiler_rt and LIBCOMPILER_RT.

Please welcome casperd daemon. It (and its services) will be responsible for
giving access to functionality that is not available in capability mode
sandbox. The functionality can be precisely restricted.

Start with the following services:
- system.dns - provides API compatible to:
- gethostbyname(3),
- gethostbyname2(3),
- gethostbyaddr(3),
- getaddrinfo(3),
- getnameinfo(3),
- system.grp - provides getgrent(3)-compatible API,
- system.pwd - provides getpwent(3)-compatible API,
- system.random - allows to obtain entropy from /dev/random,
- system.sysctl - provides sysctlbyname(3-compatible API.

Sponsored by: The FreeBSD Foundation

Move my simple logging API to a separate library. It is now already used
by hastctl(8), hastd(8) and auditdistd(8) and will soon be also used
by casperd(8) and its services. There is no documentation and pjdlog.h
header file is not installed in /usr/include/ to keep it private.
Unfortunately we don't have /lib/private/ at this point, only
/usr/lib/private/, so the library is installed in /lib/.

Sponsored by: The FreeBSD Foundation

Bring in libnv library for managing name/value pairs. The following types
are currently supported:

- NV_TYPE_NULL - only name, no data;
- NV_TYPE_BOOL - boolean (true or false);
- NV_TYPE_NUMBER - 64bit unsigned integer;
- NV_TYPE_STRING - C string;
- NV_TYPE_NVLIST - nested nvlist;
- NV_TYPE_DESCRIPTOR - file descriptor;
- NV_TYPE_BINARY - binary data.

For detailed documentation and examples see nv(3) manual page.

Sponsored by: The FreeBSD Foundation

Rename libbsdyml to libyaml, make private, and bump
SHLIB_MAJOR to 1.0

Suggested by: des
Approved by: bapt
MFC after: 1 week

Remove BIND.

Approved by: re (gjb)

Set the correct path for LIBUNBOUND.

Approved by: re (blanket)

Build and install the Unbound caching DNS resolver daemon.

Approved by: re (blanket)

Clean up the Kerberos build by turning libheimipcc and libheimipcs into
private shared libraries, instead of hacked-together archives of PIC
objects. This makes it possible to build a static libkrb5 that works.

Reviewed by: stas
Approved by: re (gjb)

Make libldns and libssh private.

Approved by: re (blanket)

Connect libexecinfo to the build

Sponsored by: DARPA, AFRL

- Trim an unused and bogus Makefile for mount_smbfs.
- Reconnect with some minor modifications, in particular now selsocket()
internals are adapted to use sbintime units after recent'ish calloutng
switch.

Merge libzfs_core branch:
includes MFV 238590, 238592, 247580

MFV 238590, 238592:
In the first zfs ioctl restructuring phase, the libzfs_core library was
introduced. It is a new thin library that wraps around kernel ioctl's.
The idea is to provide a forward-compatible way of dealing with new
features. Arguments are passed in nvlists and not random zfs_cmd fields,
new-style ioctls are logged to pool history using a new method of
history logging.

http://blog.delphix.com/matt/2012/01/17/the-future-of-libzfs/

MFV 247580 [1]:
To address issues of several deadlocks and race conditions the locking
code around dsl_dataset was rewritten and the interface to synctasks
was changed.

User-Visible Changes:
"zfs snapshot" can create more arbitrary snapshots at once (atomically)
"zfs destroy" destroys multiple snapshots at once
"zfs recv" has improved performance

Backward Compatibility:
I have extended the compatibility layer to support full backward
compatibility by remapping or rewriting the responsible ioctl arguments.
Old utilities are fully supported by the new kernel module.

Forward Compatibility:
New utilities work with old kernels with the following restrictions:
- creating, destroying, holding and releasing of multiple snapshots
at once is not supported, this includes recursive (-r) commands

Illumos ZFS issues:
2882 implement libzfs_core
2900 "zfs snapshot" should be able to create multiple,
arbitrary snapshots at once
3464 zfs synctask code needs restructuring

References:
https://www.illumos.org/issues/2882
https://www.illumos.org/issues/2900
https://www.illumos.org/issues/3464 [1]

MFC after: 1 month
Sponsored by: Hybrid Logic Inc. [1]

Import libyaml as libbsdyml (private brand name)

LibYAML is a YAML 1.1 parser and emitter under MIT license which will
soon be used by the pkg boostrap (usr.bin/pkg) and bhyve

Reviewed by: roberto, antoine

fix leftover from an earlier experiment

Import LDNS and build it as an internal library.

Merge projects/bhyve to head.

'bhyve' was developed by grehan@ and myself at NetApp (thanks!).

Special thanks to Peter Snyder, Joe Caradonna and Michael Dexter for their
support and encouragement.

Obtained from: NetApp

With the old sade removed, libdisk is no longer used by anything in HEAD
and uses a number of problematic pre-gpart interfaces. Since it has been
entirely obsoleted by interfaces in geom, remove it.

Do not expose LIBCXXRT and LIBCPLUSPLUS in bsd.libnames.mk, if
WITHOUT_LIBCPLUSPLUS is specified.

Submitted by: Garrett Cooper <yanegomi@gmail.com>
MFC after: 3 days

Add ATF to the build. This is may be a bit rought around the egdes,
but committing it helps to get everyone on the same page and makes
sure we make progress.

Tinderbox breakages that are the result of this commit are entirely
the committer's fault -- in other words: buildworld testing on amd64
only.

Credits follow:

Submitted by: Garrett Cooper <yanegomi@gmail.com>
Sponsored by: Isilon Systems
Based on work by: keramida@
Thanks to: gnn@, mdf@, mlaier@, sjg@
Special thanks to: keramida@

Disconnect non-MPSAFE SMBFS from the build in preparation for dropping
GIANT from VFS. In addition, disconnect also netsmb, which is a base
requirement for SMBFS.

In the while SMBFS regular users can use FUSE interface and smbnetfs
port to work with their SMBFS partitions.

Also, there are ongoing efforts by vendor to support in-kernel smbfs,
so there are good chances that it will get relinked once properly locked.

This is not targeted for MFC.

Disconnect non-MPSAFE NWFS from the build in preparation for dropping
GIANT from VFS. In addition, disconnect also netncp, which is a base
requirement for NWFS.

In the possibility of a future maintenance of the code and later
readd to the FreeBSD base, maybe we should think about a better location
for netncp. I'm not entirely sure the / top location is actually right,
however I will let network people to comment on that more specifically.

This is not targeted for MFC.

Add missing LIBPROCSTAT.

- Update FreeBSD Heimdal distribution to version 1.5.1. This also brings
several new kerberos related libraries and applications to FreeBSD:
o kgetcred(1) allows one to manually get a ticket for a particular service.
o kf(1) securily forwards ticket to another host through an authenticated
and encrypted stream.
o kcc(1) is an umbrella program around klist(1), kswitch(1), kgetcred(1)
and other user kerberos operations. klist and kswitch are just symlinks
to kcc(1) now.
o kswitch(1) allows you to easily switch between kerberos credentials if
you're running KCM.
o hxtool(1) is a certificate management tool to use with PKINIT.
o string2key(1) maps a password into key.
o kdigest(8) is a userland tool to access the KDC's digest interface.
o kimpersonate(8) creates a "fake" ticket for a service.

We also now install manpages for some lirbaries that were not installed
before, libheimntlm and libhx509.

- The new HEIMDAL version no longer supports Kerberos 4. All users are
recommended to switch to Kerberos 5.

- Weak ciphers are now disabled by default. To enable DES support (used
by telnet(8)), use "allow_weak_crypto" option in krb5.conf.

- libtelnet, pam_ksu and pam_krb5 are now compiled with error on warnings
disabled due to the function they use (krb5_get_err_text(3)) being
deprecated. I plan to work on this next.

- Heimdal's KDC now require sqlite to operate. We use the bundled version
and install it as libheimsqlite. If some other FreeBSD components will
require it in the future we can rename it to libbsdsqlite and use for these
components as well.

- This is not a latest Heimdal version, the new one was released while I was
working on the update. I will update it to 1.5.2 soon, as it fixes some
important bugs and security issues.

Import libc++ / libcxxrt into base. Not build by default yet (use
MK_LIBCPLUSPLUS=yes to enable). This is a work-in-progress. It works for
me, but is not guaranteed to work for anyone else and may eat your dog.

To build C++ using libc++, add -stdlib=libc++ to your CXX and LD flags.

Bug reports welcome, bug fixes even more welcome...

Approved by: dim (mentor)

libodialog: disconnect from the build and obsolete.

Backout libinstall.a -> libpkg commit.

Discussed with: erwin, brooks, bapt

Remove libobjc and other Objective-C related components, as these are
extremely outdated, and not used by anything in the base system.

Silence from: current@

Update dialog to version 20100428. This changes the license under which
dialog is distributed from GPLv2 to LGPLv2 and introduces a number of new
features and a new and better libdialog API. The existing libdialog will
be kept temporarily as libodialog for compatibility purposes until sade,
sysinstall and tzsetup have been either updated or replaced.

__FreeBSD_version is now 900030.

Discussed on: -current
Approved by: core
Obtained from: http://invisible-island.net/dialog

Remove LIBELF_PIC.

Sponsored by: The FreeBSD Foundation

Add the definition of LIBELF_PIC.

Sponsored by: The FreeBSD Foundation

Add LIBRTLD_DB.

Sponsored by: The FreeBSD Foundation

Import of liblzma, xz, xzdec, lzmainfo from vendor branch
Add support for xz and lzma to lesspipe.sh (xzless, lzless)
Bump __FreeBSD_version

Approved by: delphij (mentor)
MFC after: 2 weeks

- Take libinstall.a out of pkg_install and make it a proper shared library.
- Rework the wrapper support to check libpkg version as well as pkg_install
version.
- Add libfetch to _prebuild_libs.
- There are no new features introduced.

Notes: the API is not stable, so basically, do not use libpkg in your
projects for now. Also there's no manpage for libpkg yet, because the API
will change drastically. I repeat, do not use libpkg for now.

Fixed missing or broken library dependencies.

libusb20 was renamed libusb several months ago.

MFC after: 1 month

Add a new library: libulog.

One of the things I really want to do, is to get rid of the limitations
of our current utmp(5) mechanism:

- It only allows 8 byte TTY device names.
- The hostname only allows 16 bytes of storage.

I'm not a big fan of <utmpx.h>, but I think we should at least try to
add parts of it. Unfortunately we cannot implement <utmpx.h>, because we
miss various fields, such as ut_id, ut_pid, etc. The API provided by
libulog shares some similarities with <utmpx.h>, so it shouldn't be too
hard to port these applications eventually. In most simple cases, it
should just be a matter of removing the ulog_ prefix everywhere.

As a bonus, it also implements a function called ulog_login_pseudo(),
which allows unprivileged applications to write log entries, provided
they have a valid file descriptor to a pseudo-terminal master device.

libulog will allow a smoother transition to a new file format by adding
a library interface to deal with utmp/wtmp/lastlog files. I initially
thought about adding the functionality to libutil, but because I'm not
planning on keeping this library around forever, we'd better keep it
separated.

Next items on the todo list:

1. Port applications in the base system (and ports) to libulog, instead
of letting them use <utmp.h>.
2. Remove <utmp.h>, implement <utmpx.h> and reimplement this library on
top.
3. Port as many applications as possible back to <utmpx.h>.

Add libjail, a (somewhat) simpler interface to the jail_set and jail_get
system calls and the security.jail.param sysctls.

Approved by: bz (mentor)

Merge OpenBSM alpha 4 from OpenBSM vendor branch to head, both
contrib/openbsm (svn merge) and src/sys/{bsm,security/audit} (manual
merge). Add libauditd build parts and add to auditd's linkage;
force libbsm to build before libauditd.

OpenBSM history for imported revisions below for reference.

MFC after: 1 month
Sponsored by: Apple Inc.
Obtained from: TrustedBSD Project

OpenBSM 1.1 alpha 4

- With the addition of BSM error number mapping, we also need to map the
local error number passed to audit_submit(3) to a BSM error number,
rather than have the caller perform that conversion.
- Reallocate user audit events to avoid collisions with Solaris; adopt a
more formal allocation scheme, and add some events allocated in Solaris
that will be of immediate use on other platforms.
- Add an event for Calife.
- Add au_strerror(3), which allows generating strings for BSM errors
directly, rather than requiring applications to map to the local error
space, which might not be able to entirely represent the BSM error
number space.
- Major auditd rewrite for launchd(8) support. Add libauditd library
that is shared between launchd and auditd.
- Add AUDIT_TRIGGER_INITIALIZE trigger (sent via 'audit -i') for
(re)starting auditing under launchd(8) on Mac OS X.
- Add 'current' symlink to active audit trail.
- Add crash recovery of previous audit trail file when detected on audit
startup that it has not been properly terminated.
- Add the event AUE_audit_recovery to indicated when an audit trail file
has been recovered from not being properly terminated. This event is
stored in the new audit trail file and includes the path of recovered
audit trail file.
- Mac OS X and FreeBSD dependent code in auditd.c is separated into
auditd_darwin.c and auditd_fbsd.c files.
- Add an event for the posix_spawn(2) and fsgetpath(2) Mac OS X system
calls.
- For Mac OS X, we use ASL(3) instead of syslog(3) for logging.
- Add support for NOTICE level logging.

OpenBSM 1.1 alpha 3

- Add two new functions, au_bsm_to_errno() and au_errno_to_bsm(), to map
between BSM error numbers (largely the Solaris definitions) and local
errno(2) values for 32-bit and 64-bit return tokens. This is required
as operating systems don't agree on some of the values of more recent
error numbers.
- Fix a bug how au_to_exec_args(3) and au_to_exec_env(3) calculates the
total size for the token. This buge.
- Deprecated Darwin constants, such as TRAILER_PAD_MAGIC, removed.

src/sys/dev/usb2/controller/uss820dci_pccard.c
src/sys/dev/usb2/core/usbdevs
src/sys/dev/usb2/include/urio2_ioctl.h
src/sys/dev/usb2/storage/ustorage2_fs.h

These files are not used any more.

src/usr.sbin/Makefile
src/etc/mtree/BSD.include.dist
src/include/Makefile
src/lib/Makefile
src/share/man/man7/hier.7
src/share/mk/bsd.libnames.mk
src/etc/mtree/BSD.include.dist

Make "usbconfig" and "libusb20" a part of the default build.

src/sys/dev/usb/rio500_usb.h
src/sys/dev/usb2/storage/urio2.c

Use common include file.

src/sys/dev/usb2/bluetooth/ng_ubt2.c

Make USB bluetooth depend on "ng_hci" module.

src/sys/dev/usb2/controller/ehci2.c
src/sys/dev/usb2/controller/ehci2.h

Patches for Marvell EHCI.

src/sys/dev/usb2/core/usb2_busdma.c

Bugfix for 64-bit platforms. Need to unload the previously loaded DMA
map and some cleanup regarding some corner cases.

src/sys/dev/usb2/core/usb2_core.h
src/sys/dev/usb2/core/usb2_dev.c
src/sys/dev/usb2/core/usb2_dev.h

Bugfix for libusb filesystem interface.

New feature: Add support for filtering device data at the expense of the
userland process.

Add some more comments.

Some minor code styling.

Remove unused function, usb2_fifo_get_data_next().

Fix an issue about "fifo_index" being used instead of "ep_index".

src/sys/dev/usb2/core/usb2_device.c
src/sys/dev/usb2/core/usb2_generic.c

Bugfix for Linux USB compat layer. Do not free non-generic FIFOs when
doing an alternate setting.

Cleanup USB IOCTL and USB reference handling.
Fix a corner case where USB-FS was left initialised after
setting a new configuration or alternate setting.

src/sys/dev/usb2/core/usb2_hub.c

Improvement: Check all USB HUB ports by default at least one time.

src/sys/dev/usb2/core/usb2_request.c

Bugfix: Make sure destination ASCII string is properly zero terminated
in all cases.

Improvement: Skip invalid characters instead of replacing with a dot.

src/sys/dev/usb2/core/usb2_util.c
src/sys/dev/usb2/image/uscanner2.c

Spelling.

src/sys/dev/usb2/include/Makefile

Share "usbdevs" with the old USB stack.

src/sys/dev/usb2/include/usb2_devid.h
src/sys/dev/usb2/include/usb2_devtable.h

Regenerate files.

Alfred: Please fix the RCS tag at the top.

src/sys/dev/usb2/include/usb2_ioctl.h

Fix compilation of "kdump".

src/sys/dev/usb2/serial/ubsa2.c
src/sys/dev/usb2/serial/ugensa2.c

Remove device ID's which will end up in a new 3G driver.

src/sys/dev/usb2/sound/uaudio2.c

Correct a debug printout.

src/sys/dev/usb2/storage/umass2.c

Sync with old USB stack.

src/lib/libusb20/libusb20.3

Add more documentation.

src/lib/libusb20/libusb20.c

Various bugfixes and improvements.

src/usr.sbin/usbconfig/dump.c
src/usr.sbin/usbconfig/usbconfig.c

New commands for dumping strings and doing custom USB requests from
the command line.

Remove keyword requirements from generated files:
"head/sys/dev/usb2/include/usb2_devid.h"
"head/sys/dev/usb2/include/usb2_devtable.h"

Add support for the Compact C Type (CTF) conversions throughout FreeBSD's
system makefiles.

Note that the CTF conversion defaults to off. We may choose to change this
default later if DTrace proves popular and people are prepared to wear
the compilation performance impact of compiling with debug symbols all the
time.

Setting NO_CTF in the make args or user environment turns off CTF conversion.
Even if we choose to default CTF generation to on later, we still need
NO_CTF so that the buildworld process can bootstrap the tools without
needlessly generating CTF data for temporary tools.

Setting WITH_CTF in the make args or user environment (and _NOT_ in
/etc/make.conf) is the only way to enable CTF data conversion. Nore that
this can't be implemented the same way that the WITH_ and WITHOUT_ stuff
is implemented throughout the buildworld because the CTF conversion needs
to work when building a simple object without a Makefile, using the
default rules in sys.mk.

Typing 'make test.o' with no makefile and just a source file test.c
should work. Also, typing 'make WITH_CTF=1 test.o without a makefile and
just a source file test.c should work and produce an object with a CTF
elf section. Typing 'make WITH_CTF=1 CFLAGS=-g test.o' without a makefile
and just a source file test.c should produce an object with both a CTF
elf section and the debug elf sections.

In the FreeBSD build where more .mk files are used than just sys.mk
which is included my make by default, the use of DEBUG_FLAGS is the
correct way to enable a debug build. The important thing to note here
is that it is the DEBUG_FLAGS setting that prevents libraries and
programs from being stripped on installation. So, for the addition of
CTF data conversion, setting DEBUG_FLAGS to contain -g, without NO_CTF,
will cause the ctfconvert and ctfmerge build programs to be executed
also with the -g arg so that debug symbols are retained rather than
being removed after the CTF data elf section has been added.

Add DTrace libraries to the list of libnames.

Fix conflicts after heimdal-1.1 import and add build infrastructure. Import
all non-style changes made by heimdal to our own libgssapi.

Add LIBELF.

Fixed "make checkdpadd" (missing library dependencies).

Approved by: re (kensmith)

Enable symbol versioning by default. Use WITHOUT_SYMVER to disable it.
Warning, after symbol versioning is enabled, going back is not easy
(use WITHOUT_SYMVER at your own risk).

Change the default thread library to libthr.

There most likely still needs to be a version bump for at least the
thread libraries. If necessary, this will happen later.

Oops, keep things sorted.

Found by: ru

Add new libraries. We may want to rename libumem to not colide with
ports/devel/umem.

Enable ncurses wide character support

Approved by: delphij (mentor)
Tested by: kris on pointyhat (early version), current@

Remove old libmytinfo link.

Approved by: delphij (mentor)
Requested by: ache

Removed libc_r build support.

Extend coverage of the MK_IPX build option to the following:

- <netipx> headers [1]
- IPX library (libipx)
- IPX support in ifconfig(8)
- IPXrouted(8)
- new MK_NCP option

New MK_NCP build option controls:

- <netncp> and <fs/nwfs> headers
- NCP library (libncp)
- ncplist(1) and ncplogin(1)
- mount_nwfs(8)
- ncp and nwfs kernel modules

User knobs: WITHOUT_IPX, WITHOUT_IPX_SUPPORT, WITHOUT_NCP.

[1] <netsmb/netbios.h> unconditionally uses <netipx> headers
so they are still installed. This needs to be dealt with.

Reimplementation of world/kernel build options. For details, see:

http://lists.freebsd.org/pipermail/freebsd-current/2006-March/061725.html

The src.conf(5) manpage is to follow in a few days.

Brought to you by: imp, jhb, kris, phk, ru (all bugs are mine)

Define $LIBBSM.

Obtained from: TrustedBSD Project

Add LIBMEMSTAT tp bsd.libnames.mk.

MFC after: 1 week

Bring a working snapshot of hwpmc(4), its associated libraries, userland utilities
and documentation into -CURRENT.

Bump FreeBSD_version.

Reviewed by: alc, jhb (kernel changes)

Remove trailing spaces.

NOCRYPT -> NO_CRYPT

Add knob NO_NIS (fka NO_YP_LIBC) and make world compileable when set.
If turned on no NIS support and related programs will be built.

Lost parts rediscovered by: Danny Braniss <danny at cs.huji.ac.il>
PR: bin/68303
No objections: des, gshapiro, nectar
Reviewed by: ru
Approved by: rwatson (mentor)
MFC after: 2 weeks

Fix build in the !NOATM case by using the begemot library in place of
using libisc which was a part of BIND8.

Discussed with: des, re, dougb
Submitted by: harti (one part)
Reviewed by: harti (previous version)

Fix the WANT_BIND_LIBS knob by correctly spelling it as WITH_BIND_LIBS
to match how similar syntax is used in the ports system. Thanks to kris
for pointing out my mistake here.

Install the lwres library unless the user defines NO_BIND, or the new
knob, NO_BIND_LIBS_LWRES. There is at least one potential customer
for this library in the wings. Thanks to nectar for the reminder.

Don't expose BIND libraries and their headers to the public by default,
but have a knob (WANT_BIND_LIBS) to build and install them in /usr/lib
and /usr/include. Rumors are that this may be useful at a later point,
let's see.

What this really means is that all BIND libraries are now internal to
buildworld (by default, unless WANT_BIND_LIBS is defined), and linked
statically into various BIND executables.

While here, removed redundant -I's from CFLAGS in lib/bind makefiles.

Sponsored by: des
OK'ed by: dougb

- Wrapped BIND 9 libraries defines into !defined(NO_BIND).

- Added forgotten LIBLWRES to fix missing dependencies
revealed by "make checkdpadd".

Switch from BIND 8 to BIND 9.

Submitted by: (in part) dougb@, trhodes@
Reviewed by: dougb@, trhodes@, re@
MFC after: 5 days

The last commit had one too many libs.

Bmake the library containing and processing the magic.

Register libarchive in bsd.libnames.mk and mdoc.local

Submitted by: ru

First round of cleanups to sys/boot/ makefiles:

- do not use PROG for what's not a real C program,
- use sys.mk transformation rules where possible,
- only create the "machine" symlink on AMD64,
- removed MAINTAINER lines in individual makefiles,
- added the LIBSTAND defitinion to <bsd.libnames.mk>,
- somewhat better contents in .depend files.

Tested on: i386, amd64
Prodded by: bde

Nothing in libypclnt depends on librpcsvc.

Reported by: lorder(1) (modified to work with libraries)

Fixed MINUSLPAM:

- Added missing NOCRYPT and NO_OPENSSL checks for Kerberos.

- Don't depend on -lcrypto and -lcrypt in pam_ssh to resolve
dependencies in pam_krb5 and pam_ksu -- the former may not
be compiled at all if NO_OPENSSH knob is enabled.

- Added missing -lcrypt to pam_ssh dependencies.

- Moved librpcsvc after libypclnt.

(The last two aren't strictly speaking necessary to resolve
the dependencies of static versions of pam_ssh and pam_unix,
respectively, but they correspond to dynamic dependencies
of libssh and libypclnt, and are put here for consistency.)

In collaboration with: bde
Reviewed by: des

GC LIBPC and LIBPLOT: they never existed in unencumbered BSD versions.

Reviewed by: bde

Add LIBPTHREAD.

1. Garbage-collected LIBDES (now in a different library), LIBPERL (banished
to ports) and LIBRESOLV (now in a different library.

2. Added comments about nonexistent libraries LIBPC and LIBPLOT.

Submitted by: ru (1)

Fixed insertion sort errors for LIBBLUETOOTH and LIBSDP.

Removed XXX comments about some libraries only being in the secure
distribution. This is not the place to document this, especially
now that the secure distribution is the normal one.

Reviewed by: ru

Add a define for libbsnmp.

Add a definition for libngatm.

Reviewed by: ru

Update Bluetooth code.

Reviewed by: M. Warner Losh <imp@bsdimp.com>; John Hay <jhay@freebsd.org>
Approved by: M. Warner Losh <imp@bsdimp.com> (mentor)

Update build infrastructure for Heimdal 0.6.

- Support for multibyte charsets in LIBICONV.
- CD9660_ICONV, NTFS_ICONV and MSDOSFS_ICONV kernel options
(with corresponding modules).
- kiconv(3) for loadable charset conversion tables support.

Submitted by: Ryuichiro Imura <imura@ryu16.org>

Libraries come.

Sort.

Build/install the PIC version of libgcc (libcc_pic.a) for use by shared
libraries that do exception unwinding.

Update some library names. Libraries come, libraries go.

Turn MAKE_KERBEROS5 into NO_KERBEROS by negating the logic. Some extra
cleanups were necessary in release/Makefile, and the tinderbox code
was syntax checked, not run checked.

Added GEOM library to the bsd.libnames.mk namespace.

KerberosIB deorbit: Remove library references.

Added UFS library to the bsd.libnames.mk namespace.

NOSHARED is meaningless in the bsd.lib.mk context, so check LDFLAGS
for the -static flag instead when constructing LIBPAM.

(This fixes false warnings from ``make checkdpadd -DNOSHARED'' in
lib/libpam/modules/.)

Submitted by: bde, ru
Approved by: re

Sort in ``phone directory'' order (except for LIBC_*).

Reflect the fact that we install our libexpat as libbsdxml.

Add LIBEXPAT definition.

Add a libnames entry for libugidfw.
Add a DPADD line for ${LIBUGIDFW} for ugidfw.

Submitted by: ru

Don't include bsd.own.mk from sys.mk, this makes it impossible
to use ``.if defined()'' inside bsd.own.mk to test for defines
in individual makefiles. For example, setting DEBUG_FLAGS in
Makefile didn't take the desired effect on the STRIP assignment.

Added bsd.init.mk (like in NetBSD) that handles the inclusion
of ../Makefile.inc and bsd.own.mk from all bsd.*.mk files that
"build something".

Back out bsd.own.mk,v 1.15: moved OBJFORMAT initialization back
to sys.mk (several source tree makefiles want to check it early)
and removed MACHINE_ARCH initialization (it's hard to see from
looking at the commitlogs what the problem was at the time, but
now it serves no purpose).

Prohibit the direct inclusion of bsd.man.mk and bsd.libnames.mk.

Protect bsd.obj.mk from repetitive inclusion. Prohibiting the
direct inclusion of bsd.obj.mk might be a good idea too.

Add a dependency on libypclnt to libpam, in preperation for making pam_unix
use it to update NIS passwords.

Add libypclnt.

Fixed broken dependency in lib/libpam/modules/pam_krb5.

Install libusbhid, and use it instead of libusb.

MFC after: 6 days

Add two new libraries which are part of sendmail 8.12. libsm is used only
for building sendmail and the associated utilities. libmilter is a new
mail filtering API for sendmail.

Add pam_ssh support to the static PAM library, libpam.a:

- Spam /usr/lib some more by making libssh a standard library.
- Tweak ${LIBPAM} and ${MINUSLPAM}.
- Garbage collect unused libssh_pic.a.
- Add fake -lz dependency to secure/ makefiles needed for
dynamic linkage with -lssh.

Reviewed by: des, markm
Approved by: markm

Add the necessary dependencies and linker flags for linking with a
static PAM library that includes pam_ssh.

Sponsored by: DARPA, NAI Labs

Add LIBSMB.

Fixed world breakage in ftpd, rshd, login, su, telnetd and telnet. LIBPAM
and MINUSLPAM must be kept in sync with the libraries linked to by libpam
to support static linkage.

Moved libmd to the end of LIBPAM and MINUSLPAM. It was before libopie,
but libopie references it, so static linkage only worked accidentally.

Removed some garbage (LIBGMP, LIBSKEY). Using LIBSKEY in LIBPAM should
have been fatal since it gave a dependency on a nonexistent file, but it
worked because of an undocumented bugfeature in make(1): missing source
files named *.a are silently assumed to be up to date.

Fixed some style bugs (formatting).

Zap LIBSS

Add LIBBZ2.

Clean up the PAM lib lists. Particularly relevant to the KRB5 case.

Kerberos updates for PAM.

Fixed some bitrot:
- the changes that renamed libf2c to libg2c had not reached here
- there were no definitions for LIBDEVINFO, LIBMENU, LIBPANEL, LIBTINFO,
LIBUSB or LIBVGL. LIBUSB was used without it being defined, and
LIBDEVINFO and LIBVGL should have been used.
- the definitions of LIBDESCRYPT, LIBGCC_PIC, LIBGPLUSPLUS, LIBKZHEAD,
LIBKZTAIL, LIBSCRYPT and LIBSCSI were garbage.

Fixed some old bugs:
- LIBC_PIC and LIBCOM_ERR were assigned to using "=" instead of "?=".
- the definition of LIBC_R was disordered.
- LIBFORM was misspelled LIBFORMS (but not actually used).

Cleaner method of making PAMable apps static (in the optional case of
wanting static apps).

Remove the recently-depricated LIBPOSIX1E

Bye-bye /usr/lib/libtelnet.a. This should fix ``make release'' brokeness.

Approved by: markm

Rewrite of the CAM error recovery code.

Some of the major changes include:

- The SCSI error handling portion of cam_periph_error() has
been broken out into a number of subfunctions to better
modularize the code that handles the hierarchy of SCSI errors.
As a result, the code is now much easier to read.

- String handling and error printing has been significantly
revamped. We now use sbufs to do string formatting instead
of using printfs (for the kernel) and snprintf/strncat (for
userland) as before.

There is a new catchall error printing routine,
cam_error_print() and its string-based counterpart,
cam_error_string() that allow the kernel and userland
applications to pass in a CCB and have errors printed out
properly, whether or not they're SCSI errors. Among other
things, this helped eliminate a fair amount of duplicate code
in camcontrol.

We now print out more information than before, including
the CAM status and SCSI status and the error recovery action
taken to remedy the problem.

- sbufs are now available in userland, via libsbuf. This
change was necessary since most of the error printing code
is shared between libcam and the kernel.

- A new transfer settings interface is included in this checkin.
This code is #ifdef'ed out, and is primarily intended to aid
discussion with HBA driver authors on the final form the
interface should take. There is example code in the ahc(4)
driver that implements the HBA driver side of the new
interface. The new transfer settings code won't be enabled
until we're ready to switch all HBA drivers over to the new
interface.

src/Makefile.inc1,
lib/Makefile: Add libsbuf. It must be built before libcam,
since libcam uses sbuf routines.

libcam/Makefile: libcam now depends on libsbuf.

libsbuf/Makefile: Add a makefile for libsbuf. This pulls in the
sbuf sources from sys/kern.

bsd.libnames.mk: Add LIBSBUF.

camcontrol/Makefile: Add -lsbuf. Since camcontrol is statically
linked, we can't depend on the dynamic linker
to pull in libsbuf.

camcontrol.c: Use cam_error_print() instead of checking for
CAM_SCSI_STATUS_ERROR on every failed CCB.

sbuf.9: Change the prototypes for sbuf_cat() and
sbuf_cpy() so that the source string is now a
const char *. This is more in line wth the
standard system string functions, and helps
eliminate warnings when dealing with a const
source buffer.

Fix a typo.

cam.c: Add description strings for the various CAM
error status values, as well as routines to
look up those strings.

Add new cam_error_string() and
cam_error_print() routines for userland and
the kernel.

cam.h: Add a new CAM flag, CAM_RETRY_SELTO.

Add enumerated types for the various options
available with cam_error_print() and
cam_error_string().

cam_ccb.h: Add new transfer negotiation structures/types.

Change inq_len in the ccb_getdev structure to
be "reserved". This field has never been
filled in, and will be removed when we next
bump the CAM version.

cam_debug.h: Fix typo.

cam_periph.c: Modularize cam_periph_error(). The SCSI error
handling part of cam_periph_error() is now
in camperiphscsistatuserror() and
camperiphscsisenseerror().

In cam_periph_lock(), increase the reference
count on the periph while we wait for our lock
attempt to succeed so that the periph won't go
away while we're sleeping.

cam_xpt.c: Add new transfer negotiation code. (ifdefed
out)

Add a new function, xpt_path_string(). This
is a string/sbuf analog to xpt_print_path().

scsi_all.c: Revamp string handing and error printing code.
We now use sbufs for much of the string
formatting code. More of that code is shared
between userland the kernel.

scsi_all.h: Get rid of SS_TURSTART, it wasn't terribly
useful in the first place.

Add a new error action, SS_REQSENSE. (Send a
request sense and then retry the command.)
This is useful when the controller hasn't
performed autosense for some reason.

Change the default actions around a bit.

scsi_cd.c,
scsi_da.c,
scsi_pt.c,
scsi_ses.c: SF_RETRY_SELTO -> CAM_RETRY_SELTO. Selection
timeouts shouldn't be covered by a sense flag.

scsi_pass.[ch]: SF_RETRY_SELTO -> CAM_RETRY_SELTO.

Get rid of the last vestiges of a read/write
interface.

libkern/bsearch.c,
sys/libkern.h,
conf/files: Add bsearch.c, which is needed for some of the
new table lookup routines.

aic7xxx_freebsd.c: Define AHC_NEW_TRAN_SETTINGS if
CAM_NEW_TRAN_CODE is defined.

sbuf.h,
subr_sbuf.c: Add the appropriate #ifdefs so sbufs can
compile and run in userland.

Change sbuf_printf() to use vsnprintf()
instead of kvprintf(), which is only available
in the kernel.

Change the source string for sbuf_cpy() and
sbuf_cat() to be a const char *.

Add __BEGIN_DECLS and __END_DECLS around
function prototypes since they're now exported
to userland.

kdump/mkioctls: Include stdio.h before cam.h since cam.h now
includes a function with a FILE * argument.

Submitted by: gibbs (mostly)
Reviewed by: jdp, marcel (libsbuf makefile changes)
Reviewed by: des (sbuf changes)
Reviewed by: ken

List libfetch.

Submitted by: bde

Add LIBPOSIX1E

Zap LIBTCL, it's been superfluous for several years.

Add ${LIBC_R}

Build the ISC library as libisc. This library comes as part of the
bind distribution, but until now was not being built as a separate
entity. For documentation, see these man pages:

assertions(3), eventlib(3), heap(3), logging(3), memcluster(3), tree(3).

Reviewed by: jdp

Remove LIBRSAGLUE, add LIBSSL

Reviewed by: kris

Replace ${LIBMYTINFO} with warning

Fixed world breakage for the NOSHARED=yes case. libpam now depends on
libopie.

Don't say that libpam.a doesn't exist.

Add libipsec.

Approved by: jkh

We have some new libraries; give them names.

Fixed breakage of static linkage of rlogind. Someone added -lutil and
-lcom_err to some libpam modules without updating LIBPAM here.

Whistle's Netgraph link-layer (sometimes more) networking infrastructure.
Been in production for 3 years now. Gives Instant Frame relay to if_sr
and if_ar drivers, and PPPOE support soon. See:
ftp://ftp.whistle.com/pub/archie/netgraph/index.html
for on-line manual pages.

Reviewed by: Doug Rabson (dfr@freebsd.org)
Obtained from: Whistle CVS tree

Add libncp to the list of known libraries.

$Id$ -> $FreeBSD$

add libhistory

New names for the list; the DES crypt and libwrap for tcp_wrappers.

Change the definition of LIBPAM to support the static PAM library.
When linking statically, LIBPAM is augmented with the extra libraries
that the PAM modules require. The idea is to centralize this
information rather than scattering it about in the Makefiles of
all the applications that use (OK, will use) PAM.

There is a new variable MINUSLPAM that should be used instead of
"-lpam". In the static case, it gets -l flags for the extra required
libraries.

This approach was suggested by <bde>, but he didn't actually review
my changes.

Add LIBPAM, LIBRADIUS, and LIBTACPLUS.

Two patches from the HARP people:

Various Makefile related fixes.

-Wformat fixes.

Submitted by: Mike Spengler <mks@networkcs.com>

Define LIBCAM and LIBDEVSTAT.

Add libperl for Perl5.

add libcipher, and update comments about libdes, libkdb, libkrb etc
which do exist but are in the secure dist rather than the base.

Fixed double slashes in pathnames.

ELF preparation step 2:

Move a.out libraries to /usr/lib/aout to make space for ELF libs.
Make rtld usr /usr/lib/aout as default library path.
Make ldconfig reject /usr/lib as an a.out library path.
Fix various Makefiles for LIBDIR!=/usr/lib breakage.

This will after a make world & reboot give a system that no
longer uses /usr/lib/*, infact one could remove all the old
libraries there, they are not used anymore.

We are getting close to an ELF make world, but I'll let this
all settle for a week or two...

Oops, forgot references in previous commit.

Submitted by: Brian Cully <shmit@erols.com>
PR: 6178

Support Objective C almost as well as C++.

Notes:
- We no longer use -fgnu-runtime in bsd.lib.mk, since it is the default
and bsd.lib.mk is the wrong place to override it.
- Gnu C doesn't have a special compiler driver for Objective C like it
does for C++. The defaults are suitable for Gnu C. Use `OBJCLIBS='
in /etc/make.conf for POC.

Add libcalendar

Add LIBZ

Add LIBALIAS.

Revert $FreeBSD$ to $Id$

Put libopie.a in libnames

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

Add a general comment about this include file.

Add LIBSTDCPLUSPLUS
(can't use the '+' char in variable names, same as LIBGPLUSPLUS)

libresolv.a doesn't exist anymore... Hmm, what's the right thing to do
here? Can we just undefine it? What about 3rd party bmakefiles?

Moved definitions of library names from bsd.prog.mk to a new central
file bsd.libnames.mk and include this file where necessary. This fixes
null library names in ${DPADD}'s in library makefiles.