libc: Purge unneeded cdefs.h

These sys/cdefs.h are not needed. Purge them. They are mostly left-over
from the $FreeBSD$ removal. A few in libc are still required for macros
that cdefs.h defines. Keep those.

Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D42385

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-NetBSD identifier is obsolete, drop -NetBSD

The SPDX folks have obsoleted the BSD-2-Clause-NetBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

libc: Fix build with -DNS_REREAD_CONF.

MFC after: 3 days

Revert "lib/libc/net/nsdispatch.c: Fix missing unlock and add locking annotations"

This commit should not have introduced any functional changes, but
apparently it did. This appears to have broken LDAP setups.
Reverting for now. Will reland once I have fixed the breakage.

This reverts commit 5245bf7b92b74e556527b4916a8deba386fe5772.
Reported By: Александр Недоцуков, brd
MFC after: immediately

lib/libc/net/nsdispatch.c: Fix missing unlock and add locking annotations

The error cases (goto fin) of _nsdispatch were missing the unlock.

This change also drops the checks for __isthreaded since the pthread stubs
are already no-ops if threads are not being used. Dropping those conditionals
allows clang's thread safety analysis to deal with the file and also makes
the code a bit more readable. While touching the file also add a few more
assertions in debug mode that the right locks are held.

Reviewed By: markj
Differential Revision: https://reviews.freebsd.org/D29372

Don't call stat(2) on nsswitch.conf(5) every time nsdispatch(3)
and dependent functions (eg getpwname(3)) get called. This can
improve performance of binaries that perform a lot of name
lookups, such as gssd(8). It also matches documented behaviour
of Linux and Solaris.

The old code is left in place, should anyone need it, guarded
by #ifdef NS_REREAD_CONF.

Reviewed by: imp, bcr
MFC after: 2 weeks
Relnotes: yes
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D17934

Don't call dlopen(3) for built-in NSS types - "cache", "compat",
"dns", "files", "db", and "nis". It saves some path lookups during
binary startup.

Reviewed by: markj
Approved by: re (gjb, kib)
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D17557

libc: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using mis-identified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Don't dlclose NSS modules from nss_atexit().

As hinted in the commit log message for r259042, this is unnecessary.
Moreover, as a result of that change we may invoke a DSO's atexit handler
after it has been unmapped.

Reviewed by: bdrewery, cem
MFC after: 1 week
Sponsored by: Dell EMC Isilon

libc: provide some bounds-checking through reallocarray(3).

reallocarray(3) is a non portable extension that originated in OpenBSD.
Given that it is already in FreeBSD's libc it is useful for the cases
where reallocation involves a multiplication.

MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D9955

Fix indentation in nss_configure().

MFC after: 1 week
Sponsored by: Dell EMC Isilon

Register nss_atexit() before parsing nsswitch.conf for the first time.

NSS modules are loaded when nsswitch.conf is parsed and may register their
own atexit handlers with libc. nss_atexit() unloads any dynamically loaded
NSS modules, so it should run only after the modules' atexit handlers have
been invoked.

MFC after: 2 weeks
Sponsored by: Dell EMC Isilon

The fallback flag in nsdispatch prevents the fallback implementation of
getgroupmembership() from invoking the correct backend in the compat case.
Replace it with a nesting depth counter so it only blocks one level (the
first is the group -> group_compat translation, the second is the actual
backend). This is one of two bugs that break getgrouplist() in the compat
case, the second being that the backend's own getgroupmembership() method
is ignored. Unfortunately, that is not easily fixable without a redesign
of our nss implementation (which is also needed to implement the +@group
syntax in /etc/passwd).

PR: 190055
MFC after: 1 week

Eliminate conf_lock and instead rely on the NSS write lock to protect
NSS configuration state.

As a side effect, this fixes a race condition which can occur if multiple
threads call nsdispatch(3) concurrently before nsswitch.conf has been
parsed. Previously, the thread holding conf_lock could cause other threads
to return from nss_configure() before nsswitch.conf had been parsed, forcing
them to fall back to the default sources for their NSS methods.

Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D994
MFC after: 1 month
Sponsored by: EMC / Isilon Storage Division

libc: Make various internal file descriptors from fopen() close-on-exec.

Remove clause 3 and 4 from TNF licenses

Per the NetBSD Foundation statement
Third parties are encouraged to change the license on any files which
have a 4-clause license contributed to the NetBSD Foundation to a
2-clause license.

Fix a problem whereby a corrupt DNS record can cause named to crash. [11:06]

Add an API for alerting internal libc routines to the presence of
"unsafe" paths post-chroot, and use it in ftpd. [11:07]

Fix a buffer overflow in telnetd. [11:08]

Make pam_ssh ignore unpassphrased keys unless the "nullok" option is
specified. [11:09]

Add sanity checking of service names in pam_start. [11:10]

Approved by: so (cperciva)
Approved by: re (bz)
Security: FreeBSD-SA-11:06.bind
Security: FreeBSD-SA-11:07.chroot
Security: FreeBSD-SA-11:08.telnetd
Security: FreeBSD-SA-11:09.pam_ssh
Security: FreeBSD-SA-11:10.pam

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Clear errno for each method dispatch.

Spotted by: Kostik Belousov <kostikbel__at__gmail.com>
MFC after: 2 weeks

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

Fix an issue when nss fallback routines are used in a multithreaded application.

Reviewed by: bushman
Approved by: dfr (mentor)

- Import the HEAD csup code which is the basis for the cvsmode work.

Include libc_private.h for the declaration of __isthreaded instead of
relying on namespace pollution in stdio.h.

MFC after: 3 days

Moved logging out of the nss_method_lookup() in order not to
flood logs with failed fallback method lookup attempts.

Implementing 'fallback' nsswitch source. 'fallback' source is used
when particular function can't be found in nsswitch-module. For
example, getgrouplist(3) will use module-supplied 'getgroupmembership'
function (which can work in an optimal way for such source as LDAP) and
will fall back to the stanard iterate-through-all-groups implementation
otherwise.

PR: ports/114655
Submitted by: Michael Hanselmann <freebsd AT hansmi DOT ch>
Reviewed by: brooks (mentor)

fix memory leak from reading nsswitch.conf

MFC after: 1 week

- Extend the nsswitch to support Services, Protocols and Rpc
databases.
- Make nsswitch support caching.

Submitted by: Michael Bushkov <bushman__at__rsu.ru>
Sponsored by: Google Summer of Code 2005

The previous commit changed the behavior of nsdispatch() in the
case where an /etc/nsswitch.conf file was present, but could not
be opened (e.g. due to permissions). Previously, the open failure
condition was suppressed, and the built-in defaults were used. In
revision 1.11, however, propagated the open failure causing all
nsdispatch() invocations to return NS_UNAVAIL, and thus many APIs
including getpwnam and gethostbyname unconditionally failed.

This commit restores the previous behavior.

Pointy hat: nectar (+1 for obstinance; ache had to use clue bat)
Reported by: ache

When a dynamic NSS module is built and linked against a thread
library, it may pull in that thread library at run time. If the
process started out single-threaded, this could cause attempts to
release locks that do not exist. Guard against this possibility by
checking __isthreaded before invoking thread primitives.

A similar problem remains if the process is linked against one thread
library, but the NSS module is linked against another. This can only
be avoided by careful design of the NSS module.

Submitted by: Sean McNeil <sean@mcneil.com> (mostly; bugs are mine)

Make this compile with -O2. A proper fix would use a struct to represent
vectors, instead of requiring the caller to keep track of element size
and count and pass them in by reference.

Whitespace nits.

Don't complain about missing NSS methods when built statically. It is
annoying and not very useful.

Sponsored by: DARPA, Network Associates Laboratories

= Implement name service switch modules (NSS modules). NSS modules
may be built into libc (`static NSS modules') or dynamically loaded
via dlopen (`dynamic NSS modules'). Modules are loaded/initialized
at configuration time (i.e. when nsdispatch is called and nsswitch.conf
is read or re-read).

= Make the nsdispatch(3) core thread-safe.

= New status code for nsdispatch(3) `NS_RETURN', currently used to
signal ERANGE-type issues.

= syslog(3) problems, don't warn/err/abort.

= Try harder to avoid namespace pollution.

= Implement some shims to assist in porting NSS modules written for
the GNU C Library nsswitch interface.

Sponsored by: DARPA, Network Associates Laboratories

Eliminate 19 warnings in libc (at level WARNS=2) of the
`implicit declaration of function' variety.

Assume __STDC__, remove non-__STDC__ code.

Submitted by: keramida

Fix the style of the SCM ID's.
I believe have made all of libc .c's as consistent as possible.

Remove __P() usage.

Fixed namespace pollution related to `err' in libc in the same way as for
`warn'. Now a whole 2 members of the err() family don't cause pollution.

This fixes world breakage in awk for NOSHARED worlds. contrib/awk/msg.c
has had its own version of err() for a long time, but this somehow
didn't cause problems until the update to awk-3.1.0.

Add nsswitch support. By creating an /etc/nsswitch.conf file, you can
configure FreeBSD so that various databases such as passwd and group can be
looked up using flat files, NIS, or Hesiod.

= Hesiod has been added to libc (see hesiod(3)).

= A library routine for parsing nsswitch.conf and invoking callback
functions as specified has been added to libc (see nsdispatch(3)).

= The following C library functions have been modified to use nsdispatch:
. getgrent, getgrnam, getgrgid
. getpwent, getpwnam, getpwuid
. getusershell
. getaddrinfo
. gethostbyname, gethostbyname2, gethostbyaddr
. getnetbyname, getnetbyaddr
. getipnodebyname, getipnodebyaddr, getnodebyname, getnodebyaddr

= host.conf has been removed from src/etc. rc.network has been modified
to warn that host.conf is no longer used at boot time. In addition, if
there is a host.conf but no nsswitch.conf, the latter is created at boot
time from the former.

Obtained from: NetBSD