MFC r303794:
Create namespace for the symbols added during 12-CURRENT cycle.

MFC r303795:
Add __cxa_thread_atexit(3) API implementation.

Approved by: re (gjb, bdrewery (?))

Copy stable/11@r303970 to releng/11.0 as part of the 11.0-RELEASE
cycle.

Prune svn:mergeinfo from the new branch, and rename it to RC1.

Update __FreeBSD_version.

Use the quarterly branch for the default FreeBSD.conf pkg(8) repo and
the dvd1.iso packages population.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

This commit addresses regression introduceded in r302177
(WITH_SYSTEM_COMPILER: Enable by default) and it's prerequisite: r300354,
caused i386 builds to fail when cross-built on an amd64 host.

Reviewed by: bdrewery, delphij, gjb
Approved by: re (gjb)

iconvctl(3): remove superfluous NULL pointer tests

convname and dst are guaranteed to be non-NULL by iconv_open(3).
src is an array. Remove these tests for NULL pointers.
While I'm here, eliminate a strlcpy with a correct but suspicious-looking
calculation for the third parameter (i.e. not a simple sizeof).
Compare the strings in-place instead of copying.

Found by: bdrewery
Found by: Coverity
CID: 1130050, 1130056
MFC after: 3 days
Sponsored by: Dell Inc.
Differential Revision: https://reviews.freebsd.org/D6338

Read the contents of the snapshot files properly

- Use fgetln instead of fgets; localize complexity related to fgetln(3)
inside the loop.
- Skip over blank lines.
- Skip over lines (properly) that start with a "#"

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

print_positional_test: Fix misuse of wchar APIs

These APIs take unit length, not byte length parameters.

Reported by: Coverity
CIDs: 1338543, 1338544, 1338545
Sponsored by: EMC / Isilon Storage Division

nss/gethostby_test: fix broken vector iteration of gethostbyaddr h_aliases

h_aliases is a NULL-terminated rather than fixed-length array. nitems() is not
a valid way to determine its end; instead, check for NULL.

Reported by: Coverity
CID: 1346578
Sponsored by: EMC / Isilon Storage Division

Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
namespacing is kept with FILES appropriately, and that this shouldn't need
to be repeated if the namespace changes -- only the definition of PACKAGE
needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
`tests`. In the event we get to the point where things can be split up
enough in the base system, it would make more sense to group the tests
with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division

libc: spelling fixes.

Mostly on comments.

Fix double fclose of `fp1` when freopen fails

freopen handles closing file descriptors on error, with the exception of
fdopen'ed descriptors, so closing an already fclose'd file descriptor is
incorrect

CID: 1338525
Differential Revision: https://reviews.freebsd.org/D6013
MFC after: 2 weeks
Reported by: Coverity
Sponsored by: EMC / Isilon Storage Division

Make sure fmemopen succeeds in :test_append_binary_pos before calling ftell
on the FILE object

This fixes potential null pointer dereferences on failure

CID: 1254952
MFC after: 2 weeks
Reported by: Coverity
Sponsored by: EMC / Isilon Storage Division

Merge the projects/release-pkg branch to head.

This allows packaging the base system with pkg(8), including
but not limited to providing the ability to provide upstream
binary update possibilities for non-tier-1 architectures.

This merge is a requirement of the 11.0-RELEASE, and as such,
thank you to everyone that has tested the project branch.

Documentation in build(7) etc. is still somewhat sparse, but
updates to those parts will follow.

Sponsored by: The FreeBSD Foundation

WITHOUT_TOOLCHAIN: Skip building of h_raw.

-fsanitize does not seem to work when a --sysroot is specified and there
is no <sysroot>/usr/lib/clang/3.8.0/lib/freebsd/libclang_rt.ubsan_standalone-*.a.

Sponsored by: EMC / Isilon Storage Division

DIRDEPS_BUILD: Connect MK_TESTS.

Sponsored by: EMC / Isilon Storage Division

Fix and connect setjmp test.

Sponsored by: EMC / Isilon Storage Division

libc: Add some tests for memcmp().

Remove dd xfer stats emitted during buildworld

They result in gratuitous differences when comparing build log output.

Fix build race after r295643.

Sponsored by: EMC / Isilon Storage Division

This seems like a very trivial bug that should have been squashed a long
time ago, but for some reason it was not. Basically, without this change
dlopen(3)'ing an empty .so file would just cause application to dump core
with SIGSEGV.

Make sure the file has enough data for at least the ELF header before
mmap'ing it.

Add a test case to check that dlopen an empty file return an error.

There were a separate discussion as to whether it should be SIGBUS
instead when you try to access region mapped from an empty file,
but it's definitely SIGSEGV now, so if anyone want to check that please
be my guest.
Reviewed by: mjg, cem
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D5112

Increase the timeout for resolv_test from the default (300 seconds) to
450 seconds

This is required on slower network connections, and on older releases
(stable/10 seems to be slower as far as name resolution goes.. not sure
why yet).

Remove an outdated comment in the Makefile from when I was working on
this code over a year ago on github

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Let tsearch()/tdelete() use an AVL tree.

The existing implementations of POSIX tsearch() and tdelete() don't
attempt to perform any balancing at all. Testing reveals that inserting
100k nodes into a tree sequentially takes approximately one minute on my
system.

Though most other BSDs also don't use any balanced tree internally, C
libraries like glibc and musl do provide better implementations. glibc
uses a red-black tree and musl uses an AVL tree.

Red-black trees have the advantage over AVL trees that they only require
O(1) rotations after insertion and deletion, but have the disadvantage
that the tree has a maximum depth of 2*log2(n) instead of 1.44*log2(n).
My take is that it's better to focus on having a lower maximum depth,
for the reason that in the case of tsearch() the invocation of the
comparator likely dominates the running time.

This change replaces the tsearch() and tdelete() functions by versions
that create an AVL tree. Compared to musl's implementation, this version
is different in two different ways:

- We don't keep track of heights; just balances. This is sufficient.
This has the advantage that it reduces the number of nodes that are
being accessed. Storing heights requires us to also access all of the
siblings along the path.

- Don't use any recursion at all. We know that the tree cannot 2^64
elements in size, so the height of the tree can never be larger than
96. Use a 128-bit bitmask to keep track of the path that is computed.
This allows us to iterate over the same path twice, meaning we can
apply rotations from top to bottom.

Inserting 100k nodes into a tree now only takes 0.015 seconds. Insertion
seems to be twice as fast as glibc, whereas deletion has about the same
performance. Unlike glibc, it uses a fixed amount of memory.

I also experimented with both recursive and iterative bottom-up
implementations of the same algorithm. This iterative top-down version
performs similar to the recursive bottom-up version in terms of speed
and code size.

For some reason, the iterative bottom-up algorithm was actually 30%
faster for deletion, but has a quadratic memory complexity to keep track
of all the parent pointers.

Reviewed by: jilles
Obtained from: https://github.com/NuxiNL/cloudlibc
Differential Revision: https://reviews.freebsd.org/D4412

Iterate down lib/libc/tests/nss...

MFC after: 1 week
X-MFC with: r292323
Sponsored by: EMC / Isilon Storage Division

Integrate tools/regression/lib/libc/nss into the FreeBSD test suite as
lib/libc/tests/nss

- Convert the testcases to ATF
- Do some style(9) cleanups:
-- Sort headers
-- Apply indentation fixes
-- Remove superfluous parentheses
- Explicitly print out debug printfs for use with `kyua {debug,report}`; for
items that were overly noisy, they've been put behind #ifdef DEBUG
conditionals
- Fix some format strings

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Add Makefile accidentally missed in r292317

MFC after: 1 week
X-MFC with: r292317
Sponsored by: EMC / Isilon Storage Division

Integrate tools/regression/lib/libc/resolv into the FreeBSD test suite as
lib/libc/tests/resolv

Convert the testcases to ATF

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Delete bogus freeing of uninitialized data

MFC after: 3 days
Reported by: cppcheck
Sponsored by: EMC / Isilon Storage Division

Add missing va_ends for corresponding va_starts to clean up variable arguments
initialized in _test_fmt(..)

MFC after: 3 days
Reported by: cppcheck
Sponsored by: EMC / Isilon Storage Division

Fix regression in r291738: This really wants -lssp.

The normal LIBADD is ssp_nonshared. This also had a DPADD on LIBSSP which
does not actually exist, it is blank.

Sponsored by: EMC / Isilon Storage Division

Initialize errno to 0 in the nul testcase before testing it

For some odd reason stable/10 requires this, otherwise it always fails
the errno == 0 check on line 196.

Sponsored by: EMC / Isilon Storage Division

Fix LDADD/DPADD that should be LIBADD.

Sponsored by: EMC / Isilon Storage Division

Follow-up r291330: h_testbits.h is only needed by xdr_test.

X-MFC-With: r291330
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Replace DPSRCS that work fine in SRCS.

This is so that 'make depend' is not a required build step in these
files.

DPSRCS is overall unneeded. DPSRCS already contains SRCS, so anything
which can safely be in SRCS should be. DPSRCS is mostly just a way to
generate files that should not be linked into the final PROG/LIB. For
headers and grammars it is safe for them to be in SRCS since they will
be excluded during linking and installation.

The only remaining uses of DPSRCS are for generating .c or .o files that
must be built before 'make depend' can run 'mkdep' on the SRCS c files
list. A semi-proper example is in tests/sys/kern/acct/Makefile where a
checked-in .c file has an #include on a generated .c file. The
generated .c file should not be linked into the final PROG though since
it is #include'd. The more proper way here is just to build/link it in
though without DPSRCS. Another example is in sys/modules/linux/Makefile
where a shell script runs to parse a DPSRCS .o file that should not be
linked into the module. Beyond those, the need for DPSRCS is largely
unneeded, redundant, and forces 'make depend' to be ran. Generally,
these Makefiles should avoid the need for DPSRCS and define proper
dependencies for their files as well.

An example of an improper usage and why this matters is in usr.bin/netstat.
nl_defs.h was only in DPSRCS and so was not generated during 'make all',
but only during 'make depend'. The files including it lacked proper
depenencies on it, which forced running 'make depend' to workaround that
bug. The 'make depend' target should mostly be used for incremental build
help, not to produce a working build. This specific example was broken in
the meta build until r287905 since it does not run 'make depend'.

The gnu/lib/libreadline/readline case is fine since bsd.lib.mk has 'OBJS:
SRCS:M*.h' when there is no .depend file.

Sponsored by: EMC / Isilon Storage Division
MFC after: 1 week

Use __MAKE_SHELL instead of HOST_SHELL when generating aton_ether_subr.c
(HOST_SHELL is used in NetBSD)

This fixes permission denied issues when gen_ether_subr is not executable

MFC after: 3 days
Reported by: José Pérez <fbl@aoek.com>
Suggested by: bdrewery, sjg

Do not print out errno if the call succeeded unexpectedly; this was a mistake
made in r290868

MFC after: 4 days
X-MFC with: r290563, r290868
Reported by: jilles
Sponsored by: EMC / Isilon Storage Division

Add some initial tests for SLIST and STAILQ macros

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Disable -Wformat with scanfloat_test when compiling with gcc to avoid a
"use of assignment suppression and length modifier together in scanf format"
warning on line 90 (it's intentional)

MFC after: 1 week
X-MFC with: r290537, r290856, r290860
Sponsored by: EMC / Isilon Storage Division

Fix -Wformat issues

X-MFC with: r290563
MFC after: 1 week
Reported by: gcc
Sponsored by: EMC / Isilon Storage Division

Remove unused variables to fix building world

Change WARNS to 2 across the board with all the libc testcases

This effectively "reverts" r290846

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Fix -Wmissing-braces warnings by adding braces around all the
testcase inputs

MFC after: 1 week
X-MFC with: r290572
Sponsored by: EMC / Isilon Storage Division

Fix -Wunused warnings

MFC after: 1 week
X-MFC with: r290572
Sponsored by: EMC / Isilon Storage Division

Bump WARNS to 2

MFC after: 1 week
X-MFC with: r290532
Sponsored by: EMC / Isilon Storage Division

Remove unused variables; sort by alignment where needed

MFC after: 1 week
X-MFC with: r290532
Sponsored by: EMC / Isilon Storage Division

Polish up iswctype_test

- Split up the testcases into C locale and ja_JP.eucJP testcases.
- Avoid a segfault in the event that setlocale fails, similar to r290843
- Replace `sizeof(x) / sizeof(*x)` pattern with `nitems(x)`

MFC after: 1 week
X-MFC with: r290532
Sponsored by: EMC / Isilon Storage Division

Polish up the tests a bit more after projects/collation was merged to head

Provide more meaningful diagnostic messages if LC_CTYPE can't be set properly
instead of segfaulting, because setlocale returns NULL and strcmp(NULL, b) will
always segfault

Split up the testcases so one failing (in this case en_US.ISO8859-15) won't
cause the rest of the testcases to be skipped

Remove some unused variables

MFC after: 1 week
X-MFC with: r290532
Sponsored by: EMC / Isilon Storage Division

Fix the Indian numbering system (hi_IN.ISCII-DEV) tests

Submitted by: ache
X-MFC with: r290494 (if that ever happens)
Sponsored by: EMC / Isilon Storage Division

Add missing licensing boilerplate to test-fnmatch.c

Carry over licensing author info from fnmatch_test.c (jilles@)

MFC after: 1 week
X-MFC with: r290572
Sponsored by: EMC / Isilon Storage Division

Integrate tools/regression/lib/libc/gen into the FreeBSD test suite
as lib/libc/tests/gen

The code in test-fnmatch that was used for generating:

- bin/sh/tests/builtins/case2.0
- bin/sh/tests/builtins/case3.0

has been left undisturbed. The target `make sh-tests` has been moved over
from tools/regression/lib/libc/gen/Makefile to
lib/libc/tests/gen/Makefile and made into a PHONY target

case2.0 and case3.0 test input generation isn't being done automatically.
This needs additional discussion.

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Fix some TAP -> ATF conversion errors

- Remove a leftover printf from when this was a TAP based testcase
- Catch mmap failures properly

MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division

Integrate tools/regression/lib/libc/net into the FreeBSD test suite
as lib/libc/tests/net

Also, fix eui64_aton_test:test_str(..). The test was comparing the result
of eui64_aton to a pointer of the expected result.

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Delete leftover printfs from when these were TAP tests

MFC after: 1 week
X-MFC with: r290532
Sponsored by: EMC / Isilon Storage Division

Convert print_positional_test over to ATF

Somehow missed in r290537

X-MFC with: r290537
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

printfloat_test and scanfloat_test need symbols from msun; these are automatically
provided on amd64, but not i386. Add libm to DPADD/LDADD to unbreak the i386
tinderbox

Pointyhat to: ngie
MFC after: 1 week
X-MFC with: r290538
Sponsored by: EMC / Isilon Storage Division

Integrate tools/regression/lib/libc/string into the FreeBSD test suite
as lib/libc/tests/string

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Integrate tools/regression/lib/libc/stdlib into the FreeBSD test suite
as lib/libc/tests/stdlib

- Make the code a bit more style(9) compliant
- Convert a sizeof(x)/sizeof(x[0]) to nitems

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Integrate tools/regression/lib/libc/stdio into the FreeBSD test suite
as lib/libc/tests/stdio

- Fix some whitespace
- Convert the testcases to ATF
- Convert "/dev/null" to _PATH_DEVNULL

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Integrate tools/regression/lib/libc/locale into the FreeBSD test suite
as lib/libc/tests/locale

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Add _test suffix to multiple tests in lib/libc to conform to the design noted
in the FreeBSD Test Suite wiki

MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division

Remove unused variable (SRCDIR)

MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division

Not all targets support by clang have a tested or enabled ubsan yet.

Only enable h_raw on x86 targets for today so that a buildworld runs to
completion for clang enabled targets that are not x86. This should be
removed when validation of the sanitizer has occured for all targets
supported by FreeBSD and clang.

Disable h_raw/h_read with gcc

I forgot that these testcases fail with gcc 4.2.1; add a note to that effect

MFC after: never
Sponsored by: EMC / Isilon Storage Division

- Re-enable h_raw with clang 3.7.0+
- Fix the compiler check to allow the test to be compiled for gcc

PR: 196430
MFC after: never
Sponsored by: EMC / Isilon Storage Division

Integrate contrib/netbsd-tests/lib/libc/rpc into the FreeBSD test suite
as lib/libc/rpc

This testcase requires rpcbind be up in running; otherwise the testcases
will time out and be skipped

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Refactor the test/ Makefiles after recent changes to bsd.test.mk (r289158) and
netbsd-tests.test.mk (r289151)

- Eliminate explicit OBJTOP/SRCTOP setting
- Convert all ad hoc NetBSD test integration over to netbsd-tests.test.mk
- Remove unnecessary TESTSDIR setting
- Use SRCTOP where possible for clarity

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Divison

In this context fclose() can never fail, so assert it in the test
case.

Add missing CLEANFILES.

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Enable mincore_test on arm64, we now have a working pmap_mincore.

PR: 202307
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

Fix t_spawnattr test for attributes handling by posix_spawn(3).
Connect it to the build.

The code assumed that SCHED_* constants form a contiguous set of
numbers, remove the assumption by using schedulers[] array in
get_different_scheduler(). This is no-op on FreeBSD, but improves
code portability.

The selection of different priority used the min/max priority range of
the current scheduler class, instead of the priority to be changed to.
The bug caused the test failure.

Remove duplication of POSIX_SPAWN_SETSIGDEF flag and now unused
duplications of MIN/MAX definitions.

Reviewed by: jilles, pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D3533

On arm64 disable three tests that hang or panic

Each issue has a PR open to track. This workaround allows us to run the
tests to investigate the failures and avoid any new regressions.

PR: 202304, 202305, 202307
Reviewed by: ngie
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3378

Fix and re-enable UTF-8 tests.

Actually disable the invalid test

Add support for makecontext. This supports up to 8 arguments as this
simplifies the code, these can be passed in registers.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

Disable the tests that use makecontext on arm64, it still needs to be
written.

Assuming a system has /bin/csh on it is a bad idea (especially it being
optional on FreeBSD). Look for /bin/cat instead

MFC after: 3 days

Enable utimensat tests from NetBSD.

As with other tests from c063, a required #include <sys/stat.h> was missing.

Link lib/libc/c063 tests to the build.

Some files lack required #include <sys/stat.h>. The #ifdef is per ngie's
request; the includes are clearly necessary for struct stat.

The faccessat test fails because it tries to use AT_SYMLINK_NOFOLLOW with
faccessat(), which is not specified by POSIX.1-2008.

Differential Revision: https://reviews.freebsd.org/D1411
Reviewed by: ngie

Don't install h_raw if dealing with clang 3.5.0+ to unbreak the tests2 Jenkins
job

The h_raw application doesn't do proper bounds checking without the option
being supplied via the build, which means that it doesn't throw signals and
fail as expected

PR: 196430
X-MFC with: r276479

Build/install lib/libc/tests/hash/t_sha2 if MK_OPENSSL == yes

Reported by: Beeblebrox <zaphod@berentweb.com>

For now, disable using -fsanitize=bounds for the libc ssp tests, when
using clang 3.5.0, until the runtime support (via compiler-rt) is added.
Otherwise, this would lead to link errors about missing support
libraries.

Convert tools/regression/lib/libc/stdio/test-fpclassify into an ATF testcase and
Rename as lib/libc/stdio/fpclassify2_test

Sponsored by: EMC / Isilon Storage Division

Convert tools/regression/lib/libc/stdio/test-fmemopen into an ATF testcase and
rename as lib/libc/stdio/fmemopen2_test

Sponsored by: EMC / Isilon Storage Division

Convert tools/regression/lib/libc/gen/test-arc4random into an ATF testcase and
rename as lib/libc/gen/arc4random_test

Sponsored by: EMC / Isilon Storage Division

Add reachover Makefiles for contrib/netbsd-tests/lib/libc; this adds approximately
500 new testcases

Various TODOs have been sprinkled around the Makefiles for items that even need
to be ported (missing features), testcases have issues with building/linking, or
issues at runtime.

A variant of this code has been tested extensively on amd64 and i386
10-STABLE/11-CURRENT for several months without issue. It builds on other
architectures, but the code will remain off until I have prove it works on
virtual hardware or real hardware on other architectures

In collaboration with: pho, Casey Peel <casey.peel@isilon.com>
Sponsored by: EMC / Isilon Storage Division