MFC r319800,r319806:

r319800:

Don't explicitly get the class to PART in gctl_test_helper.c

This will allow the tool to be used with arbitrary geom(4) classes, like GEOM.

Specify class=PART explicitly in the tester to keep existing behavior.

r319806:

Improve handling with system state

- Always unlink $cmd after exit via END block.
- The tests don't function well if kern.geom.debugflags != 0. Save debugflags,
then restore them at the end of the test.

MFC r319015:

Use calloc instead of malloc + memset

MFC r317313:

gctl_test.t: catch errors with the mdcfg directive

While here, add a note about certain testcases relying on `count=1024` in
the "create" portion.

MFC r317290,r317291,r317292,r317293,r317294,r317295,r317304,r317306,r317307,r317308,r317309:

r317290:

Fix -Wimplicit-function-declaration compilation warning by moving libgeom.h
#include below the stdio.h #include.

gctl_dump(3) needs stdio.h, per reasoning noted in r317289.

PR: 218809

r317291:

Rename gctl.t to gctl_test.t and test.c to gctl_test_helper.c

This is being done to reduce ambiguity and to make the tests more portable
in the future to other locations in the source tree.

r317292:

gctl_test.t: use make to compile gctl_test_helper instead of calling cc directly

r317293:

gctl_test_helper: apply polish

- Staticize variables to fix warnings.
- Sprinkle asserts around for calls that can fail
- Apply style(9) for main(..) definition.
- ANSIify usage(..) definition.

r317294:

Bump WARNS to 6 per previous commits which fixed warnings

Tested with: clang (4.0), gcc (4.2.1, 6.3.0)

r317295:

The GPT class no longer exists; use the PART class instead

r317304:

gctl_test_helper: add diagnostic output for parse_retval(..)

This will help end-users better diagnose issues with the function.

r317306:

gctl_test.t: minor tweaks

- Declare $count with the `my` scope operator to permit `use strict`.
- Add `use strict`.
- Use `use warnings` instead of using `-w` in the shebang.
- Don't unlink $cmd when done (prevents unnecessary rebuilding).
- Improve the error message when running with insufficient permissions, e.g.,
non-root.

r317307:

Use verb=delete not verb=remove

The `remove` verb hasn't been present in geom_part*(4) for well
over a decade, if ever. I couldn't find any references to it in
^/stable/5 at least, which is around the timeframe that this test
was written.

r317308:

gctl_test.t: more tweaks to try and update the code and get it functional (again?)

- Make the logfile for $out be built off the basename for $cmd, instead of $cmd.
(r317292 broke this assumption).
- Rename $mntpt to $mntpt_prefix for clarity, as this variable is a prefix for
mountpoints.
- Reindent the umount directive block while here to match the rest of the code.

r317309:

gctl_test.t: improve error reporting with mdcfg and mount directives

If the commands had failed previously, it would press on and result in a
series of cascading failures. Fail early and continue on to the next case
instead of executing additional commands after a previously failed series
of steps.

MFC r298881, 298882, 298883, 298885:

Minor spelling fixes in:
tools, share, bluetooth, pmcstat, etc

Many of these have user-visible strings.

MFC r295385: semget(): Check for [EEXIST] error first.

Although POSIX literally permits failing with [EINVAL] if IPC_CREAT and
IPC_EXCL were both passed, the semaphore set already exists and has fewer
semaphores than nsems, this does not allow an application to retry safely:
if the [EINVAL] is actually because of the semmsl limit, an infinite loop
would result.

PR: 206927

MFC r293821:

Integrate
tools/regression/geom_{concat,eli,gate,mirror,nop,raid3,shsec,stripe,uzip}
in to the FreeBSD test suite as
tests/sys/geom/class/{concat,eli,gate,mirror,nop,raid3,shsec,stripe,uzip}

The tools/regression/geom and tools/regression/geom_part testcases are being
left alone because both test sets are both currently broken.

The majority of this work was done on ^/user/ngie/more-tests2 . The differences
are as follows:
- tests/sys/geom/class/Makefile.inc is not present; it was
inlined into the class's Makefiles for explicitness.
- The testcases officially require root via kyua
- The geom_gate(4) tests don't use the pidfile changes proposed in
https://reviews.freebsd.org/D4836 .

MFC r293621,r293622,r293815:

r293621:

- Delete non-TAP testcases
- Add a conf.sh file for executing common functions with geom_gate
- Use attach_md for attaching md(4) devices
- Don't hardcode /tmp for temporary files, which violates the kyua sandbox
- Add/increase sleeps to try and improve synchronization
- Add debug output for when checksums fail

test-1.t:
- Use pkill for killing ggated

r293622:

Remove Makefile now that the testcases are all TAP based and
prove -rv can be used on them

r293815:

Add conf.sh file missed in r293621

MFC r293443:

- Make test-1.sh into a TAP testable testcase
- Delete test-2.sh as it was an incomplete testcase, and the contents were
basically a subset of test-1.sh
- Add a conf.sh file for executing common functions with geom_uzip
- Use attach_md for attaching md(4) devices
- Don't hardcode /tmp for temporary files, which violates the kyua sandbox

MFC r293442:

- Add a geom_stripe specific cleanup function and trap on that function at
exit so things are cleaned up properly
- Use attach_md for attaching md(4) devices
- Don't hardcode /tmp for temporary files, which violates the kyua sandbox

MFC r293441:

- Add a geom_shsec specific cleanup function and trap on that function at
exit so things are cleaned up properly
- Use attach_md for attaching md(4) devices
- Don't hardcode /tmp for temporary files, which violates the kyua sandbox

MFC r293438:

- Add a geom_raid3 specific cleanup function and trap on that function at
exit so things are cleaned up properly
- Use attach_md for attaching md(4) devices
- Don't hardcode /tmp for temporary files, which violates the kyua sandbox

MFC r293437:

- Add a conf.sh file for executing common functions with gnop
- Use attach_md for attaching md(4) devices
- Don't hardcode /tmp for temporary files, which violates the kyua sandbox

MFC r293436:

- Add a conf.sh file for executing common functions with geli
-- Use linear probing to find the first unique md(4) device, unlike the other
code which uses attach_md, as geli(8) allocates the md(4) devices itself
- Don't hardcode /tmp for temporary files, which violates the kyua sandbox

MFC r293434:

- Use attach_md for memory disks so they can be tracked.
- Add a geom_concat specific cleanup function and trap on that function at
exit so things are cleaned up properly
- Don't hardcode /tmp for temporary files, which violates the kyua sandbox

MFC r294565: sem: Don't free nameinfo that is still in list when open()
fails.

This bug could be reproduced easily by calling sem_open() with O_CREAT |
O_EXCL on a semaphore that is already open in the process. The struct
sem_nameinfo would be freed while still in sem_list and later calls to
sem_open() or sem_close() could access freed memory.

PR: 206396

MFC r293073:

- Use attach_md instead of hardcoding md(4) provider unit numbers
- Implement a gmirror_test_cleanup function, which in turn calls
geom_test_cleanup to clean up all md(4) providers allocated in the test
run.
- Remove duplicate logic in test scripts for removing md(4) providers.
- Don't create files in /tmp (outside the kyua sandbox); use the current
directory instead

MFC r294125:

Test for EPROTOTYPE not EPROTONOSUPPORT

- `SOCK_RAW` is the implied supported type parameter for socket(2) per route(4)
- localsw in `sys/kern/uipc_usrreq.c` doesn't have an entry for `SOCK_RAW`, so
the prototype is invalid (this isn't explicitly documented anywhere I could
find)

MFC r293048,r293444:

r293048:

- Use a temporary file for the temporary md(4) devices instead of
hardcoding it
- Remove the temporary file in the cleanup routine

r293444:

- Move functions that might be used in class-specific cleanup functions
(geom_test_cleanup, etc) down so the testcases don't emit noise when
bailing
- Conform to the TAP protocol better when dealing with classes that can't
be loaded and with temporary files that can't be allocated for tracking
md(4) devices.

MFC r294126:

Fix warnings with gcc 5.0

reconnect.c:
- Convert the K&R prototype of main to an ANSI prototype to mute a
warning from gcc 4.2.1
- Close s_sock2 after finishing off the last test to plug a leak and
mute a warning from gcc 5.0 about a -Wunused-but-set variable

sendfile.c:
- Fix a -Wunused-but-set warning with gcc 5.0 with pagesize in main(..)

MFC r294127:

Fix warnings with clang/gcc

- Get rid of unused argc/argv variables in main
- Exit on failure with a return code of 1 instead of -1 with err/errx as a
return code of -1 is implementation dependent
- Bump WARNS to 6

MFC r294128:

Fix -Wunused warning with clang/gcc

- Get rid of unused argc/argv variables in main
- Bump WARNS to 6

MFC r294103:

- Check for accf_filter before running the tests, otherwise it will always
fail at subtest 9/11
- Use strncpy instead of strcpy with afa.af_name

MFC r294123:

Add missing newline to message about requiring root privileges

This will help ensure that scripts/parsers don't get confused when the message
is printed out

Remove tools/regression/lib/msun (follow up to r292497)

Sponsored by: EMC / Isilon Storage Division

MFC r293032:

Use randomly generated device names in testcases via mktemp -u instead of using
the hardcoded device name, "test"

MFC r293028,r293029:

r293028:

- Use geom <class> load instead of g<class> load; g<class> doesn't exist
for all geom classes, e.g. geom_uzip(4)
- These tests require root. Skip all of the tests if they're run as non-root

r293029:

Add functions for managing md(4) devices and cleaning up said md(4) devices

These will be used soon in the various test scripts that source geom_subr.sh

MFC r292914,r292957:

r292914:

Integrate tools/regression/sockets/unix_passfd into the FreeBSD test
suite as tests/sys/kern/unix_passfd_test

- Convert testcases to ATF
- Fix an alignment issues
- Mark rights_creds_payload(..) as an expected failure (see PR # 181741)

https://reviews.freebsd.org/D689

Submitted by: markj

r292957:

Rename `recvfd` and `sendfd` variables in recvfd/sendfd functions to avoid
-Wshadow issues with gcc

MFC r292328:

Integrate a number of testcases from tools/regression/lib/msun
into the FreeBSD test suite

There's no functional change with these testcases; they're purposely
being left in TAP format for the time being

Other testcases which crash on amd64/i386 as-is have not been
integrated yet (they need to be retested on a later version of
CURRENT, as I haven't used i386 in some time)

MFC r292530,r292546:

r292530:

- Use 1 for an exit code instead of -1 with err, errx, and exit
- Add unistd.h for getuid(3)
- Sort #includes

r292546:

- Convert testcase to TAP format
- Use nitems(x) instead of handrolled sizeof(x) / sizeof(*x) macro
- Do not mark count != 0 case with bsde_get_rule_count as a failure; this
generates false positives on systems with ugidfw rules set on it

MFC r292814:

- Explicitly initialize ch to 0
- Delete some spurious whitespace
- Use calloc instead of malloc in the last test to ensure that
sendspace is properly zero'ed out

Submitted by: markj

MFC r292531,r292532,r292533,r292545:

r292531:

Make test_matches.sh into a series of TAP testcases

Use temporary filesystems / memory disks instead of a hardcoded path
which doesn't exist on test systems

r292532:

Mark `subject matching jailid` testcase as an unexpected failure with
TODO to ensure that the testcase isn't marked as a failure

PR: 205481

r292533:

Skip the testcases if mac_bsdextended(4) isn't detected on the
system

r292545:

Redo the TAP integration so it works with Kyua

Kyua needs numbers in the TAP results :/, but prove doesn't

MFC r292569:

Make the mac_portacl testcases work / more robust

- A trap(1) call has been added to the test scripts to better
ensure that the tests do a better job at trying to restore the
test host state at the end of the tests (if the test was
interrupted before it would leave the system in an odd state,
potentially making the test results for subsequent runs
non-deterministic).
- Add root user checks
- Fix nc(1) usage:
-- -o is deprecated
-- Using `-w 10` will make the call timeout after 10 seconds so it
doesn't block indefinitely
- Use local variables
- Be more terse in the error messages
- Parameterize out "127.0.0.1"

Sponsored by: EMC / Isilon Storage Division

MFC r292489:

Disable test-ctrig.t testcases which fail assertions on i386

Tested on: stable/10 (amd64/i386), head (amd64/i386)
PR: 205446
X-MFC to: stable/10
Sponsored by: EMC / Isilon Storage Division

MFC r292492,r292495,r292647:

r292492:

- Use nitems instead of handrolling the macro
- Use a separate variable for tracking the testcase count instead
of hardcoding the offset for the testcases

Sponsored by: EMC / Isilon Storage Division

r292495:

Initialize j so it doesn't print out a garbage index

Use it consistently instead of i in the first loop

Sponsored by: EMC / Isilon Storage Division

r292647:

Use j instead of a hardcoded index (9) and increment it after
running the NaNs testcases

Pointyhat to: ngie
Sponsored by: EMC / Isilon Storage Division

MFC r292491,r292493,r292496:

r292491:

Disable test-ctrig.t testcases which fail assertions on i386 [*]

Also, don't compile the exp2l(3) testcases on platforms that don't support the
libcall (technically only x86 right now). This makes this test buildable on
arm*/mips*/powerpc*

Tested on: stable/10 (amd64/i386), head (amd64/i386)
PR: 205446 [*]
X-MFC to: stable/10
Sponsored by: EMC / Isilon Storage Division

r292493:

Don't run test-fma on i386

It completely fails all assertions on i386 on both stable/9 and stable/10

PR: 205448
X-MFC to: stable/10
Sponsored by: EMC / Isilon Storage Division

r292496:

Skip the testcases on i386 (all the assertions fail) [*]

Also, don't compile the ldexpl(3) testcases on platforms that don't support
the libcall (technically only x86 right now). This makes this test buildable on
arm*/mips*/powerpc*

PR: 205449 [*]
Tested on: stable/10 (amd64/i386), head (amd64/i386)
Sponsored by: EMC / Isilon Storage Division

MFC r292493:

Don't run test-fma on i386

It completely fails all assertions on i386 on both stable/9 and stable/10

PR: 205448
X-MFC to: stable/10
Sponsored by: EMC / Isilon Storage Division

MFC r292316:

Remove hosts that don't resolve properly with the nss and resolv tests

Sponsored by: EMC / Isilon Storage Division

MFC r292327:

Use fabsl instead of fabs to mute -Wabsolute-value warnings from clang
because `nums[]` is an array of long doubles

Sponsored by: EMC / Isilon Storage Division

MFC r291982:

Skip the MAC portacl tests if MAC_PORTACL support is missing instead of
marking them failed

Sponsored by: EMC / Isilon Storage Division

MFC r291983,r291984:

r291983:

Fix compilation warnings by adding unistd.h #include and missing return
statements

Sponsored by: EMC / Isilon Storage Division

r291984:

Add missing stdlib.h header

Apply some minor style(9) fixes

Sponsored by: EMC / Isilon Storage Division

Remove stale tools/regression directory that should have been deleted
in r291840

Sponsored by: EMC / Isilon Storage Division

MFC r291359,r291362:

r291359:

Skip over lines that start with # (comments)

r291362:

r291359 was incorrect. Skip over tokens that start with `#' as fgetln can
return more than one '\n' delimited line in a buffer

Handle empty lines too, just in case

MFC r291363:

Clean up trailing whitespace

MFC r291358:

Remove cnftp.bjpu.edu.cn (it no longer resolves)
Add localhost to resolv/mach for parity with nss/mach

MFC r264737:

Discussed with: jilles

r264737 (by jilles):

libc/stdio: Fail fdopen() on an execute-only fd.

An execute-only fd (opened with O_EXEC) allows neither read() nor write()
and is therefore incompatible with all stdio modes. Therefore, the [EINVAL]
error applies.

Also adjust the similar check in freopen() with a NULL path, even though
this checks an fd which is already from a FILE.

MFC r290914:

Integrate tools/regression/pipe in to the FreeBSD test suite as
tests/sys/kern/pipe

- Fix style(9) bugs
- Fix compiler warnings
- Use `nitems(x)` instead of `sizeof(x) / sizeof(*x)` pattern

The testcases will be converted over to ATF eventually, but for now will be
integrated in as plain C tests

Sponsored by: EMC / Isilon Storage Division

MFC r290532,r290561,r290843,r290844,r290845:

r290532:

Integrate tools/regression/lib/libc/locale into the FreeBSD test suite
as lib/libc/tests/locale

Sponsored by: EMC / Isilon Storage Division

r290561:

Delete leftover printfs from when these were TAP tests

Sponsored by: EMC / Isilon Storage Division

r290843:

Polish up the tests a bit more after projects/collation was merged to head

Provide more meaningful diagnostic messages if LC_CTYPE can't be set properly
instead of segfaulting, because setlocale returns NULL and strcmp(NULL, b) will
always segfault

Split up the testcases so one failing (in this case en_US.ISO8859-15) won't
cause the rest of the testcases to be skipped

Remove some unused variables

Sponsored by: EMC / Isilon Storage Division

r290844:

Polish up iswctype_test

- Split up the testcases into C locale and ja_JP.eucJP testcases.
- Avoid a segfault in the event that setlocale fails, similar to r290843
- Replace `sizeof(x) / sizeof(*x)` pattern with `nitems(x)`

Sponsored by: EMC / Isilon Storage Division

r290845:

Remove unused variables; sort by alignment where needed

Sponsored by: EMC / Isilon Storage Division

MFC r290563,r290868,r291038:

r290563:

Integrate tools/regression/lib/libc/net into the FreeBSD test suite
as lib/libc/tests/net

Also, fix eui64_aton_test:test_str(..). The test was comparing the result
of eui64_aton to a pointer of the expected result.

Sponsored by: EMC / Isilon Storage Division

r290868:

Fix -Wformat issues

Reported by: gcc
Sponsored by: EMC / Isilon Storage Division

r291038:

Do not print out errno if the call succeeded unexpectedly; this was a mistake
made in r290868

Reported by: jilles
Sponsored by: EMC / Isilon Storage Division

MFC r290905,r290922:

r290905:

Integrate acct(2) testcase in as tests/sys/kern/acct/acct_test

The :encode_tv_random_million testcase fails the epsilon tests a few thousand
times out of one million, so expect the testcase to fail

Submitted by: keramida
Sponsored by: EMC / Isilon Storage Division

r290922:

Remove directory made obsolete by r290905

X-MFC note: stable/10 only
Sponsored by: EMC / Isilon Storage Division

Remove tools/regression/acltools

Should have been done with r290894

Sponsored by: EMC / Isilon Storage Division

MFC r289195:

Integrate the tests from lib/libarchive, usr.bin/cpio, and usr.bin/tar in to
the FreeBSD test suite

functional_test.sh was ported from bin/sh/tests/functional_test.sh, as a
small wrapper around libarchive_test, bsdcpio_test, and bsdtar_test provided
by upstream.

A handful of testcases in lib/libarchive/tests have been disabled as they
were failing when run with kyua test (see BROKEN_TESTS in
lib/libarchive/tests/Makefile)

As a sidenote: this removes the check/test targets from the Makefiles as they
don't match the pattern used in the rest of the FreeBSD test suite.

Sponsored by: EMC / Isilon Storage Division

Conflicts:
lib/libarchive/test
usr.bin/cpio/test

MFC r290538:

Integrate tools/regression/lib/libc/stdlib into the FreeBSD test suite
as lib/libc/tests/stdlib

- Make the code a bit more style(9) compliant
- Convert a sizeof(x)/sizeof(x[0]) to nitems

Sponsored by: EMC / Isilon Storage Division

MFC r290539:

Integrate tools/regression/lib/libc/string into the FreeBSD test suite
as lib/libc/tests/string

Sponsored by: EMC / Isilon Storage Division

MFC r289441:

Integrate tools/test/posixshm and tools/regression/posixshm into the FreeBSD
test suite as tests/sys/posixshm

Some other highlights:
- Convert the testcases over to ATF
- Don't use hardcoded paths to /tmp (which violate the ATF/kyua samdbox); use
mkstemp to generate temporary paths for non-SHM_ANON shm objects.

Sponsored by: EMC / Isilon Storage Division

MFC r290190,r290251:

r290190:

Fix compiler warnings with open_to_operation.c

Other sidenotes:
- Remove unused variables with main(..)
- Convert errx/exit with -1 to errx/exit with 1
- Fix a bogus test in try_directory_open
(expected_errno == expected_errno -> errno == expected_errno) [*]
- Fix some warnings related to discarded qualifiers
- Remove a bogus else-statement at the end of check_mmap_exec(..) in the
successful case. mmap(2), POSIX, Linux, etc all don't state what the
behavior is when mixing O_WRONLY + PROT_EXEC, so assume success for now to
get the test program to pass again.

PR: 201286 [*]
Submitted by: David Binderman <dcb314@hotmail.com>
Sponsored by: EMC / Isilon Storage Division

r290251:

Use nitems(x) instead of sizeof(x)/sizeof(x[0])

Sponsored by: EMC / Isilon Storage Division

MFC r289300:

Integrate tools/regression/vfs into the FreeBSD test suite as tests/sys/vfs

Sponsored by: EMC / Isilon Storage Division

MFC r264965:
r264965 (by des):

Note that the bug was fixed, and when.

MFC r289332:

Fix test-fenv:test_dfl_env when run on some amd64 CPUs

Compare the fields that the AMD [1] and Intel [2] specs say will be
set once fnstenv returns.

Not all amd64 capable processors zero out the env.__x87.__other field
(example: AMD Opteron 6308). The AMD64/x64 specs aren't explicit on what the
env.__x87.__other field will contain after fnstenv is executed, so the values
in env.__x87.__other could be filled with arbitrary data depending on how the
CPU-specific implementation of fnstenv.

1. http://support.amd.com/TechDocs/26569_APM_v5.pdf
2. http://www.intel.com/Assets/en_US/PDF/manual/253666.pdf

Discussed with: kib, Anton Rang <anton.rang@isilon.com>
Reviewed by: Daniel O'Connor <darius@dons.net.au> (earlier patch; pre-generalization)
Sponsored by: EMC / Isilon Storage Division
Reported by: Bill Morchin <wmorchin@isilon.com>

MFC r282072,r283018:

r282072:

- Fix compilation (MAP_INHERIT's dead)
- Fix warnings
- Use mkstemp instead of tmpnam

r283018:

Fix more warnings related to missing headers

MFC: r285543

Add a missing break statement, which made the code default to IPv6.

PR: 201285
Submitted by: David Binderman
Approved by: re (gjb)

MFC 281887:
Reassign copyright statements on several files from Advanced
Computing Technologies LLC to Hudson River Trading LLC.

MFC r283357:

Remove directory for test that has been integrated in under tests/sys/...

MFC r281593,r282071,r282074,r282133,r282134,r282135,r282136,r282137,r282138:

r282071:

Integrate tools/regression/mqueue into the FreeBSD test suite as
tests/sys/mqueue

r282074:

Integrate tools/regression/aio/aiotest and tools/regression/aio/kqueue into the
FreeBSD test suite as tests/sys/aio

r282133:

Fill in the copyright boilerplate for the test program

r282134:

Add initial (unpolished) macros for interfacing with the FreeBSD test suite

This is very rough, but will be replaced/redesigned some time soon after I fix
the Jenkins breakage I introduced

r282135:

Use ATF_REQUIRE_KERNEL_MODULE instead of aio_available function

r282136:

- Use ATF_REQUIRE_KERNEL_MDOULE to require aio(4)
- Don't use /tmp as a basis for temporary files as it's outside of the ATF
sandbox
- Don't override MAX macro in sys/param.h

r282137:

Use PLAIN_REQUIRE_KERNEL_MODULE to require "mqueuefs"

r282138:

Adjust CFLAGS to find freebsd_test_suite/macros.h

MFC r280894,r280895:

r280894:

Minor cleanup before converting to ATF testcases

- Remove blank (tab-only) lines.
- Fix -Wunused warnings.
- Bump up to WARNS= 6

r280895:

- Fix -Wsign issue
- Bump up to WARNS=6

MFC r282076,r282104:

r282076:

Integrate tools/regression/mmap into the FreeBSD test suite as
tests/sys/kern/mmap_test

r282104:

Move tests/sys/kern/mmap_test to tests/sys/vm/mmap_test

As jhb noted, the actual mmap(2) implementation is under sys/vm, not
sys/kern/, so the correct logical place is tests/sys/vm/, not
tests/sys/kern/

MFC r282075:

Convert this testcase over to a TAP format testcase

MFC r282073:

- Fix style(9) a bit
-- Fix whitespace
-- Use err/errx
-- Remove superfluous braces
- Be a bit more defensive with input from the end-user
- Don't throw a floating point exception by dividing by 0 when processing a
zero-byte file

MFC r282067:

Integrate tools/regression/fifo into the FreeBSD test suite as tests/sys/fifo
and tools/regression/file into the FreeBSD test suite as tests/sys/file

MFC r282063:

Integrate tools/regression/kqueue into the FreeBSD test suite as
tests/sys/kqueue

MFC r282062:

Fix -Wformat warnings by using proper format string qualifiers for
long and unsigned[ long] types

MFC r261550,r281354,r281355,r281356,r281358,r281359,r281360,r281361,r281362,r281391,r281392,r281393,r281394,r281395,r281397,r281398,r281399,r281400,r281401,r281402,r281403,r281404,r281407,r281408,r281409,r281410,r281411:

r261550 (by glebius):

Add test case for kern/181741. Right now test fails.

PR: 181741
Sponsored by: Nginx, Inc.

r281354:

Fix warnings, fix a typo in a testcase description, bump WARNS to 3

- Remove argc/argv (-Wunused)
- Cast len in comparison to size_t (-Wsign-compare)

Sponsored by: EMC / Isilon Storage Division

r281355:

Fix -Wunused warnings, bump WARNS to 6

The testcase fails today on subtest # 9

The output is still broken if prove -rv is run and the testcase aborts
prematurely (the testcase doesn't really conform to TAP protocol properly,
except when it completes fully)

Sponsored by: EMC / Isilon Storage Division

r281356:

Fix -Wunused warnings, bump WARNS to 6

The output is still broken if prove -rv is run and the testcase aborts
prematurely with fail_assertion (the testcase doesn't really conform to TAP
protocol properly, except when it completes fully)

Sponsored by: EMC / Isilon Storage Division

r281358:

- Parameterize out the number of accept/connect attempts
- Randomize the bind port to allow 2+ consecutive calls in < 10 minutes, and
to also not fail if (for instance) there's a server already listening on port
8080
- Don't leak the listening socket / fds into the child process
- Fix warnings:
-- Remove argc/argv (-Wunused)
-- Mark sig __unused (-Wunused)
-- Mark quit static (-Wmissing-variable-declarations)

Sponsored by: EMC / Isilon Storage Division

r281359:

Remove argc/argv (-Wunused)

Sponsored by: EMC / Isilon Storage Division

r281360:

Fix warnings

- Remove argc/argv (-Wunused)
- Mark some parameters to socket_listen_update __unused (-Wunused)

Sponsored by: EMC / Isilon Storage Division

r281361:

Remove argc/argv (-Wunused)

Sponsored by: EMC / Isilon Storage Division

r281362:

Use _exit, not exit in forked process

Sponsored by: EMC / Isilon Storage Division

r281391:

- Use static buffers for temporary file paths instead of strdup of constant strings
- Don't use /tmp because it's outside ATF's prescribed sandbox
- Use mkstemp instead of mktemp to eliminate warning

Sponsored by: EMC / Isilon Storage Division

r281392:

- Garbage collect argc/argv (-Wunused)
- Bump WARNS to 6

Sponsored by: EMC / Isilon Storage Division

r281393:

Fix warnings and bump WARNS to 6
- Garbage collect argc/argv (-Wunused)
- sleep(3) will always return an unsigned int; don't check for return codes <0
(-Wsign-compare)

Sponsored by: EMC / Isilon Storage Division

r281394:

- Don't use /tmp because it's outside ATF's prescribed sandbox
- Replace a hardcoded PATH_MAX value with sizeof(path)
- Use path like an array, not a pointer, and always try to unlink it in cleanup

Sponsored by: EMC / Isilon Storage Division

r281395:

Fix a -Wuninitialized warning by setting the socket to -1 and bump WARNS to 6

Sponsored by: EMC / Isilon Storage Division

r281397:

Mark signum unused in signal_handler; bump WARNS to 6

Sponsored by: EMC / Isilon Storage Division

r281398:

Garbage collect argc/argv and bump WARNS to 6

Sponsored by: EMC / Isilon Storage Division

r281399:

Fix warnings and bump WARNS to 6
- Staticize variables as needed
- Garbage collect argc/argv
- Fix -Wsign-compare warnings by casting small sizeof to (int)

Sponsored by: EMC / Isilon Storage Division

r281400:

- Garbage collect argc/argv; bump WARNS to 6
- Make the socket path random and move it out of /tmp as that's outside ATF's
prescribed path

Sponsored by: EMC / Isilon Storage Division

r281401:

- Garbage collect argc/argv
- Use random paths instead of one in /tmp

Sponsored by: EMC / Isilon Storage Division

r281402:

Garbage collect argc/argv and bump WARNS to 6

Sponsored by: EMC / Isilon Storage Division

r281403:

Garbage collect argc/argv and bump WARNS to 6

Sponsored by: EMC / Isilon Storage Division

r281404:

Generate temporary files with mkstemp instead of mktemp

Sponsored by: EMC / Isilon Storage Division

r281407:

Fix the knob twiddling to work properly per src.opts.mk

Sponsored by: EMC / Isilon Storage Division

r281408:

- Remove the .t wrapper and put the "magic" of determining the number of
testcases into the .c file
- Require root for now because it fails with SOCK_RAW without root privileges
- Increment the test count properly on socket create failure

Sponsored by: EMC / Isilon Storage Division

r281409:

Fix warnings, bump WARNS to 6, and use a temporary socket instead of one in /tmp

Sponsored by: EMC / Isilon Storage Division

r281410:

Fix more warnings I didn't catch in the first go-around

Sponsored by: EMC / Isilon Storage Division

r281411:

Fix even more warnings..

Sponsored by: EMC / Isilon Storage Division

MFC r280893:

Cleanup and do minor refactoring before converting testcases to ATF

- Convert errx(-1, ..) to errx(1, ..)
- Move the aio(4) checks to a single function (aio_available); use modfind(2)
instead of depending on SIGSYS (doesn't work when aio(4) support is missing,
not documented in the aio syscall manpages).
- Use aio_available liberally in the testcase functions
- Use mkstemp(3) + unlink(2) instead of mktemp(3)
- Fix some -Wunused warnings
- Bump WARNS to 6

Submitted by: mjohnston [*]
Sponsored by: EMC / Isilon Storage Division

MFC r280830: wordexp(): Add testcase for non-default IFS in environment.

The non-default IFS is expected to be used.

MFC r280232:
Cosmetics.

Merge an applicable subset of r263234 from HEAD to stable/10:

Update most userspace consumers of capability.h to use capsicum.h instead.

auditdistd is not updated as I will make the change upstream and then do a
vendor import sometime in the next week or two.

Note that a significant fraction does not apply, as FreeBSD 10 doesn't
contain a Capsicumised ping, casperd, libcasper, etc. When these features
are merged, the capsicum.h change will need to be merged with them.

Sponsored by: Google, Inc.

MFC r279779: env: Fix crash when -S string is not empty but no operand
follows.

split_spaces() set argc in main() incorrectly, which caused trouble for
getopt().

Examples:
env -S '\c'
env -S -i

PR: 197769

MFC r279722: env: Fix testsuite for additional variables set by sh.

MFC r277527:

r277527 (by ngie):

Make this compile with WARNS=6 and clang/gcc

Sponsored by: EMC / Isilon Storage Division

MFC r277453:

r277453 (by ngie):

Garbage collect a prove test wrapper

Sponsored by: EMC / Isilon Storage Division

MFC r267004,r274592:

r267004 (by gahr):

- Use strlen instead of hardcoding a number
- Terminate a sentence with a period

Approved by: cognet

r274592 (by ngie):

Convert tools/regression/lib/libc/stdio/test-fmemopen into an ATF testcase and
rename as lib/libc/stdio/fmemopen2_test

Sponsored by: EMC / Isilon Storage Division

MFC r274075,r274581,r274582,r274595:

r274075 (by ngie):

Add reachover Makefiles for contrib/netbsd-tests/lib/libc; this adds approximately
500 new testcases

Various TODOs have been sprinkled around the Makefiles for items that even need
to be ported (missing features), testcases have issues with building/linking, or
issues at runtime.

A variant of this code has been tested extensively on amd64 and i386
10-STABLE/11-CURRENT for several months without issue. It builds on other
architectures, but the code will remain off until I have prove it works on
virtual hardware or real hardware on other architectures

In collaboration with: pho, Casey Peel <casey.peel@isilon.com>
Sponsored by: EMC / Isilon Storage Division

r274581 (by ngie):

Convert tools/regression/lib/libc/gen/test-arc4random into an ATF testcase and
rename as lib/libc/gen/arc4random_test

Sponsored by: EMC / Isilon Storage Division

r274582 (by ngie):

Remove test-arc4random from this Makefile so others can continue to use
this as-is for the time being

r274595 (by ngie):

Convert tools/regression/lib/libc/stdio/test-fpclassify into an ATF testcase and
Rename as lib/libc/stdio/fpclassify2_test

Sponsored by: EMC / Isilon Storage Division

MFC discussed with: jilles, -developers

MFC r266971:

- Return NULL and set errno to EINVAL if size is 0 (as required by POSIX).
Update the manpage to reflect this change.
- Always set the current position to the first null-byte when opening in append
mode. This makes the implementation compatible with glibc's. Update the test
suite.

Reported by: pho
Approved by: cognet

MFC r264400,r265836:

r264400:

NO_MAN= has been deprecated in favor of MAN= for some time, go ahead
and finish the job. ncurses is now the only Makefile in the tree that
uses it since it wasn't a simple mechanical change, and will be
addressed in a future commit.

r265836:

Remove last two NO_MAN= in the tree. In both of these cases, MAN= is
what is needed.

MFC r271796 (by will):

Use 'cc' for the C compiler instead of 'gcc'.

Sponsored by: Spectra Logic
MFSpectraBSD: 1079507 on 2014/07/24

MFC r273627,r273628:

r273627:

- Print out "Bail out!" in die(..) so prove terminates immediately
- Handle the output from newer versions of openssl md5, similar to what
pjd@ did in r248304

Sponsored by: EMC / Isilon Storage Division

r273628:

Move the redirection to stderr out of the cmd variable assignment

Putting 2>/dev/null in cmd= escapes the redirection operation, which causes
mdconfig to think it's a filename

Sponsored by: EMC / Isilon Storage Division

MFC r271349:

Add diagnostic printfs and disable test # 4 on i386

Reviewed by: jmmv, rpaulo
Phabric: D749
PR: 191676
Sponsored by: EMC / Isilon Storage Division

MFC r271336:

Expand the tests structure in test_small(..) to workaround the
"initializer not constant" warning with gcc

Approved by: jmmv (mentor)
Phabric: D744
Sponsored by: EMC / Isilon Storage Division

MFC r271459:

Fix ACL tests to correctly work with ZFS; previous version used wrong
paths.

PR: 191545
Approved by: re (gjb)
Sponsored by: The FreeBSD Foundation

MFC r270180:

r269906:

Add missing BSD.tests.dist entry for lib/libutil to unbreak installworld with
MK_TESTS == no

Phabric: D555
Approved by: jmmv (mentor, implicit)
Pointyhat to: ngie

r269904:

Integrate lib/libutil into the build/kyua

Remove the .t wrappers

Rename all of the TAP test applications from test-<test> to
<test>_test to match the convention described in the TestSuite
wiki page

humanize_number_test.c:

- Fix -Wformat warnings with counter variables
- Fix minor style(9) issues:
-- Header sorting
-- Variable declaration alignment/sorting in main(..)
-- Fit the lines in <80 columns
- Fix an off by one index error in the testcase output [*]
- Remove unnecessary `extern char * optarg;` (this is already provided by
unistd.h)

Phabric: D555
Approved by: jmmv (mentor)
Obtained from: EMC / Isilon Storage Division [*]
Submitted by: Casey Peel <cpeel@isilon.com> [*]
Sponsored by: EMC / Isilon Storage Division

MFC r269534:

Integrate lib/libmp into the build/kyua

- Remove the .t wrapper
- Fix -Wreturn-type warnings with clang

This change has been tested on amd64/i386

Phabric: D530
Reviewed by: jmmv
Approved by: jmmv (co--mentor)
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

MFC note: src.opts.mk in the original commit was changed to
bsd.own.mk.

MFC r269098:

Use the right lengths.

Submitted by: Sascha Wildner

MFC r268385:
Make this compilable on latest Linux'es without warnings.

MFC various moves of tools/regressions/ tests to the new infrastructure.

- r263220 Migrate tools/regression/sbin/ to the new tests layout.
- r263222 Add Makefile missed in r263220.
- r263226 Migrate tools/regression/{usr.bin/lastcomm,usr.sbin}/ to the new tests layout.
- r263227 Migrate most of tools/regression/usr.bin/ to the new tests layout.
- r263345 Expand tabs that sneaked in into spaces.
- r263346 Migrate tools/regression/usr.bin/make/ to the new tests layout.
- r263348 Add Makefiles missed in r263346.
- r263351 Migrate tools/regression/usr.bin/pkill/ to the new tests layout.
- r263388 Mark multi_test as requiring /usr/share/dict/words.
- r263814 Fix path to the run.pl script to let these tests run.
- r264742 Prevent building tests when bootstrapping make.

This is 'make tinderbox' clean.

MFC various fixes to the tools/regression/ tests.

- r262953 Fix m4 tests so that they run cleanly with prove.
- r262954 Fix printf tests so that they run cleanly with prove.
- r262959 Fix sed tests so that they run cleanly with prove.
- r262960 Fix yacc tests so that they run cleanly with prove.
- r262961 Fix pkill tests so that they run cleanly with prove.
- r262962 Fix ncal tests so that they run cleanly with prove.
- r263081 Fix lastcomm tests under amd64.
- r263082 Only run the make tests when make is fmake.
- r263083 Fix sa tests.
- r263084 Turn a test precondition into a skip in the mdconfig tests.
- r263085 Make the strerror tests work without libtap.
- r263087 Remove broken tests for eui64_line.
- r263221 Change etcupdate tests to return 1 on test failures.
- r263352 Make the priv test program exit with non-zero if any failures are detected.
- r263353 errx prepends the program name to the message; don't do it by hand.
- r263362 Include strings.h so that bpf_filter.c can be built in userland.

Sync sh(1) in stable/10 to head.

This is a MFC of all the commits listed below.

My original goal of this change was to only merge the move of the tests
from tools/regression/bin/ into the new layout (which include tests for
sh(1) and other tools as well). However, doing so is tricky due to the
ongoing work in sh(1) and, especially, the many changes to its tests
since stable/10 was first branched.

Merging everything is the simplest way to achieve this goal and, as a
bonus point, we get various fixes and miscellaneous improvements into
the branch.

Per jilles' suggestion, I'm avoiding the merge of a couple of changes
(r256850 and r257506) that required depending kernel changes. I'm also
avoiding very recent changes that have not had a long enough time to be
validated in current.

This is "make tinderbox" clean.

r256735 sh: Remove one syscall when waiting for a foreground job.
r257399 sh: Allow trapping SIGINT/SIGQUIT after ignore because of '&'.
r257504 sh: Reorder union node to reduce its size on 64-bit platforms.
r257920 sh: Add a test case for would-be assignments that are not due to quoting.
r257929 sh: Properly quote alias output from command -v.
r258489 sh: Add tests for the </dev/null implicit in a background command.
r258533 sh: Add more tests for the </dev/null implicit in a background command.
r258535 sh: Make <&0 disable the </dev/null implicit in a background command.
r258776 sh: Prefer memcpy() to strcpy() in most cases. Remove the scopy macro.
r259047 sh: Split set -x output into a separate function.
r259210 Migrate tools/regression/bin/ tests to the new layout.
r259844 sh: Remove an unused variable.
r259846 sh: Initialize OPTIND=1 even if it came from the environment.
r259874 sh: Simplify code related to PPID variable.
r259946 sh: Don't check input for non-whitespace if history is disabled.
r260246 sh(1): Discourage use of -e.
r260506 Run the sh(1) and test(1) tests as unprivileged.
r260586 Mark the bin/pax tests as requiring perl.
r260634 Use TAP_TESTS_PERL to register the legacy_test in bin/pax.
r260635 Replace hand-crafted Kyuafiles with automatic generation.
r260654 sh: Remove SIGWINCH handler and just check for resize before every read.
r261121 sh: Add test for nested alias.
r261125 sh: Solve the alias recursion problem in a less hackish way.
r261141 sh: Do not depend on parse/execute split in new alias test.
r261160 sh: Add tests for alias names after another alias.
r261192 sh: Allow aliases to force alias substitution on the following word.
r262533 sh: Make expari() static.
r262565 sh: Do not corrupt internal representation if LINENO inner expansion fails.
r262697 sh: Simplify expari().

Reviewed by: jilles

MFC r261081, r261133, and r261615

r261081
Replace the old unix_seqpacket and unix_seqpacket_exercise
tests, which were a little broken and not automatable, with
unix_seqpacket_test. It's coverage is a superset of the old
tests and it uses ATF. It includes test cases for bugs
kern/185813 and kern/185812.

r261133
Fix the Makefiles so that the tests I submitted in r261081 will
actually get built.

r261615
tests/sys/Makefile
use TESTS_SUBDIRS for kern instead of SUBDIRS. I don't
think it makes a difference in this case, but
TESTS_SUBDIRS is generally correct for subdirectories
that contain tests.

MFC: r261566

Use CAP_EVENT instead of the deprecated CAP_POLL_EVENT.

PR: 185382 (based on)
Submitted by: Loganaden Velvindron
Reviewed by: pjd

MFC r258352: fsx: add an option to randomly call msync(MS_INVALIDATE)

MFC r258351: fsx: new option to disable msync(MS_SYNC) after each write

MFC 258063,258064,258066,258097,258185,259134:
- Add an -s option to the regression tests that specifies a path to an
alternate etcupdate.sh script to test. This allows a non-installed
version of the script to be tested more easily.
- Add a pre-world mode of updating similar to the -p option that can be
passed to mergemaster.
- Fix a couple of issues with -F:
- Fix ALWAYS_INSTALL to take precedence over the FreeBSD ID checks.
- Fix the -F option in the case that the only upstream change is a
change in the FreeBSD ID and the local file is removed.
- Refresh /etc/localtime after each update using tzsetup -r.
- Regenerate /var/db/services.db when /etc/services changes.

MFC r259132,r259133,r259143:
Per the resolution of POSIX bug 0000779 (note 0002050) add support for using 'i'
as a case insensitive flag.

Add regression test for recently added 'i' flag in r259132.

the i modifier was added in r259132 since POSIX recently agreed to add
it. Switch uses of /I to /i.

PR: standards/184641

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

don't assert on bad args, instead return an error..

Since so many programs don't check return value, always NUL terminate
the buf...

fix rounding when using base 1024 (the bug that started it all)...

add a set of test cases so we can make sure that things don't break
in the future...

Thanks to Clifton Royston for testing and the test program...

Approved by: re (hrs, glebius)
MFC after: 1 week

Bring in the new iSCSI target and initiator.

Reviewed by: ken (parts)
Approved by: re (delphij)
Sponsored by: FreeBSD Foundation

Add a sendfile regression test for transmit length > file size.

This test identified the issue fixed in FreeBSD-SA-13:11.sendfile.

Sponsored by: The FreeBSD Foundation
Approved by: re (glebius)

libc/stdio: Allow fopen/freopen modes in any order (except initial r/w/a).

Austin Group issue #411 requires 'e' to be accepted before and after 'x',
and encourages accepting the characters in any order, except the initial
'r', 'w' or 'a'.

Given that glibc accepts the characters after r/w/a in any order and that
diagnosing this problem may be hard, change our libc to behave that way as
well.

libc/stdio: Run mkostemp test using prove.

libc/stdio: Provide proper TAP output for fmemopen/open_[w]memstream.

A *.t file should provide Test Anything Protocol output so that it can be
run using the Perl "prove" tool.

sh: Make return return from the closest function or dot script.

Formerly, return always returned from a function if it was called from a
function, even if there was a closer dot script. This was for compatibility
with the Bourne shell which only allowed returning from functions.

Other modern shells and POSIX return from the function or the dot script,
whichever is closest.

Git 1.8.4's rebase --continue depends on the POSIX behaviour.

Reported by: Christoph Mallon, avg

sh: Recognize "--" as end of options in type builtin.

This implementation makes minimal changes: command names starting with "-"
(other than "--") can still be queried normally.

Fix fcntl F_GETFL F_SETFL for files opened execute-only (O_EXEC).

The FFLAGS and OFLAGS now work correctly also for files opened with O_EXEC.
Except possibly fuse, the other users pass values without O_EXEC set. fuse
appears to assume O_EXEC is handled correctly.

Although F_SETFL may not be commonly used for execute-only file descriptors,
F_GETFL may be useful to find the access mode.

sh: Recognize "--" as end of options in alias builtin.

Aliases starting with "-" (which are non-POSIX) will need to be preceded by
an alias not starting with "-" or the newly added "--".

sh: Disallow empty simple commands.

As per POSIX, a simple command must have at least one redirection,
assignment word or command word.

These occured in rare cases such as eval "f()" .

The extension of allowing no commands inside { }, if, while, for, etc.
remains.

sh: Reject ++ and -- in arithmetic.

POSIX does not require ++ and -- in arithmetic. It is probably more useful
to reject them than to treat ++x and --x as x silently.

Note that the behaviour of increment and decrement can be obtained via
(x+=1), ((x+=1)-1), (x-=1) and ((x-=1)+1).

PR: bin/176444

sh: Recognize "--" as end of options in bg/fg/jobid builtins.

sh: Add test for the non-standard jobid builtin.

Add tests for dup3().

sh: Recognize "--" as end of options in local builtin.

sh: Allow a lone redirection before '|', ';;' or ';&'.

Example: </dev/null | :

PR: 181240
MFC after: 1 week

fnmatch(): Add test for r254091 (pattern with single backslash).

This test cannot be converted to an sh(1) test because the syntax would be
invalid.

PR: 181129
MFC after: 1 week

Add mkostemp() and mkostemps().

These are like mkstemp() and mkstemps() but allow passing open(2) flags like
O_CLOEXEC.

wordexp(): Fix syntax validation for backslashes in single-quotes.

Fix several warnings.
Fix crash in aio_pty_cleanup() by initializing the pointer before
dereferencing.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Include stdint.h when use intmax_t.

Reported by: bde

Replace PRId64 with "jd" in a printf call. Cast the corresponding value to
intmax_t, because the original type is off_t.

Reported by: bde

Add missing headers.

Use PRId64 instead of %gd to print an int64_t.

Need to define NO_MAN here.

Use MAN= instead of NO_MAN=

For some reason it still tries to install a priv.1 when using NO_MAN,
even though there isn't one yet.

Add missing header needed by sleep(3).

sh: Do not read from stdin if an error occurs during -i -c cmd.

Although using -i with -c does not seem very useful, it seems inappropriate
to read commands from the terminal in this case.

Side effect: if the -s -c extension is used and the -s option is turned off
using 'set +s' during the interactive part, the shell now exits after an
error or interrupt. Note that POSIX only specifies -s as option to sh, not
to set.

See also Austin Group issue #718.

Restore "all rights reserved" (spelled correctly). This was actually part of the standard text of the license which I did not realize prior.

Approved by: bushman

Remove lines declaring "All rights reserved" or similar comments: they
are not true as the files are actually under the BSD-2 license

Approved by: bushman

sh: Add tests for 'local -' (save shell options).

Fix some bugs in the complex trig tests so that they test both double
and float precision properly.

Add testing utility for behavior of atomic ops.

This small utility performs a sequence of atomic operations with random
parameters on an atomic variable. For every type, we also create 16
variables, to ensure that we test the correctness at different
alignments.

sh: Return status 127 for unknown jobs in wait builtin.

This is required by POSIX, at least for pids that are not known child
processes.

Other problems with job specifications still cause wait to abort with
exit status 2.

PR: 176916

sh: Allow multiple operands in wait builtin.

This is only part of the PR; the behaviour for unknown/invalid pids/jobs
remains unchanged (aborts the builtin with status 2).

PR: 176916
Submitted by: Vadim Goncharov

Match the options of the kernel.

Add libiconv based versions of *c16*() and *c32*().

I initially thought wchar_t was locale independent, but this seems to be
only the case on Linux. This means that we cannot depend on the *wc*()
routines to implement *c16*() and *c32*(). Instead, use the Citrus
libiconv that is part of libc.

I'll see if there is anything I can do to make the existing functions
somewhat useful in case the system is built without libiconv in the
nearby future. If not, I'll simply remove the broken implementations.

Reviewed by: jilles, gabor

Add more tests for log functions. A few are commented out because the
long double versions don't pass yet. (They are rather nit-picky cases,
so there's ongoing discussion with Bruce about whether it is worth the
performance cost.)

Factor out some common code from the libm tests. This is a bit messy
because different tests have different ideas about what it means to be
"close enough" to the right answer, depending on the properties of the
function being tested. In the process, I fixed some warnings and
added a few more 'volatile' hacks, which are sufficient to make all
the tests pass at -O2 with clang.

test: Remove -ntXY and -otXY primaries.

This reverts commit r247274.

As maintainer of sh, I disapprove of this feature addition.

It is too specific and can be done without easily using find(1) or stat(1).
I will add some hints to the test(1) man page shortly.

In general, FreeBSD sh is not the place to invent new shell language
features. This is how it has been maintained and adding features randomly
does not work with that.

The new syntax (e.g. [ FILE1 -ntca FILE2 ]) looks cryptic to me.

sh: Add test cases for break outside a loop.

In most shells (including our sh), break outside a loop does nothing with
status 0, or at least does not abort. Therefore, scripts sometimes (buggily)
depend on this.

Different approach to making all compilers happy.

Allow building with clang (which is being really stupid here...).

Fix some harmless bugs in a test.

Basic tests for complex inverse trig and hyperbolic functions.

Fix cexp regression tests that have an infinite real part. The signs of the
result depend on the cosine and sine of the imaginary part.
Small values are used in the new tests such that cosine and sine are well
defined.

Reviewed by: das

Fix some regressions caused by the switch from gcc to clang. The fixes
are workarounds for various symptoms of the problem described in clang
bugs 3929, 8100, 8241, 10409, and 12958.

The regression tests did their job: they failed, someone brought it
up on the mailing lists, and then the issue got ignored for 6 months.
Oops. There may still be some regressions for functions we don't have
test coverage for yet.

Add missing #includes, to keep Clang silent.

Add <uchar.h>.

The <uchar.h> header, part of C11, adds a small number of utility
functions for 16/32-bit "universal" characters, which may or may not be
UTF-16/32. As our wchar_t is already ISO 10646, simply add light-weight
wrappers around wcrtomb() and mbrtowc().

While there, also add (non-yet-standard) _l functions, similar to the
ones we already have for the other locale-dependent functions.

Reviewed by: theraven

popen(): Add 'e' mode character to set close-on-exec on the new fd.

If 'e' is used, the kernel must support the recently added pipe2() system
call.

The use of pipe2() with O_CLOEXEC also fixes race conditions between
concurrent popen() calls from different threads, even if the close-on-exec
flag on the fd of the returned FILE is later cleared (because popen() closes
all file descriptors from earlier popen() calls in the child process).
Therefore, this approach should be used in all cases when pipe2() can be
assumed present.

The old version of popen() rejects "re" and "we" but treats "r+e" like "r+".

Add a test program for popen().

Tidy up some CVS workarounds.

Add simple testcases for fcntl(F_DUP2FD_CLOEXEC).

Add simple testcases for fcntl(F_DUPFD_CLOEXEC).

Add missing argument to fcntl(F_DUPFD) in regression test.

Add a few xargs tests related to -0, -n and quoting.

sh: Remove racy test case for read builtin.

This test case sometimes fails because of an EINTR-related race condition.
Fixing this race condition likely requires an extra system call per byte,
which would make the read builtin even slower than it already is, or very
complicated trickery. Therefore, remove the test case for now.

sh: Improve error handling in read builtin:

* If read -t times out, return status as if interrupted by SIGALRM
(formerly 1).
* If a trapped signal interrupts read, return status 128+sig (formerly 1).
* If [EINTR] occurs but there is no trap, retry the read (for example
because of a SIGWINCH in interactive mode).
* If a read error occurs, write an error message and return status 2.

As before, a variable assignment error returns 2 and discards the remaining
data read.

Style cleanups.

- Correct mispellings of the word occurrence

Submitted by: Christoph Mallon <christoph.mallon@gmx.de> (via private mail)

sh: Don't modify exit status when break/continue/return passes !.

This matches what would happen if ! P were to be replaced with
if P; then false; else true; fi.

Example:
f() { ! return 0; }; f

sh: Add a variation on builtins/eval4.0 where the cmdsubst returns 0.

wordexp(): Remove wrong IFS usage.

Words in shell script are separated by spaces or tabs independent of the
value of IFS. The value of IFS is only relevant for the result of
substitutions. Therefore, there should be a space between 'wordexp' and the
words to be expanded, not an IFS character.

Paranoia might dictate that the shell ignore IFS from the environment (even
though our sh currently uses it), so do not depend on it in the new test
case.

Update regression tests after adding chflagsat(2).

Sponsored by: The FreeBSD Foundation

- Make 'flags' argument to chflags(2), fchflags(2) and lchflags(2) of type
u_long. Before this change it was of type int for syscalls, but prototypes
in sys/stat.h and documentation for chflags(2) and fchflags(2) (but not
for lchflags(2)) stated that it was u_long. Now some related functions
use u_long type for flags (strtofflags(3), fflagstostr(3)).
- Make path argument of type 'const char *' for consistency.

Discussed on: arch
Sponsored by: The FreeBSD Foundation

Update the tests now that absence of the O_APPEND flag requires CAP_SEEK
capability. Add some more tests.

Sponsored by: The FreeBSD Foundation

The mode argument for open(2)/openat(2) only makes sense if the O_CREAT flag
was given.

Sponsored by: The FreeBSD Foundation

sh: Recognize "--" and explicitly reject options in wait builtin.

If syntactically invalid job identifiers are to be taken as jobs that exited
with status 127, this should not apply to options, so that we can add
options later if need be.

Make file name generation to work with both new and old versions of OpenSSL.

Sponsored by: The FreeBSD Foundation

sh: When executing a trap, keep exit status along with evalskip.

This ensures 'return' in a trap returns the correct status to the caller.

If evalskip is not set or if it is overridden by a previous evalskip, keep
the old behaviour of restoring the exit status from before the trap.

If all ioctls are allowed, cap_ioctls_get(2) will return CAP_IOCTLS_ALL.
Update regression tests.

Add support for bindat(2) and connectat(2).

Sponsored by: The FreeBSD Foundation

Add regression tests for the new Capsicum system calls.

Sponsored by: The FreeBSD Foundation

Update existing regression tests after Capsicum overhaul.

Add an implementation of open_memstream() and open_wmemstream(). These
routines provide write-only stdio FILE objects that store their data in a
dynamically allocated buffer. They are a string builder interface somewhat
akin to a completely dynamic sbuf.

Reviewed by: bde, jilles (earlier versions)
MFC after: 1 month

Enhance test(1) by adding provision to compare any combination of the
access, birth, change and modify times of two files, instead of only
being able to compare modify times. The builtin test in sh(1) will
automagically acquire the same expansion.

Approved by: grog
MFC after: 2 weeks

sh: If a SIGINT or SIGQUIT interrupts "wait", return status 128+sig.

sh: Test that the exit status is 1 if read encounters EOF.

Major update for unix_cmsg from Andrey Simonenko.

Quoting the submitter:
- Added tests for SCM_BINTIME, LOCAL_PEERCRED, cmsghdr.cmsg_len
- Code that checks correctness of groups was corrected (getgroups(2) change)
- unix_cmsg.c was completely redesigned and simplified
- Use less timeout value in unix_cmsg.c for faster work
- Added support for not sending data in a message, not sending data and
data array associated with a cmsghdr structure in a message
- Existent tests were improved
- unix_cmsg.t was redesigned and simplified

Correctness of unix_cmsg verified on 7.1-STABLE, 9.1-STABLE and 10-CURRENT.

PR: bin/131567
Submitted by: Andrey Simonenko <simon@comsys.ntu-kpi.kiev.ua>
MFC after: 2 weeks

- Fix more style(9)-related issues (copyright header, spaces after function
names, unnecessary casts)
- Change type of boolean variable from char to bool

Suggested by: jhb, zont, jmallett
Reviewed by: cognet
Approved by: cognet

- Remove underscores from the internal structure name, as it doesn't collide
with the user's namespace.

- Correct size and position variables type from long to size_t.

- Do not set errno to ENOMEM on malloc failure, as malloc already does so.

- Implement the concept of "buffer data length", which mandates what SEEK_END
refers to and the allowed extent for a read.

- Use NULL as read-callback if the buffer is opened in write-only mode.
Conversely, use NULL as write-callback when opened in read-only mode.

- Implement the handling of the ``b'' character in the mode argument. A binary
buffer differs from a text buffer (default mode if ``b'' is omitted) in that
NULL bytes are never appended to writes and that the "buffer data length"
equals to the size of the buffer.

- Remove shall from the man page. Use indicative instead. Also, specify that
the ``b'' flag does not conform with POSIX but is supported by glibc.

- Update the regression test so that the ``b'' functionality and the "buffer
data length" concepts are tested.

- Minor style(9) corrections.

Suggested by: jilles
Reviewed by: cognet
Approved by: cognet

Add fmemopen(3), an interface to get a FILE * from a buffer in memory, along
with the respective regression test.
See http://pubs.opengroup.org/onlinepubs/9699919799/functions/fmemopen.html

Reviewed by: cognet
Approved by: cognet

sh: Pass $? to command substitution containing compound/multiple commands.

Example:
false; echo $(echo $?; :)

sh: Add testcase that $? is preserved into a simple command substitution.

The test builtins/trap6.0 already uses this but having it separate eases
diagnosis if this would break.

sh: Add some testcases related to subshells.

These failed in earlier attempts to execute more subshells without forking.
The patches are uncommitted.

sh: Fix crash when parsing '{ } &'.

MFC after: 1 week

sh: Don't lose $? when backquoted command ends with semicolon or newline.

An empty simple command was added and overwrote the exit status with 0.

This affects `...` but not $(...).

Example:
v=`false;`; echo $?

sh: Fix a Subversion property.

Test scripts should have keywords; only pure output that must match exactly
should have keywords disabled.

Add an additional regression tests for other cases to ensure these do not get fixed by accident.

POSIX requires that non-existent or null arguments be treated as if a
zero argument were supplied.

Add a regression test to catch this case as well.

PR: bin/174521
Submitted by: Daniel Shahaf <danielsh@elego.de> (pr)
Submitted by: Mark Johnston <markjdb@gmail.com> (initial patch)
Reviewed by: jilles
Approved by: cperciva (implicit)
MFC after: 3 weeks

sh: Detect and flag write errors on stdout in builtins.

If there is a write error on stdout, a message will be printed (to stderr)
and the exit status will be changed to 2 if it would have been 0 or 1.

PR: bin/158206

libc: Add a missing header to a test program.

Usage of dup(), mkstemp() and unlink() needs <unistd.h>.

Non-void function should return a value.

Found by: clang

Remove unused variable.

Use '%zu' and '%zd' as appropriate for size_t / ssize_t.

Zero the whole struct not just the size of a pointer.

Found by: clang

Use '%zd' format specifier for ssize_t

Found by: clang

Use '%zd' printf format for ssize_t.

sh: Apply rlimits to parser/alias10.0 so it fails fast.

Requested by: uqs

sh: Add tests for modifying an alias (r242766).

Note: parser/alias10.0 will eat a lot of memory/cpu time when it fails (with
the old sh).

sh: Test that a redefined alias works.

Test both active and non-active cases.

Iterate rather than use recursion. We can blow out the kernel stack if there
is a long chain of fork(2)s.

remove duplicate semicolons where possible.

Approved by: cperciva
MFC after: 1 week

Covert regression test to python 3

Approved by: cperciva
MFC after: 3 days

Revert previous commit...

Pointyhat to: kevlo (myself)

Prefer NULL over 0 for pointers

Add regression test for kern/172075.

Sponsored by: ADARA Networks
PR: kern/172075

sh: Add some tests for aliasing a utility to itself.

nftw(): POSIX says directories causing loops should be silently skipped.

Formerly, loops caused nftw() to abort the traversal with ELOOP.

sh: Add a simple test for the "local" builtin.

sh: Expand assignment-like words specially for export/readonly/local.

Examples:
export x=~
now expands the tilde
local y=$1
is now safe, even if $1 contains IFS characters or metacharacters.

For a word to "look like an assignment", it must start with a name followed
by an equals sign, none of which may be quoted.

The special treatment applies when the first word (potentially after
"command") is "export", "readonly" or "local". There may be quoting
characters but no expansions. If "local" is overridden with a function there
is no special treatment ("export" and "readonly" cannot be overridden with a
function).

If things like
local arr=(1 2 3)
are ever allowed in the future, they cannot call a "local" function. This
would either be a run-time error or it would call the builtin.

This matches Austin Group bug #351, planned for the next issue of POSIX.1.

PR: bin/166771

sh: Add tests where "export" does not parse differently.

It is planned to expand variable assignments as assignments (no word
splitting, different tilde expansion) when they follow a "declaration
utility" (export, readonly or local). However, a quoted character cannot be
part of a "name" so things like \v=~ are not assignments, and the existing
behaviour applies.

The etcupdate utility is a tool for managing updates to files that are
not updated as part of `make installworld' such as files in /etc. It
manages updates by doing a three-way merge of changes made to these files
against the local versions. It is also designed to minimize the amount
of user intervention with the goal of simplifying upgrades for clusters
of machines.

The primary difference from mergemaster is that etcupdate requires less
manual work. The primary difference from etcmerge is that etcupdate
updates files in-place similar to mergemaster rather than building a
separate /etc tree.

Requested by: obrien, kib, theraven, joeld (among others)

Remove the "funny targets" make check. We no longer need embedded :: targets
to build FreeBSD (they are used in Perl man pages). We never needed embedded
"!" in targets that I can find.

We got this from OpenBSD and I cannot find any other make that supports
such things -- contrary to their commit message claim: "This behaviour
is also consistent with other versions of make.".

Recognize 'none' or '0' as no flags.

Support also running a 32-bit test on 64-bit platforms.

Provide a mechanism to not clear out the work and output dirs when
developing tests.

Add a test for the :tl & :tu modifiers.

sh: Do not assume that SIGPIPE will only kill a subshell in builtins/wait3.0
test.

POSIX says that SIGPIPE affects a process and therefore a SIGPIPE caused and
received by a subshell environment may or may not affect the parent shell
environment.

The change assumes that ${SH} is executed in a new process. This must be the
case if it contains a slash and everyone appears to do so anyway even though
POSIX might permit otherwise.

This change makes builtins/wait3.0 work in ksh93.

Correct examples to the latest version I had.

Add a regression test for filemon(4) [r236592].

Add regression test for '-V' command line option (requires r236346).

make it easier to test newly-built make.

Import byacc from invisible island, it brings us lots of compatibilities with
bison, keeping full compatibility with our previous yacc implementation.

Also bring the ability to create reentrant parser

This fix bin/140309 [1]

PR: bin/140309 [1]
Submitted by: Philippe Pepiot <ksh@philpep.org> [1]
Approved by: des (mentor)
MFC after: 1 month

According to shared memory man pages, advice including <sys/types.h>
instead of <sys/param.h>

Import m4 regression tests from OpenBSD

Obtained from: OpenBSD
Approved by: des (mentor)

Add some tests from PR 166463. Also make sure that all of the tests,
old and new, check the sign bits of both the remainder and the
quotient.

sh: Add a test for variables with underscores in arithmetic.

Things like $((_x+1)) are broken in stable/8 sh but work in stable/9 and
head.

Add regression test for USB enumeration.

MFC after: 1 week

Fix order of arguments to lseek().

PR: 160867
Submitted by: Henning Petersen <henning.petersen@t-online.de>
MFC after: 1 week

Check fchmod()/fchown() in fifo_misc test.

libc: Add some tests for fmtmsg().

Add regression tests for the setsockopt(2) SO_SETFIB socket option.

Check that the expected domain(9) families all handle the socket option
correctly and do proper bounds checks. This would catch bugs as fixed
in (r230938,)r230981.

Sponsored by: Cisco Systems, Inc.

Include target names in diagnostic output.

Submitted by: Garrett Cooper

sh: Make 'hash' return 1 if at least one utility is not found.

Reported by: lme

More mdconfig(8) tests.

Move mdconfig(8) tests under regression/sbin/, where they belong.

Submitted by: jh@

Add some basic regression tests for mdconfig(8).

sh: Fix $? in the first command of a 'for'.

In the first command of a 'for', $? should be the exit status of the last
pipeline (command substitution in the word list or command before 'for'),
not always 0.

These tests check whether the compiler evaluates floating-point
expressions properly. Some of the tests depend on the compiler
implementing C99's FENV_ACCESS pragma, and only commercial compilers
do; those tests are currently skipped. If any of the enabled tests
fail, then odds are the libm regression tests will fail also.
This should make it easier to diagnose reported problems on platforms
I don't have.

Currently, gcc passes all the tests that don't depend on FENV_ACCESS
on amd64 and sparc64. Clang fails a few on amd64 (see clang bug
11406). Both gcc and clang fare poorly on i386, which has well-known
issues.

sh: Fix execution of multiple statements in a trap when evalskip is set

Before this fix, only the first statement of the trap was executed if
evalskip was set. This is for example the case when:
o "-e" is set for this shell
o a trap is set on EXIT
o a function returns 1 and causes the script to abort

Reviewed by: jilles
MFC after: 2 weeks

sh: Test EXIT trap with multiple statements in it

Reviewed by: jilles
MFC after: 2 weeks

sh: Fix some bugs with exit status from case containing ;&.

Also, rework evalcase() to not evaluate any tree. Instead, return the
NCLISTFALLTHRU node and handle it in evaltree().

Fixed bugs:

* If a ;& list with non-zero exit status is followed by an empty ;; or final
list, the exit status of the case command should be equal to the exit
status of the ;& list, not 0.

* An empty ;& case should not reset $?.

sh: Fix two bugs with case and exit status:

* If no pattern is matched, POSIX says the exit status shall be 0 (even if
there are command substitutions).
* If a pattern is matched and there are no command substitutions, the first
command should see the $? from before the case command, not always 0.

sh: Add testcases that should not be broken by future optimizations.

Add .t files for tests, missed in prior checkins, so that prove(1) works
in this directory.

Update the tests for arm and other ports where long double is the same
as double, similar to r178141.

Fix a test that doesn't work on architectures where long double is no
wider than double. Thanks to Ian Lepore for catching the bug.

sh: Properly show "Not a directory" error in cd builtin.

The errno message display added in r222292 did not take attempting to
cd to a non-directory or something that cannot be stat()ed into account.

PR: bin/164070
MFC after: 10 days

sh: Avoid possible echo options in a testcase.

Convert files over to UTF-8.

These files contained various combinations of Big5, eucJP and KOI8-U
encoded strings. The byte representations of their respective encodings
have been translated to $'...' escape sequences as understood by our sh(1).

With help from: jilles

Adapt testsuite following change in Domain Search error handling

In this testsuite, warning() and error() have the same behaviour.

PR: bin/163431
Sponsored by: Yakaz (http://www.yakaz.com)

Spelling fixes for tools/

Add some $FreeBSD$ tags so svn will allow the commit.

Reencode files from latin1 to UTF-8.

sh: Allow quoting ^ and ] in bracket expressions.

sh: Add some testcases for pasting $*/$@ directly to a literal.

This also passes on stable/8.

Fix last-minute typo.

Add job error output test.

make(1) with openpty() currently fails this test: there's a race condition
and error output is sometimes lost.

Add a test for r228510.

- Add a test for PR 151758.
- While here, make this compile and work on non-i386:
- Use CMSG_SPACE(), CMSG_LEN(), and CMSG_FIRSTHDR() instead of ignoring
padding between 'struct cmsghdr' and control message payloads.
- Don't initialize the control message before calling recvmsg().
Instead, check that we get a valid control message on return from
recvmsg().
- Use errx() instead of err() for some errors that don't report failures
that set errno.

Requested by: kib (1)

Fix some uninitialized variables in pipe regression tests that result in
failure.

PR: misc/161175
Submitted by: gianni@
Approved by: nwhitehorn@
MFC after: 1 week

Support domain-search in dhclient(8)

The "domain-search" option (option 119) allows a DHCP server to publish
a list of implicit domain suffixes used during name lookup. This option
is described in RFC 3397.

For instance, if the domain-search option says:
".example.org .example.com"
and one wants to resolve "foobar", the resolver will try:
1. "foobar.example.org"
2. "foobar.example.com"

The file /etc/resolv.conf is updated with a "search" directive if the
DHCP server provides "domain-search".

A regression test suite is included in this patch under
tools/regression/sbin/dhclient.

PR: bin/151940
Sponsored by Yakaz (http://www.yakaz.com)

Add a simple test for pipe inode numbers reported by fstat(2).

Submitted by: gianni
MFC after: 1 week

Add three execution tests for make(1):
- plus: execute "+command" when run with -jX -n
- ellipsis: ellipsis ("...") from variable
- empty: empty command (from variable)

Currently make(1) fails all three tests:
- plus: segmentation fault due to incorrect command list handling
- ellipsis: works in compat mode but fails in job (-jX) mode
- empty:
- compat mode: prints error message
- job mode: works but prints empty string

test: Add more testcases.

The new testcases pass even on old stable/7, but some other implementations
manage to get them wrong.

Also remove a few duplicate testcases.

sh: Add tests for some corner cases of 'case' exit status.

These already work properly.

sh: Allow unsetting OPTIND.

Note that only assigning the decimal value 1 resets getopts, as before.

A regression test to ensure that arc4random returns different sequences
in parent and child processes after a fork.

fifo_misc test: Fix swapped lseek arguments.

It worked regardless because SEEK_CUR happens to be 1.

sh: Add test for exit status of for loop without items.

POSIX says the exit status of a for loop without any items shall be 0. There
are no exceptions if the exit status of the previous command was not 0 or if
the item list contains a command substitution with non-zero exit status.

Fix resize regressions tests - we need to destroy BSDlabel before we can
create GPT.

MFC after: 3 days

Add regression tests for modf{,f,l}().

Tests for complex trig and hyperbolic functions.

Tests for cancellation in fma(). Also include more tests for 128-bit
long doubles. Thanks for clusteradm (simon) for making the needed
hardware available.

Fix some memory errors in *at() regression tests.

Add some tests for corner cases of log() in unusual rounding modes.
I wrote these ages ago, but they've been failing until now.

Add some tests for double-rounding bugs in fma().

Add support for IPv6 to ipfw fwd:
Distinguish IPv4 and IPv6 addresses and optional port numbers in
user space to set the option for the correct protocol family.
Add support in the kernel for carrying the new IPv6 destination
address and port.
Add support to TCP and UDP for IPv6 and fix UDP IPv4 to not change
the address in the IP header.
Add support for IPv6 forwarding to a non-local destination.
Add a regession test uitilizing VIMAGE to check all 20 possible
combinations I could think of.

Obtained from: David Dolson at Sandvine Incorporated
(original version for ipfw fwd IPv6 support)
Sponsored by: Sandvine Incorporated
PR: bin/117214
MFC after: 4 weeks
Approved by: re (kib)

Test process descriptors.

Ensure that process descriptors work as expected. We should be able to:
- pdfork(), like regular fork(), but producing a process descriptor
- pdgetpid() to convert a PD into a PID
- pdkill() to send signals to a process identified by a PD

Approved by: re (kib), mentor (rwatson)
Sponsored by: Google Inc

poll(2) implementation for capabilities.

When calling poll(2) on a capability, unwrap first and then poll the
underlying object.

Approved by: re (kib), mentor (rwatson)
Sponsored by: Google Inc

Rename CAP_*_KEVENT to CAP_*_EVENT.

Change the names of a couple of capability rights to be less
FreeBSD-specific.

Approved by: re (kib), mentor (rwatson)
Sponsored by: Google Inc

Test *at(2) calls with capability-mode lookup.

This commit adds regression testing for openat(), fstatat(), etc. with
capability scoping ("strict relative" lookup), which applies:
- in capability mode
- when performing any *at() lookup relative to a capability

These tests will fail until the *at() code is committed; on my local
instance, with the *at() changes, they all pass.

Approved by: re (kib), mentor (rwatson)
Sponsored by: Google Inc

Use the right printf() format string without a cast to maxint_t.

As per kib's suggestion, we also change test_count from a size_t to an int;
its value at the moment is 4, and we only expect it to go up to 7.

Approved by: re (kib), mentor (rwatson)
Sponsored by: Google Inc

Second-to-last commit implementing Capsicum capabilities in the FreeBSD
kernel for FreeBSD 9.0:

Add a new capability mask argument to fget(9) and friends, allowing system
call code to declare what capabilities are required when an integer file
descriptor is converted into an in-kernel struct file *. With options
CAPABILITIES compiled into the kernel, this enforces capability
protection; without, this change is effectively a no-op.

Some cases require special handling, such as mmap(2), which must preserve
information about the maximum rights at the time of mapping in the memory
map so that they can later be enforced in mprotect(2) -- this is done by
narrowing the rights in the existing max_protection field used for similar
purposes with file permissions.

In namei(9), we assert that the code is not reached from within capability
mode, as we're not yet ready to enforce namespace capabilities there.
This will follow in a later commit.

Update two capability names: CAP_EVENT and CAP_KEVENT become
CAP_POST_KEVENT and CAP_POLL_KEVENT to more accurately indicate what they
represent.

Approved by: re (bz)
Submitted by: jonathan
Sponsored by: Google Inc

Properly initialise the "len" argument to getsockname(2) in the tcpdrop
regression test so that it works (more) consistently.

Approved by: re (bz)
Sponsored by: Juniper Networks

Expect fchflags(2) to fail with EOPNOTSUPP on NFS.

Even if we have CAP_FCHFLAGS, fchflags(2) fails on NFS. This is normal
and expected, so don't fail the test because of it.

Note that, whether or not we are on NFS, fchflags(2) should always fail
with ENOTCAPABLE if we are using a capability that does not have the
CAP_FCHFLAGS right.

Approved by: re (kib), mentor (rwatson)
Sponsored by: Google Inc

Flesh out the cap_test regression test.

Add more regression testing, some of which is expected to fail until we
commit more kernel implementation.

Approved by: re (kib), mentor (rwatson)
Sponsored by: Google Inc

Flesh out the cap_test regression test.

Add more regression testing, some of which is expected to fail until we
commit more kernel implementation.

Approved by: re (kib), mentor (rwatson)
Sponsored by: Google Inc

sh: Add a test for a fairly obscure case with aliases.

This also passes on stable/8.

posix_spawn: If an error is detected in the child process, reap the zombie.

Formerly, in this case an error was returned but the pid was also returned
to the application, requiring the application to use unspecified behaviour
(the returned pid in error situations) to avoid zombies.

Now, reap the zombie and do not return the pid.

MFC after: 2 weeks

Clarify the meaning of a test.

Rather than using err() if either of two failure conditions
fires (which can produce spurious error messages), just use
errx() if the one condition that really matters fires.

In practice, this single test is enough to detect the failure
mode we're looking for (kqueue being inherited across fork).

Approved by: mentor (rwatson), re (Capsicum blanket)
Sponsored by: Google Inc

Ensure that kqueue is not inherited across fork().

Modify the existing unit test (from libkqueue) which already exercises process events via
fork() and kill(). Now, the child process simply checks that the 'kqfd' descriptor is invalid.

Some minor modifications were required to make err() work correctly. It seems that this test
was imported using the output of a configure script, but config.h was not included in key
places, nor was its syntax correct (need '#define HAVE_FOO 1' rather than '#define HAVE_FOO').

Finally, change main() to run the "proc" suite by default, but widened the '#if TODO' in
proc.c to include the non-functioning test event_trigger().

Approved by: mentor (rwatson), re (Capsicum blanket)
Sponsored by: Google Inc

sh: Test that '!' is literal if quoted and first char of bracket expression

This also works on stable/8.

sh: Add test for r223282.

sh: Add case statement fallthrough (with ';&' instead of ';;').

Replacing ;; with the new control operator ;& will cause the next list to be
executed as well without checking its pattern, continuing until a list ends
with ;; or until the end of the case statement. This is like omitting
"break" in a C "switch" statement.

The sequence ;& was formerly invalid.

This feature is proposed for the next POSIX issue in Austin Group issue
#449.

sh: Skip variables with invalid names in "set", "export -p", "readonly -p".

This ensures the output of these commands is valid shell input.

sh: Reduce unnecessary forks with eval.

The eval special builtin now runs the code with EV_EXIT if it was run
with EV_EXIT itself.

In particular, this eliminates one fork when a command substitution contains
an eval command that ends with an external program or a subshell.

This is similar to what r220978 did for functions.

sh: Add support for named character classes in bracket expressions.

Example:
case x in [[:alpha:]]) echo yes ;; esac

sh: Add test for LC_COLLATE-based character ranges in case.

sh: Add test for case pattern matching with iso-8859-1 charset.

This also passes on stable/8.

sh: Do parameter expansion on ENV before using it.

This is required by POSIX, and allows things like ENV=\$HOME/.shrc.

Note that tilde expansion is explicitly not performed.

sh: Do parameter expansion before printing PS4 (set -x).

The function name expandstr() and the general idea of doing this kind of
expansion by treating the text as a here document without end marker is from
dash.

All variants of parameter expansion and arithmetic expansion also work (the
latter is not required by POSIX but it does not take extra code and many
other shells also allow it).

Command substitution is prevented because I think it causes too much code to
be re-entered (for example creating an unbounded recursion of trace lines).

Unfortunately, our LINENO is somewhat crude, otherwise PS4='$LINENO+ ' would
be quite useful.

sh: Add simple tests for set -x and PS4.

etire the cpumask_t type and replace it with cpuset_t usage.

This is intended to fix the bug where cpu mask objects are
capped to 32. MAXCPU, then, can now arbitrarely bumped to whatever
value. Anyway, as long as several structures in the kernel are
statically allocated and sized as MAXCPU, it is suggested to keep it
as low as possible for the time being.

Technical notes on this commit itself:
- More functions to handle with cpuset_t objects are introduced.
The most notable are cpusetobj_ffs() (which calculates a ffs(3)
for a cpuset_t object), cpusetobj_strprint() (which prepares a string
representing a cpuset_t object) and cpusetobj_strscan() (which
creates a valid cpuset_t starting from a string representation).
- pc_cpumask and pc_other_cpus are target to be removed soon.
With the moving from cpumask_t to cpuset_t they are now inefficient
and not really useful. Anyway, for the time being, please note that
access to pcpu datas is protected by sched_pin() in order to avoid
migrating the CPU while reading more than one (possible) word
- Please note that size of cpuset_t objects may differ between kernel
and userland. While this is not directly related to the patch itself,
it is good to understand that concept and possibly use the patch
as a reference on how to deal with cpuset_t objects in userland, when
accessing kernland members.
- KTR_CPUMASK is changed and now is represented through a string, to be
set as the example reported in NOTES.

Please additively note that no MAXCPU is bumped in this patch, but
private testing has been done until to MAXCPU=128 on a real 8x8x2(htt)
machine (amd64).

Please note that the FreeBSD version is not yet bumped because of
the upcoming pcpu changes. However, note that this patch is not
targeted for MFC.

People to thank for the time spent on this patch:
- sbruno, pluknet and Nicholas Esborn (nick AT desert DOT net) tested
several revision of the patches and really helped in improving
stability of this work.
- marius fixed several bugs in the sparc64 implementation and reviewed
patches related to ktr.
- jeff and jhb discussed the basic approach followed.
- kib and marcel made targeted review on some specific part of the
patch.
- marius, art, nwhitehorn and andreast reviewed MD specific part of
the patch.
- marius, andreast, gonzo, nwhitehorn and jceel tested MD specific
implementations of the patch.
- Other people have made contributions on other patches that have been
already committed and have been listed separately.

Companies that should be mentioned for having participated at several
degrees:
- Yahoo! for having offered the machines used for testing on big
count of CPUs.
- The FreeBSD Foundation for having sponsored my devsummit attendance,
which has been instrumental.
- Sandvine for having offered offices and infrastructure during
development.

(I really hope I didn't forget anyone, if it happened I apologize in
advance).

sh: Fix $? in heredocs on simple commands.

PR: bin/41410

sh: Add already working testcases for $? in here-document.

If the here-document is attached to a compound command or subshell, $?
already works properly. This is both a workaround for bin/41410 and a
requirement for a true fix for bin/41410.

PR: bin/41410
MFC after: 1 week

Add a very simple IPDIVERT test, which creates IP divert sockets and
checks for collision/non-collision properties in binding them. This
test would have identified a bug recently reported on current@
involding my disaggregation of the pcbinfo lock.

It would be nice if this test also exercised packet diversion and
injection, but that is for another day.

MFC after: 3 days
Sponsored by: Juniper Networks, Inc.

sh: Reduce more needless differences between error messages.

sh: Honour -n while processing -c string.

sh: Add tests for -n flag. These already pass.

sh: Add tests for some somewhat obscure aspects of function definitions.

Upgrade jail(2) to latest jail(2) API to make the regression test work
again. Eventually should switch to jail_set(2).

Reported by: rwatson
MFC after: 10 days

Rework TIMEWAIT regression test so that kernel-allocated port numbers are
used rather than a fixed userspace one, avoiding conflicts between the two
test runs.

MFC after: 3 days
Sponsored by: Juniper Networks, Inc.

Add missing include of stdio.h.

MFC after: 3 days
Sponsored by: Juniper Networks, Inc.

In the tcpdrop regression test, allow the kernel to allocate us a port
rather than using a fixed port number. This means that the regression test
can be run many times in a row without waiting on TIMEWAIT to release a
hard-coded port number.

MFC after: 3 days
Sponsored by: Juniper Networks, Inc.

Add missing #include of err.h.

MFC after: 3 days
Sponsored by: Juniper Networks, Inc.

sh: Add test for 'set +o'.

printf: Allow multibyte characters for '<char> form, avoid negative codes.

Examples:
LC_ALL=en_US.UTF-8 printf '%d\n' $(printf \'\\303\\244)
LC_ALL=en_US.ISO8859-1 printf '%d\n' $(printf \'\\344)
Both of these should print 228.

Like some other shells, incomplete or invalid multibyte characters yield the
value of the first byte without a warning.

Note that there is no general way to go back from the character code to the
character.

sh: Correct criterion for using CDPATH in cd.

CDPATH should be ignored not only for pathnames starting with '/' but also
for pathnames whose first component is '.' or '..'.

The man page already describes this behaviour.

sh: Add simple CDPATH test.

sh: Fix unquoted $@/$* if IFS=''.

If IFS is null, unquoted $@/$* should still expand to separate words.
This differs from quoted $@ (which does not depend on IFS) in that pathname
generation is performed and empty words are removed.

sh: Add test for r222173.

sh: Expand aliases after assignments and redirections.

sh: Add test for positional parameters with more than one digit.

This also passes on stable/8.

sh: Implement the cd -e flag proposed for the next POSIX issue.

This reflects failure to determine the pathname of the new directory in the
exit status (1). Normally, cd returns successfully if it did chdir() and the
call was successful.

In POSIX, -e only has meaning with -P; because our -L is not entirely
compliant and may fall back to -P mode, -e has some effect with -L as well.

sh: Allow terminating a heredoc with a terminator at EOF without a newline.

This is sometimes used with eval or old-style command substitution, and most
shells other than ash derivatives allow it.

It can also be used with scripts that violate POSIX's requirement on the
application that they end in a newline (scripts must be text files except
that line length is unlimited).

Example:
v=`cat <<EOF
foo
EOF`
echo $v

This commit does not add support for the similar construct with new-style
command substitution, like
v=$(cat <<EOF
foo
EOF)
This continues to require a newline after the terminator.

sh: Add tests for lines that look like heredoc delimiters but are not.

sh: Add \u/\U support (in $'...') for UTF-8.

Because we have no iconv in base, support for other charsets is not
possible.

Note that \u/\U are processed using the locale that was active when the
shell started. This is necessary to avoid behaviour that depends on the
parse/execute split (for example when placing braces around an entire
script). Therefore, UTF-8 encoding is implemented manually.

sh: Add UTF-8 support to pattern matching.

?, [...] patterns match codepoints instead of bytes. They do not match
invalid sequences. [...] patterns must not contain invalid sequences
otherwise they will not match anything. This is so that ${var#?} removes the
first codepoint, not the first byte, without putting UTF-8 knowledge into
the ${var#pattern} code. However, * continues to match any string and an
invalid sequence matches an identical invalid sequence. (This differs from
fnmatch(3).)

sh: Add UTF-8 support to ${#var}.

If the current locale uses UTF-8, ${#var} counts codepoints (more precisely,
bytes b with (b & 0xc0) != 0x80).

sh: Add $'quoting' (C-style escape sequences).

A string between $' and ' may contain backslash escape sequences similar to
the ones in a C string constant (except that a single-quote must be escaped
and a double-quote need not be). Details are in the sh(1) man page.

This construct is useful to include unprintable characters, tabs and
newlines in strings; while this can be done with a command substitution
containing a printf command, that needs ugly workarounds if the result is to
end with a newline as command substitution removes all trailing newlines.

The construct may also be useful in future to describe unprintable
characters without needing to write those characters themselves in 'set -x',
'export -p' and the like.

The implementation attempts to comply to the proposal for the next issue of
the POSIX specification. Because this construct is not in POSIX.1-2008,
using it in scripts intended to be portable is unwise.

Matching the minimal locale support in the rest of sh, the \u and \U
sequences are currently not useful.

Exp-run done by: pav (with some other sh(1) changes)

sh: Apply set -u to variables in arithmetic.

Note that this only applies to variables that are actually used.
Things like (0 && unsetvar) do not cause an error.

Exp-run done by: pav (with some other sh(1) changes)