tests/netgraph: Fix build error due to missing directory

While MFCing the tests into stable/11 the test directory needs to be
added, because earlier netgraph tests were not MFCed to this train.

PR: 256773
Reported by: Arrigo Marchiori
Submitted by: Arrigo Marchiori
Tested by: Arrigo Marchiori

This is a direct commit into the stable/11 branch.

Git Hash: db7136d6b97f53e58976d20b454f7bb71322b07d
Git Author: donner@FreeBSD.org

MFC r351836, r351866, r354328: patch(1) /dev/null testing+improvement

r351836: patch(1): add some basic tests

Summary:
- basic: test application of patches created by diff -u at the
beginning/middle/end of file, which have differing amounts of context
before and after chunks being added
- limited_ctx: stems from PR 74127 in which a rogue line was getting added
when the patch should have been rejected. Similar behavior was
reproducible with larger contexts near the beginning/end of a file. See
r326084 for details
- file_creation: patch sourced from /dev/null should create the file
- file_nodupe: said patch sourced from /dev/null shouldn't dupe the contents
when re-applied (personal vendetta, WIP, see comment)
- file_removal: this follows from nodupe; the reverse of a patch sourced
from /dev/null is most naturally deleting the file, as is expected based
on GNU patch behavior (WIP)

r351866: patch(1): fix the file removal test, strengthen it a bit

To remain compatible with GNU patch, we should ensure that once we're
removing empty files after a reversed /dev/null patch we don't remove files
that have been modified. GNU patch leaves these intact and just reverses the
hunk that created the file, effectively implying --remove-empty-files for
reversed /dev/null patches.

r354328: patch(1): give /dev/null patches special treatment

We have a bad habit of duplicating contents of files that are sourced from
/dev/null and applied more than once... take the more sane (in most ways)
GNU route and complain if the file exists and offer reversal options.

This still falls short a little bit as selecting "don't reverse, apply
anyway" will still give you duplicated file contents. There's probably other
issues as well, but awareness is the first step to happiness.

MFC r352263:

[jail] removal by jid doesn't trigger pre/post stop scripts

This commit fixes bug: command "jail -r" didn't trigger pre/post stop
commands (and others) defined in config file if jid is specified insted of
name. Also it adds basic tests for usr.sbin/jail to avoid regression.

Reviewed by: jamie, kevans, ray
Differential Revision: https://reviews.freebsd.org/D21328

MFC r346545-r346546, r346680, r346700, r346705

r346545:
libbe(3): allow creation of arbitrary depth boot environments

libbe currently only provides an API to create a recursive boot environment,
without any formal support for intentionally limiting the depth. This
changeset adds an API, be_create_depth, that may be used to arbitrarily
restrict the depth of the new BE.

r346546:
libbe(3): Add a test for be creation

r346680:
libbe(3): Copy received properties as well

This was inherently broken on send|recv datasets.

r346700:
libbe(3): Fix mis-application of patch (SHLIBDIR)

Rob's patch in D18564 cemented the SHLIBDIR because bsd.own.mk (included by
src.opts.mk) sets it to /usr/lib. r346546 did somehow not apply this part of
the patch, leaving it to get installed to the wrong place and subsequently
removed via ObsoleteFiles.

r346705:
libbe(3): Fix libcompat build

SHLIBDIR should still be optionally set, just before src.opts.mk is included
so that libcompat can properly override it. This fixes lib32 failures
reported by both Jenkins and Michael Butler.

MFC bectl(8)/libbe(3): r337663-337664,337667,337697-337699,337800,337805,
337915-337918,337921,337924,337947,337993-337995,338221-338222,338303,
338417,339047,339972,339994,340334,340507-340508,340592-340594,
340635-340636,340722-340723,340974,342466,342849,342903,342911,343335,
343543,343977,343993-343994,344034,344067,344084,345302,345769,
345845-345846,345848,346082

There are simply too many small changes to enumerate; in summary:

bectl(8)/libbe(3) has been introduced from current state in -CURRENT and
added to the stable/11 rescue build. bectl(8) is a tool for managing ZFS
boot environments, largely inspired by beadm. It includes features such as
being able to jail a boot environment or easily mount it for modification.

Relnotes: probably

MFC r342649, r342650

netmap: add suite of unit tests

Import the unit tests from upstream (https://github.com/luigirizzo/netmap
ba02539859d46d33), and make them ready for use with Kyua.
There are currently 38 regression tests, which test the kernel control ABI
exposed by netmap to userspace applications:

1: test for port info get
2-5: tests for basic port registration
6-9: tests for VALE
10-11: tests for getting netmap allocator info
12-15: tests for netmap pipes
16: test on polling mode
17-18: tests on options
19-27: tests for sync-kloop subsystem
28-39: tests for null ports
31-38: tests for the legacy NIOCREGIF registers

Reviewed by: ngie
Sponsored by: Sunny Valley Networks
Differential Revision: https://reviews.freebsd.org/D18490

MFC r336728:

Introduce test program for auditpipe(4)

Submitted by: aniketp
Sponsored by: Google, Inc. (GSoC 2018)
Differential Revision: https://reviews.freebsd.org/D16395

MFC r334360, r334362, r334388, r334395

r334360:
Add initial set of tests for audit(4)

This change includes the framework for testing the auditability of various
syscalls, and includes changes for the first 12. The tests will start
auditd(8) if needed, though they'll be much faster if it's already running.
The syscalls tested in this commit include mkdir(2), mkdirat(2), mknod(2),
mknodat(2), mkfifo(2), mkfifoat(2), link(2), linkat(2), symlink(2),
symlinkat(2), rename(2), and renameat(2).

Submitted by: aniketp
Sponsored by: Google, Inc (GSoC 2018)
Differential Revision: https://reviews.freebsd.org/D15286

r334362 by emaste:
Temporarily disconnect audit tests

Audit tests added in r334360 broke the build on a number of archs.
Remove the subdir from the top level tests/sys/Makefile until they're
fixed.

r334388:
Fix OpenBSM with GCC with -Wredundant-decls

Upstream change ed47534 consciously added some redundant functional
declarations, and I'm not sure why. AFAICT they were never required. On
FreeBSD, they break the build with GCC (but not Clang) for any program
including libbsm.h with WARNS=6.

Fix by cherry-picking upstream change
https://github.com/openbsm/openbsm/commit/0553c27

Reported by: emaste
Reviewed by: cem
Obtained from: OpenBSM
Pull Request: https://github.com/openbsm/openbsm/pull/31

r334395:
Revert r334362

Reconnect tests/sys/audit now that the GCC issue is fixed by 334388

X-MFC-With: 334362, 334360, 334388

MFC r335287, r335290

r335287:
praudit(1): return 0 on success

Cherry pick https://github.com/openbsm/openbsm/commit/ed83bb3

Submitted by: aniketp
Reviewed by: rwatson, 0mp
Obtained from: OpenBSM
Sponsored by: Google, Inc. (GSoC 2018)
Pull Request: https://github.com/openbsm/openbsm/pull/32

r335290:
praudit(1): add tests

Submitted by: aniketp
X-MFC-With: 335287
Sponsored by: Google, Inc. (GSoC 2018)
Differential Revision: https://reviews.freebsd.org/D15751

MFC r330090:

Add 'usr.bin/seq' to tests mtree after r330086

PR: 217149

MFC 332657:
Properly do a deep copy of the ioctls capability array for fget_cap().

fget_cap() tries to do a cheaper snapshot of a file descriptor without
holding the file descriptor lock. This snapshot does not do a deep
copy of the ioctls capability array, but instead uses a different
return value to inform the caller to retry the copy with the lock
held. However, filecaps_copy() was returning 1 to indicate that a
retry was required, and fget_cap() was checking for 0 (actually
'!filecaps_copy()'). As a result, fget_cap() did not do a deep copy
of the ioctls array and just reused the original pointer. This cause
multiple file descriptor entries to think they owned the same pointer
and eventually resulted in duplicate frees.

The only code path that I'm aware of that triggers this is to create a
listen socket that has a restricted list of ioctls and then call
accept() which calls fget_cap() with a valid filecaps structure from
getsock_cap().

To fix, change the return value of filecaps_copy() to return true if
it succeeds in copying the caps and false if it fails because the lock
is required. I find this more intuitive than fixing the caller in
this case. While here, change the return type from 'int' to 'bool'.

Finally, make filecaps_copy() more robust in the failure case by not
copying any of the source filecaps structure over. This avoids the
possibility of leaking a pointer into a structure if a similar future
caller doesn't properly handle the return value from filecaps_copy()
at the expense of one more branch.

I also added a test case that panics before this change and now passes.

MFC r330696, r330709, r330742, r331358

r330696:
Add some functional tests for tftpd(8)

tftpd(8) is difficult to test in isolation due to its relationship with
inetd. Create a test program that mimics the behavior of tftp(1) and
inetd(8) and verifies tftpd's response in several different scenarios.

These test cases cover all of the basic TFTP protocol, but not the optional
parts.

PR: 157700
PR: 225996
PR: 226004
PR: 226005
Differential Revision: https://reviews.freebsd.org/D14310

r330709:
Commit missing file from r330696

X-MFC-With: 330696

r330742:
tftpd: fix the build of tests on i386 after 330696

It's those darn printf format specifiers again

Reported by: cy, kibab
X-MFC-With: 330696

r331358:
tftpd: misc Coverity cleanup in the tests

A bunch of unchecked return values from open(2) and read(2)

Reported by: Coverity
CID: 1386900, 1386911, 1386926, 1386928, 1386932, 1386942
CID: 1386961, 1386979
X-MFC-With: 330696

MFC r329874:

Add tests for lagg(4) and other cloned network interfaces

Unfortunately, most of the tests are disabled because they fairly frequently
trigger panics.

Sponsored by: Spectra Logic Corp

MFC r303452,r303482,r303483,r303484,r303485,r303487,r303489,r303498,r303499,r303502,r303523,r303525,r303570,r303571,r303588,r303596,r303597,r303598,r303599,r303600,r303601,r303625,r303629,r303718,r304651,r304684,r304686,r305983,r309217,r309219,r309341,r309342,r309343,r309382,r309415,r309417,r309418,r309419,r310863,r311141,r314613,r318471,r321382,r321383,r321396:

indent(1): avoid calling write(2) with a negative second parameter.
indent(1): Avoid out of bound access of array codebuf.
indent(1): Avoid potential use-after-free.
indent(1): Fix breakage caused by single comment following "else".
indent(1) simply wasn't taught that "else" may be followed by a comment
without any opening brace anywhere on the line, so it was very confused
in such cases.
indent(1): fix struct termination detection.
indent(1): fix struct termination detection.
indent(1): Removed whitespace shouldn't be considered in column calculations.
indent(1): Support "f" and "F" floating constant suffixes.
indent(1): Use NULL instead of zero for pointers.
indent(1): Attempt to preserve some consistent style.
indent(1): Yet more style issues.
indent(1): Consistently indent declarations.
indent(1): Bail out if there's no more space on the parser stack.
indent(1): Remove dead code relating to unix-style comments.
indent(1): Simplify pr_comment().
indent(1): Fix wrapping of some lines in comments.
indent(1): Untangle the connection between pr_comment.c and io.c.
indent(1): Don't newline on cpp lines like #endif unless -bacc is on.
indent(1): replace function call to bzero with memset.
indent(1): Rearrange option parsing code to squelch clang's static analyzer.
indent(1): Use a dash in the license headers.
indent(1): accept offsetof(3) as a keyword.
indent(1): add some comments to quiet down Coverity.
indent(1): remove dead assignments.
indent(1): have the memset invocation somewhat more canonical.
indent(1): Capsicumify
indent(1): minor off-by-one error.
indent(1): fix regression introduced in r303596.
indent(1): Avoid out of bound access of array in_buffer
indent(1): Don't ignore newlines after comments that follow braces.
indent(1): Don't unnecessarily add a blank before a comment ends.
indent(1): Do not define opchar unless it will be used.
indent(1): Optimize parser stack usage.
indent(1): Remove an extra newline added in a previous commit.
indent(1): Avoid out-of-bound accesses of arrays.
indent(1): Avoid out-of-bound accesses of array ps.p_stack.
indent(1): Avoid out of bounds access of array ps.paren_indents
indent(1): add a piece missed in r311138.
indent(1): Support binary integer literals.
indent(1): don't produce unnecessary blank lines.
indent(1): rename the profile file.
indent(1): better alignment of comments on code.

MFC r329273, r329275, r329277, r329284, r329344

r329273:
geli: append "/eli" to the underlying provider's physical path

If the underlying provider's physical path is null, then the geli device's
physical path will be, too. Otherwise, it will append "/eli". This will make
geli work better with zfsd(8).

PR: 224962
Differential Revision: https://reviews.freebsd.org/D13979

r329275:
gpart: append partition name to the underlying provider's physical path

If the underlying provider's physical path is null, then the gpart device's
physical path will be, too. Otherwise, it will append the partition name,
such as "/p1" or "/s1/a". This will make gpart work better with zfsd(8).

PR: 224965
Differential Revision: https://reviews.freebsd.org/D14010

r329277:
Add mtree entry for 329275

X-MFC-With: 329275
Sponsored by: Spectra Logic Corp

r329284:
zfsd: Allow zfsd to work on any type of GEOM provider

cddl/usr.sbin/zfsd/zfsd_event.cc
Remove the check for da and ada devices. This way zfsd can work on md,
geli, glabel, gstripe, etc devices. geli in particular is useful
combined with ZFS. gnop is also useful for simulating drive pulls in
the ZFSD test suite.

Also, eliminate the DevfsEvent class entirely. Move its
responsibilities into GeomEvent. We can get everything we need to know
just from listening to GEOM events.

lib/libdevdctl/event.cc
Fix GeomEvent::DevName for CREATE events. Oddly, the relevant field is
named "cdev" for CREATE events but "devname" for disk events.

Relnotes: Yes (probably worth mentioning the geli part)
Sponsored by: Spectra Logic Corp

r329344:
Optimize zfsd for the happy case

If there are no damaged pools, then ignore all GEOM events. We only use
them to fix damaged pools. However, still pay attention to ZFS events.

X-MFC-With: 329284
Sponsored by: Spectra Logic Corp

MFC r320930,r324286: Add tests for hexdump(1)

MFC r320930:
Add some basic tests for hexdump(1)'s various output flags. Formatting
tests are omitted for this initial run as there are still some bugs to work
out there.

This covers -s flag testing on devices and non-devices that would have
caught breakage found in PR 219173 as well as other subtle breakage caused
locally.

MFC r324286: hexdump: Remove expected test failures for now succeeding tests

r323990 fixed the -s flag breakage reported by PR 219173. Mark the
corresponding hexdump tests accordingly, since they should now both succeed.

PR: 222781

MFC r326289:

Add basic tests for ctfconvert(1), fold(1) and rs(1)

Add basic command line parsing test coverage for these utilities. The tests
were automatically generated based on their man pages. These tests can be
expanded by hand for more thorough coverage. The aim is to generate very
basic amount of test coverage for all the utilities in the base system.

Tests generated via: https://github.com/shivansh/smoketestsuite/

Submitted by: shivansh
Reviewed by: asomers
Differential Revision: https://reviews.freebsd.org/D12424

MFC r326039:

Add ATF tests for head(1)

Submitted by: Fred Schlecter <https://github.com/fjs-github>
Reviewed by: asomers, jilles
Differential Revision: https://github.com/freebsd/freebsd/pull/127

MFC r323275, r324112

r323275:
Add basic tests for chflags, mkdir, rcp, and rmdir

Add basic command line parsing test coverage for these utilities. The tests
were automatically generated based on their man pages. These tests can be
expanded by hand for more thorough coverage. The aim is to generate very
basic amount of test coverage for all the utilities in the base system.

Submitted by: shivansh
Reviewed by: asomers, brooks
Sponsored by: Google, Inc (GSoC 2017)
Differential Revision: https://reviews.freebsd.org/D12036

r324112:
Fix Makefile entries from r323275

Reported by: Vladimir Zakharov <zakharov.vv@gmail.com>
Reviewed by: ngie
X-MFC-With: 323275

MFC r305626,r305629,r307863,r322447,r322448,r322449,r322450,r322451:

r305626 (by oshogbo):

Move libcasper tests from regression/capsicum/libcasper/ to
lib/libcasper/service/${service_name}/tests.

r305629 (by jkim):

Add new directories added in r305626 to fix "make installworld".

r307863 (by emaste):

Set SHLIBDIR before .including src.opts.mk in libcapser services

bsd.own.mk (included from src.opts.mk) sets SHLIBDIR?=${LIBDIR}, so
SHLIBDIR must be set before including either one of them.

MFC with: 305626

r322447:

Fix result printing

- Flushing stdout prevents the buffer from being printed twice, fixing
issues with stdout printing out the testplan, etc, twice.
- Don't print out raw source/line numbers; hide them behind comments.

r322448:

Make root-privileges a requirement for the test

Some of the testcases try to manipulate sysctls that require root privileges,
e.g., "kern.sync_on_panic". Make root-privileges a hard requirement so the
tests don't raise false positives due to privilege issues when calling
sysctlbyname(3) on writable sysctls.

r322449:

Use hardcoded IPv4/IPv6 addresses for google-public-dns-a.google.com instead
of freefall.freebsd.org to unbreak the DNS tests

The address allocations for freefall.freebsd.org have changed in the past 4 years.
Use a more stable set of hardcoded addresses for now to make the tests succeed
reliably.

The hostname should be resolved dynamically instead of hardcoding the addresses in
the future. This is just a bandaid.

r322450:

Integrate the tests moved in r305626 in to the FreeBSD test suite

The reachover Kyuafiles were never added, and thus the tests were installed
as standalone tests, and not integrated into the full suite.

MFC with: r305626, 305629, r307863, r322447, r322448, r322449

r322451:

TESTSDIR isn't required; remove it

MFC with: r322450

MFC r320702,r320703:

r320702:

Formalize LEAPSECONDS and OLDTIMEZONES in share/zoneinfo/... as
`MK_ZONEINFO_LEAPSECONDS_SUPPORT == yes` and
`MK_ZONEINFO_OLD_TIMEZONES_SUPPORT == yes`.

Keep `LEAPSECONDS` and `OLDTIMEZONES` for backwards compatibility,
but print out a warning notifying users that they should use the new
variables, in an effort to migrate them to the variables. This is being
done mostly for automated build tools, etc, that might rely on these
variables being set. The variables will be removed in the future on
^/head, e.g., after ^/stable/12 is cut.

Relnotes: yes

r320703:

Add tests to help verify Links functionality for .../contrib/tzdata/backwards

MFC with: r320702

MFC r320443,r320444:

r320443:

Add kyua TAP test integration examples

The examples are patterned loosely after the ATF examples, similar
to the plain test examples.

r320444:

Commit the corresponding mtree file change for the TAP test examples

MFC with: r320443

MFC r319850:

Add some initial basic tests for du(1)

Tests that exercise the following flags are added in this commit:
- -A
- -H
- -I
- -g
- -h
- -k
- -m

Additional tests will be added soon.

MFC r303212,r319642,r319830:

r303212 (by bdrewery):

Move chown tests to proper path

r319642:

Add some basic tests for chmod(1)

r319830:

Add more simple positive tests for chown(1)

The tests are largely symmetric with the tests for chmod(1)--added in r319642.

Remove chown-f_test (added in r268030) since the test coverage is now being
provided by `chown_test`.

MFC r316557:

sbuf(3): add some basic functional tests for the library

Areas not covered still [positive functionality wise] are:
- sbuf_{clear,get,set}_flags
- sbuf_new (in particular, with fixed buffers, etc).

Some basic negative testing has been added, but more will be added in the
future.

This work was in part to validate work done by cem in r288223, and ian
before that.

MFC r319834,r319841,r320723,r320724:

r319834:

Write up some basic tests for readlink(1)

The tests exercise -f (f_flag), -n (n_flag), and no arguments (basic).

r319841:

Add initial tests for stat(1)

Testcases for -H, -L, and -f haven't been implemented yet, in part due
to additional complexity needed to validate the features:
* -H and -f will require an external "helper" program to display/modify
the state/permissions for a given path.
* -L is being covered partially via the -n testcase today.

r320723:

Use %e instead of %d with x_output_date(..)

stat -x doesn't 0-fill days so %d is inappropriate. %e is correct.

MFC with: r319841

r320724:

:l_flag:: be more aggressive when normalizing whitespace

Save output from ls -ldT and stat -l, then normalize all repeating whitespace using
sed to single column spaces.

This makes the test flexible with single-digit days, etc, similar to r320723. This
approach is just a bit more of a hammer approach because of how the columns are
ordered/spaced in both ls and stat.

MFC with: r319841

MFC r318325:

Start writing up some basic feature tests for procstat

These tests query a running process for information related to the -b,
-c, -e, and -f flags; the -f testcase is largely stubbed out, pending
additional work to determine a good, deterministic descriptor.

Core file test support is coming soon--it requires a bit more effort
due to the fact that:
- coredumps can be disabled (kern.coredump=0).
- corefiles can be put in different directories than the current
directory, or be named something other than `<prog>.core`
(`kern.corefile`).

MFC r319626:

Add basic tests for echo(1)

Verify that echo(1) does not...
- ... print the trailing newline character with option '-n'.
- ... print the trailing newline character when '\c' is appended to
the end of the string.

MFC r319714, r319854

r319714:
Add tests for ln(1)

* Verify that when creating a hard link to a symbolic link, '-L' option
creates a hard link to the target of the symbolic link
* Verify that when creating a hard link to a symbolic link, '-P' option
creates a hard link to the symbolic link itself
* Verify that if the target file already exists, '-f' option unlinks it so
that link may occur
* Verify that if the target file or directory is a symbolic link, '-shf'
option prevents following the link
* Verify that if the target file or directory is a symbolic link, '-snf'
option prevents following the link
* Verify that '-s' option creates a symbolic link
* Verify that '-w' option produces a warning if the source of a symbolic
link does not currently exist

Submitted by: shivansh
Reviewed by: asomers, ngie
Sponsored by: Google, Inc (GSoC 2017)
Differential Revision: https://reviews.freebsd.org/D11084

r319854:
bin/ln: Set umask appropriately before creating files for testing

These changes were missed in D11084

Submitted by: shivansh
Reviewed by: asomers
X-MFC-With: 319714
Sponsored by: Google, Inc (GSoC 2017)
Differential Revision: https://reviews.freebsd.org/D11158

MFC r318436:

usr.bin/getconf: add some initial tests

Items tested via this commit are:
- Some basic POSIX constants.
- Some valid programming environments with -v.
- Some invalid programming environments via -v.

NOTE: this test makes assumptions about ILP32/LP32 vs LP64 that are
currently not true on all architectures to avoid hardcoding some
architectures in the tests. I'm working on improving getconf(1) to be
more sane about handling ILP32/LP32 vs LP64. Future commits are coming
soon to address this.

Tested with: amd64, i386

MFC r318591: compress: Add basic tests.

MFC r316099:

lib/libkvm: start adding basic tests for kvm(3)

- kvm_close: add a testcase to verify support for errno = EINVAL / -1
(see D10065) when kd == NULL is provided to the libcall.
- kvm_geterr:
-- Add a negative testcase for kd == NULL returning "" (see D10022).
-- Add two positive testcases:
--- test the error case using kvm_write on a O_RDONLY descriptor.
--- test the "no error" case using kvm_read(3) and kvm_nlist(3) as
helper routines and by injecting a bogus error message via
_kvm_err (an internal API) _kvm_err was used as there isn't a
formalized way to clear the error output, and because
kvm_nlist always returns ENOENT with the NULL terminator today.
- kvm_open, kvm_open2:
-- Add some basic negative tests for kvm_open(3) and kvm_open2(3).
Testing positive cases with a specific
`corefile`/`execfile`/`resolver` requires more work and would require
user intervention today in order to reliably test this out.

MFC r316501, r316523

r316501:
Fix file descriptor and memory leaks in pr(1)

Also, hook NetBSD's pr test into the build, and add three more test cases.

Reported by: Coverity, Valgrind
CID: 271650 271651 271652 271653 271654 271655 271656 271656
CID: 271657 271658 271659 1006939 1006940 1006941 1006942 1009098
Reviewed by: ngie
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D9137

r316523:
Remove a duplicate fclose() that snuck into r316501

Reported by: Coverity
CID: 1373338
X-MFC-With: 316501
Sponsored by: Spectra Logic Corp

MFC r314892:
r314892 (by cy):

Fix install due to incorrect placement of pwait dir in r314886.

MFC r314886,r314943,r314944:

r314886:
pwait: Add a -t flag to specify a timeout before exiting, and tests.
r314943:
Remove unneeded -x from tests.
r314944:
Rename some tests to end in _test.

MFC r315320:

Start adding basic tests for cam(3)

This change contains several negative and positive tests for:
- cam_open_device
- cam_close_device
- cam_getccb
- cam_freeccb

This also contains a test for the failure case noted in bug 217649,
i.e., O_RDWR must be specified because pass(4) requires it.

This test unfortunately cannot assume that cam-capable devices are
present, so the user must explicitly provide a device via
`test_suites.FreeBSD.cam_test_device`. In the future, a test kernel
module might be shipped, or ctl(4) might be used, as a test device
when testing out libcam, which will allow the tests to do away with
having to specify an explicit test device.

MFC r313439,r314450:

r313439:

Merge content from ^/projects/netbsd-tests-upstream-01-2017 into ^/head

The primary end-goal of this drop is ease future merges with NetBSD and
collaborate further with the NetBSD project.

The goal was (largely, not completely as some items are still oustanding
in the NetBSD GNATS system) achieved by doing the following:
- Pushing as many changes required to port contrib/netbsd-tests
back to NetBSD as possible, then pull the upstream applied changes
back in to FreeBSD.
- Diff reduce with upstream where possible by:
-- Improving libnetbsd header, etc compat glue.
-- Using _SED variables to modify test scripts on the fly for items
that could not be upstreamed to NetBSD.

As a bonus for this work, this change also introduces testcases for
uniq(1).

Many thanks to Christos for working with me to get many of the changes
back into the NetBSD project.

In collaboration with: Christos Zoulas <christos@netbsd.org>

r314450:

Add additional __FreeBSD_version guards around the hsearch_r testcases

The reasoning for this is the same as r276046: to ease MFCing the tests
to ^/stable/10 .

This was accidentally missed in r313439

MFC r311572, r311895, r311928, r311985, r312395, r312417

r311572:
Fix file descriptor leaks in cmp(1)

Also, add a few test cases

Reported by: Coverity
CID: 271624 275338
Reviewed by: ngie
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D9074

r311895:
Fix memory leaks during "tail -r" of an irregular file

* Rewrite r_buf to use standard tail queues instead of a hand-rolled
circular linked list. Free dynamic allocations when done.
* Remove an optimization for the case where the file is a multiple of 128KB
in size and there is a scarcity of memory.
* Add ATF tests for "tail -r" and its variants.

Reported by: Valgrind
Reviewed by: ngie
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D9067

r311928:
Fix build of usr.bin/tail with GCC

Submitted by: pluknet
Reported by: pluknet
MFC after: 27 days
X-MFC-with: 311895
Sponsored by: Spectra Logic Corp

r311985:
Fix uninitialized variable CIDs in route6d

The variables in question are actually return arguments, but it's still good
form to initialize them.

Reported by: Coverity
CID: 979679 979680
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp

r312395:
Fix several Coverity CIDs in devd

CID 1362055, 1362054: File descriptor leaks during shutdown
CID 1362013: Potential null-termination fail with long network device names
CID 1362097: Uncaught exception during memory pressure
CID 1362017, 1362016: Unchecked errors, possibly resulting in weird behavior
if two devd instances start at the same time.
CID 1362015: Unchecked error that will probably never fail

Reported by: Coverity
CID: 1362055 1362054 1362013 1362097 1362017 1362016 1362015
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp

r312417:
Fix build of devd with GCC 4.2

Reported by: olivier
Pointy-hat-to: asomers
MFC after: 27 days
X-MFC-with: 312395
Sponsored by: Spectra Logic Corp

MFC r311349, r311445

r311349:
tabs -> spaces in etc/mtree

MFC after: 4 weeks

r311445:
Fix typo from r311349

Reported by: lwhsu
Pointy-hat-to: asomers
MFC after: 4 weeks
X-MFC-with: 311349

MFC r306030,r306031,r306033,r306036,r306038,r307190,r307196,r307204,r307205,r307701,r307702:

r306030:

Port vnode_leak_test:main to FreeBSD

Use a simpler way of dumping kern.maxvnodes, i.e. `sysctl -n kern.maxvnodes`

The awk filtering method employed in NetBSD doesn't work on FreeBSD

r306031:

Port contrib/netbsd-tests/fs/h_funcs.subr to FreeBSD

Use kldstat -m to determine whether or not a filesystem is loaded. This works
well with tmpfs, ufs, and zfs

r306033:

Port sizes_test and statvfs_test to FreeBSD

Similar to r306030, use a simpler method for getting the value of
`hw.pagesize`, i.e. `sysctl -n hw.pagesize`. The awk filtering method doesn't
work on FreeBSD

r306036:

Port to mknod_test and readdir_test to FreeBSD

The `mknod <file> p` command doesn't exist on FreeBSD, like on NetBSD. Use
mkfifo instead to create named pipes (FIFOs).

r306038:

Port vnd_test to FreeBSD

Use mdmfs/mdconfig instead of vndconfig/newfs. vndconfig doesn't exist on FreeBSD.

TODO: need to parameterize out the md(4) device as it's currently hardcoded to "3"
(in both the FreeBSD and NetBSD cases).

r307190:

Skip :uchg on FreeBSD

Unfortunately removing files with uchg set always succeeds with root on
FreeBSD. Unfortunately running the test as an unprivileged user isn't doable
because mounting tmpfs requires root

PR: 212861

r307196:

Port contrib/netbsd-tests/fs/tmpfs/h_tools.c to FreeBSD

- Add inttypes.h #include for PRId64 macro
- Use FreeBSD's copy of getfh(2), which doesn't include a `fh_size` parameter.
Use sizeof(fhandle_t) instead as the size of fhp is always fixed as
fhandle_t, unlike NetBSD's copy of fhp, which is void*.

r307204:

Expect :large to fail on FreeBSD

FreeBSD doesn't appear to validate large -o size values like
NetBSD does

PR: 212862

r307205:

Change atf_skip call to atf_expect_fail to make it clear that a failure is
expected

PR: 212861
Suggested by: jmmv

r307701:

Expect tests/sys/fs/tmpfs/link_test:kqueue to fail

It fails with: "dir/b did not receive NOTE_LINK"

Also, add needed cleanup logic to cleanup the mountpoint after the fact

PR: 213662

r307702:

Integrate contrib/netbsd-tests/fs/tmpfs into the FreeBSD test suite
as tests/sys/fs

These testcases exercise tmpfs support

MFC r304797,r305467,r305468,r305483:

r304797 (by jmmv):

Make use of Kyua's work directories.

Change the vnode tests to use the current directory when creating temporary
files, which we can assume is a volatile work directory, and then make the
kqueue_test.sh driver _not_ abandon the directory created by Kyua.

This makes the various kqueue tests independent of each other, and ensures
the temporary file is cleaned up on failure.

Problem spotted by asomers@ when reviewing D4254.

r305467:

Move tests/sys/kqueue/... to tests/sys/kqueue/libkqueue/...

This is being done to clearly distinguish the libkqueue tests
from the (soon to be imported) NetBSD tests.

r305468:

Port contrib/netbsd-tests/kernel/kqueue/... as tests/sys/kqueue/...

proc2_test must be skipped because the invariant tested
(`ke.fflags & NOTE_TRACKERR`) doesn't pass.

r305483:

Fix tests/sys/kqueue NetBSD tests on 32-bit platforms by using proper
format specifier for pointers when printing them out with printf(3)

Pointyhat to: ngie

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Fix "make installworld" with MK_CDDL == no after r300906 by
adding a missing entry for ${TESTSBASE}/cddl/sbin

X-MFC with: r300906
Pointyhat to: asomers
Reported by: Shawn Webb <shawn.webb@hardenedbsd.org>
Sponsored by: EMC / Isilon Storage Division

zfsd(8), the ZFS fault management daemon

Add zfsd, which deals with hard drive faults in ZFS pools. It manages
hotspares and replements in drive slots that publish physical paths.

cddl/usr.sbin/zfsd
Add zfsd(8) and its unit tests

cddl/usr.sbin/Makefile
Add zfsd to the build

lib/libdevdctl
A C++ library that helps devd clients process events

lib/Makefile
share/mk/bsd.libnames.mk
share/mk/src.libnames.mk
Add libdevdctl to the build. It's a private library, unusable by
out-of-tree software.

etc/defaults/rc.conf
By default, set zfsd_enable to NO

etc/mtree/BSD.include.dist
Add a directory for libdevdctl's include files

etc/mtree/BSD.tests.dist
Add a directory for zfsd's unit tests

etc/mtree/BSD.var.dist
Add /var/db/zfsd/cases, where zfsd stores case files while it's shut
down.

etc/rc.d/Makefile
etc/rc.d/zfsd
Add zfsd's rc script

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c
Fix the resource.fs.zfs.statechange message. It had a number of
problems:

It was only being emitted on a transition to the HEALTHY state.
That made it impossible for zfsd to take actions based on drives
getting sicker.

It compared the new state to vdev_prevstate, which is the state that
the vdev had the last time it was opened. That doesn't make sense,
because a vdev can change state multiple times without being
reopened.

vdev_set_state contains logic that will change the device's new
state based on various conditions. However, the statechange event
was being posted _before_ that logic took effect. Now it's being
posted after.

Submitted by: gibbs, asomers, mav, allanjude
Reviewed by: mav, delphij
Relnotes: yes
Sponsored by: Spectra Logic Corp, iX Systems
Differential Revision: https://reviews.freebsd.org/D6564

iconvctl(3): remove superfluous NULL pointer tests

convname and dst are guaranteed to be non-NULL by iconv_open(3).
src is an array. Remove these tests for NULL pointers.
While I'm here, eliminate a strlcpy with a correct but suspicious-looking
calculation for the third parameter (i.e. not a simple sizeof).
Compare the strings in-place instead of copying.

Found by: bdrewery
Found by: Coverity
CID: 1130050, 1130056
MFC after: 3 days
Sponsored by: Dell Inc.
Differential Revision: https://reviews.freebsd.org/D6338

MFV r299425:

Update libarchive to 3.2.0

New features:
- new bsdcat command-line utility
- LZ4 compression (in src only via external utility from ports)
- Warc format support
- 'Raw' format writer
- Zip: Support archives >4GB, entries >4GB
- Zip: Support encrypting and decrypting entries
- Zip: Support experimental streaming extension
- Identify encrypted entries in several formats
- New --clear-nochange-flags option to bsdtar tries to remove noschg and
similar flags before deleting files
- New --ignore-zeros option to bsdtar to handle concatenated tar archives
- Use multi-threaded LZMA decompression if liblzma supports it
- Expose version info for libraries used by libarchive

Patched files (fixed compiler warnings):

contrib/libarchive/cat/bsdcat.c (vendor PR #702)
contrib/libarchive/cat/bsdcat.h (vendor PR #702)
contrib/libarchive/libarchive/archive_read_support_format_mtree.c (PR #701)
contrib/libarchive/libarchive_fe/err.c (vendor PR #703)

MFC after: 1 month
Relnotes: yes

install: Add some tests.

Improve performance and functionality of the bitstring(3) api

Two new functions are provided, bit_ffs_at() and bit_ffc_at(), which allow
for efficient searching of set or cleared bits starting from any bit offset
within the bit string.

Performance is improved by operating on longs instead of bytes and using
ffsl() for searches within a long. ffsl() is a compiler builtin in both
clang and gcc for most architectures, converting what was a brute force
while loop search into a couple of instructions.

All of the bitstring(3) API continues to be contained in the header file.
Some of the functions are large enough that perhaps they should be uninlined
and moved to a library, but that is beyond the scope of this commit.

sys/sys/bitstring.h:
Convert the majority of the existing bit string implementation from
macros to inline functions.

Properly protect the implementation from inadvertant macro expansion
when included in a user's program by prefixing all private
macros/functions and local variables with '_'.

Add bit_ffs_at() and bit_ffc_at(). Implement bit_ffs() and
bit_ffc() in terms of their "at" counterparts.

Provide a kernel implementation of bit_alloc(), making the full API
usable in the kernel.

Improve code documenation.

share/man/man3/bitstring.3:
Add pre-exisiting API bit_ffc() to the synopsis.

Document new APIs.

Document the initialization state of the bit strings
allocated/declared by bit_alloc() and bit_decl().

Correct documentation for bitstr_size(). The original code comments
indicate the size is in bytes, not "elements of bitstr_t". The new
implementation follows this lead. Only hastd assumed "elements"
rather than bytes and it has been corrected.

etc/mtree/BSD.tests.dist:
tests/sys/Makefile:
tests/sys/sys/Makefile:
tests/sys/sys/bitstring.c:
Add tests for all existing and new functionality.

include/bitstring.h
Include all headers needed by sys/bitstring.h

lib/libbluetooth/bluetooth.h:
usr.sbin/bluetooth/hccontrol/le.c:
Include bitstring.h instead of sys/bitstring.h.

sbin/hastd/activemap.c:
Correct usage of bitstr_size().

sys/dev/xen/blkback/blkback.c
Use new bit_alloc.

sys/kern/subr_unit.c:
Remove hard-coded assumption that sizeof(bitstr_t) is 1. Get rid of
unrb.busy, which caches the number of bits set in unrb.map. When
INVARIANTS are disabled, nothing needs to know that information.
callapse_unr can be adapted to use bit_ffs and bit_ffc instead.
Eliminating unrb.busy saves memory, simplifies the code, and
provides a slight speedup when INVARIANTS are disabled.

sys/net/flowtable.c:
Use the new kernel implementation of bit-alloc, instead of hacking
the old libc-dependent macro.

sys/sys/param.h
Update __FreeBSD_version to indicate availability of new API

Submitted by: gibbs, asomers
Reviewed by: gibbs, ngie
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D6004

Add sdiff test directory to the tests mtree

Add ATF tests for usr.sbin/extattr

Add ATF tests for the existing behavior of setextattr, rmextattr, lsextattr,
and getextattr.

Reviewed by: ngie
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D5889

Fix and connect setjmp test.

Sponsored by: EMC / Isilon Storage Division

Integrate
tools/regression/geom_{concat,eli,gate,mirror,nop,raid3,shsec,stripe,uzip}
in to the FreeBSD test suite as
tests/sys/geom/class/{concat,eli,gate,mirror,nop,raid3,shsec,stripe,uzip}

The tools/regression/geom and tools/regression/geom_part testcases are being
left alone because both test sets are both currently broken.

The majority of this work was done on ^/user/ngie/more-tests2 . The differences
are as follows:
- tests/sys/geom/class/Makefile.inc is not present; it was
inlined into the class's Makefiles for explicitness.
- The testcases officially require root via kyua
- The geom_gate(4) tests don't use the pidfile changes proposed in
https://reviews.freebsd.org/D4836 .

MFC after: 1 month
Sponsored by: EMC / Isilon Storage Division

"source routing" in rpcbind

Fix a bug in rpcbind for multihomed hosts. If the server had interfaces on
two separate subnets, and a client on the first subnet contacted rpcbind at
the address on the second subnet, rpcbind would advertise addresses on the
first subnet. This is a bug, because it should prefer to advertise the
address where it was contacted. The requested service might be firewalled
off from the address on the first subnet, for example.

usr.sbin/rpcbind/check_bound.c
If the address on which a request was received is known, pass that
to addrmerge as the clnt_uaddr parameter. That is what addrmerge's
comment indicates the parameter is supposed to mean. The previous
behavior is that clnt_uaddr would contain the address from which the
client sent the request.

usr.sbin/rpcbind/util.c
Modify addrmerge to prefer to use an IP that is equal to clnt_uaddr,
if one is found. Refactor the relevant portion of the function for
clarity, and to reduce the number of ifdefs.

etc/mtree/BSD.tests.dist
usr.sbin/rpcbind/tests/Makefile
usr.sbin/rpcbind/tests/addrmerge_test.c
Add unit tests for usr.sbin/rpcbind/util.c:addrmerge.

usr.sbin/rpcbind/check_bound.c
usr.sbin/rpcbind/rpcbind.h
usr.sbin/rpcbind/util.c
Constify some function arguments

Reviewed by: imp
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D4690

Integrate tools/regression/mac/mac_bsdextended and
tools/regression/mac/mac_portacl into the FreeBSD test suite as
tests/sys/mac/bsdextended and tests/sys/mac/portacl, respectively

MFC after: 1 month
Sponsored by: EMC / Isilon Storage Division

Integrate tools/regression/lib/libc/nss into the FreeBSD test suite as
lib/libc/tests/nss

- Convert the testcases to ATF
- Do some style(9) cleanups:
-- Sort headers
-- Apply indentation fixes
-- Remove superfluous parentheses
- Explicitly print out debug printfs for use with `kyua {debug,report}`; for
items that were overly noisy, they've been put behind #ifdef DEBUG
conditionals
- Fix some format strings

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Integrate tools/regression/lib/libc/resolv into the FreeBSD test suite as
lib/libc/tests/resolv

Convert the testcases to ATF

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Integrate tools/regression/pipe in to the FreeBSD test suite as
tests/sys/kern/pipe

- Fix style(9) bugs
- Fix compiler warnings
- Use `nitems(x)` instead of `sizeof(x) / sizeof(*x)` pattern

The testcases will be converted over to ATF eventually, but for now will be
integrated in as plain C tests

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Integrate acct(2) testcase in as tests/sys/kern/acct/acct_test

The :encode_tv_random_million testcase fails the epsilon tests a few thousand
times out of one million, so expect the testcase to fail

MFC after: 1 week
Submitted by: keramida
Sponsored by: EMC / Isilon Storage Division

Add some basic tests that exercise cputime limits with limits(1)

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Integrate contrib/netbsd-tests/lib/libc/rpc into the FreeBSD test suite
as lib/libc/rpc

This testcase requires rpcbind be up in running; otherwise the testcases
will time out and be skipped

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Correctly reintroduce the rudimentary smoke tests I botched up
in r289684

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Revert r289694

I committed some other undesirable local changes by accident

Add some rudimentary [smoke] testcases for makefs

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Integrate contrib/netbsd-tests/bin/dd into the FreeBSD test suite as
bin/dd/tests

Ensure fdescfs is mounted on /dev/fd/ for the length testcase as it's used
in validating the characters read from /dev/zero

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Integrate tools/regression/acltools into the FreeBSD test suite as tests/sys/acl

- Make the requirements more complete for the testcases
- Detect prerequisites so the tests won't fail (zfs.ko is loaded, zpool(1)
is available, ACL support is enabled with UFS, etc).
- Work with temporary files/directories/mountpoints that work with atf/kyua
- Limit the testcases to work on temporary filesystems to reduce tainting the
test host

MFC after: 2 weeks
Reviewed by: trasz (earlier version)
Differential Revision: https://reviews.freebsd.org/D3810

Integrate tools/test/posixshm and tools/regression/posixshm into the FreeBSD
test suite as tests/sys/posixshm

Some other highlights:
- Convert the testcases over to ATF
- Don't use hardcoded paths to /tmp (which violate the ATF/kyua samdbox); use
mkstemp to generate temporary paths for non-SHM_ANON shm objects.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Integrate tools/regression/vfs into the FreeBSD test suite as tests/sys/vfs

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Integrate the tests from lib/libarchive, usr.bin/cpio, and usr.bin/tar in to
the FreeBSD test suite

functional_test.sh was ported from bin/sh/tests/functional_test.sh, as a
small wrapper around libarchive_test, bsdcpio_test, and bsdtar_test provided
by upstream.

A handful of testcases in lib/libarchive/tests have been disabled as they
were failing when run with kyua test (see BROKEN_TESTS in
lib/libarchive/tests/Makefile)

As a sidenote: this removes the check/test targets from the Makefiles as they
don't match the pattern used in the rest of the FreeBSD test suite.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Integrate the tests from libxo into the FreeBSD test suite

The functional_test.sh harness for each test subdir was inspired
by the version in bin/sh/tests/functional_test.sh

Some gymnastics were required to deal with implicit rules for
.c / .o -> .out as the suffix transformation rules were
incorrectly trying to create the test outputs from some of the
source files

Sponsored by: EMC / Isilon Storage Division

Add initial testcases for bin/ls

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Add ATF functional tests for fstyp(8). No ZFS or GELI tests yet.

Reviewed by: trasz, ngie
MFC after: 2 weeks
Sponsored by: SpectraLogic
Differential Revision: https://reviews.freebsd.org/D2801

Replace GNU RCS ident with a BSD license ident

Rationale: ident(1) is useful out of RCS, lot of scripts are using ident(1) and
failing when base is built WITHOUT_RCS.

This version is:
- fully compatible with RCS 5.7 ident.
- fully compatible with RCS 5.9 ident.
- passes all ident test from GNU RCS 5.9 test suite

This version has support for: svn extension for the Keyword id (double colon and
# before last $)

Différences with GNU RCS ident:
- no long options as found in GNU RCS 5.9 (but not commented there).
- '-V' reports nothing but has been added for compatibility.

Differential Revision: https://reviews.freebsd.org/D3200
Reviewed by: pfg

Add regression test about reverse line feed to col(1)

Add regression tests for soelim(1)

Move tests/sys/kern/mmap_test to tests/sys/vm/mmap_test

As jhb noted, the actual mmap(2) implementation is under sys/vm, not
sys/kern/, so the correct logical place is tests/sys/vm/, not
tests/sys/kern/

X-MFC with: r282076
MFC after: 6 days

Integrate tools/regression/aio/aiotest and tools/regression/aio/kqueue into the
FreeBSD test suite as tests/sys/aio

MFC after: 1 week

Integrate tools/regression/mqueue into the FreeBSD test suite as
tests/sys/mqueue

MFC after: 1 week

Integrate tools/regression/fifo into the FreeBSD test suite as tests/sys/fifo
and tools/regression/file into the FreeBSD test suite as tests/sys/file

MFC after: 1 week

Integrate tools/regression/kqueue into the FreeBSD test suite as
tests/sys/kqueue

MFC after: 1 week

Integrate tools/regression/execve into the FreeBSD test suite as
tests/sys/kern/execve

MFC after: 1 week

Add infrastructure to integrate the DTrace test suite with Kyua.

For each test category, we generate a script containing ATF test cases for
the tests under that category. Each test case simply runs dtest.pl (the
upstream test harness) with the corresponding test files. The exclude.sh
script is used to record info about tests which should be skipped or are
expected to fail; it is used to generate atf_skip and atf_expect_fail calls.
The genmakefiles.sh script can be used to regenerate the test makefiles when
new tests are brought it from upstream.

The test suite is currently not connected to the build as there is a small
number of lingering test issues which still need to be worked out. In the
meantime however, the test suite can be easily built and installed
manually from cddl/usr.sbin/dtrace/tests.

Reviewed by: ngie
Sponsored by: EMC / Isilon Storage Division

Add tests/etc/rc.d to mtree.

Submitted by: stefanf
MFC after: 1 week
MFC with: 277627

Integrate contrib/netbsd-tests/bin/expr into the build/kyua as bin/expr/tests

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Add makefile for the "osmtest" utility. While at it:
- Fix depend target by removing a space after an "-I" inclusion option.
- Fix some minor compile issues in the "osmtest" utility.

MFC after: 3 days
PR: 196580
Sponsored by: Mellanox Technologies

Integrate bin/cat/tests from NetBSD into atf/kyua

Sponsored by: EMC / Isilon Storage Division

Integrate sbin/ifconfig/tests from NetBSD into atf/kyua

Sponsored by: EMC / Isilon Storage Division

Fix building/installing tests when TESTSBASE != /usr/tests

The work in r258233 hardcoded the assumption that tests was the last component
of the tests tree by pushing tests as an explicit prefix for the paths in
BSD.tests.dist and /usr was the prefix for all tests, per BSD.usr.dist and all
of the mtree calls used in Makefile.inc1. This assumption breaks if/when one
provides a custom TESTSBASE "prefix", e.g. TESTSBASE=/mytests .

One thing that r258233 did properly though was remove "/usr/tests" creation
from BSD.usr.dist -- that should have not been there in the first place. That
was an "oops" on my part for the work that was originally committed in r241823

MFC after: 2 weeks
Phabric: D1301
Reviewed by: imp
Sponsored by: EMC / Isilon Storage Division

Add some new modes to OpenCrypto. These modes are AES-ICM (can be used
for counter mode), and AES-GCM. Both of these modes have been added to
the aesni module.

Included is a set of tests to validate that the software and aesni
module calculate the correct values. These use the NIST KAT test
vectors. To run the test, you will need to install a soon to be
committed port, nist-kat that will install the vectors. Using a port
is necessary as the test vectors are around 25MB.

All the man pages were updated. I have added a new man page, crypto.7,
which includes a description of how to use each mode. All the new modes
and some other AES modes are present. It would be good for someone
else to go through and document the other modes.

A new ioctl was added to support AEAD modes which AES-GCM is one of them.
Without this ioctl, it is not possible to test AEAD modes from userland.

Add a timing safe bcmp for use to compare MACs. Previously we were using
bcmp which could leak timing info and result in the ability to forge
messages.

Add a minor optimization to the aesni module so that single segment
mbufs don't get copied and instead are updated in place. The aesni
module needs to be updated to support blocked IO so segmented mbufs
don't have to be copied.

We require that the IV be specified for all calls for both GCM and ICM.
This is to ensure proper use of these functions.

Obtained from: p4: //depot/projects/opencrypto
Relnotes: yes
Sponsored by: FreeBSD Foundation
Sponsored by: NetGate

Add reachover Makefiles for contrib/netbsd-tests/lib/librt

A variant of this code has been tested on amd64/i386 for some time by
EMC/Isilon on 10-STABLE/11-CURRENT. It builds on other architectures, but the
code will remain off until it's proven it works on virtual hardware or real
hardware on other architectures

Sponsored by: EMC / Isilon Storage Division

Add reachover Makefiles for contrib/netbsd-tests/lib/librt

A variant of this code has been tested on amd64/i386 for some time by
EMC/Isilon on 10-STABLE/11-CURRENT. It builds on other architectures, but the
code will remain off until it's proven it works on virtual hardware or real
hardware on other architectures

Sponsored by: EMC / Isilon Storage Division

Add reachover Makefiles for contrib/netbsd-tests/lib/libpthread as
lib/libthr/tests

A variant of this code has been tested on amd64/i386 for some time by
EMC/Isilon on 10-STABLE/11-CURRENT. It builds on other architectures, but the
code will remain off until it's proven it works on virtual hardware or real
hardware on other architectures

Original work by: pho
Sponsored by: EMC / Isilon Storage Division

Create a directory for the PAM tests.

Add reachover Makefiles for contrib/netbsd-tests/lib/libc; this adds approximately
500 new testcases

Various TODOs have been sprinkled around the Makefiles for items that even need
to be ported (missing features), testcases have issues with building/linking, or
issues at runtime.

A variant of this code has been tested extensively on amd64 and i386
10-STABLE/11-CURRENT for several months without issue. It builds on other
architectures, but the code will remain off until I have prove it works on
virtual hardware or real hardware on other architectures

In collaboration with: pho, Casey Peel <casey.peel@isilon.com>
Sponsored by: EMC / Isilon Storage Division

Integrate pjdfstest test suite execution into kyua

pjdfstest execution is opt-in and must be done as root due to some of the
assumptions made by the test suite and lack of error checking in the non-root
case

A description of how to execute pjdfstest with kyua is provided in
share/pjdfstest/README

Phabric: D824 (an earlier prototype patch)
MFC after: 1 month
Relnotes: yes
Sponsored by: EMC / Isilon Storage Division

Add regression tests for the timeout(1) utility

They are modeled over the regression tests that are provided for the GNU
coreutils timeout(1) utility

Integrate usr.bin/gzip/tests from NetBSD into atf/kyua

Sponsored by: EMC / Isilon Storage Division

Integrate usr.bin/diff/tests from NetBSD into atf/kyua at gnu/usr.bin/diff/tests

Sponsored by: EMC / Isilon Storage Division

Integrate usr.bin/grep/tests from NetBSD into atf/kyua

Sponsored by: EMC / Isilon Storage Division

Integrate usr.bin/cut/tests from NetBSD into atf/kyua

Sponsored by: EMC / Isilon Storage Division

Integrate bin/sleep/tests from NetBSD into atf/kyua

Sponsored by: EMC / Isilon Storage Division

Integrate usr.bin/dirname/tests from NetBSD into atf/kyua

In collaboration with: pho, sjg
Sponsored by: EMC / Isilon Storage Division

Integrate usr.bin/cmp/tests from NetBSD into atf/kyua

In collaboration with: sjg
Sponsored by: EMC / Isilon Storage Division

Integrate usr.sbin/basename/tests from NetBSD into atf/kyua

In collaboration with: pho
Sponsored by: EMC / Isilon Storage Division

Set the autoindent to 4 spaces with vim in BSD.tests.dist

This will prevent vim users from accidentally checking in buggy mtree files
(mixed tabs/spaces).

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Integrate usr.sbin/nmtree/tests from NetBSD into atf/kyua

In collaboration with: pho
Sponsored by: EMC / Isilon Storage Division

- Add a test for bug 191427 where pw(8) will go into an infinite loop

Reviewed by: will
MFC after: 1 month

Register /usr/tests/lib/libproc to fix build.

Missed in r271937, reviewed in D710.

Add unit tests for mkimg(1):

Convert devd's client socket to type SOCK_SEQPACKET.

This change consists of two merges from projects/zfsd/head along with the
addition of an ATF test case for the new functionality.

sbin/devd/tests/Makefile
sbin/devd/tests/client_test.c
Add ATF test cases for reading events from both devd socket types.

r266519:
sbin/devd/devd.8
sbin/devd/devd.cc
Create a new socket, of type SOCK_SEQPACKET, for communicating with
clients. SOCK_SEQPACKET sockets preserve record boundaries,
simplying code in the client. The old SOCK_STREAM socket is retained
for backwards-compatibility with existing clients.

r269993:
sbin/devd/devd.8
Fix grammar bug.

CR: https://reviews.freebsd.org/rS266519
MFC after: 5 days
Sponsored by: Spectra Logic

Add missing BSD.tests.dist entry for lib/libutil to unbreak installworld with
MK_TESTS == no

Phabric: D555
X-MFC with: r269904
Approved by: jmmv (mentor, implicit)
Pointyhat to: ngie

Integrate lib/libnv into the build/kyua

Rename all of the TAP test applications from <test> to <test>_test
to match the convention described in the TestSuite wiki page

Phabric: D538
Approved by: jmmv (mentor)
Sponsored by: EMC / Isilon Storage Division

Integrate lib/libmp into the build/kyua

- Remove the .t wrapper
- Fix -Wreturn-type warnings with clang

This change has been tested on amd64/i386

Phabric: D530
Reviewed by: jmmv
Approved by: jmmv (co-mentor)
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Add the mtree entry for yacc tests

Fix indentation

units(1): Add basic tests
Add some tests to help avoid breaking units

chown: add a test

Add a test for the chown utility. This sets up chown(8) to be capable of being
tested. As such, only add one test for now as an example.

Unbreak installation of the rtld tests.

X-MFC-With: r267679

Test RTLD's new LD_LIBRARY_PATH_FDS variable.

Test LD_LIBRARY_PATH_FDS by linking a binary that requires a shared
library that isn't in any of the usual search paths. Ensure this fails
when we don't supply LD_LIBRARY_PATH_FDS or we pass invalid information
in it. Ensure it works when we pass the correct directory in various
places in the variable.

Approved by: rwatson (mentor)
MFC after: 3 weeks
Sponsored by: DARPA/AFRL

truncate: Detect integer overflow, fix relative sizes, add tests.

The change to expand_number (r204654) broke detection of too large sizes and
relative sizes ('+'/'-').

Also add some tests.

PR: 190735
Submitted by: Kirk Russell
MFC after: 1 week

Move atf-sh from /usr/bin/ to /usr/libexec/

In r266650, we made libatf-c and libatf-c++ private libraries so that no
components outside of the source tree could unintendedly depend on them.

This change does the same for the "atf-sh library" by moving the atf-sh
interpreter from its public location in /usr/bin/ to the private location
in /usr/libexec/. Our build system will ensure that our own test programs
use the right binary, but users won't be able to depend on atf-sh by
"mistake".

Committing this now to ride the UPDATING notice added with r267172 today.

Change libatf-c and libatf-c++ to be private libraries.

We should not be leaking these interfaces to the outside world given
that it's much easier for third-party components to use the devel/atf
package from ports.

As a side-effect, we can also drop the ATF pkgconfig and aclocal files
from the base system. Nothing in the base system needs these, and it
was quite ugly to have to get them installed only so that a few ports
could build. The offending ports have been fixed to depend on
devel/atf explicitly.

Reviewed by: bapt

Move old fmake tests into bmake and hook them to the build.

This first step is mostly to prevent the code from rotting even further
and to ensure these do not get wiped when fmake's code is removed from
the tree.

These tests are currently being skipped because they detect the underlying
make is not fmake and thus disable themselves -- and the reason is that
some of the tests fail, possibly due to legitimate bugs. Enabling them to
run against bmake will come separately.

Lastly, it would be ideal if these tests were fed upstream but they are
not ready for that yet. In the interim, just put them under usr.bin/bmake/
while we sort things out. The existence of a different unit-tests directory
within here makes me feel less guilty about this.

Change confirmed working with a clean amd64 build.

Remove the ifconfig test added in rev 263445. After discussion with
melifaro, we agreed that ifconfig's behavior was not a bug. The main
motivation for bin/187551 was to partially resolve kern/187549, but we
resolved kern/187549 in a different way instead.

ObsoleteFiles.inc
etc/mtree/BSD.tests.dist
sbin/ifconfig/tests/fibs_test.sh
sbin/ifconfig/tests/Makefile
sbin/ifconfig/Makefile
Remove /usr/tests/sbin/ifconfig

PR: bin/187551
MFC after: 3 days
Sponsored by: Spectra Logic

Add placeholder Kyuafiles for various top-level hierarchies.

This change adds tests/ directories in the source tree to create various
subdirectories in /usr/tests/ and to install placeholder Kyuafiles for
them.

the relevant hierarchies are: cddl, etc, games, gnu and secure.

The reason for this is to simplify the addition of new test programs for
utilities or libraries under any of these directories. Doing so on a
case by case basis is unnecessary and is quite an obscure process.

Add atf m4 files from the vendor branch.

These were originally deleted as "not important" but, actually we need them
in place if we want to be able to use autoconf on software that provides
atf-based tests. (That includes being able to rebuild autotest from scratch
on the Kyua cluster machines, as the automated setup does.)

Add several ATF tests that deal with multiple fibs. They're described in
several different PRs, but the tests share some common code, so I'm
committing them together.

sbin/ifconfig/tests
sbin/ifconfig/tests/fibs_test.sh
sbin/ifconfig/tests/Makefile
sbin/ifconfig/Makefile
Add fibs_test.sh, which regresses bin/187551

tests/sys/netinet
tests/sys/netinet/fibs_test.sh
tests/sys/netinet/udp_dontroute.c
tests/sys/netinet/Makefile
tests/sys/Makefile
Add fibs_test.sh, which regresses kern/167947, kern/187552
kern/187549, kern/187550, and kern/187553

etc/mtree/BSD.tests.dist
Add newly created directories

PR: bin/187551
PR: kern/167947
PR: kern/187552
PR: kern/187549
PR: kern/187550
PR: kern/187553
Discussed with: melifaro
MFC after: 3 weeks
Sponsored by: Spectra Logic Corporation

Migrate tools/regression/usr.bin/pkill to the new tests layout.

Interestingly, the pkill tool lives in bin, not usr.bin. Haven't bothered
to check if this is because the tool moved or because the tests were
originally added in the wrong place.

Migrate tools/regression/usr.bin/make/ to the new tests layout.

Note that these tests are for fmake, not bmake, and thus they are not
installed nor run when bmake is selected (the default). Yes, I have
wasted a *ton* of time on moving tests for no real reason other than
ensuring they are not left behind.

But maybe, just maybe, it was not work in vain: the majority of these
tests also work with bmake and the few that don't may point at broken
stuff. For example, the tests for the "archive" feature do not work
with bmake, but bmake's manpage and source tree seem to imply that they
should. So... to be investigated later; need to poke sjg@.

Expand tabs that sneaked in into spaces.

Problem introduced by r263227. Spotted by Alan Somers.

Migrate most of tools/regression/usr.bin/ to the new tests layout.

I'm starting with the easy cases. The leftovers need to be looked at a
bit more closely.

Note that this change _does_ modify the code of the old tests. This is
required in order to allow the code to locate the data files in the
source directory instead of the current directory, because Kyua
automatically changes the latter to a temporary directory.

Also note that at least one test is known to be broken here. Actually,
the test is not really broken: it's marked as a TODO but unfortunately
Kyua's TAP parser currently does not understand that. Will have to be
fixed separately.

Migrate tools/regression/{usr.bin/lastcomm,usr.sbin}/ to the new tests layout.

This change was originally going to only migrate the usr.sbin tests but, as
it turns out, the usr.sbin/sa/ tests require files from usr.bin/lastcomm/
so it's better to just also migrate the latter at the same time. The other
usr.bin tests will be moved separately.

To make these tests work within the test suite, some of them have required
changes to prevent modifying the source directory and instead just rely on
the current directory for file manipulation.

Migrate tools/regression/sbin/ to the new tests layout.

Pretty much all that this change does is shuffles the code around and hooks
it into the regular build. The code of the old tests has not changed.

Replace the old unix_seqpacket and unix_seqpacket_exercise tests, which
were a little broken and not automatable, with unix_seqpacket_test.
It's coverage is a superset of the old tests and it uses ATF. It
includes test cases for bugs kern/185813 and kern/185812.

PR: kern/185812
PR: kern/185813
Sponsored by: Spectra Logic
MFC after: 2 weeks

Respect the original layout of the atf-{c,c++} tests.

Put test programs for internal modules into a 'detail' subdirectory of the
libatf-c and libatf-c++ test directories, just as the upstream distribution
does. This is necessary because the tests assume such layout to find the
process_helper program, and currently fail because of this divergence.

MFC after: 1 week

Migrate tools/regression/bin/ tests to the new layout.

This change is a proof of concept on how to easily integrate existing
tests from the tools/regression/ hierarchy into the /usr/tests/ test
suite and on how to adapt them to the new layout for src.

To achieve these goals, this change:

- Moves tests from tools/regression/bin/<tool>/ to bin/<tool>/tests/.
- Renames the previous regress.sh files to legacy_test.sh.
- Adds Makefiles to build and install the tests and all their supporting
data files into /usr/tests/bin/.
- Plugs the legacy_test test programs into the test suite using the new
TAP backend for Kyua (appearing in 0.8) so that the code of the test
programs does not have to change.
- Registers the new directories in the BSD.test.dist mtree file.

Reviewed by: freebsd-testing
Approved by: rpaulo (mentor)

Add some sample test programs.

This change adds some sample test cases to share/examples/tests/
demonstrating the basic usage of the atf and plain interfaces.

These test programs are fully-functional and are installed as part
of the test suite, which guarantees that the sample code remains
correct. However, they currently mostly serve as a placeholder for
additional examples and may be incomplete (depending on how you
look at them). I will see what else can be useful while working on
documentation.

As a bonus, the addition of these tests exercise the *.test.mk files,
one of which (plain.test.mk) was not yet in use, and also demonstrates
that it's possible to mix different kinds of test programs into the
same test suite.

Approved by: rpaulo (mentor)

Move all atf directories to the tests mtree.

This is to ensure that test-related directories don't get needlessly
created (and later deleted) when MK_TESTS=no.

Problem found by jhb@.

Approved by: rpaulo (mentor)

Build and install the atf tests.

Reviewed by: freebsd-testing
Approved by: rpaulo (mentor)

Set up the /usr/tests hierarchy.

Populate /usr/tests with the only test programs that currently live
in the tree (those in lib/libcrypt/tests/) and add all the build
machinery to accompany this change.

In particular:

- Add a WITHOUT_TESTS variable that users can define to request that
no tests be put in /usr/tests.
- Add a top-level Kyuafile for /usr/tests and a way to create similar
Kyuafiles in top-level subdirectories.
- Add a BSD.tests.dist file to define the directory layout of
/usr/tests.

Submitted by: Julio Merino jmmv google.com
Reviewed by: sjg
MFC after: 2 weeks