bsdgrep: fix -w flag matching with an empty pattern

MFC r317703: bsdgrep: fix -w flag matching with an empty pattern

-w flag matching with an empty pattern was generally 'broken', allowing
matches to occur on any line whether or not it actually matches -w
criteria.

This fix required a good amount of refactoring to address. procline()
is altered to *only* process the line and return whether it was a match
or not, necessary to be able to short-circuit the whole function in case
of this matchall flag. -m flag handling is moved out as well because it
suffers from the same fate as context handling if we bypass any actual
pattern matching.

The matching context (matches, mostly) didn't previously exist outside
of procline(), so we go ahead and create context object for file
processing bits to pass around. grep_printline() was created due to
this, for the scenarios where the matches don't actually matter and we
just want to print a line or two, a la flushing the context queue and
no -o or --color specified.

Damage from this broken behavior would have been mitigated by the fact
that it is unlikely users would invoke grep -w with an empty pattern.

This was identified while checking PR 105221 for problems it this may
cause in BSD grep, but PR 105221 is *not* a report of this behavior.

MFC r317741: bsdgrep: correct uninitialized variable introduced in r317703

MFC r317842: bsdgrep: don't ouptut matches with -c, -l, -L

Refactoring done in r317703 broke -c, -l, and -L flags implying
suppression of match printing. Fortunately this is just a matter of not
doing any printing of the resulting matches and context printing was not
broken in this refactoring.

Add some regression tests since this area may still see further
refactoring, include different context flags as well even though they
were not broken in this case.

PR: 219077
Approved by: emaste (mentor, blanket MFC)

bsdgrep: Use implied working directory for -r if no directories are passed

MFC r317050: bsdgrep: for -r, use the working directory if none specified

This is more sensible than the previous behaviour of grepping stdin,
and matches newer GNU grep behaviour.

MFC r317300 (ngie): Only expect :grep_r_implied to pass with bsdgrep(1)

The test fails with gnu grep from base and ports.

MFC r319002 (ngie): :rgrep : use atf-check to check the exit code/save the
output of grep -r instead of calling grep -r without it, and saving the
output to a file

This ensures that any errors thrown via grep -r are caught, not lost, and
uses existing atf-sh idioms for saving files.

PR: 216307
Approved by: emaste (mentor, blanket MFC)
Relnotes: yes

bsdgrep: Fix matching behavior and add regression tests

MFC r316477: bsdgrep: fix matching behaviour

- Set REG_NOTBOL if we've already matched beginning of line and we're
examining later parts

- For each pattern we examine, apply it to the remaining bits of the
line rather than (potentially) smaller subsets

- Check for REG_NOSUB after we've looked at all patterns initially
matching the line

- Keep track of the last match we made to later determine if we're
simply not matching any longer or if we need to proceed another byte
because we hit a zero-length match

- Match the earliest and longest bit of each line before moving the
beginning of what we match to further in the line, past the end of the
longest match; this generally matches how gnugrep(1) seems to behave,
and seems like pretty good behavior to me

- Finally, bail out of printing any matches if we were set to print all
(empty pattern) but -o (output matches) was set

MFC r316489: bsdgrep: Initialize vars to avoid a false positive GCC warning

MFC r316491: bsdgrep: revert color changes from r316477

r316477 changed the color output to match exactly the in-tree GNU grep,
but introduces unnecessary escape sequences.

MFC r316536: bsdgrep: create additional tests for coverage on recent fixes

Create additional tests to cover regressions that were discovered by
PRs linked to reviews D10098, D10102, and D10104.

It is worth noting that neither bsdgrep(1) nor gnugrep(1) in the base
system currently pass all of these tests, and gnugrep(1) not quite being
up to snuff was also noted in at least one of the PRs.

MFC r317052: bsdgrep: fix zero-length matches without the -o flag

r316477 broke zero-length matches when not using the -o flag, by
skipping over them entirely.

Add a regression test so that it doesn't break again in the future.

PR: 175314, 180990, 181263, 195763, 197531, 197555, 202022, 209116
Approved by: emaste (mentor, blanket MFC)
Relnotes: yes

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
namespacing is kept with FILES appropriately, and that this shouldn't need
to be repeated if the namespace changes -- only the definition of PACKAGE
needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
`tests`. In the event we get to the point where things can be split up
enough in the base system, it would make more sense to group the tests
with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division

Merge the projects/release-pkg branch to head.

This allows packaging the base system with pkg(8), including
but not limited to providing the ability to provide upstream
binary update possibilities for non-tier-1 architectures.

This merge is a requirement of the 11.0-RELEASE, and as such,
thank you to everyone that has tested the project branch.

Documentation in build(7) etc. is still somewhat sparse, but
updates to those parts will follow.

Sponsored by: The FreeBSD Foundation

Refactor the test/ Makefiles after recent changes to bsd.test.mk (r289158) and
netbsd-tests.test.mk (r289151)

- Eliminate explicit OBJTOP/SRCTOP setting
- Convert all ad hoc NetBSD test integration over to netbsd-tests.test.mk
- Remove unnecessary TESTSDIR setting
- Use SRCTOP where possible for clarity

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Divison

Integrate usr.bin/grep/tests from NetBSD into atf/kyua

Sponsored by: EMC / Isilon Storage Division