#
3e2d96ac |
|
07-Feb-2021 |
Kyle Evans <kevans@FreeBSD.org> |
grep: fix -A handling in conjunction with -m match limitation The basic issue here is that grep, when given -m 1, would stop all line processing once it hit the match count and exit immediately. The problem with exiting immediately is that -A processing only happens when subsequent lines are processed and do not match. The fix here is relatively easy; when bsdgrep matches a line, it resets the 'tail' of the matching context to the value supplied to -A and dumps anything that's been queued up for -B. After the current line has been printed and tail is reset, we check our mcount and do what's needed. Therefore, at the time that we decide we're doing nothing, we know that 'tail' of the context is correct and we can simply continue on if there's still more to pick up. With this change, we still bail out immediately if there's been no -A flag. If -A was supplied, we signal that we should continue on. However, subsequent lines will not even bothere to try and process the line. We have reached the match count, so even if the next line would match then we must process it if it hadn't. Thus, the loop in procfile() can short-circuit and just process the line as a non-match until procmatches() indicates that it's safe to stop. A test has been added to reflect both that we should be picking up the next line and that the next line should be considered a non-match even if it should have been. PR: 253350 MFC-after: 3 days
|
#
f823c6dc |
|
04-Feb-2021 |
Kyle Evans <kevans@FreeBSD.org> |
grep: fix null pattern and empty pattern file behavior The null pattern semantics were terrible because I tried to match gnugrep, but I got it wrong. Let's unwind that: - The null pattern should match every line if neither -w nor -x. - The null pattern should match empty lines if -x. - The null pattern should not match any lines if -w. The first two will stop processing (shortcut) even if additional patterns are specified. In any other case, we will continue processing other patterns. If no other patterns are specified beside a null pattern, then we match if neither -w nor -x or set and do not match if either of those are specified. The justification for -w is that it should match on a whole word, but the null pattern deos not have a whole word to match on. Empty pattern files should never match anything, and more importantly, -v should cause everything to be written. PR: 253209 MFC-after: 4 days
|
#
63c8336d |
|
01-Oct-2020 |
Eric van Gyzen <vangyzen@FreeBSD.org> |
zgrep: fix exit status with multiple files zgrep should exit with success when given multiple files and the pattern is found in at least one file. Prior to this change, it would exit with success only if the pattern was found in _every_ file. Reviewed by: dab ngie MFC after: 2 weeks Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D26616
|
#
c4cbf1fb |
|
20-Jul-2020 |
Craig Leres <leres@FreeBSD.org> |
Fix some regressions with the zgrep(1) wrapper. - Handle whitespace with long flags that take arguments: echo 'foo bar' > test zgrep --regexp='foo bar' test - Do not hang reading from stdin with patterns in a file: echo foobar > test echo foo > pattern zgrep -f pattern test zgrep --file=pattern test - Handle any flags after -e: echo foobar > test zgrep -e foo --ignore-case < test These two are still outstanding problems: - Does not handle flags that take an argument if there is no whitespace: zgrep -enfs /etc/rpc - When more than one -e pattern used matching should occur for all patterns (similar to multiple patterns supplied with -f file). Instead only the last pattern is used for matching: zgrep -e rex -e nfs /etc/rpc (This problem is masked in the unpatched version by the "any flags after -e" problem.) Add tests for the above problems. Update the mange and add references to gzip(1) and zstd(1) and also document the remaining known problems. PR: 247126 Approved by: markj MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D25613
|
#
38325e2a |
|
25-Sep-2019 |
Kyle Evans <kevans@FreeBSD.org> |
bsdgrep(1): various fixes of empty pattern/exit code/-c behavior When an empty pattern is encountered in the pattern list, I had previously broken bsdgrep to count that as a "match all" and ignore any other patterns in the list. This commit rectifies that mistake, among others: - The -v flag semantics were not quite right; lines matched should have been counted differently based on whether the -v flag was set or not. procline now definitively returns whether it's matched or not, and interpreting that result has been kicked up a level. - Empty patterns with the -x flag was broken similarly to empty patterns with the -w flag. The former is a whole-line match and should be more strict, only matching blank lines. No -x and no -w will will match the empty string at the beginning of each line. - The exit code with -L was broken, w.r.t. modern grep. Modern grap will exit(0) if any file that didn't match was output, so our interpretation was simply backwards. The new interpretation makes sense to me. Tests updated and added to try and catch some of this. This misbehavior was found by autoconf while fixing ports found in PR 229925 expecting either a more sane or a more GNU-like sed. MFC after: 1 week
|
#
52cf8965 |
|
07-Jun-2018 |
Kyle Evans <kevans@FreeBSD.org> |
netbsd-tests: bsdgrep(1): Add a test for -m, too
|
#
ce024bdc |
|
07-Jun-2018 |
Kyle Evans <kevans@FreeBSD.org> |
netbsd-tests: grep(1): Add test for -c flag Someone might be inclined to accidentally break this. someone might have written said test because they broke it locally.
|
#
acfa114e |
|
29-Jan-2018 |
Kyle Evans <kevans@FreeBSD.org> |
Remove t_grep:mmap_eof_not_eol test The test was marked as an expected failure in r320414 after r319971's import of a newer jemalloc removed an essential feature (opt.redzone) for reproducing the behavior it was testing. Since then, no way has been found or demonstrated to reliably test the behavior, so remove the test. PR: 220309
|
#
c552f48b |
|
23-Aug-2017 |
Kyle Evans <kevans@FreeBSD.org> |
bsdgrep: add some additional tests for fgrep Previously added tests only check that fgrep is somewhat sane and works. Add some more tests that check that the implementation is basically functional and not producing incorrect results with various flags. Reviewed by: cem, emaste, ngie Approved by: emaste (mentor) MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D12056
|
#
0e957942 |
|
24-Jul-2017 |
Kyle Evans <kevans@FreeBSD.org> |
bsdgrep(1): Don't exit before processing every file Given an empty pattern (i.e. grep "" A B), bsdgrep(1) would previously exit() with the appropriate exit code upon encountering an empty file. Likely intended as an optimization, but this behavior is technically incorrect since an empty pattern should match every line. PR: 220924 Reviewed by: emaste, cem (earlier version), ngie Approved by: emaste (mentor) Differential Revision: https://reviews.freebsd.org/D11698
|
#
69a3d9e7 |
|
27-Jun-2017 |
Enji Cooper <ngie@FreeBSD.org> |
Expect :mmap_eof_not_eol to fail It relies on a jemalloc feature (opt.redzone) no longer available after r319971. MFC with: r318908, r319971 PR: 220309
|
#
d7b65474 |
|
25-May-2017 |
Ed Maste <emaste@FreeBSD.org> |
bsdgrep: add --mmap tests Basic sanity tests as well as coverage for the bug fixed in r318565. Submitted by: Kyle Evans <kevans91@ksu.edu> Reviewed by: bapt, ngie Differential Revision: https://reviews.freebsd.org/D10827
|
#
6d635d3b |
|
20-May-2017 |
Ed Maste <emaste@FreeBSD.org> |
bsdgrep: Correct per-line line metadata printing Metadata printing with -b, -H, or -n flags suffered from a few flaws: 1) -b/offset printing was broken when used in conjunction with -o 2) With -o, bsdgrep did not print metadata for every match/line, just the first match of a line 3) There were no tests for this Address these issues by outputting this data per-match if the -o flag is specified, and prior to outputting any matches if -o but not --color, since --color alone will not generate a new line of output for every iteration over the matches. To correct -b output, fudge the line offset as we're printing matches. While here, make sure we're using grep_printline in -A context. Context printing should *never* look at the parsing context, just the line. The tests included do not pass with gnugrep in base due to it exhibiting similar quirky behavior that bsdgrep previously exhibited. Submitted by: Kyle Evans <kevans91@ksu.edu> Reviewed by: cem Differential Revision: https://reviews.freebsd.org/D10580
|
#
fe8c9d5b |
|
19-May-2017 |
Ed Maste <emaste@FreeBSD.org> |
bsdgrep: emit more than MAX_LINE_MATCHES per line We should not set an arbitrary cap on the number of matches on a line, and in any case MAX_LINE_MATCHES of 32 is much too low. Instead, if we match more than MAX_LINE_MATCHES, keep processing and matching from the last match until all are found. For the regression test, we produce 4096 matches (larger than we expect we'll ever set MAX_LINE_MATCHES) and make sure we actually get 4096 lines of output with the -o flag. We'll also make sure that every distinct line is getting its own line number to detect line metadata not being printed as appropriate along the way. PR: 218811 Submitted by: Kyle Evans <kevans91@ksu.edu> Reported by: jbeich Reviewed by: cem Differential Revision: https://reviews.freebsd.org/D10577
|
#
f701bdc5 |
|
15-May-2017 |
Ed Maste <emaste@FreeBSD.org> |
bsdgrep: add more tests for different binary flags The existing 'binary' test in netbsd-tests/ does a basic check of the default treatment for binary behavior, but not much more than that. Given some opportunity for breakage recently that did not trigger any failures, add some tests to cover the three different binary file behaviors (a, -I, -U) and their --binary-files= equivalent values. Submitted by: Kyle Evans <kevans91@ksu.edu> Reviewed by: cem, ngie Differential Revision: https://reviews.freebsd.org/D10620
|
#
b5fc583c |
|
15-May-2017 |
Ed Maste <emaste@FreeBSD.org> |
bsdgrep: don't allow negative -A / -B / -C Previously, when given a negative -A/-B/-C argument bsdgrep would overflow the respective context flag(s) and exhibited surprising behavior. Fix this by removing unsignedness of Aflag/Bflag and erroring out if we're given a value < 0. Also adjust the type used to track 'tail' context in procfile() so that it accurately reflects the Aflag value rather than overflowing and losing trailing context. This also fixes an inconsistency previously existing between -n and -C "n" behavior. They are now both limited to LLONG_MAX, to be consistent. Add some test cases to make sure grep errors out properly for both negative context values as well as non-numeric context values rather than giving bogus matches. Submitted by: Kyle Evans <kevans91@ksu.edu> Reviewed by: cem Differential Revision: https://reviews.freebsd.org/D10675
|
#
a41afc82 |
|
08-May-2017 |
Enji Cooper <ngie@FreeBSD.org> |
Remove expected failure that no longer fails with gnu grep in base Reported by: Jenkins Submitted by: Kyle Evans <kevans91@ksu.edu> Sponsored by: Dell EMC Isilon
|
#
e2127de8 |
|
05-May-2017 |
Ed Maste <emaste@FreeBSD.org> |
bsdgrep: don't ouptut matches with -c, -l, -L Refactoring done in r317703 broke -c, -l, and -L flags implying suppression of match printing. Fortunately this is just a matter of not doing any printing of the resulting matches and context printing was not broken in this refactoring. Add some regression tests since this area may still see further refactoring, include different context flags as well even though they were not broken in this case. PR: 219077 Submitted by: Kyle kevans91@ksu.edu Reported by: markj Reviewed by: cem, ngie Differential Revision: https://reviews.freebsd.org/D10607
|
#
a4f3f02b |
|
02-May-2017 |
Ed Maste <emaste@FreeBSD.org> |
bsdgrep: fix -w flag matching with an empty pattern -w flag matching with an empty pattern was generally 'broken', allowing matches to occur on any line whether or not it actually matches -w criteria. This fix required a good amount of refactoring to address. procline() is altered to *only* process the line and return whether it was a match or not, necessary to be able to short-circuit the whole function in case of this matchall flag. -m flag handling is moved out as well because it suffers from the same fate as context handling if we bypass any actual pattern matching. The matching context (matches, mostly) didn't previously exist outside of procline(), so we go ahead and create context object for file processing bits to pass around. grep_printline() was created due to this, for the scenarios where the matches don't actually matter and we just want to print a line or two, a la flushing the context queue and no -o or --color specified. Damage from this broken behavior would have been mitigated by the fact that it is unlikely users would invoke grep -w with an empty pattern. This was identified while checking PR 105221 for problems it this may cause in BSD grep, but PR 105221 is *not* a report of this behavior. Submitted by: Kyle Evans <kevans91 at ksu.edu> Differential Revision: https://reviews.freebsd.org/D10433
|
#
befd089c |
|
02-May-2017 |
Ed Maste <emaste@FreeBSD.org> |
bsdgrep: revise test case which will soon become a failure Work in progress (D10315) is going to make egrep_empty_invalid an actually invalid regex, to be consistent with the equivalent BRE "{" behavior, when using regex(3). Any non-0 exit value is acceptable, depending on how the installed grep interprets the expression. GNU grep interprets it as non-matching, and in the future BSD grep will interpret it is an error. Submitted by: Kyle Evans <kevans91 at ksu.edu> Reviewed by: cem, ngie Differential Revision: https://reviews.freebsd.org/D10572`
|
#
945fc991 |
|
01-May-2017 |
Ed Maste <emaste@FreeBSD.org> |
bsdgrep: fix -w -v matching improperly with certain patterns -w and -v flag matching was mostly functional but had some minor problems: 1. -w flag processing only allowed one iteration through pattern matching on a line. This was problematic if one pattern could match more than once, or if there were multiple patterns and the earliest/ longest match was not the most ideal, and 2. Previous work "fixed" things to not further process a line if the first iteration through patterns produced no matches. This is clearly wrong if we're dealing with the more restrictive -w matching. #2 breakage could have also occurred before recent broad rewrites, but it would be more arbitrary based on input patterns as to whether or not it actually affected things. Fix both of these by forcing a retry of the patterns after advancing just past the start of the first match if we're doing more restrictive -w matching and we didn't get any hits to start with. Also move -v flag processing outside of the loop so that we have a greater change to match in the more restrictive cases. This wasn't strictly wrong, but it could be a little more error prone. While here, introduce some regressions tests for this behavior and fix some excessive wrapping nearby that hindered readability. GNU grep passes these new tests. PR: 218467, 218811 Submitted by: Kyle Evans <kevans91 at ksu.edu> Reviewed by: cem, ngie Differential Revision: https://reviews.freebsd.org/D10329
|
#
5199917c |
|
22-Apr-2017 |
Enji Cooper <ngie@FreeBSD.org> |
Add more sanity tests for grep, egrep, and fgrep The test suite currently lacks basic sanity checks to ensure that egrep, fgrep, and grep are actually matching the right expression types, i.e. passing the right flags to regcomp(3). Amend the test suite to make sure that not only are the individual versions doing the right thing, but also that we don't have some kind of frankenregex situation happening where egrep is accepting a BRE or grep an ERE. I've chosen to not expand the 'basic' test but to add the 'grep_sanity' checks to their own test case since this is testing for more than just 'grep matches things', but actual expression types. Differential Revision: D10444 Reviewed by: emaste, ngie Submitted by: Kyle Evans <kevans91@ksu.edu> Tested with: bsdgrep, gnu grep (base, ports) Sponsored by: Dell EMC Isilon
|
#
6e5e4dbd |
|
22-Apr-2017 |
Enji Cooper <ngie@FreeBSD.org> |
Remove the expected failures for :context and :context2 with bsdgrep(1) They're no longer needed after recent fixes made to bsdgrep(1). Submitted by: Kyle Evans <kevans91@ksu.edu> (via a previous diff in D10433) Sponsored by: Dell EMC Isilon
|
#
e06ffa32 |
|
17-Apr-2017 |
Ed Maste <emaste@FreeBSD.org> |
bsdgrep: fix zero-length matches without the -o flag r316477 broke zero-length matches when not using the -o flag, by skipping over them entirely. Add a regression test so that it doesn't break again in the future. Submitted by: Kyle Evans <kevans91 at ksu.edu> Reviewed by: cem emaste ngie Differential Revision: https://reviews.freebsd.org/D10333
|
#
9f67a481 |
|
12-Apr-2017 |
Enji Cooper <ngie@FreeBSD.org> |
Fix expectations for testcases per bsdgrep vs gnu grep from base/ports The following failures occur with various versions of grep: BSD grep: - :context - :context2 GNU grep (base): - :color - :oflag_zerolen GNU grep (ports): - :recurse_symlink Tested with: bsdgrep (^/head@r316542), gnu grep (base/2.5.1), gnu grep (ports/2.27) Reported by: Jenkins (bsdgrep failures) Sponsored by: Dell EMC Isilon
|
#
799c5faa |
|
05-Apr-2017 |
Ed Maste <emaste@FreeBSD.org> |
bsdgrep: create additional tests for coverage on recent fixes Create additional tests to cover regressions that were discovered by PRs linked to reviews D10098, D10102, and D10104. It is worth noting that neither bsdgrep(1) nor gnugrep(1) in the base system currently pass all of these tests, and gnugrep(1) not quite being up to snuff was also noted in at least one of the PRs. PR: 175314 202022 195763 180990 197555 197531 181263 209116 Submitted by: Kyle Evans <kevans91@ksu.edu> Reviewed by: cem, ngie, emaste MFC after: 1 month Differential Revision: https://reviews.freebsd.org/D10112
|
#
eaae77f8 |
|
27-Jan-2016 |
Alan Somers <asomers@FreeBSD.org> |
Fix grep_test:recurse with ZFS and TMPFS tmpdirs contrib/netbsd-tests/usr.bin/grep/t_grep.sh Fix grep_test:recurse when /tmp is either zfs or tmpfs. The test was relying on an implicit ordering of directory recursion which happens to be true when using UFS. grep's specification requires no such ordering. The solution is to ignore the order of grep's results. Reviewed by: ngie MFC after: 32 days Sponsored by: Spectra Logic Corp Differential Revision: https://reviews.freebsd.org/D4925
|
#
1115e598 |
|
21-Dec-2015 |
Enji Cooper <ngie@FreeBSD.org> |
Use stable output to a test file instead of depending on the OS name being grep'able in /bin/sh This fixes the situation where the OS has been rebranded to something other than `FreeBSD` MFC after: 1 week Obtained from: Isilon OneFS (^/onefs/head@r511419) Reviewed by: cem, Daniel O'Connor <darius@dons.net.au> Sponsored by: EMC / Isilon Storage Division
|