diff: Nits in tests.

Sponsored by: Klara, Inc.

diff: honour -B flag with -q

PR: 278988
Reviewed by: bapt
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45220

diff: Integrate libdiff from OpenBSD GoT.

This adds support for two new diff algorithms, Myers diff and Patience
diff.

These algorithms perform a different form of search compared to the
classic Stone algorithm and support escapes when worst case scenarios
are encountered.

Add the -A flag to allow selection of the algorithm, but default to
using the new Myers diff implementation.

The libdiff implementation currently only supports a subset of input and
output options supported by diff. When these options are used, but the
algorithm is not selected, automatically fallback to the classic Stone
algorithm until support for these modes can be added.

Based on work originally done by thj@ with contributions from kevans@.

Sponsored by: Klara, Inc.
Reviewed by: thj
Differential Revision: https://reviews.freebsd.org/D44302

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

diff: fix side-by-side output with tabbed input

The previous logic conflated some things... in this block:
- j: input characters rendered so far
- nc: number of characters in the line
- col: columns rendered so far
- hw: column width ((h)ard (w)idth?)

Comparing j to hw or col to nc are naturally wrong, as col and hw are
limits on their respective counters and nc is already brought down to hw
if the input line should be truncated to start with.

Right now, we end up easily truncating lines with tabs in them as we
count each tab for $tabwidth lines in the input line, but we really
should only be accounting for them in the column count. The problem is
most easily demonstrated by the two input files added for the tests,
the two tabbed lines lose at least a word or two even though there's
plenty of space left in the row for each side.

Reviewed by: bapt, pstef
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D37676

diff: tests: loosen up requirements for report_identical

This test cannot run without an unprivileged_user being specified
anyways, so just run as the unprivileged user. Revoking read permisions
works just as well if you're guaranteed non-root.

Reviewed by: pstef
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D34950

Revert "Temporarily skip usr.bin.diff.diff_test.functionname in CI"

This reverts commit 85eeb6ea62d45c5df893a16b87969bd7313a3dbb.

The issue has been fixed by 4be7d087c2b566f4910683836be279d55c1a81c6.

PR: 262086

Temporarily skip usr.bin.diff.diff_test.functionname in CI

PR: 262086
Sponsored by: The FreeBSD Foundation

diff: fix failed compare when ignoring file case

With --ignore-file-name-case we need to compare files regardless of
case. We need to propigate both names down to diffit so we can look up
the correct file when the names differ based on case, otherwise we try
to look up the file using the case from the a tree which might not be
discoverable if its case is different in the b tree.

Reviewed by: bapt
Sponsored by: Klara Inc.
Differential Revision: https://reviews.freebsd.org/D34201

Remove surplus check for which diff is being tested

Must have left this when I was testing the test invocation for
8f79bd9b85716c495c2741ac25db37e8d71f22f7

Pointy hat: thj
Sponsored by: Klara Inc.

diff: add --no-dereference flag

When diffing files and directories, don't follow symbolic links, instead
compare where the links point to.

Reviewed by: bapt
Sponsored by: Klara Inc.
Differential Revision: https://reviews.freebsd.org/D34203

diff: Detect Objective-C methods

When searching back for function definitions, consider lines starting
with '+' and '-', this allows us to pick up Objective-C methods as well
as C style function definitions.

Reviewed by: bapt
Sponsored by: Klara Inc.
Differential Revision: https://reviews.freebsd.org/D34202

diff: read whole files to determine if they are ASCII text

Before this change, only the first BUFSIZE bytes were checked.

Reviewed by: bapt (previous version)
Differential Revision: https://reviews.freebsd.org/D31639

diff: don't output carriage returns that were stripped on input

--strip-trailing-cr worked as intended for comparison between files,
but the characters were still present in final output.

diff tests: Require root for report_identical

This test switches users and either hangs or fails when run as an
unprivileged user.

MFC after: 1 week
Sponsored by: The FreeBSD Foundation

diff: Use unprivileged_user with report_identical test

Approved by: bapt
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D28466

diff: add a test case about the non regular file support

diff: add a test case for failed -s option

diff: honour flags with -q

Previously -q (just print a line when files differ) ignored flags like
-w (ignore whitespace). Avoid the D_BRIEF short-circuit when flags are
in effect.

PR: 252515
Reported by: Scott Aitken
Reviewed by: kevans
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D28064

diff: fix side_by_side after r363679

It's currently unclear to me how this could have worked previously; \n here
is not a literal newline but actual '\' 'n', and was getting passed to the
underlying regex engine as such. regex(3) does not translate this to a
newline, and this became an error because we don't really allow escaping
of arbitrary ordinary characters anymore.

Run the pattern strings through printf to make sure we're dealing with real
newlines before passing them through to atf_check, which ultimately feeds
them directly to regcomp(3).

This fix is different than that will be needed for sed, in that this is the
proper way to inject newlines into search strings as long as regex(3)
won't combine \ + n as folks might expect.

Reported by: Jenkins via lwhsu
MFC after: 1 week

diff: restore compatibility with GNU diff regarding -N option

When -N is used the missing files are treated as empty.

PR: 233402
Submitted by: Fehmi Noyan Isi <fnoyanisi@yahoo.com>
Reported by: Roman Neuhauser <roman@sigpipe.cz>
MFC after: 3 days
Differential Revision: D25081

Restore compatibility with GNU diff regarding --label

Various options to "diff(1)" show filenames, and traditionally make use of the
"--label" parameter, if set.

Restore this behaviour in BSD diff.

While here add a regression test

PR: 244533
Submitted by: Jamie Landeg-Jones <jamie@catflap.org>
MFC after: 3 days

diff(1): don't reject specifying the same format multiple times

This may happen, for instance, if one happens to have an alias of diff to
diff -up and attempts to specify the amount of context on top of that.

Aliases like this may cause other problems, but if they're really not ever
generating non-unified diffs then we should at least not break that
use-case.

In addition, we'll now pick up a format mismatch if -p is specified with
!contextual && !unified && !unset.

Fix up a small trailing whitespace nit in the tests while we're here, and
add tests to make sure that we can double up all the formatting options.

Reported by: jbeich
MFC after: 3 days

diff(1): reject conflicting formatting options

This matches GNU diff(1) behavior and, more importantly, eliminates any
source of confusion if multiple formatting options are specified.

Note that the committed diff differs slightly from the submitted: I've
modified it so that we initialize diff_format to something that isn't an
accepted format option so that we can also reject --normal -c and -c
--normal, which would've otherwise been accepted because the default was
--normal. After option parsing we default it to D_NORMAL if it's still
unset.

PR: 243975
Submitted by: fehmi noyan isi
MFC after: 1 week

diff: fix segfault with --tabsize and no/malformed argument

--tabsize was previously listed as optional_argument, but didn't account for
the optionality of it in the argument handling. This is irrelevant -- the
manpage doesn't indicate that the argument is optional, and indeed there's
no clear interpretation of omitting the argument because there's no other
side effect of --tabsize.

The "malformed" argument part of the header on this message is simply
referring to usage like this:

% diff --tabsize 4 A B

With an optional_argument, the argument must be attached to the parameter
directly (e.g. --tabsize=4), so the argument is effectively NULL with the
above invocation as if no argument had been passed.

PR: 243974
Submitted by: fehmi noyan isi <fnoyanisi yahoo com> (diff.c portion)
MFC after: 3 days

diff: implement -y (--side-by-side) along with -W and --suppress-common-lines

PR: 219933
Submitted by: fehmi noyan isi <fnoyanisi@yahoo.com>
MFC after: 3 weeks

diff(1): Implement -B/--ignore-blank-lines

As noted by cem in r338035, coccinelle invokes diff(1) with the -B flag.
This was not previously implemented here, so one was forced to create a link
for GNU diff to /usr/local/bin/diff

Implement the -B flag and add some primitive tests for it. It is implemented
in the same fashion that -I is implemented; each chunk's lines are scanned,
and if a non-blank line is encountered then the chunk will be output.
Otherwise, it's skipped.

MFC after: 2 weeks

Improve --strip-trailing-cr handling:

- Advance ctold for f1 and ctnew for f2
- ungetc() if the character is unexpected
- Don't break early when we hit the combination on one side

PR: 230049
Reported by: maskray <emacsray gmail com>
Reviewed by: bapt, maskray
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D16451

Add tests that exercise -q, like -rq and add tests that test -q like -Nrq

MFC after: 2 months
MFC with: r321076, r321077, r321078

Add some tests for brief (--brief/-q) format

MFC after: 2 months
MFC with: r321076

Add some testcases for `diff --side-by-side` support

These are were created proactively, in anticipation of the support being
fully implemented sometime in the future.

The tests currently fail on ^/head@r319845, however. Expect them to fail.

PR: 219933
Tested with: gdiff

Implement a basic --changed-group-format

etcupdate(8) requires that option, while GNU diff supports many more variation
of that options, their behaviour beside the simple verion implemented here are
quite inconsistent as such I do not plan to implement those.

The only special keyword supported by this implementation are: %< and %>
%= is not implemented as the documentation of GNU diff says: common lines, but
it actually when tested print the changes from the first file

Add a regression test for diff -D

diff: Show nanoseconds in -u/-c header line.

Show nanoseconds in the -u/-c header line.

The present portability conditionals cannot handle the POSIX standard
st_mtim, so remove them and unconditionally use st_mtim.

PR: 218018
Reported by: jbeich
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D10145

diff: Fix mtime of file1 in -u/-c header line.

PR: 218018
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D10140

Rename tests from <foo> to <foo>_test to match the FreeBSD test suite
naming scheme

usr.bin/diff/diff_test was renamed to usr.bin/diff/netbsd_diff_test
to avoid collisions with the renamed FreeBSD test.

MFC after: 1 week
Sponsored by: Dell EMC Isilon