uniq: Error out if writing to the output failed.

MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D43407

uniq: Clean up and test obsolete options.

MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D43402

uniq: Fix interactive use.

Output a line as soon as it is possible to determine that it will have
to be output. For the basic case, this means output each line as it is
read unless it is identical to the previous one. For the -d case, it
means output the first instance as soon as the second is read, unless
the -c option was also given. The -D and -u cases were already fine.

Add test cases for interactive use with no options and with -d.

Explicitly ignore -d when -D is also specified.

MFC after: 1 week
Sponsored by: Klara, Inc.
Reviewed by: rew, kevans
Differential Revision: https://reviews.freebsd.org/D43382

Remove copyright strings ifdef'd out

We've ifdef'd out the copyright strings for some time now. Go ahead and
remove the ifdefs. Plus whatever other detritis was left over from other
recent removals. These copyright strings are present in the comments and
are largely from CSRG's attempt at adding their copyright to every
binary file (which modern interpretations of the license doesn't
require).

Sponsored by: Netflix

usr.bin: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

Purge more stray embedded $FreeBSD$ strings

These do not use __FBSDID but instead use bare char arrays.

Reviewed by: imp, emaste
Differential Revision: https://reviews.freebsd.org/D41957

uniq(1): use strtonum to parse options

Previously strtol was used and the result was directly cast to an int
without checking for an overflow. Use strtonum instead since it is
safer and tells us what went wrong.

Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/643

Support --all-repeats in uniq(1) for compatibility with gnu coreutils.

This adds a new -D/--all-repeats option to uniq(1), which outputs each copy
of any repeated lines (as opposed to a single copy of a repeated line). You
can specify a separator option to output a blank line before or after each
group of repeated lines. This adds compatibility with the GNU coreutils
version of uniq(1).

This change also re-groups the -c, -d, -D, -u options in the usage display
and man page to indicate that they are mutally exclusive of each other. This
matches the posix/opengroup definition of uniq(1) command line args. Note
that this change does NOT actually enforce the mutual exclusion in the code,
for now, it simply documents that the arguments should be considered
exclusive with each other.

Differential Revision: https://reviews.freebsd.org/D22262

capsicum: use a new capsicum helpers in tools

Use caph_{rights,ioctls,fcntls}_limit to simplify the code.

Use capsicum helpers to cache NLS data.

Convert `cap_enter() < 0 && errno != ENOSYS` to `caph_enter() < 0`.

No functional change intended.

uniq(1): Add some long options

These match GNU uniq(1) where appropriate for compatibility's sake.

While here, re-sort options alphabetically by the short-option.

MFC after: 1 month

General further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

uniq: allow -c to be used with -d or -u

Bring in some bits from NetBSD and lift the restriction in uniq(1) that
-c cannot be used with the -d and -u options. This restriction seems
unnecessary and is supported at least by GNU, OpenBSD, and NetBSD. Lift
the restriction and simplify the show() logic a little bit to maintain
functionality when -c is provided with -d/-u.

Also with this change, -d and -u are now actually a mutually exclusive,
albeit valid, combination. Given that they both indicate opposite
behavior, uniq(1) will no longer output anything if both -d and -u are
supplied. This is in line with NetBSD as well as GNU.

Adjust the man page and usage() to reflect that -c is its own standalone
option.

PR: 200553
Submitted by: Kyle Evans <kevans91@ksu.edu>
Reviewed by: cem, emaste
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D10694

Renumber copyright clause 4

Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96

Remove usage of _WITH_GETLINE from usr.bin

Update most userspace consumers of capability.h to use capsicum.h instead.

auditdistd is not updated as I will make the change upstream and then do a
vendor import sometime in the next week or two.

MFC after: 3 weeks

Change the cap_rights_t type from uint64_t to a structure that we can extend
in the future in a backward compatible (API and ABI) way.

The cap_rights_t represents capability rights. We used to use one bit to
represent one right, but we are running out of spare bits. Currently the new
structure provides place for 114 rights (so 50 more than the previous
cap_rights_t), but it is possible to grow the structure to hold at least 285
rights, although we can make it even larger if 285 rights won't be enough.

The structure definition looks like this:

struct cap_rights {
uint64_t cr_rights[CAP_RIGHTS_VERSION + 2];
};

The initial CAP_RIGHTS_VERSION is 0.

The top two bits in the first element of the cr_rights[] array contain total
number of elements in the array - 2. This means if those two bits are equal to
0, we have 2 array elements.

The top two bits in all remaining array elements should be 0.
The next five bits in all array elements contain array index. Only one bit is
used and bit position in this five-bits range defines array index. This means
there can be at most five array elements in the future.

To define new right the CAPRIGHT() macro must be used. The macro takes two
arguments - an array index and a bit to set, eg.

#define CAP_PDKILL CAPRIGHT(1, 0x0000000000000800ULL)

We still support aliases that combine few rights, but the rights have to belong
to the same array element, eg:

#define CAP_LOOKUP CAPRIGHT(0, 0x0000000000000400ULL)
#define CAP_FCHMOD CAPRIGHT(0, 0x0000000000002000ULL)

#define CAP_FCHMODAT (CAP_FCHMOD | CAP_LOOKUP)

There is new API to manage the new cap_rights_t structure:

cap_rights_t *cap_rights_init(cap_rights_t *rights, ...);
void cap_rights_set(cap_rights_t *rights, ...);
void cap_rights_clear(cap_rights_t *rights, ...);
bool cap_rights_is_set(const cap_rights_t *rights, ...);

bool cap_rights_is_valid(const cap_rights_t *rights);
void cap_rights_merge(cap_rights_t *dst, const cap_rights_t *src);
void cap_rights_remove(cap_rights_t *dst, const cap_rights_t *src);
bool cap_rights_contains(const cap_rights_t *big, const cap_rights_t *little);

Capability rights to the cap_rights_init(), cap_rights_set(),
cap_rights_clear() and cap_rights_is_set() functions are provided by
separating them with commas, eg:

cap_rights_t rights;

cap_rights_init(&rights, CAP_READ, CAP_WRITE, CAP_FSTAT);

There is no need to terminate the list of rights, as those functions are
actually macros that take care of the termination, eg:

#define cap_rights_set(rights, ...) \
__cap_rights_set((rights), __VA_ARGS__, 0ULL)
void __cap_rights_set(cap_rights_t *rights, ...);

Thanks to using one bit as an array index we can assert in those functions that
there are no two rights belonging to different array elements provided
together. For example this is illegal and will be detected, because CAP_LOOKUP
belongs to element 0 and CAP_PDKILL to element 1:

cap_rights_init(&rights, CAP_LOOKUP | CAP_PDKILL);

Providing several rights that belongs to the same array's element this way is
correct, but is not advised. It should only be used for aliases definition.

This commit also breaks compatibility with some existing Capsicum system calls,
but I see no other way to do that. This should be fine as Capsicum is still
experimental and this change is not going to 9.x.

Sponsored by: The FreeBSD Foundation

Close uniq(1) in the capability mode sandbox and limit descriptors using
capability rights.

Add missing static keywords to uniq(1)

Remove the advertising clause from UCB copyrighted files in usr.bin. This
is in accordance with the information provided at
ftp://ftp.cs.berkeley.edu/pub/4bsd/README.Impt.License.Change

Also add $FreeBSD$ to a few files to keep svn happy.

Discussed with: imp, rwatson

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

Add SIZE_MAX overflow check

1) Reimplement (differently) unlimited line length restricted in prev.
commit.

2) Honor missing the very last \n (if absent) on output.

Remove vestiges of old %-format which prevents build on amd64

1) Rewrite input processing to not exit with error on the first EILSEQ found
in the input data but fallback to "binary equal" check instead.

POSIX says: "The input file shall be a text file", nothing more,
so the text file with illegal sequence is valid input.
BTW, GNU sort does not fails on EILSEQ too.

2) Speedup input processing a bit in complex cases like skipping fields,
chars or ignore case.

3) Enforce the implied LINE_MAX limit (from POSIX definition of "text file"
and POSIX uniq(1) description).

Sync getline() with comm(1):

- Prevent overflowing of the buffer length variable in getline() by
limiting its maximum value.
- Exit if reallocf(3) fails in getline(). Failure was silently
considered as end-of-file.

Reviewed by: ghelmer
Approved by: trasz (mentor)

- Import the HEAD csup code which is the basis for the cvsmode work.

Fix truncation of lines at LINE_MAX characters by dynamically
extending line buffers.

PR: bin/76578

Fix typo.

o) Correct for missing whitespace.
o) We don't need to check if ifp == stdin to give the filename, since we already
know that ifn will be "stdin" if it is.

Fix confusing misindentation of a closing-brace. (It goes with the switch, not
with the while.)

Remove a bogus check that caused empty lines not to be counted when the
-c option was given.

Noticed by: sf

Add support for multibyte characters.

ANSIify function definitions.
Add some constness to avoid some warnings.
Remove use register keyword.
Deal with missing/unneeded extern/prototypes.
Some minor type changes/casts to avoid warnings.

Reviewed by: md5

Use err instead of errx when malloc fails. "malloc" is not a helpful
error message.

Skip fields in the manner required by POSIX, and the way V7 did it.

MFC after: 1 week

Newline characters should not participate in line comparisons. Only apparent
when -s is used or the last line of the file is missing a newline.
Noticed by the textutils test suite.

MFC after: 1 week

Back out rev 1.19 because

1) It breaks uniq for real life languages when "substitute" directive used in
the collating table.
2) It breaks uniq usage in tool chain with other localized utilities which
use collate.
3) To follow LC_COLLATE it is directly allowed for uniq
by POSIX P1003.1 Draft7 (7.3.2). It means that rev 1.19 gains no additional
POSIX conformance.

Compare lines with strcmp(), not strcoll(). We are interested only in
equality, not ordering.

Sync usage() with manual page synopsis.

Accept an input file name of "-" to mean standard input, as required by
P1003.2.

Fields should be separated by <blank>s, not <space>s according to P1003.2.

Don't bother trying to handle "-" arguments ourselves, getopt(3) already
does this for us.

Use LC_ALL to pick collate

Noticed by: tjr

remove __P

Warns cleanups. Add FreeBSD ID to Makefile.

fix fatal typo

toupper -> tolower to match changed behaviour of new grep case fold

Cosmetique: use standard prototypes scheme
Back out prev. change: toupper is more compatible with sort -f

toupper->tolower to match what strcasecmp does

Use strcoll to provide the same results as sort and comm
Use LINE_MAX for max line size (as comm does)

$Id$ -> $FreeBSD$

Localize it

Teach comm(1) and uniq(1) about an option for case-insensitive work.

PR: 3042
Submitted by: graphix@iastate.edu (Kent Vander Velden)

Use err(3) instead of local redefinition.

compare return value from getopt against -1 rather than EOF, per the final
posix standard on the topic.

Import some parts of CSRG 4.4BSD-Lite2 usr.bin sources to fix tree build.

BSD 4.4 Lite Usr.bin Sources