Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Replace index() and rindex() calls with strchr() and strrchr().

The index() and rindex() functions were marked LEGACY in the 2001
revision of POSIX and were subsequently removed from the 2008 revision.
The strchr() and strrchr() functions are part of the C standard.

This makes the source code a lot more consistent, as most of these C
files also call into other str*() routines. In fact, about a dozen
already perform strchr() calls.

Build tr(1) with WARNS=6.

Remove the advertising clause from UCB copyrighted files in usr.bin. This
is in accordance with the information provided at
ftp://ftp.cs.berkeley.edu/pub/4bsd/README.Impt.License.Change

Also add $FreeBSD$ to a few files to keep svn happy.

Discussed with: imp, rwatson

tr: Fix '[=]=]' equivalence class.

A closing bracket immediately after '[=' should not be treated as special.

Different from the submitted patch, a string ending with '[=' does not cause
access beyond the terminating '\0'.

PR: bin/150384
Submitted by: Richard Lowe
MFC after: 2 weeks

Revert most part of 200420 as requested, as more review and polish is
needed.

Remove unneeded header includes from usr.bin/ except contributed code.

Tested with: make universe

o Remove duplicate includes.

Obtained from: Slava Semushin via NetBSD

tr(1) attempts to convert \n[n][n] sequences into octal digits, but doesn't
check to see that a given digit is actually an octal digit. This leads to
unusual consequences if passed in values like \9.

Reported by: Joseph Davison (OpenDarwin project)
MFC after: 1 week

Add support for multibyte characters. The challenge here was to use
data structures that scale better with large character sets, instead of
arrays indexed by character value:
- Sets of characters to delete/squeeze are stored in a new "cset" structure,
which is implemented as a splay tree of extents. This structure has the
ability to store character classes (ala wctype(3)), but this is not
currently fully utilized.
- Mappings between characters are stored in a new "cmap" structure, which
is also a splay tree.
- The parser no longer builds arrays containing all the characters in a
particular class; instead, next() determines them on-the-fly using
nextwctype(3).

Back out [:upper:] and [:lower:] classes sorting, it is not required
by POSIX and gains nothing with current code.

No functional changes, just code reorganization from prev. commit, it
makes one malloc unneeded, removes two bzero's and makes code more readable.

"Bright ideas comes only _after_ commits."

POSIX require complex processing of 'c-c' ranges: if one of the endpoints
is octal sequence, range is taken in the byte values order, for non-octal
endpoints range is taken in the sorted collation order.

Implement it.

1) Fix -C - it was broken since introduced, wrong array sorted
2) Fix last (repeated) char after [:class:], it was \0 in original code

POSIX requires 'c-c' must conform collate and be in collation order

This patch address two problems.

1st one is relatively minor: according our own manpage, upper and lower
classes must be sorted, but currently not.

2nd one is serious:
tr '[:lower:]' '[:upper:]'
(and vice versa) currently works only if upper and lower classes
have exact the same number of elements. When it is not true, like for
many ISO8859-x locales which have bigger amount of lowercase letters,
tr may do nasty things.

See this page
http://www.opengroup.org/onlinepubs/007908799/xcu/tr.html
for detailed description of desired tr behaviour in such cases.

Use err instead of errx when malloc fails. "malloc" is not a helpful
error message.

Improve parsing of character and equivalence classes:
[:*] and [=*] are parsed as `infinitely many repetitions of :' (or *)
instead of literal characters (SUSv3)

Don't treat the trailing ']' of an equivalence class expression as a
character in the set. tr -d '[=a=]' was deleting ]'s as well as a's.
Noticed by the textutils test suite.

Implement support for equivalence classes ([=e=]) when the mapping is
one-to-one (SUSv3)

remove __P

WARNS=2 fixes, use __FBSDID(), kill register keyword.

$Id$ -> $FreeBSD$

Use err(3) instead of local redefinition. Cosmetic in usage().

Cast char's to (u_char) before passing them to isctype() functions.

Fix a couple of sign-extension bugs.

Submitted by: serg@bcs1.bcs.zaporizhzhe.ua (Sergey Shkonda)

Updated to BSD4.4lite2. Fixes PR836. `echo abcd | tr a-d A-BC-D' now
works.

Fix broken charclass handling
Add setlocale LC_CTYPE

Remove declamations which <ctype.h> already does for us.

Added #include <ctype.h>

Fix print class mistype

This commit was generated by cvs2svn to compensate for changes in r1590,
which included commits to RCS files with non-trunk default branches.

BSD 4.4 Lite Usr.bin Sources