Copy stable/10@r272459 to releng/10.1 as part of
the 10.1-RELEASE process.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Add libiconv based versions of *c16*() and *c32*().

I initially thought wchar_t was locale independent, but this seems to be
only the case on Linux. This means that we cannot depend on the *wc*()
routines to implement *c16*() and *c32*(). Instead, use the Citrus
libiconv that is part of libc.

I'll see if there is anything I can do to make the existing functions
somewhat useful in case the system is built without libiconv in the
nearby future. If not, I'll simply remove the broken implementations.

Reviewed by: jilles, gabor

Add missing #includes, to keep Clang silent.

Add <uchar.h>.

The <uchar.h> header, part of C11, adds a small number of utility
functions for 16/32-bit "universal" characters, which may or may not be
UTF-16/32. As our wchar_t is already ISO 10646, simply add light-weight
wrappers around wcrtomb() and mbrtowc().

While there, also add (non-yet-standard) _l functions, similar to the
ones we already have for the other locale-dependent functions.

Reviewed by: theraven

Remove regression tests for the obsolete rune interface.

Switch over to a different, more flexible test output protocol that's
understood by Perl's Test::Harness module and prove(1) commands.

Update README to describe the new protocol. The work's broken down into
two main sets of changes.

First, update the existing test programs (shell scripts and C programs)
to produce output in the ok/not ok format, and to, where possible, also
produce a header describing the number of tests that are expected to be
run.

Second, provide the .t files that actually run the tests. In some cases
these are copies of, or very similar too, scripts that already existed.
I've kept the old scripts around so that it's possible to verify that
behaviour under this new system (in terms of whether or not a test fails)
is identical to the behaviour under the old system.

Add a TODO file.

Add test programs for mbsnrtowcs() and wcsnrtombs().

Oops, test error behaviour of wctrans(), not wctype().

Test accumulation of partial multibyte characters in conversion
state objects.

Reset internal conversion state after triggering conversion errors;
the state is undefined according to the standards, and our undefined
behaviour has changed.

Add regression tests for sgetrune() and sputrune().

Regression tests for wctype(), iswctype(), wctrans(), towctrans().

Test that if the buffer argument (`s') to wcrtomb() is NULL then
the value of the supplied wide character is ignored and L'\0' is used
instead. Remove incorrect comments about "internal buffer" since wcrtomb()
does not have one (wctomb() does).

Add test cases for btowc() and wctob() in multibyte locales.

Test for the zero-length buffer bug corrected in mbrtowc.c revision 1.3.

Add regression tests for mblen(), mbtowc(), wctomb(), mbstowcs() and
wcstombs(). These tests have already found two libc bugs.

Add missing #includes of <stdio.h>. Correct a few typos in comments.

Simplify by using inference rules properly.

Add tests for btowc(), mbrlen(), mbsrtowcs() and wcsrtombs().

Add regression tests for mbrtowc(3) and wcrtomb(3).