Revert r330897:

This was intended to be a non-functional change. It wasn't. The commit
message was thus wrong. In addition it broke arm, and merged crypto
related code.

Revert with prejudice.

This revert skips files touched in r316370 since that commit was since
MFCed. This revert also skips files that require $FreeBSD$ property
changes.

Thank you to those who helped me get out of this mess including but not
limited to gonzo, kevans, rgrimes.

Requested by: gjb (re)

Partial merge of the SPDX changes

These changes are incomplete but are making it difficult
to determine what other changes can/should be merged.

No objections from: pfg

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Improve collation string and locales support

Merge collation support from Illumos and DragonflyBSD.

Locales are now generated with the new localedef(1) tool from CLDR POSIX files.
The generated files are now identified as "BSD 1.0" format.

The libc now only read "BSD 1.0" locales definitions, all other version will be
set to "C"
The localedef(1) tool has been imported from Illumos and modified to use tree(3)
instead of the CDDL avl(3)
A set of tool created by edwin@ and extended by marino@ for dragonfly has been
added to be able to generate locales and the Makefiles from the vanilla CLDR
unicode databases + a universal UTF-8 charmap (by marino@)
Update the locales to unicode v27
Given our regex(3) does not support multibyte (yet) it has been forced to always
use locale C
Remove now unused colldef(1) and mklocale(1)
Finish implementing the numeric BSD extension for ctypes
The number of supported locales has grown from 175 to 250 locales. Among the new
locales: 6 Arabic locales (AE EG JO MA QA SA), Different variations of spanish
locales.
Added new 3 components locales for mn_Cyrl_MN, sr_Cyrl_RS sr_Latn_RS,
zh_Hans_CN, zh_Hant_HK and zh_Hant_TW. Some aliases has been for 2 components
version when possible.

Thanks: Garrett D'Amore (Illumos) who made sure all his work was done under
BSD license!, Edwin Groothuis (edwin@) for the work he made on tools to be able
to generate locales definition usable in freebsd sources out of vanilla CLDR
definitions, John Marino (DragonflyBSD) who first merge the Illumos work into
Dragonfly and spent hours tracking down bugs.

Use ANSI C prototypes. Eliminates -Wold-style-definition warnings.

The problem is: currently our single byte ctype(3) functions are broken
for wide characters locales in the argument range >= 0x80 - they may
return false positives.

Example 1: for UTF-8 locale we currently have:
iswspace(0xA0)==1 and isspace(0xA0)==1
(because iswspace() and isspace() are the same code)
but must have
iswspace(0xA0)==1 and isspace(0xA0)==0
(because there is no such character and all others in the range
0x80..0xff for the UTF-8 locale, it keeps ASCII only in the single byte
range because our internal wchar_t representation for UTF-8 is UCS-4).

Example 2: for all wide character locales isalpha(arg) when arg > 0xFF may
return false positives (must be 0).
(because iswalpha() and isalpha() are the same code)

This change address this issue separating single byte and wide ctype
and also fix iswascii() (currently iswascii() is broken for
arguments > 0xFF).
This change is 100% binary compatible with old binaries.

Reviewied by: i18n@

Per Regents of the University of Calfornia letter, remove advertising
clause.

# If I've done so improperly on a file, please let me know.

Cosmetic - remove unneded brackets and #undef

Fix the style of the SCM ID's.
I believe have made all of libc .c's as consistent as possible.

There is a problem in that one cannot use ctype.h at the same time as parts
of the C++ stdlib. Our ctype.h uses symbols of the form _<X> to denote the
various character classes. Our ctype.h also extends the usual ctype.h
offering by adding the "_T" (special) class. Problem is parts of the STL
also use the symbol "_T" as its parameterized type. These two uses are
incompatible.

Thus change the form of the symbols used in ctype to something that fixes
the current problem and is less likely to cause conflicts in the future.

Requested by: Tomoaki NISHIYAMA <tomoaki@biol.s.u-tokyo.ac.jp>
Ok'ed by: JKH

Back up following macros by functions: ishexnumber, isideogram, isnumber,
isphonogram, isrune, isspecial. Fix ordering.

Reviewed by: bde

Use revived __maskrune for digittoint
Minor formatting

__maskrune --> __istype

Submitted by: Sin'ichiro MIYATANI / Phase One, Inc <siu@phaseone.co.jp>
Basic support for the Shift JIS encoding of japanese.
(and one tiny typo fixed in a comment)

Reviewed by: ache and wollman (long ago)

isctype.c:
o The tolower() and toupper() functions duplicated too much code
and were out of date (surprise). This didn't matter because
it was difficult to call them.
o Change formatting to be more like that in <ctype.h> (with
extra parentheses as in the macros). Perhaps this file should
be machine generated or everything should be handled like
__tolower() so that no code is repeated.

nomacros.c:
o Instead of looking at _USE_CTYPE_INLINE_ to see what <ctype.h>
has done, set _EXTERNALIZE_CTYPE_INLINES_ to tell <ctype.h>
what to do, so that we don't have anything left to do. Note
that code is now generated even if inlines are used by default.
This allows users to switch to non-inline versions.

This commit was generated by cvs2svn to compensate for changes in r1573,
which included commits to RCS files with non-trunk default branches.

BSD 4.4 Lite Lib Sources