#
341631 |
|
06-Dec-2018 |
yuripv |
MFC r339827: localedef: define characters in "space" class also as "print", except for the known conflicts ("control" characters can't be "print"able). POSIX doesn't explicitly forbid this, and actually includes <space> character in "print".
PR: 225692 Reviewed by: bapt, cem (previous version), pfg (previous version) Differential Revision: https://reviews.freebsd.org/D17467
|
#
315224 |
|
14-Mar-2017 |
pfg |
MFC r314974, r315006: localedef(1): Fix small coverity issues.
- Operands don't affect result (CONSTANT_EXPRESSION_RESULT) - Buffer not null terminated (BUFFER_SIZE_WARNING)
CID: 1338557, 1338565
Obtained from: illumos
|
#
308330 |
|
05-Nov-2016 |
bapt |
MFC r306782-r306783
r306782: localedef: Fix ctype dump (fixed wide spread errors)
This commit is from John Marino in dragonfly with the following commit log:
==== This was a CTYPE encoding error involving consecutive points of the same ctype. It was reported by myself to Illumos over a year ago but I was unsure if it was only happening on BSD. Given the cause, the bug is also present on Illumos.
Basically, if consecutive points were of the exact same ctype, they would be defined as a range regardless. For example, all of these would be considered equivalent:
<A> ... <C>, <H> (converts to <A> .. <H>) <A>, <B>, <H> (converts to <A> .. <H>) <A>, <J> ... <H> (converts to <A> .. <H>)
So all the points that shouldn't have been defined got "bridged" by the extreme points.
The effects were recently reported to FreeBSD on PR 213013. There are countless places were the ctype flags are misdefined, so this is a major fix that has to be MFC'd. ====
This reveals a bad change I did on the testsuite: while 0x07FF is a valid unicode it is not used yet (reserved for future use)
PR: 213013 Submitted by: marino@ Reported by: Kurtis Rader <krader@skepticism.us> Obtained from: Dragonfly MFC after: 1 month
r306783: localedef: Improve cc_list parsing
original commit log: ===== I had originally suspected the parsing of ctype definition files as being the source of the ctype flag mis-definitions, but it wasn't. In the process, I simplified the cc_list parsing so I'm committing the no-impact improvement separately. It removes some parsing redundancies and won't parse partial range definitions anymore. ====
Submitted by: marino Obtained from: Dragonfly MFC after: 1 month
|
#
302408 |
|
07-Jul-2016 |
gjb |
Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle. Prune svn:mergeinfo from the new branch, as nothing has been merged here.
Additional commits post-branch will follow.
Approved by: re (implicit) Sponsored by: The FreeBSD Foundation |
#
297057 |
|
20-Mar-2016 |
pfg |
localedef(1): minor sorting to match Illumos.
Illumos recently included space in 'print' class. We already had this but the code had slight sorting differences. Move it some lines up to reduce diffs with Illumos.
No functional change.
Reference: https://illumos.org/issues/5227
|
#
290517 |
|
07-Nov-2015 |
bapt |
Fix build of localedef(1) on arm where wchar_t is an unsigned int
|
#
290494 |
|
07-Nov-2015 |
bapt |
Improve collation string and locales support
Merge collation support from Illumos and DragonflyBSD.
Locales are now generated with the new localedef(1) tool from CLDR POSIX files. The generated files are now identified as "BSD 1.0" format.
The libc now only read "BSD 1.0" locales definitions, all other version will be set to "C" The localedef(1) tool has been imported from Illumos and modified to use tree(3) instead of the CDDL avl(3) A set of tool created by edwin@ and extended by marino@ for dragonfly has been added to be able to generate locales and the Makefiles from the vanilla CLDR unicode databases + a universal UTF-8 charmap (by marino@) Update the locales to unicode v27 Given our regex(3) does not support multibyte (yet) it has been forced to always use locale C Remove now unused colldef(1) and mklocale(1) Finish implementing the numeric BSD extension for ctypes The number of supported locales has grown from 175 to 250 locales. Among the new locales: 6 Arabic locales (AE EG JO MA QA SA), Different variations of spanish locales. Added new 3 components locales for mn_Cyrl_MN, sr_Cyrl_RS sr_Latn_RS, zh_Hans_CN, zh_Hant_HK and zh_Hant_TW. Some aliases has been for 2 components version when possible.
Thanks: Garrett D'Amore (Illumos) who made sure all his work was done under BSD license!, Edwin Groothuis (edwin@) for the work he made on tools to be able to generate locales definition usable in freebsd sources out of vanilla CLDR definitions, John Marino (DragonflyBSD) who first merge the Illumos work into Dragonfly and spent hours tracking down bugs.
|
#
289590 |
|
19-Oct-2015 |
bapt |
Actually only T_ISDIGIT should be flagged as _E4
|
#
289586 |
|
19-Oct-2015 |
bapt |
With regard to ctype, digits (e.g. 0 to 9) and xdigits (the 0 to 9 portion of hexidecimal numbers) are all considered "numbers". (Note that while all digits are numbers, not all numbers are digits).
Enhance localedef to automatically set the "number" characteristic when it encounters a digit or xdigit definition. This fixes malfunctionning isalnum(3)
Obtained from: DragonflyBSD
|
#
289264 |
|
13-Oct-2015 |
bapt |
eliminate need for "print" definition
By having space automatically classified as "print" type, we can eliminate the print section from ctype src files completely (they are just "graph" plus "<space>".
Obtained from: Dragonfly
|
#
289263 |
|
13-Oct-2015 |
bapt |
Commit log from Dragonfly:
FreeBSD extended ctypes to include numbers (e.g. isnumber()) but never actually implemented it. The isnumber() function was equivalent to the isdigit() function in every case.
Now that DragonFly's ctype source files have number definitions, the number ctype can finally be implemented. It's given a new flag _CTYPE_N. The isalnum() and iswalnum() functions have been changed to use this flag rather than the _CTYPE_D digit flag.
While isalnum(), isnumber(), and their wide equivalents now return different values in locale cases, the ishexnumber() and iswhexnumber() functions are unchanged. They are still aliases for isxdigit() and iswxdigit().
Also change ctype.h for isdigit and isxdigit to use sbistype like the other functions.
Obtained from: dragonfly
|
#
286484 |
|
08-Aug-2015 |
bapt |
Convert localedef(1) from avl to RB trees
|
#
286482 |
|
08-Aug-2015 |
bapt |
Prefer static generation of functions
|
#
286481 |
|
08-Aug-2015 |
bapt |
Convert ctype generation to Red Black tree
|
#
286432 |
|
07-Aug-2015 |
bapt |
Add localedef(1), a locale definition generator tool
The localedef tool can read entire (and unmodified) CLDR posix definition files, and generate all 6 LC categories: LC_COLLATE, LC_CTYPE, LC_TIME, LC_NUMERIC, LC_MONETARY and LC_MESSAGES.
This tool has a long history with Solaris. The Nexenta developers modified it to read CLDR files and created the much richer collation formats. The libc collation functions have to be modified to read the new format (called "BSD-1.0") and to handle the new data structures.
The result will be that locale-sensitive tools and functions will now properly sort multibyte and unicode strings.
Obtained from: Dragonfly
|