History log of /freebsd-10.0-release/lib/libc/iconv/bsd_iconv.c
Revision Date Author Comments
(<<< Hide modified files)
(Show modified files >>>)
# 259065 07-Dec-2013 gjb

- Copy stable/10 (r259064) to releng/10.0 as part of the
10.0-RELEASE cycle.
- Update __FreeBSD_version [1]
- Set branch name to -RC1

[1] 10.0-CURRENT __FreeBSD_version value ended at '55', so
start releng/10.0 at '100' so the branch is started with
a value ending in zero.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

# 258750 29-Nov-2013 gjb

MFC r258537, r258587:

r258537 (hrs):
Add ICONV_{GET,SET}_ILSEQ_INVALID iconvctl. GNU iconv returns
EILSEQ when there is an invalid character in the output codeset
while it is valid in the input. However, POSIX requires iconv()
to perform an implementation-defined conversion on the character.
So, Citrus iconv converts such a character to a special character
which means it is invalid in the output codeset.

This is not a problem in most cases but some software like libxml2
depends on GNU's behavior to determine if a character is output
as-is or another form such as a character entity (&#NNN;).

r258587 (peter):
Move the iconv wrapper source from libc_nonshared to libc/iconv so
that it is all in the one place again. Rename libc/iconv/iconv.c
to bsd_iconv.c. Compile the wrappers into libc.a so that
WITHOUT_DYNAMICROOT works again.

Approved by: re (kib)
Sponsored by: The FreeBSD Foundation


# 258587 25-Nov-2013 peter

Move the iconv wrapper source from libc_nonshared to libc/iconv so that
it is all in the one place again. Rename libc/iconv/iconv.c to
bsd_iconv.c. Compile the wrappers into libc.a so that WITHOUT_DYNAMICROOT
works again.

Discussed with: kib (and partly stolen from his patch)


# 258537 24-Nov-2013 hrs

Add ICONV_{GET,SET}_ILSEQ_INVALID iconvctl. GNU iconv returns EILSEQ
when there is an invalid character in the output codeset while it is
valid in the input. However, POSIX requires iconv() to perform an
implementation-defined conversion on the character. So, Citrus iconv converts
such a character to a special character which means it is invalid in the
output codeset.

This is not a problem in most cases but some software like libxml2 depends
on GNU's behavior to determine if a character is output as-is or another form
such as a character entity (&#NNN;).


# 258283 17-Nov-2013 peter

Attempt to move the POSIX iconv* symbols out of runtime linker space.
FreeBSD systems usually implemented this as a third party module and
our implementation hasn't played as nicely with the old way as it could
have.

To that end:
* Rename the iconv* symbols in libc.so.7 to have a __bsd_ prefix.
* Provide .symver compatability with existing 10.x+ binaries that
referenced the iconv symbols. All existing binaries should work.
* Like on Linux/glibc systems, add a libc_nonshared.a to the ldscript
at /usr/lib/libc.so.
* Move the "iconv*" wrapper symbols to libc_nonshared.a

This should solve the runtime ambiguity about which symbols resolve
to where. If you compile against the iconv in libc, your runtime
dependencies will be unambiguous.

Old 9.x libraries and binaries will always resolve against their
libiconv.so.3 like they did on 9.x. They won't resolve against libc.

Old 10.x binaries will be satisified by the .symver helpers.

This should allow ports to selectively compile against the libiconv
port if needed and it should behave without ambiguity now.

Discussed with: kib


# 257583 03-Nov-2013 peter

Remove the WITH_LIBICONV_COMPAT hack that seems to do more harm than
good. This caused libc to spoof the ports libiconv namespace and
provide a colliding libiconv.so.3 to fool rtld. This should have
been removed some time ago.


# 254273 13-Aug-2013 peter

The iconv in libc did two things - implement the standard APIs, the GNU
extensions and also tried to be link time compatible with ports libiconv.
This splits that functionality and enables the parts that shouldn't
interfere with the port by default.

WITH_ICONV (now on by default) - adds iconv.h, iconv_open(3) etc.
WITH_LIBICONV_COMPAT (off by default) adds the libiconv_open etc API, linker
symbols and even a stub libiconv.so.3 that are good enough to be able
to 'pkg delete -f libiconv' on a running system and reasonably expect it
to work.

I have tortured many machines over the last few days to try and reduce
the possibilities of foot-shooting as much as I can. I've successfully
recompiled to enable and disable the libiconv_compat modes, ports that use
libiconv alongside system iconv etc. If you don't enable the
WITH_LIBICONV_COMPAT switch, they don't share symbol space.

This is an extension of behavior on other system. iconv(3) is a standard
libc interface and libiconv port expects to be able to run alongside it on
systems that have it.

Bumped osreldate.


# 252547 03-Jul-2013 peter

Replace the #define for "iconv" so it is for the function name instead of
a macro with parameters. Remove a __DECONST hack and add consts instead
for gnu libiconv API compatability. This makes it work with things like
devel/boost-libs that expects to use "iconv" as though it were a pointer.


# 252458 01-Jul-2013 peter

__weak_alias() doesn't exist on FreeBSD. Use __weak_reference();

Expose iconv functions as weak symbols as well as their internal
remapped #define names. This is necessary for autoconf compatability -
on Linux it appears that #include <iconv.h> isn't a link time
prerequisite for their version that's built into glibc.

Initialize the pthread rwlock. Note that upstream has three
separate locks. The file-local static lock appears intentional.

I'm using this as a ports-compatible compile-time substitute for
converters/libiconv on one of my personal machines.


# 250981 25-May-2013 ed

Make some tiny improvements to iconv_open().

- Remove an unneeded variable.
- Fix whitespace bugs.
- Fix typoes in comment.
- Improve string handling a bit. Don't handroll strstr() and don't
terminate a strdup()'ed string. Instead, simply strndup() the part we
need.


# 223296 19-Jun-2011 kan

Put a quick bandaid on internal citrus locking.

The code is not quite right still, but it programs from deadlocking
on themselves if one enables new citrus code by mistake.


# 219019 24-Feb-2011 gabor

Add the BSD-licensed Citrus iconv to the base system with default off
setting. It can be built by setting the WITH_ICONV knob. While this
knob is unset, the library part, the binaries, the header file and
the metadata files will not be built or installed so it makes no impact
on the system if left turned off.

This work is based on the iconv implementation in NetBSD but a great
number of improvements and feature additions have been included:

- Some utilities have been added. There is a conversion table generator,
which can compare conversion tables to reference data generated by
GNU libiconv. This helps ensuring conversion compatibility.
- UTF-16 surrogate support and some endianness issues have been fixed.
- The rather chaotic Makefiles to build metadata have been refactored
and cleaned up, now it is easy to read and it is also easier to add
support for new encodings.
- A bunch of new encodings and encoding aliases have been added.
- Support for 1->2, 1->3 and 1->4 mappings, which is needed for
transliterating with flying accents as GNU does, like "u.
- Lots of warnings have been fixed, the major part of the code is
now WARNS=6 clean.
- New section 1 and section 5 manual pages have been added.
- Some GNU-specific calls have been implemented:
iconvlist(), iconvctl(), iconv_canonicalize(), iconv_open_into()
- Support for GNU's //IGNORE suffix has been added.
- The "-" argument for stdin is now recognized in iconv(1) as per POSIX.
- The Big5 conversion module has been fixed.
- The iconv.h header files is supposed to be compatible with the
GNU version, i.e. sources should build with base iconv.h and
GNU libiconv. It also includes a macro magic to deal with the
char ** and const char ** incompatibility.
- GNU compatibility: "" or "char" means the current local
encoding in use
- Various cleanups and style(9) fixes.

Approved by: delphij (mentor)
Obtained from: The NetBSD Project
Sponsored by: Google Summer of Code 2009