locale kit: allow creating a catalog without entry_ref

The catalogs are loaded from separate files, so there is no need to have
an app entry_ref to load them, just a MIME type is enough.

The implementation is a bit simplified: only the default catalog format
is allowed (unlike when loading from entry_ref, where extra catalog
formats can be added in add-ons).

Unrelated cleanup: remove unused code to load catalogs from attributes
of an application. We considered this when designing the locale kit, but
using resources or separate files works better.

Use this in Cortex, where some strings are in a static library, so they
don't have an associated executable or library or add-on to identify
them. The code in Cortex is not complete localization, several parts
should use StringForRate, BStringFormat, etc.

Change-Id: I09be22b1f50891250c4497c51e1db8dcee279140
Reviewed-on: https://review.haiku-os.org/c/haiku/+/3172
Reviewed-by: Kacper Kasper <kacperkasper@gmail.com>

Locale kit: Fix PVS 550, 551

Use BStackOrHeapArray instead of applying auto_ptr to array.

Change-Id: I204dcb97a1bc2d047783bc6015f86b36ca0cc15b
Reviewed-on: https://review.haiku-os.org/725
Reviewed-by: Adrien Destugues <pulkomandy@pulkomandy.tk>

Remove B_COMMON*_DIRECTORY and kCommon*Directory constants

* This does intentionally break source compatibility, so that a review
of concerned code is forced.
* Binary compatibility should be maintained in most cases. The values
of the constants for the writable directories are now used for the
writable system directories. The values for the non-writable
directories are mapped to "/boot/system/data/empty/...", an empty or
non-existent directory, so that they will simply be skipped in search
paths. Only code that explicitly expects to find something in a
B_COMMON_* directory, will fail.

Fix catalogs stored in resources
We used the hash of the language code as a key for identifying catalogs.
However, hash do collide, and in particular, "en" and "fi" have the same hash.
Identify catalog resources by name instead.

Adjust Locale Kit to no longer use syslog

Compilation and 64-bit fixes to libbe.so sources.

Fixed the usual issues - printf format strings, uint32 instead of
addr_t, etc. One thing that isn't so nice is several places where
BList is used to store (u)int32, these require a double cast to addr_t
then void* to silence a warning on x86_64.

Cleanup BCatalogAddOn.

* rename BCatalogAddOn to BCatalogData, since it doesn't represent an
add-on, but rather the catalog data provided by an add-on
* move BCatalogData out of Catalog.{h,cpp} into its own header and
implementation file
* drop BCatalogData::MarkForTranslation() methods, they're not needed
* drop BCatalog::GetNoAutoCollectString() methods, they're not being
used anywhere
* cleanup the B_TRANSLATE_... macros somewhat
* add versions of the B_TRANSLATE_MARK_... macros that are meant to be
used in void context (when the string isn't being used by the program,
just meant to be picked up by collectcatkeys).
* adjust several apps to use B_TRANSLATE_MARK_..._VOID where needed
* adjust users of BCatalogAddOn accordingly

More cleanup in locale kit, this time regarding namespaces.

* it's bad practice to do a 'using <namespace>' in a header, as that
is very likely to have unintended effects, so drop those from a couple
of private Locale headers
* adjust files all over the locale kit in order to fix the problems
(by explicitly importing the required classes in the implementation
files)

* Use const references instead of pointers for the read from/write to
attribute/resource method in locale kit catalogs
* Only load the embedded catalog if nothing else was found, so it can easily
be overridden
* Change the resource type to 'CADA' (CAtalog DAta) for embedded catalogs, and
use a hash of the language code as the resource ID. This allows multiple
languages to be stored in the same file and does not interfere with the user
storing his own BMessages as resources.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43057 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Coding style
* Stub out the unneeded code in the buildtool version of DefaultCatalog.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43022 a95241bf-73f2-0310-859d-f6bbb57e9c96

Rework the handling of catalog loading in locale kit :
Instead of computing the mime signature and giving this to the catalog system,
give an entry_ref instead. The default catalog add-on can thus look at the
right place when searching local catalogs (embedded as resources, or stored
as files next to the executable.
* This allows different versions of the same app to each have their own
catalog set,
* And also make the embedded/local catalog searching work for add-ons and
libs, instead it only worked for apps because of a getAppInfo call.

Fix cpufrequency to make use of it properly (that wouldhave worked without the
change, but nowit's mandatory, since loading a catlog by mimesignature is not
possible anymore).

Should fix #8037.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43021 a95241bf-73f2-0310-859d-f6bbb57e9c96

Try to load resource-embedded catalog from application. Should fix #8037.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@42950 a95241bf-73f2-0310-859d-f6bbb57e9c96

Closing #6721: avoid unescaping when linking catalogs together
* add SetRawString() to both implementations of DefaultCatalog which sets
the translated string as given
* use SetRawString() in both (standard- and build-)versions of linkcatkeys


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@41167 a95241bf-73f2-0310-859d-f6bbb57e9c96

* some cleanup in BLocaleRoster

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39018 a95241bf-73f2-0310-859d-f6bbb57e9c96

Largish cleanup sweep concerning the Locale Kit (sorry it got so big):
* refactored private/mutable stuff out of LocaleRoster into MutableLocaleRoster
* moved management of Locale/Time settings file and broadcasting of any changes
out of preflets and into MutableLocaleRoster
* added proper sorting to the listviews of the Locale preflet
* the Time preflet no longer overlaps long timezone names into the actual time
* several fixes with respect to leaking ICU objects, esp. in BCountry
* the locale roster no longer passes out references to its own BCountry object,
but uses copies, instead - this makes locking superfluous, as the clients'
BCountry objects can no longer be changed by the setting a new default
country in the locale roster
* removed pretty useless POSIX-style symbol fetching from BCountry - if we
need that at all, it should live in the dedicated formatter classes
* adjusted readonlybootprompt, dstcheck and Deskbar to the changed Locale API
* refactored existing Time-formatter into TimeUnitFormat and DurationFormat
(the latter of which is now used by AboutSystem)
* added stubs for Date, DateTime and Time formatters
* lots of coding style fixes throughout the Locale Kit and the Locale and Time
preflets
This will probably break most external apps making use of the Locale Kit - it
does break WebPositive.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37831 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Implement a function to get all the available languages for a given application
* use it in readonlybootprompt instead of previous workaround code.
This makes the readonlybootprompt faster and lighter as it doesn't open and parse all the catalogs anymore, but only checks for their existence.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36276 a95241bf-73f2-0310-859d-f6bbb57e9c96

* LocaleRoster.h was not self-containing.
* Moved the locale stuff to the data directory as well (some parts were in /etc).
* The DefaultCatalog will now also scan the user directory for catalogs.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33983 a95241bf-73f2-0310-859d-f6bbb57e9c96

Cleanup of the locale kit tools :
-Jamrule for collectcatkeys no longer print all the strings, only a message when it can't parse one (happens for TR(variable) basically)
-Added fingerprint check in the plaintext catalog ReadFromFile. However, the adler checksum is different each time the catalog is loaded because it relies on the string being iterated always in the same order, but this is not always the case with an HashMap ! Some rethinking is needed, so disabled the check for now so it does not breaks the build
-Some try to debug the bluetooth preflet localization. Still buggy, but I wanted to commit all this mess before I break everything up again.
-Also sorted the fr.catkeys files to be in the same order as the autogenerated en.catkeys (this is useless but makes them easier to check) and updated their fingerprint even if they are still not checked.
-Miscelaneous style fixes, small bugfixes, more error checking and error messages saying where they come from.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33322 a95241bf-73f2-0310-859d-f6bbb57e9c96

* reintegrated gsoc-locale-kit branch into trunk - there's more
work to do, but it's about time to give this code more exposure.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33176 a95241bf-73f2-0310-859d-f6bbb57e9c96

err, fix for my fix:
* its __GNUC__, not __GCC__, tsk!

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30572 a95241bf-73f2-0310-859d-f6bbb57e9c96

fix gcc2 build:
* __gnu_cxx namespace is unknown to gcc2

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30571 a95241bf-73f2-0310-859d-f6bbb57e9c96

Move using directives into the cpp files so they don't leak into apps making use of the locale kit.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30570 a95241bf-73f2-0310-859d-f6bbb57e9c96

* followed advice by Rene to explicitly import the used std-classes instead
of simply importing the whole namespace - thanks for proofreading!


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30568 a95241bf-73f2-0310-859d-f6bbb57e9c96

* applied several fixes to make locale kit buildable with gcc4, too
* actually, all those fixes were already contained in PulkoMandys patch,
but I just missed that :-/


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30566 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Copied imported OpenTracker Locale Kit files from the vendor branch
into their new homes (at least for now, might need some adjustment).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30540 a95241bf-73f2-0310-859d-f6bbb57e9c96

Remove B_COMMON*_DIRECTORY and kCommon*Directory constants

* This does intentionally break source compatibility, so that a review
of concerned code is forced.
* Binary compatibility should be maintained in most cases. The values
of the constants for the writable directories are now used for the
writable system directories. The values for the non-writable
directories are mapped to "/boot/system/data/empty/...", an empty or
non-existent directory, so that they will simply be skipped in search
paths. Only code that explicitly expects to find something in a
B_COMMON_* directory, will fail.

Fix catalogs stored in resources
We used the hash of the language code as a key for identifying catalogs.
However, hash do collide, and in particular, "en" and "fi" have the same hash.
Identify catalog resources by name instead.

Adjust Locale Kit to no longer use syslog

Compilation and 64-bit fixes to libbe.so sources.

Fixed the usual issues - printf format strings, uint32 instead of
addr_t, etc. One thing that isn't so nice is several places where
BList is used to store (u)int32, these require a double cast to addr_t
then void* to silence a warning on x86_64.

Cleanup BCatalogAddOn.

* rename BCatalogAddOn to BCatalogData, since it doesn't represent an
add-on, but rather the catalog data provided by an add-on
* move BCatalogData out of Catalog.{h,cpp} into its own header and
implementation file
* drop BCatalogData::MarkForTranslation() methods, they're not needed
* drop BCatalog::GetNoAutoCollectString() methods, they're not being
used anywhere
* cleanup the B_TRANSLATE_... macros somewhat
* add versions of the B_TRANSLATE_MARK_... macros that are meant to be
used in void context (when the string isn't being used by the program,
just meant to be picked up by collectcatkeys).
* adjust several apps to use B_TRANSLATE_MARK_..._VOID where needed
* adjust users of BCatalogAddOn accordingly

More cleanup in locale kit, this time regarding namespaces.

* it's bad practice to do a 'using <namespace>' in a header, as that
is very likely to have unintended effects, so drop those from a couple
of private Locale headers
* adjust files all over the locale kit in order to fix the problems
(by explicitly importing the required classes in the implementation
files)

* Use const references instead of pointers for the read from/write to
attribute/resource method in locale kit catalogs
* Only load the embedded catalog if nothing else was found, so it can easily
be overridden
* Change the resource type to 'CADA' (CAtalog DAta) for embedded catalogs, and
use a hash of the language code as the resource ID. This allows multiple
languages to be stored in the same file and does not interfere with the user
storing his own BMessages as resources.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43057 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Coding style
* Stub out the unneeded code in the buildtool version of DefaultCatalog.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43022 a95241bf-73f2-0310-859d-f6bbb57e9c96

Rework the handling of catalog loading in locale kit :
Instead of computing the mime signature and giving this to the catalog system,
give an entry_ref instead. The default catalog add-on can thus look at the
right place when searching local catalogs (embedded as resources, or stored
as files next to the executable.
* This allows different versions of the same app to each have their own
catalog set,
* And also make the embedded/local catalog searching work for add-ons and
libs, instead it only worked for apps because of a getAppInfo call.

Fix cpufrequency to make use of it properly (that wouldhave worked without the
change, but nowit's mandatory, since loading a catlog by mimesignature is not
possible anymore).

Should fix #8037.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43021 a95241bf-73f2-0310-859d-f6bbb57e9c96

Try to load resource-embedded catalog from application. Should fix #8037.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@42950 a95241bf-73f2-0310-859d-f6bbb57e9c96

Closing #6721: avoid unescaping when linking catalogs together
* add SetRawString() to both implementations of DefaultCatalog which sets
the translated string as given
* use SetRawString() in both (standard- and build-)versions of linkcatkeys


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@41167 a95241bf-73f2-0310-859d-f6bbb57e9c96

* some cleanup in BLocaleRoster

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@39018 a95241bf-73f2-0310-859d-f6bbb57e9c96

Largish cleanup sweep concerning the Locale Kit (sorry it got so big):
* refactored private/mutable stuff out of LocaleRoster into MutableLocaleRoster
* moved management of Locale/Time settings file and broadcasting of any changes
out of preflets and into MutableLocaleRoster
* added proper sorting to the listviews of the Locale preflet
* the Time preflet no longer overlaps long timezone names into the actual time
* several fixes with respect to leaking ICU objects, esp. in BCountry
* the locale roster no longer passes out references to its own BCountry object,
but uses copies, instead - this makes locking superfluous, as the clients'
BCountry objects can no longer be changed by the setting a new default
country in the locale roster
* removed pretty useless POSIX-style symbol fetching from BCountry - if we
need that at all, it should live in the dedicated formatter classes
* adjusted readonlybootprompt, dstcheck and Deskbar to the changed Locale API
* refactored existing Time-formatter into TimeUnitFormat and DurationFormat
(the latter of which is now used by AboutSystem)
* added stubs for Date, DateTime and Time formatters
* lots of coding style fixes throughout the Locale Kit and the Locale and Time
preflets
This will probably break most external apps making use of the Locale Kit - it
does break WebPositive.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37831 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Implement a function to get all the available languages for a given application
* use it in readonlybootprompt instead of previous workaround code.
This makes the readonlybootprompt faster and lighter as it doesn't open and parse all the catalogs anymore, but only checks for their existence.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36276 a95241bf-73f2-0310-859d-f6bbb57e9c96

* LocaleRoster.h was not self-containing.
* Moved the locale stuff to the data directory as well (some parts were in /etc).
* The DefaultCatalog will now also scan the user directory for catalogs.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33983 a95241bf-73f2-0310-859d-f6bbb57e9c96

Cleanup of the locale kit tools :
-Jamrule for collectcatkeys no longer print all the strings, only a message when it can't parse one (happens for TR(variable) basically)
-Added fingerprint check in the plaintext catalog ReadFromFile. However, the adler checksum is different each time the catalog is loaded because it relies on the string being iterated always in the same order, but this is not always the case with an HashMap ! Some rethinking is needed, so disabled the check for now so it does not breaks the build
-Some try to debug the bluetooth preflet localization. Still buggy, but I wanted to commit all this mess before I break everything up again.
-Also sorted the fr.catkeys files to be in the same order as the autogenerated en.catkeys (this is useless but makes them easier to check) and updated their fingerprint even if they are still not checked.
-Miscelaneous style fixes, small bugfixes, more error checking and error messages saying where they come from.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33322 a95241bf-73f2-0310-859d-f6bbb57e9c96

* reintegrated gsoc-locale-kit branch into trunk - there's more
work to do, but it's about time to give this code more exposure.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33176 a95241bf-73f2-0310-859d-f6bbb57e9c96

err, fix for my fix:
* its __GNUC__, not __GCC__, tsk!

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30572 a95241bf-73f2-0310-859d-f6bbb57e9c96

fix gcc2 build:
* __gnu_cxx namespace is unknown to gcc2

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30571 a95241bf-73f2-0310-859d-f6bbb57e9c96

Move using directives into the cpp files so they don't leak into apps making use of the locale kit.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30570 a95241bf-73f2-0310-859d-f6bbb57e9c96

* followed advice by Rene to explicitly import the used std-classes instead
of simply importing the whole namespace - thanks for proofreading!


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30568 a95241bf-73f2-0310-859d-f6bbb57e9c96

* applied several fixes to make locale kit buildable with gcc4, too
* actually, all those fixes were already contained in PulkoMandys patch,
but I just missed that :-/


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30566 a95241bf-73f2-0310-859d-f6bbb57e9c96

* Copied imported OpenTracker Locale Kit files from the vendor branch
into their new homes (at least for now, might need some adjustment).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30540 a95241bf-73f2-0310-859d-f6bbb57e9c96