1# $FreeBSD$ 2 3TYPE ROWCOL 4NAME DEVANAGA/UCS 5SRC_ZONE 0x00-0xFA 6OOB_MODE ILSEQ 7DST_ILSEQ 0xFFFE 8DST_UNIT_BITS 16 9#======================================================================= 10# File name: DEVANAGA.TXT 11# 12# Contents: Map (external version) from Mac OS Devanagari 13# encoding to Unicode 2.1 and later. 14# 15# Copyright: (c) 1995-2002, 2005 by Apple Computer, Inc., all rights 16# reserved. 17# 18# Contact: charsets@apple.com 19# 20# Changes: 21# 22# c02 2005-Apr-05 Update header comments; add section on 23# roundtrip considerations. Matches internal 24# xml <c1.1> and Text Encoding Converter 2.0. 25# b3,c1 2002-Dec-19 Update URLs. Matches internal utom<b1>. 26# b02 1999-Sep-22 Update contact e-mail address. Matches 27# internal utom<b1>, ufrm<b1>, and Text 28# Encoding Converter version 1.5. 29# n04 1998-Feb-05 First version; matches internal utom<n9>, 30# ufrm<n15>. 31# 32# Standard header: 33# ---------------- 34# 35# Apple, the Apple logo, and Macintosh are trademarks of Apple 36# Computer, Inc., registered in the United States and other countries. 37# Unicode is a trademark of Unicode Inc. For the sake of brevity, 38# throughout this document, "Macintosh" can be used to refer to 39# Macintosh computers and "Unicode" can be used to refer to the 40# Unicode standard. 41# 42# Apple Computer, Inc. ("Apple") makes no warranty or representation, 43# either express or implied, with respect to this document and the 44# included data, its quality, accuracy, or fitness for a particular 45# purpose. In no event will Apple be liable for direct, indirect, 46# special, incidental, or consequential damages resulting from any 47# defect or inaccuracy in this document or the included data. 48# 49# These mapping tables and character lists are subject to change. 50# The latest tables should be available from the following: 51# 52# <http://www.unicode.org/Public/MAPPINGS/VENDORS/APPLE/> 53# 54# For general information about Mac OS encodings and these mapping 55# tables, see the file "README.TXT". 56# 57# Format: 58# ------- 59# 60# Three tab-separated columns; 61# '#' begins a comment which continues to the end of the line. 62# Column #1 is the Mac OS Devanagari code or code sequence 63# (in hex as 0xNN or 0xNN+0xNN) 64# Column #2 is the corresponding Unicode or Unicode sequence 65# (in hex as 0xNNNN or 0xNNNN+0xNNNN). 66# Column #3 is a comment containing the Unicode name or sequence 67# of names. In some cases an additional comment follows the 68# Unicode name(s). 69# 70# The entries are in two sections. The first section is for pairs of 71# Mac OS Devanagari code points that must be mapped in a special way. 72# The second section maps individual code points. 73# 74# Within each section, the entries are in Mac OS Devanagari code order. 75# 76# Control character mappings are not shown in this table, following 77# the conventions of the standard UTC mapping tables. However, the 78# Mac OS Devanagari character set uses the standard control characters 79# at 0x00-0x1F and 0x7F. 80# 81# Notes on Mac OS Devanagari: 82# --------------------------- 83# 84# This is a legacy Mac OS encoding; in the Mac OS X Carbon and Cocoa 85# environments, it is only supported via transcoding to and from 86# Unicode. 87# 88# Mac OS Devanagari is based on IS 13194:1991 (ISCII-91), with the 89# addition of several punctuation and symbol characters. However, 90# Mac OS Devanagari does not support the ATR (attribute) mechanism of 91# ISCII-91. 92# 93# 1. ISCII-91 features in Mac OS Devanagari include: 94# 95# a) Overloading of nukta 96# 97# In addition to using the nukta (0xE9) like a combining dot below, 98# nukta is overloaded to function as a general character modifier. 99# In this role, certain code points followed by 0xE9 are treated as 100# a two-byte code point representing a character which may be 101# rather different than the characters represented by either of 102# the code points alone. For example, the character DEVANAGARI OM 103# (U+0950) is represented in ISCII-91 as candrabindu + nukta. 104# 105# b) Explicit halant and soft halant 106# 107# A double halant (0xE8 + 0xE8) constitutes an "explicit halant", 108# which will always appear as a halant instead of causing formation 109# of a ligature or half-form consonant. 110# 111# Halant followed by nukta (0xE8 + 0xE9) constitutes a "soft 112# halant", which prevents formation of a ligature and instead 113# retains the half-form of the first consonant. 114# 115# c) Invisible consonant 116# 117# The byte 0xD9 (called INV in ISCII-91) is an invisible consonant: 118# It behaves like a consonant but has no visible appearance. It is 119# intended to be used (often in combination with halant) to display 120# dependent forms in isolation, such as the RA forms or consonant 121# half-forms. 122# 123# d) Extensions for Vedic, etc. 124# 125# The byte 0xF0 (called EXT in ISCII-91) followed by any byte in 126# the range 0xA1-0xEE constitutes a two-byte code point which can 127# be used to represent additional characters for Vedic (or other 128# extensions); 0xF0 followed by any other byte value constitutes 129# malformed text. Mac OS Devanagari supports this mechanism, but 130# does not currently map any of these two-byte code points to 131# anything. 132# 133# 2. Mac OS Devanagari additions 134# 135# Mac OS Devanagari adds characters using the code points 136# 0x80-0x8A and 0x90-0x91 (the latter are some Devanagari additions 137# from Unicode). 138# 139# 3. Unused code points 140# 141# The following code points are currently unused, and are not shown 142# here: 0x8B-0x8F, 0x92-0xA0, 0xEB-0xEF, 0xFB-0xFF. In addition, 143# 0xF0 is not shown here, but it has a special function as described 144# above. 145# 146# Unicode mapping issues and notes: 147# --------------------------------- 148# 149# 1. Mapping the byte pairs 150# 151# If one of the following byte values is encountered when mapping 152# Mac OS Devanagari text - 0xA1, 0xA6, 0xA7, 0xAA, 0xDB, 0xDC, 0xDF, 153# 0xE8, or 0xEA - then the next byte (if there is one) should be 154# examined. If the next byte is 0xE9 - or also 0xE8, if the first 155# byte was 0xE8 - then the byte pair should be mapped using the 156# first section of the mapping table below. Otherwise, each byte 157# should be mapped using the second section of the mapping table 158# below. 159# 160# - The Unicode Standard, Version 2.0, specifies how explicit 161# halant and soft halant should be represented in Unicode; 162# these mappings are used below. 163# 164# If the byte value 0xF0 is encountered when mapping Mac OS 165# Devanagari text, then the next byte should be examined. If there 166# is no next byte (e.g. 0xF0 at end of buffer), the mapping 167# process should indicate incomplete character. If there is a next 168# byte but it is not in the range 0xA1-0xEE, the mapping process 169# should indicate malformed text. Otherwise, the mapping process 170# should treat the byte pair as a valid two-byte code point with no 171# mapping (e.g. map it to QUESTION MARK, REPLACEMENT CHARACTER, 172# etc.). 173# 174# 2. Mapping the invisible consonant 175# 176# It has been suggested that INV in ISCII-91 should map to ZERO 177# WIDTH NON-JOINER in Unicode. However, this causes problems with 178# roundtrip fidelity: The ISCII-91 sequences 0xE8+0xE8 and 0xE8+0xD9 179# would map to the same sequence of Unicode characters. We have 180# instead mapped INV to LEFT-TO-RIGHT MARK, which avoids these 181# problems. 182# 183# 3. Additional loose mappings from Unicode 184# 185# These are not preserved in roundtrip mappings. 186# 187# U+0958 0xB3+0xE9 # DEVANAGARI LETTER QA 188# U+0959 0xB4+0xE9 # DEVANAGARI LETTER KHHA 189# U+095A 0xB5+0xE9 # DEVANAGARI LETTER GHHA 190# U+095B 0xBA+0xE9 # DEVANAGARI LETTER ZA 191# U+095C 0xBF+0xE9 # DEVANAGARI LETTER DDDHA 192# U+095D 0xC0+0xE9 # DEVANAGARI LETTER RHA 193# U+095E 0xC9+0xE9 # DEVANAGARI LETTER FA 194# 195# 4. Roundtrip considerations when mapping to decomposed Unicode 196# 197# Both ISCII-91 (hence Mac OS Devanagari) and Unicode provide multiple 198# ways of representing certain Devanagari consonants. For example, 199# DEVANAGARI LETTER NNNA can be represented in Unicode as the single 200# character 0x0929 or as the sequence 0x0928 0x093C; similarly, this 201# consonant can be represented in Mac OS Devanagari as 0xC7 or as the 202# sequence 0xC6 0xE9. This leads to some roundtrip problems. First 203# note that we have the following mappings without such problems: 204# 205# ISCII/ standard decomposition of reverse mapping 206# Mac OS Unicode mapping standard mapping of decomposition 207# ------ ----------------------- ---------------- ---------------- 208# 0xC6 0x0928 ... LETTER NA 0x0928 (same) 0xC6 209# 0xCD 0x092F ... LETTER YA 0x092F (same) 0xCD 210# 0xCF 0x0930 ... LETTER RA 0x0930 (same) 0xCF 211# 0xD2 0x0933 ... LETTER LLA 0x0933 (same) 0xD2 212# 0xE9 0x093C ... SIGN NUKTA 0x093C (same) 0xE9 213# 214# However, those mappings above cause roundtrip problems for the 215# the following mappings if they are decomposed: 216# 217# ISCII/ standard decomposition of reverse mapping 218# Mac OS Unicode mapping standard mapping of decomposition 219# ------ ----------------------- ---------------- ---------------- 220# 0xC7 0x0929 ... LETTER NNNA 0x0928 0x093C 0xC6 0xE9 221# 0xCE 0x095F ... LETTER YYA 0x092F 0x093C 0xCD 0xE9 222# 0xD0 0x0931 ... LETTER RRA 0x0930 0x093C 0xCF 0xE9 223# 0xD3 0x0934 ... LETTER LLLA 0x0933 0x093C 0xD2 0xE9 224# 225# One solution is to use a grouping transcoding hint with the four 226# decompositions above to mark the decomposed sequence for special 227# treatment in transcoding. This yields the following mappings to 228# decomposed Unicode: 229# 230# ISCII/ decomposed 231# Mac OS Unicode mapping 232# ------ ---------------- 233# 0xC7 0xF860 0x0928 0x093C 234# 0xCE 0xF860 0x092F 0x093C 235# 0xD0 0xF860 0x0930 0x093C 236# 0xD3 0xF860 0x0933 0x093C 237# 238# Details of mapping changes in each version: 239# ------------------------------------------- 240# 241################## 242# Section 1: Map the following byte pairs as indicated: 243# (ZWNJ means ZERO WIDTH NON-JOINER, ZWJ means ZERO WIDTH JOINER) 244# (Also see note about 0xF0 in comments above) 245# Section 2: Map the remaining bytes as follows: 246# 247# 248# 249# 250BEGIN_MAP 2510x00 - 0x7F = 0x0000 - 2520x80 = 0x00D7 2530x81 = 0x2212 2540x82 = 0x2013 2550x83 = 0x2014 2560x84 = 0x2018 2570x85 = 0x2019 2580x86 = 0x2026 2590x87 = 0x2022 2600x88 = 0x00A9 2610x89 = 0x00AE 2620x8A = 0x2122 2630x90 = 0x0965 2640x91 = 0x0970 2650xA1 = 0x0901 266#0xA1+0xE9 = 0x0950 2670xA2 = 0x0902 2680xA3 = 0x0903 2690xA4 = 0x0905 2700xA5 = 0x0906 2710xA6 = 0x0907 272#0xA6+0xE9 = 0x090C 2730xA7 = 0x0908 274#0xA7+0xE9 = 0x0961 2750xA8 = 0x0909 2760xA9 = 0x090A 2770xAA = 0x090B 278#0xAA+0xE9 = 0x0960 2790xAB = 0x090E 2800xAC = 0x090F 2810xAD = 0x0910 2820xAE = 0x090D 2830xAF = 0x0912 2840xB0 = 0x0913 2850xB1 = 0x0914 2860xB2 = 0x0911 2870xB3 = 0x0915 2880xB4 = 0x0916 2890xB5 = 0x0917 2900xB6 = 0x0918 2910xB7 = 0x0919 2920xB8 = 0x091A 2930xB9 = 0x091B 2940xBA = 0x091C 2950xBB = 0x091D 2960xBC = 0x091E 2970xBD = 0x091F 2980xBE = 0x0920 2990xBF = 0x0921 3000xC0 = 0x0922 3010xC1 = 0x0923 3020xC2 = 0x0924 3030xC3 = 0x0925 3040xC4 = 0x0926 3050xC5 = 0x0927 3060xC6 = 0x0928 3070xC7 = 0x0929 3080xC8 = 0x092A 3090xC9 = 0x092B 3100xCA = 0x092C 3110xCB = 0x092D 3120xCC = 0x092E 3130xCD = 0x092F 3140xCE = 0x095F 3150xCF = 0x0930 3160xD0 = 0x0931 3170xD1 = 0x0932 3180xD2 = 0x0933 3190xD3 = 0x0934 3200xD4 = 0x0935 3210xD5 = 0x0936 3220xD6 = 0x0937 3230xD7 = 0x0938 3240xD8 = 0x0939 3250xD9 = 0x200E 3260xDA = 0x093E 3270xDB = 0x093F 328#0xDB+0xE9 = 0x0962 3290xDC = 0x0940 330#0xDC+0xE9 = 0x0963 3310xDD = 0x0941 3320xDE = 0x0942 3330xDF = 0x0943 334#0xDF+0xE9 = 0x0944 3350xE0 = 0x0946 3360xE1 = 0x0947 3370xE2 = 0x0948 3380xE3 = 0x0945 3390xE4 = 0x094A 3400xE5 = 0x094B 3410xE6 = 0x094C 3420xE7 = 0x0949 3430xE8 = 0x094D 344#0xE8+0xE8 = 0x094D+0x200C 345#0xE8+0xE9 = 0x094D+0x200D 3460xE9 = 0x093C 3470xEA = 0x0964 348#0xEA+0xE9 = 0x093D 3490xF1 = 0x0966 3500xF2 = 0x0967 3510xF3 = 0x0968 3520xF4 = 0x0969 3530xF5 = 0x096A 3540xF6 = 0x096B 3550xF7 = 0x096C 3560xF8 = 0x096D 3570xF9 = 0x096E 3580xFA = 0x096F 359END_MAP 360