Subject: [Ned Walsh: Re: coding standards]

From: Valk v/d M <[email-masked]>
Date: Fri, 4 Oct 1991 16:57:51 MET

In his original message Martijn van der Valk <[email-masked]> wrote:

> GB
> BIG5
> Unicode
> JIS 1st and 2nd level
>
> What I'd like to know is: what order is used to assign codes to characters.

While several people have replied explaining what the codes are,
nobody has said what order is used in each.  I'll give it a stab, and
hope others will correct my mistakes.  I'm lifting most of this info
from J. and T. Huang's "An Introduction to Chinese, Japanese and
Korean Computing" (World Scientific, 1989; a kind of flaky book, I
know, but it's what I've got :).

The Japanese Industrial Standard includes 6,353 characters, divided
into two levels:  (1) 2,965 in the first level, arranged by a "repre-
sentative on-kun reading," which appears to be a somewhat arbitrary
selection and pairing of a character's assorted on and kun readings (I
assume they chose the most common readings).  (2) 3,388 characters,
arranged by radical, then stroke count, then representative on
reading.  Parent characters precede variants.  This code also includes
147 special characters (space, parentheses, scientific symbols, etc.);
the numerals 0-9; the 52 letters of the Roman alphabet; 83 hiragana;
86 katakana; 48 Greek letters; 66 Russian letters; and 32 "ruled line
elements" (assorted symbols for drawing lines).  I'm not certain, but
I think all except the ruled line elements precede the characters in
the order I've listed; the ruled line elements come last in the code.
I could easily be mistaken.

The Guo Biao code includes 6,763 simplified characters, also divded
into two levels:  (1) 3,755 more frequently used characters, arranged
in order by pinyin (I don't know how they decided among multiple
readings); (2) 3,008 arranged by the PRC's radical system, then
presumably by stroke count.  I don't know what order is used for
characters with the same pronunciation, or the same
radical/stroke-count.  The code also includes 202 general characters
(space, punctuation marks, etc.); 60 serial numbers (e.g. 1-20 with
periods after them, 1-20 in circles, etc.); the Roman alphabet;
Japanese kana; the Cyrillic alphabet; pinyin symbols with tone marks;
and the bopomofo syllabary.  I'm not sure where they all fit in.

The Big 5 code (essentially developed and promoted by the ROC
government) includes 13,051 traditional characters divded into two
sets:  (1) 5,401 most frequently used characters and (2) 7,650 next
frequently used characters, both arranged in order by stroke count
then by radical.  Again, I'm not sure how they ordered characters with
the same stroke/radical index, but it seems to follow the order they
appear in the Zhongwen da zidian.  It includes all the extras that
come with Guo Biao, except for pinyin with tone marks.

I suppose having most frequently used characters appear first in these
codes speeds up character searches.  But the ordering of characters
used in all three codes is absolutely useless when it comes to sorting
a list of characters.  For one thing, all divide the character set in
two, so that characters seem to appear in a set order until halfway
through a list when the ordering seems to start afresh.  JIS and Guo
Biao even change ordering systems on you in the second set.  Their
reliance on phonetic readings to order the most frequently used set of
charcters isn't terribly helpful, since so many characters have
multiple readings.  Big 5's use of a stroke/radical index seems pretty
ass-backwards to me; most people can tell a character's radical at a
glance, then estimate the remaining stroke count fairly accurately;
going the other way around leaves most people counting from the
outset.

Has anyone come up with a weighted table for sorting JIS/Big5/GuoBiao
characters by radical/stroke count?

Maybe Unicode or one of the other new international coding standards
being debated will solve all this.  Mr. Feng would know, but he's off
fermenting in China.:)

/Ned

----- End of forwarded messages