Subject: a flexible encoding standard

From: Richard S Cook <[email-masked]>
Date: Wed, 29 Jan 1997 15:47:14 -0500

After recently reading the short article in WIRED magazine titled "Lost in
Translation" (Sept. '96, pp.130-1, Jose Manuel Tesoro <[email-masked]>), I
would like to offer a few comments.  I think that from the perspective of
anyone interested in Chinese literature it is a great pity that so many of
the great classical texts are not readily available in computerized form,
due to lack of a suitable standard.  As was noted by a contributor on the
CCNET listserv recently, Chinese characters are truly infinite.  This
should not prevent an adequate standard from being developed for the main
Classical texts. (Readers interested in descriptions of encoding schemes
now in use might check out the following:
http://www.iijnet.or.jp/iriz/irizhtml/kanjibas/cefintro.htm.)  What is
truly needed however is a flexible encoding standard, permitting users to
create unusual characters as needed.  Given the fact that most if not all
square-script characters follow rules of composition based on a limited
number of possible components, it should be possible to create most any
character (in scaleable outline format suitable for laser printing) simply
by defining the relative proportions and placement of its components.
Thus, for the purpose of entering a classical text, all the standard or
currently used chars. could be entered (e.g. in Big5), and then
non-standard ones built-up and entered in as needed, according to a
standard for composing supplemental graphs.  Such a standard could also
significantly reduce the size of the actual fonts, since rather than
redundant info for rendering a particular "radical", for example, being
given in every character in the font in which it occurs, a link would
exist between all characters having the same components.  Some roman
TrueType fonts (for example) employ a similar technique in adding
diacritics to standard roman characters.  Thus, for example, Fontographer
permits typographers to draw each lower-case vowel only once, and each
kind of diacritic only once, and create various "new" characters
(combinations of vowel and diacritic) by means of "linked references".
This can substantially reduce the ultimate font size.  It seems to me that
a similar technique would be well employed in Chinese typography.  Users
could compose needed high resolution characters on the fly, and embedded
in each such character would be info necessary to render, sort and access
it.

Just a few thoughts ... any comments or references would be welcome.

Richard

---

From: Georges KO <[email-masked]>
Date: Thu, 30 Jan 1997 23:34:04 +0800

Richard S Cook <[email-masked]> writes:

[...About composing characters...etc.]
> Just a few thoughts ... any comments or references would be welcome.

        Have you looked at the Boshiamy input method? It can be a good
starting  point... Basically, this method  consists  of splitting each
character into small  components that  can be input according to their
shape, to their meaning or   to their sound, without  Chinese-specific
keyboard.

        For example, my  name  in Pinyin is  `ke' (mu4  ke3),  so it's
input by `tto', `t'  as `t'ree and  `to'  for `ke3' (shape). If  there
were  characters  composed of  the  `water' radical or `grass' radical
with my name, then they would be input  with `wtto' and `rtto' (`r' as
`r'oots).  No more `huh... which tone is it?' :) !

        Of  course,  it's  just  an  input  method,  but  it's  a good
reference for an exhaustive list of possible components.

        For  more information, look  at http://www.boshiamy.com.tw and
any IFCSS mirror for this input method.

        G.
--
 Georges KO [ [email-masked] http://mtc.ntnu.edu.tw:52021/~m52021 ]
                              Home page closed from Feb. 1st to Feb. 16th

---

From: Jacob Morrison <[email-masked]>
Date: Fri, 31 Jan 1997 01:03:27 +0800

I agree that Bosiamy is a nice input method (that's what I use), but
I don't think it slices things finely enough for use with the many
character variations. For example, you would need to encode the
difference between a two-dot water radical and a three-dot water
radical. Another big issue is making a canonical decomposition and order,
allowing a character to be identified unambiguously from its components.

Someone from mainland china put together
an input method that handles all the 22,000 CJK characters in Unicode.
I might be able to find the info on it somewhere.

There was talk on the Unicode (see http://unicode.org) mailing list a
while back about this problem. They would certainly be interested in a
solution.

Regards,
Jake

Control Data Asia/Pacific             Jacob.Morrison @ cdc.com

---

From: Richard S Cook <[email-masked]>
Date: Thu, 30 Jan 1997 13:12:09 -0500

Just a brief note of clarification ... as some responding to my earlier
comments have seen, I was suggesting a different sort of encoding scheme,
one which would define certain components univerally occurring (or
possible) in Chinese characters, define configurations of these components
for common characters, and define a method for composing uncommon
characters as needed.  I was not simply talking about another input
method, though some have noted that certain input methods do function
along componential lines.

To further explain, I think the current state of Chinese encoding is just
a brute strength approach ... something similar to what would be required
if an English typographer were required to anticipate every word which a
potential typist might ever want to type when using the font being
designed.  Certainly, when kerning is considered, a roman typographer does
try to take into account what combinations of letters are probable .. he
doesn't kern useless pairs which are unlikely ever to be adjacent.  Why is
it necessary in Chinese typography to have incredibly enormous character
sets to satisfy the huge periodic and local graphical divergences?  Would
we like to type English if the fonts were built on a word by word basis,
so that to type "electronic", we'd have to sort out things like
"elecampane" and "eleemosynary"?  And what if Shakespearian, Chaucerian or
Beowulfian spellings were included?  An English font too might have
millions of "characters" ... one per variant spelling of each vocabulary
item. Really ... the best way cannot be ever bigger and bigger character
sets ...

---

From: Werner Lemberg <[email-masked]>
Date: Thu, 30 Jan 1997 19:46:43 +0100

> along componential lines.

Maybe SKIP is what you want. Jack Halpern developed it for his
Japanese-English dictionary. It's a very convenient searching method for
CJK characters:

you have basically 4 classes: characters which can be separated
horizontally, characters which can be separated vertically, characters
which are enclosed (at least on two sides) and characters which do not fit
in any of these categories. This yields a number between 1 and 4. The next
step is simply to count the number of strokes on the left and right (upper
and lower/inner and outer) side; this process yields another two numbers
(Halpern further refines this scheme but this is not so important here).
Of course this description is rudimentary.

Doing this you get a very fine-grained splitting into groups; if I
remember correctly the largest group consists of about 20 characters.
And you don't need to learn radicals... Nevertheless, this method is only
suitable to search characters you know how it looks.

> Why is
> sets to satisfy the huge periodic and local graphical divergences?

It's irrelevant IMHO how the characters are stored; maybe it's possible to
logically split CJK characters into smaller units. But if you want to
print them you again need the whole glyph since it's impossible to compose
a CJK glyph by describing the components alone. You need further
information where and how the components are positioned and scaled (this
is the way how some composite CJK TrueType fonts are done).

>  Would
> sets.

In our computer age it's _really_ irrelevant how large the character
set(s) are. Much more important is to investigate how to _access_ the
characters in an easy-to-understand way (see above SKIP).

    Werner