Subject: Re:Encoding Chinese text for e-mail etc

From: Ed Lai <[email-masked]>
Date: Mon, 12 Feb 90 21:57:17 PST

This is a composite response to all the discussions on this topic.

There are three approaches to text exchanges.
1. We can do straight text exchange.
2. We can just use the file format of a single popular WP.
3. We can came up with a rich text/graphic/multi-media standard.

For number 2, if I am sending a document to someone who I know uses
the same WP, that certainly is not a problem. As far as encoding
is concerned, it is no different from sending any other binary file,
so there is no need to invent anything. But if I want to send to a
large group of people, there is no WP out there that is so popular
that it can serve as an exchange standard. I guess everybody want
the WP they are using to be the standard, so we can hardly agree
on one. So this solution is not viable.

For number 1, people would say it is the lowest common denominator,
and that it is technology backward, and so we should shoot for option
3. My answer to that is that this may very well be true, but before
we arrive this wonderful standard, can we in the mean time use the
good old text standard. I do not want to hold my breath waiting for
the new wonderful standard. After all, what are we doing here right
now? We are exchanging information using simple ASCII text. It may
not be ideal but it can be working now.

Next we have to decide on the internal code. I think in the long
run Unicode is the way to go, but until the software is available
we have to stick with less elegant solution. So we have to stick
with some of the existing standard. Three standards come into mind,
GB, Big-5, CCCII (may be also the telegraph code should be one too,
the problem is that everybody has modified the telegraph code so in
reality there is not a single telegraph code anymore). CCCII being
the superset of every thing else may look like a good candidate.
But it is three byte code which a lot of people (including myself)
do not like. But no exchange table is easily available and it is
not easy to build one since it is so massive. As for GB and Big-5,
when I proposed 'zW' I choose GB over Big-5 for several reasons.
However, I certainly would not say GB is 'art' and Big-5 is a mess.
The choice of characters is certainly more careful in GB than Big-5,
and Big-5 is the standard that everyone seem to love to hate. However
Big-5 at least is consistent in that in both level 1 and 2, it is
arrange in increasing stroke count order (and radical as secondary
key),unlike GB that uses one ordering scheme in one level and
another scheme in another level.

Actually there are good reason that the coding standard should be
based on a traditional char coding because they should be a superset
of the simplfied one. However I pick GB over Big-5 for several in
'zW' for several reasons. Outside Taiwan, GB is used more than
Big-5; to transmit big-5 in 7-bit printable really require 3 bytes;
the pre-decessor of 'zW' was proposed in May 89, the hope was that
there is a more efficient medium to sent Chinese data into China
so the decoding should be trival to do inside China.

Actually 'zW' and other comparable encoding standard in the guts are
very similar: Chinese character is transmitted as pure GB (pure in
the sense that it is as defined in GB and not as modified as it is
being used in any commerial systems). The fact that it is based on
GB does not mean that Big-5 system cannot use it, only that it would
require a bigger encoding/decoding program since a lookup table is
required.

But because the traditional/simplifed mapping is not one to one,
using traditional character with GB is not completely satisfactory.
What we need is an extended GB system, I shall not comment on this
because other people have more expertise on this, I just want to
point out that is what they do in commerical system that can switch
between the traditional/simplified font such as TianMa. This is an
area that is on the 'to-do' list on 'zW' but is put on hold because
of lack of sufficent interest.

Talking about lack of interest it is indeed discouraging to work on
an encoding standard. Workable encoding standard certainly exist and
'zW' is not even the only one. But none of them are being used and
at the same time people lament about that they cannot encode
Chinese in mail/news. There are recently discussion on Chinese
couplets in soc.culture.hongkong, and they are almost unreadable.

One more thing I want to mention is that 'zW' is not designed merely
as an encoding standard, one major goal is that it can also be used
at the same time for Chinese terminals, and it has certainly work
that way, there exists a Chinese terminal emulator that works for
'zW' (and also 'hz').

Paul Hui once forward me an article on how they do it in Japan, I
have forward it to a few other individuals and they all agree it
was intereting. I no longer have that article. It would be
interesting for anyone who still have that to post it to CCNET.

/* Disclaimer: All statments and opinions expressed are my own */
/* Edmund K. Lai                                               */
/* Apple Computer, MS77-E                                      */
/* 20525 Mariani Ave,                                          */
/* Cupertino, CA 95014                                         */
/* (408)974-6272                                               */
zW@h9cOi