Subject: GB versus BIG-5

From: Stephen G Simpson <[email-masked]>
Date: Sun, 24 Nov 1991 13:58:46 -0500

In an earlier CCNET posting, I wrote:

 > I think a lot of people have wondered about GB versus BIG-5 character
 > codes.  BIG-5 is in a sense much better, because it includes about 13000
 > characters, compared to about 7000 for GB.  Also, since BIG-5 is keyed to
 > traditional rather than simplified characters, it is more suitable for
 > classical Chinese literature and for everyday use in the United States
 > and indeed most parts of the world, except mainland China.

Perhaps I was being overly enthusiastic here, and I certainly didn't
mean to start a debate.  (My enthusiasm was triggered by Carlos
McEvilly's kindness in providing a Pin Yin BIG-5 input method for Yi
Tian word processor.)

I don't want to get into a political or "religious" discussion, but
there may be some value in a further clarification of the issues.

The choice of GB versus BIG-5 depends on a lot of factors, including:

    1. intended application

    2. software availability

    3. disk space and memory availability

    4. personal preference

    5. politics

Factors (2) and (3) are becoming less important as time goes on.
Software, cheap disk space, and cheap memory are becoming more and
more available.  Moreover, if you want to use BIG-5 but have a severe
disk space or memory problem, you can always use the stripped down
version of BIG-5, containing only the 5401 most frequently used
characters.  This is adequate for everyday business or scientific use.
Therefore, I think factors (2) and (3) are not decisive.

Because of the nature of this list, I won't comment on factors (4) and
(5), politics and "religion".

I think that the most important factor is (1), intended application.

Benjamin Ao <ao-b.cis.ohio-state.edu> posted some thoughtful arguments
in favor of GB codes and a smaller character set.  I agree with
Benjamin that more is not necessarily better.  If the GB character set
contains all the characters you need for your intended application,
then there is no reason to switch to BIG-5.

However, I think BIG-5 is more suitable for many applications.

The main problem is that GB-2312 simply does not have codes for many
commonly used characters.  An example is Mian4 ("wheat") as in Mian4
Bao4 ("bread").  GB users have to substitute Mian4 ("face") in this
context.  This substitution is acceptable in mainland China, but it is
not acceptable in, for instance, Chinese newspapers published in the
United States.

In addition, GB does not have codes for a great number of uncommon but
occasionally useful characters.  For example, a couple of weeks ago, a
colleague from the mainland told me that her given name was a very
rare character that could not be found in any dictionary or
typesetting system.  The character in question did not exist in GB,
but I easily found it in the second part of the BIG-5 table.  (The ET
BIG-5 hex code is F2C9.)

Finally, if one wants to publish a document in both traditional and
simplified characters, BIG-5 would seem to be the better choice.  This
is because the mapping from traditional characters to simplified
characters is many-to-one, so automatic BIG-5 to GB conversion is at
least possible in principle, while the opposite sometimes presents
severe difficulties (see for instance Ned Walsh's recent complaints on
the Chpoem list).

If there is a version of GB that addresses these problems, I would
like to learn about it.

Benjamin pointed out the following:

 > According to a 1985 Chinese character frequency assessment, whose
 > scale is the largest in history, a total of 7,754 characters appear at
 > least once in a literature composed of 11,873,453 characters, which
 > was published in mainland China from 1977 to 1982 covering subjects in
 > natural and social sciences.

At best this means that GB may be suitable for natural and social
sciences publishing in mainland China.  What about natural and social
sciences in the rest of the world?  What about newspapers?  Perhaps
most significantly, what about literature, poetry, and scholarship in
other fields?  Judging from recent postings on the Zhongwen and Chpoem
lists, European and American and even mainland Chinese scholars in
Chinese literature can't use GB-2312, because GB-2312 lacks a lot of
characters that they need.  It appears that most of the Europeans and
Americans use BIG-5.  I don't know what the mainlanders do.

 > Allowing for some variance, 8000 is
 > probably a close estimate of the number of characters used in modern
 > Chinese.

I don't think the study justifies this conclusion about modern Chinese
as a whole.  As Benjamin pointed out, the study was restricted to
natural and social science publications on mainland China.  And even
in that restricted domain, it may be that the mainland scientists
would have liked to use certain characters, or even do use them in
handwritten notes, but were prevented from using them in publications,
because of the lack of suitable software.

 > In other words,
 > about half of all the characters defined in BIG5 will probably never
 > be used.

This conclusion seems completely unjustified.  For example, if your
name happens to be one of the many characters (common or rare) that
are missing from the GB character set, then you will not be able to
typeset it using GB software, but you will use it nevertheless.

Benjamin went on to discuss other coding schemes, including Unicode
and a hypothetical scheme involving a modified GB with 8000
characters, presumably including all of the common characters that are
missing from GB-2312.  These proposals are interesting, and more
information would be welcome.

Stephen G. Simpson               [email-masked]
Department of Mathematics        +1 814 863-0775
Pennsylvania State University    home: +1 814 238-2274
University Park, PA 16802