Subject: CNS converter. Re: 7-bit clean big5

From: Jing-Shin Chang <[email-masked]>
Date: Fri, 29 Apr 1994 19:54:27 +0800

Dear CCNET netters,

You don't need to invent another wheel when it is available!!

         Z@D>]BDbFnGB]_D!T6oGDMD'

(especially when there are so many wheels -^^-)
(What are those  c*D$D,t`N{  words in the above line.)

There are many suggestions in the latest ccnet-l discussions
concering the "7-bit clean Big5" (or CNS) and proposals for
an extended HZ standard for the new chinese.* news hierachy.

 From technical points of view, I think this might be a
retrospect from the 8-bit (and 2-4 bytes) encoding trends
back to the ancient times since 8-bit clean environment is
increasingly polular. People just don't know their Sparcs,
RS/6000, and HPUXs are mostly 8-bit clean. (Hardware as well
as applications.)

OK. I will not talk about those 8-bit stuffs. Let's started
with the 7-bit envionments.

I am strongly in favor of using the CNS code as the exchange
code for the chinese.* newsgroups and use plug-in conversion
programs, as mentioned by one of Ning's posting, between
the CNS and the reader's favorite coding system, so as to
hide the readers from the exchange code. In addition, the
ISO 2022 standard should be followed.  The configuration
would then look like this:

        (HZ Users) HZ <-> | Big5 <-> CNS posting | <-> Big5 (Big5 Users)
                          +--- news readers -----+

(HZ <-> Big5 and CNS <-> Big5 are available.)

The major reasons are:

        1) the news readers can use whatever Chinese terminals
                they used, either 7-bit HZ or 8-bit Big5 terminals

        2) a larger code space which prevent one from using
                variant characters frequently

        3) there is no n-to-1 mapping problem for Big5 postings
                although HZ postings still have such a problem
                for Big5 readers.

                (In the present act.big5 newsgroup, even a Big5
                posting may become messed due to the conversion
                to HZ and the reconversion back to Big5.)

        4) conversion programs are readily available and the
                ISO 2022 standard is followed.

The decision, of course, depends on many other factors.

The 3rd line of this email is an encoded CNS message, which
is the chinese translation of the 2nd line. This encoding
scheme is widely used in the mail and news systems in the
SEEDNET of the ROC. And it is probably what you are seeking
for as a 7-bit clean code.

(The SeedNet is the *.com.tw, *.org.tw and *.net.tw parts of the
Internet. However, only mail access is available due to some
legal problems in the ROC telecommunication laws.)

Thanks Simpson. Thank you for your consideration for the Big5
users. You may want to get a code conversion package from the ftp site:
iiidns.iii.org.tw(140.92.1.45):/pub/seed/chcode, which provide code
conversion functions among major Taiwanese Chinese coding schemes,
including BIG5, IBM5550, NSC (should be the CNS-11643 internal code)
NSC-P (the "7-bit clean" version of NSC, this is probably what
everybody is expecting for), EUC (extended UNIX code, which involve
the ASCII and CNS-11643 in the front part of its code space,
2 or 4 bytes, widely used in Workstations in Taiwan, and probably in
Japan and Korea too), TCA (Taipei Computer Association recommended
code, which avoids code confliction better than the Big5 code does),
IBM Host code and Telgram code (old stuffs!!).

See the "README" and "chcode.doc" in the same directory for a brief
introduction on the encoding schemes. Also, you may like to trace
the GetNSCPcode() and PutNSCPcode() functions in code.c for details.

(Note: The above distributed version has a few bugs, and the interface
is not good as a filter. I had fixed some bugs and personally made a
few changes to make it a filter. I could contribute the revised
version if necessary.)

There are also free CNS fonts made available by the ROC government
on Jan. (They are 24x24 and 40x40 fonts though. See the attached
uuencoded file (Big5). Sorry for those persons who cannot view the
Big5 file!)

You could also get a PC version of SUCCESS, which integrate, PPP
communication program, news reader, TCP/IP supports (ftp, telnet,
email), from III, which supports BIG5, CNS, EUC code conversion.
(Since I am a workstation user, this information is not testified
by myself.)

Because III (which is exactly the organization who invents the Big5
code) is a govenment-supported organization, you can get these stuffs
for free and they will continue on improving the software without extra
charge. I think it is therefore more suitable in comparison with other
sharewares.

Using a standard that conforms to ISO 2022 is probably a better solution
for combining our applications with other multi-lingual applications as
mentioned in a few previous postings. So, instead of using private
standard as the exchange code, I prefer not to invent another wheel.
This is also adopted by those workstation giants like Sun, Dec, IBM.
In fact, the technical trends for these giants are to move toward 8-bit
clean environment (More exactly, toward MLNS, Mutli-Language National
Supports.) When they have to retrospect to the old environment, they
resorted to available standards. One example is the Sun ALE (Asia
language Environment, including HLE for Traditional Chinese, CLE for
Simplified Chinese, JLE for Japanese and KLE for Korean). It was made
8-bit clean as early as 1990 (probably earlier?) for the terminal
or workstation side. However, they use an ISO-2022 conformant approach
for sending Chinese email among HLE workstations (See the attached
appendix from Sun's Training courses.) Therefore, I suggest to use
the CNS coding scheme and make it ISO conformat when designing the
exchange code for the news groups.

        Jing-Shin -^^-

=== Appendix I. Sun's Approaches for Sending Chinese/Japanese/Korean email ===
/usr/ucb/mail was modified for:

Sending
        EUC convert to ISO 2022 with self-announcement,
        resulting stream is 7-bit, send to mail spool
Receving
        8-bit EUC (e.g., mail folder) unaffected
        7-bit ISO from mail spool converted to EUC
        self-announcement indicates need for EUC
Saving message
        mail going into mail folder is EUC
        7-bit -> 8-bit conversion done if necessary
Printing
        Mailtool allows filter definition for printing EUC text
        In .mailrc
                set print2="pr|htops|lpr"

        mailtool - EUC -> ucbmail - ISO -> sendmail - IP net or UUCP ->
        sendmail -> binmail -> mail spool - ISO -> ucbmail - EUC -> mailtool

Locale (language) of mail embedded in message text
Example, for codeset 1:
ESC $ ) F
        F: '0' (HLE/Taiwan), 'A' (CLE/China), 'B' (JLE/Japan), 'C' (KLE/Korean)

        Note: Taiwan(ROC) does not have entry in ISO Standard Registry.
        HLE mail is currently using entry in ISO private registry space.
        Possibly may conflict with other private entry mail originating
        outside Taiwan.

==== Appendix II ==== Big5 News: Free CNS fonts available ====