From simpson@math.psu.edu Tue Apr 26 09:51:15 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA24843; Tue, 26 Apr 94 09:51:15 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id KAA24643; Tue, 26 Apr 1994 10:49:12 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Tue, 26 Apr 1994 10:49:10 -0400 Date: Tue, 26 Apr 1994 10:49:10 -0400 Message-Id: <199404261449.KAA07466@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: HZ+S proposal for 7-bit merged GB/Big5 Reply-To: simpson@math.psu.edu Status: RO The soft-authors group has been silent lately, so I forgot about its existence. But it just occurred to me that maybe the discussion of HZ+S should be in soft-authors rather than ccnet-l. To bring soft-authors up to date on HZ+S, here is my latest ccnet-l posting. -- Steve ------------- >From: Stephen G Simpson Sender: Chinese Computing Network To: Multiple recipients of list CCNET-L Subject: HZ+S proposal for 7-bit merged GB/Big5 Date: Tue, 26 Apr 1994 10:17:12 -0400 In response to my earlier proposal, Ross Paterson made the following suggestions for improvement. > A variation on this would be to use CNS to represent Big5. The > conversion is only slightly more complicated than the one proposed > above, and has much the same effect. > Another variation would be to use the ISO-2022 final characters (G > and H) to identify the sets. That would make conversion to and > from ISO-2022 particularly simple. Further extension would also be > easy. These "variations" are actually excellent suggestions for improvement. We can always count on Ross for superb technical insight. I hereby incorporate Ross's suggestions into the HZ+S proposal. So, the HZ+S proposal for 7-bit merged GB/Big5 now reads as follows: The purpose of HZ+S is to merge GB, Big5, and ASCII into a standard 7-bit representation for convenient e-mail transmission, etc. GB already has a standard 7-bit representation, implemented in HZ. Big5 also has a standard 7-bit representation, namely ROC Taiwan national standard CNS-11643, here called CNS. Just like Big5, CNS consists of two levels. Level 1 contains the more frequently used Chinese characters as well as miscellanous numerals, Roman and Greek alphabets, etc. Level 2 contains the less frequently used Chinese characters. Each of the two CNS levels is a subset of the 94 x 94 range (2 printable ASCII bytes). What we need is a way to distinguish between CNS Level 1, CNS Level 2, GB, and ASCII. We can do so by using four tilde escape sequences: ~{ to switch into GB (just as in the current HZ standard) ~} to switch into ASCII (just as in the current HZ standard) ~G{ to switch into Big5-CNS Level 1 (this is new) ~H{ to switch into Big5-CNS Level 2 (this is new) We provisionally call this proposal HZ+S. (If it becomes widely accepted, we may call it HZ+.) HZ+S has several advantages: 1. HZ+S is a simple extension of the existing HZ standard. 2. Under HZ+S, the ~H{ escape sequence would be very seldom used, because it is needed only to switch into Big5-CNS Level 2, the less frequently used Chinese characters. Most of the time, we would only use ~G{ to switch into Big5-CNS Level 1, plus ~{ and ~} as in HZ. 3. Looking to the future, it would be possible to extend HZ+S to incorporate other character sets such as Japanese, Korean, ... in a way that is compatible with ISO-2022. A similar scheme is already implemented in the 7-bit Mule-Junet-ISO-2022 multilingual coding scheme. (Mule is available for anonymous FTP at etlport.etl.go.jp.) 4. HZ+S is somewhat similar to Ya-Gui Wei's unreleased HZ-2 proposal. I would encourage Ya-Gui to release his proposal for discussion. I welcome comments on HZ+S. Let's continue this discussion on CCNET-L. -- Steve Stephen G. Simpson Department of Mathematics, Pennsylvania State University 333 McAllister Building, University Park, State College, PA 16802 Office phone +1 814 863-0775 Home phone +1 814 238-2274 E-mail simpson@math.psu.edu FAX +1 814 865-3735 World Wide Web http://www.math.psu.edu/simpson/ From xiaokun@stats.gla.ac.uk Tue Apr 26 10:44:32 1994 Return-Path: Received: from hillhead.cent.gla.ac.uk by ifcss.org (4.1/IFCSS-Mailer) id AA26796; Tue, 26 Apr 94 10:44:32 CDT Received: from bb-gateway.stats.gla.ac.uk by jess.glasgow.ac.uk with SMTP (PP) id <12646-0@jess.glasgow.ac.uk>; Tue, 26 Apr 1994 16:41:12 +0100 From: Xiaokun Zhu Date: Tue, 26 Apr 94 16:44:29 BST Message-Id: <13389.9404261544@jupiter.stats> To: soft-authors@ifcss.org Subject: Re: HZ+S proposal for 7-bit merged GB/Big5 Status: O Steve writes: > ~{ to switch into GB (just as in the current HZ standard) > > ~} to switch into ASCII (just as in the current HZ standard) > > ~G{ to switch into Big5-CNS Level 1 (this is new) > > ~H{ to switch into Big5-CNS Level 2 (this is new) I prefer keep escape sequences in the same length :-) ~G to switch into Big5-CNS Level 1 (this is new) ~H to switch into Big5-CNS Level 2 (this is new) PS: why do you use 'G' and 'H' as escape characters? ---xiaokun From mcevilly@netcom.com Tue Apr 26 10:46:02 1994 Return-Path: Received: from netcom.com (netcom13.netcom.com) by ifcss.org (4.1/IFCSS-Mailer) id AA26835; Tue, 26 Apr 94 10:46:02 CDT Received: by netcom.com (8.6.8.1/SMI-4.1/Netcom) id IAA12004; Tue, 26 Apr 1994 08:45:19 -0700 From: mcevilly@netcom.com (Carlos McEvilly) Message-Id: <199404261545.IAA12004@netcom.com> Subject: Re: HZ+S proposal for 7-bit merged GB/Big5 To: soft-authors@ifcss.org Date: Tue, 26 Apr 1994 08:45:19 -0700 (PDT) Cc: ccnet-l@uga.bitnet In-Reply-To: <199404261449.KAA07466@boole.math.psu.edu> from "Stephen G Simpson" at Apr 26, 94 10:49:10 am X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 857 Status: O > From: Stephen G Simpson > What we need is a way to distinguish between CNS Level 1, CNS Level > 2, GB, and ASCII. We can do so by using four tilde escape > sequences: This is looking good. > ~G{ to switch into Big5-CNS Level 1 (this is new) > ~H{ to switch into Big5-CNS Level 2 (this is new) Picky point: I would suggest we omit the 'Big5' from this to avoid confusion-- and to avoid creating yet another standard (with a new name, there is temptation to take new liberties and implement our own subset of CNS, thus creating a new codeset). How about just calling these levels CNS Level 1 and CNS Level 2? (Not to be confused with the seven zi4mian4 in CNS). Regards, Carlos McEvilly | email for info on Bamboo Helper Chinese text mcevilly@netcom.com | tools, or ftp to ftp.netcom.com:/pub/mcevilly From simpson@math.psu.edu Tue Apr 26 12:09:28 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA00187; Tue, 26 Apr 94 12:09:28 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id NAA25804; Tue, 26 Apr 1994 13:07:25 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Tue, 26 Apr 1994 13:07:23 -0400 Date: Tue, 26 Apr 1994 13:07:23 -0400 Message-Id: <199404261707.NAA07705@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: Re: HZ+S proposal for 7-bit mixed GB/Big5/ASCII In-Reply-To: <13389.9404261544@jupiter.stats> References: <13389.9404261544@jupiter.stats> Reply-To: simpson@math.psu.edu Status: O > PS: why do you use 'G' and 'H' as escape characters? Because G and H are the official, registered ISO-2022 "final characters" which designate CNS Level 1 and CNS Level 2 respectively. By using G and H here in HZ+S, we get a degree of compatibility with ISO-2022. The use of G and H was suggested by Ross Paterson. The underlying idea was, if somebody in the future wants to extend HZ+S to cover more character sets such as JIS Japanese and KSC Korean, they can continue to use the corresponding ISO-2022 final characters as part of the escape sequences. Some of the official ISO-2022 final characters are A for GB-2312, B for JIS-X-0208, C for KSC-5601. So in addition to ~G{ and ~H{ there would be escape sequences like ~B{ for JIS. And ~A{ would be a redundant alternative to ~{. I like the idea of leaving the way open for future extensions of HZ+S which would be compatible with ISO-2022. But all this is for the future. Right now the Chinese Internet community needs a simple, practical, convenient 7-bit standard for mixing GB, Big5, and ASCII. It seems to me that HZ+S has the potential of becoming such a standard. > I prefer keep escape sequences in the same length :-) > ~G to switch into Big5-CNS Level 1 (this is new) > ~H to switch into Big5-CNS Level 2 (this is new) This suggestion has merit. But for the time being, let's stick to ~G{ and ~H{. I don't want to make too many changes too quickly. Before I make any more changes in the HZ+S proposal, I'd like to get input from people like Ya-Gui Wei and F.F. Lee and M.C. Pong. Carlos writes: > > ~G{ to switch into Big5-CNS Level 1 (this is new) > > ~H{ to switch into Big5-CNS Level 2 (this is new) > > Picky point: > I would suggest we omit the 'Big5' from this to avoid confusion-- > and to avoid creating yet another standard (with a new name, there > is temptation to take new liberties and implement our own subset > of CNS, thus creating a new codeset). How about just calling these > levels CNS Level 1 and CNS Level 2? (Not to be confused with the > seven zi4mian4 in CNS). Yes, what I really mean is CNS Level 1 and CNS Level 2. I see big advantages in adhering strictly to the established, official, 7-bit CNS-11643 standard. This is a major advantage of the HZ+S proposal as it now stands, thanks to Ross. The only reason I used the terminology "Big5-CNS" is that some people are unaware of CNS, and some other people are under the mistaken impression that CNS is very different from Big5. I wanted to emphasize that CNS really *IS* just a 7-bit form of Big5. In future discussions of HZ+S, let's use the correct terminology, CNS Level 1 and CNS Level 2. But let's also keep in mind and talk up the fact that CNS Levels 1 and 2 are identical to 7-bit Big5. I would like to suggest another terminological correction. Instead of describing HZ+S as a 7-bit standard for merged GB/Big5/ASCII, let's describe it as "mixed" (as in the title of this posting) rather than "merged". "Merged" may suggest something like Unicode, but the HZ+S approach is very different from the Unicode approach. HZ+S is much closer to ISO-2022 than to Unicode. (See recent postings on ccnet-l about Unicode versus ISO-2022.) ISO-2022 is "mixed". Unicode is "merged". HZ+S is "mixed", not "merged". Sorry for the confusion if there was any. -- Steve Stephen G. Simpson Department of Mathematics, Pennsylvania State University 333 McAllister Building, University Park, State College, PA 16802 Office phone +1 814 863-0775 Home phone +1 814 238-2274 E-mail simpson@math.psu.edu FAX +1 814 865-3735 World Wide Web http://www.math.psu.edu/simpson/ From simpson@math.psu.edu Tue Apr 26 12:45:11 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA01270; Tue, 26 Apr 94 12:45:11 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id NAA26165; Tue, 26 Apr 1994 13:42:57 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Tue, 26 Apr 1994 13:42:56 -0400 Date: Tue, 26 Apr 1994 13:42:56 -0400 Message-Id: <199404261742.NAA07760@boole.math.psu.edu> To: yawei@ifcss.org Cc: soft-authors@ifcss.org Subject: proposals for 7-bit standard for mixed GB/Big5/ASCII Reply-To: simpson@math.psu.edu Status: O To: Ya-Gui Wei Dear Ya-Gui, Right now ccnet-l and soft-authors are seeing a lot of discussion about HZ+S, my proposed 7-bit standard for mixed GB/Big5/ASCII, which as I acknowledged is very similar to your unreleased HZ-2 proposal. Since you moved to Minnesota recently, let me ask you, are you still receiving ccnet-l and soft-authors? Would you like me to send you the recent postings? I would value your input and feedback. In your letter to me in September 1993 about HZ-2, you said: > this extension may brought confusion to HZ: maybe it is a better > idea to reserve HZ extensions for future extensions of GB itself. I have been loudly proposing HZ+S as an extension to HZ. You have been almost silent. Am I making a mistake? What is the difficulty? It seems to me that if we design HZ+ the right way, then it should be possible to incorporate any possible future extensions of GB, just as in ISO-2022. Recent discussions on ccnet-l about Chinese newsgroups and e-mail indicate a definite need for something called HZ+, be it HZ+S or HZ-2 or something similar. I think it would be a mistake to hold back because of worries about unknown future extensions of GB. What kind of extensions? Can't we design HZ+ to accommodate them? Best wishes, -- Steve Stephen G. Simpson Department of Mathematics, Pennsylvania State University 333 McAllister Building, University Park, State College, PA 16802 Office phone +1 814 863-0775 Home phone +1 814 238-2274 E-mail simpson@math.psu.edu FAX +1 814 865-3735 World Wide Web http://www.math.psu.edu/simpson/ From jiang Tue Apr 26 15:38:39 1994 Return-Path: Received: by ifcss.org (4.1/IFCSS-Mailer) id AA08192; Tue, 26 Apr 94 15:38:39 CDT Date: Tue, 26 Apr 94 15:38:39 CDT From: jiang (Yuan Jiang) Message-Id: <9404262038.AA08192@ifcss.org> To: soft-authors Subject: Re: HZ+S proposal for 7-bit merged GB/Big5 Status: O >I prefer keep escape sequences in the same length :-) > >~G to switch into Big5-CNS Level 1 (this is new) > >~H to switch into Big5-CNS Level 2 (this is new) ~G and ~H may have high likelihood occuring in ASCII files. Unix Mail uses ~(letter) at command sequences. Yuan . From simpson@math.psu.edu Tue Apr 26 16:57:29 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA11260; Tue, 26 Apr 94 16:57:29 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id RAA28484; Tue, 26 Apr 1994 17:55:31 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Tue, 26 Apr 1994 17:55:29 -0400 Date: Tue, 26 Apr 1994 17:55:29 -0400 Message-Id: <199404262155.RAA08203@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: Re: HZ+S, a proposal to extend HZ to include both GB and Big5 Reply-To: simpson@math.psu.edu Status: O > Very good proposal. Thanks. But maybe you are not so happy about the latest version, using ~G{ and ~H{. The question of tilde escapes in e-mail has been raised. I just looked in the SunOS 4.1.3 manual and found that ~G and ~H and ~G{ and ~H{ and ~{ and ~} are not among the tilde escapes defined for /usr/ucb/mail. But ~A and ~| are among them. Perhaps we need to be careful, especially so since international Big5 e-mail is an important possible application of HZ+S. -- Steve From mcpong@cs.ust.hk Tue Apr 26 22:56:10 1994 Return-Path: Received: from cssu2.cs.ust.hk ([143.89.40.2]) by ifcss.org (4.1/IFCSS-Mailer) id AA23325; Tue, 26 Apr 94 22:56:10 CDT Received: by cssu2.cs.ust.hk (4.1/SMI-4.1) id AA26252; Wed, 27 Apr 94 11:53:36 HKT Date: Wed, 27 Apr 94 11:53:36 HKT From: mcpong@cs.ust.hk (Dr. Man-Chi PONG) Message-Id: <9404270353.AA26252@cs.ust.hk> X-Mailer: Mail User's Shell (7.2.5 10/14/92) To: soft-authors@ifcss.org Subject: Re: proposals for 7-bit standard for mixed GB/Big5/ASCII Status: RO -- "Stephen G Simpson " wrote on 1994 Apr 26 (Tue): > > What we need is a way to distinguish between CNS Level 1, CNS Level > 2, GB, and ASCII. We can do so by using four tilde escape > sequences: > > ~{ to switch into GB (just as in the current HZ standard) > > ~} to switch into ASCII (just as in the current HZ standard) > > ~G{ to switch into Big5-CNS Level 1 (this is new) > > ~H{ to switch into Big5-CNS Level 2 (this is new) > > We provisionally call this proposal HZ+S. (If it becomes widely > accepted, we may call it HZ+.) HZ+S has several advantages: > > 1. HZ+S is a simple extension of the existing HZ standard. > > 2. Under HZ+S, the ~H{ escape sequence would be very seldom used, > because it is needed only to switch into Big5-CNS Level 2, the less > frequently used Chinese characters. Most of the time, we would only > use ~G{ to switch into Big5-CNS Level 1, plus ~{ and ~} as in HZ. > > ... I think the above HZ+S scheme is a simple and workable way to have mixed GB/Big5/ASCII in 7-bit data communication. One point is that, though CNS & Big5 describes roughly the same set of (traditional) Chinese characters, the code-points of many Chinese charcters are different in CNS & Big5. Currently HZ+S has evolved to use ~G{ & ~H{ to switch into CNS coding. ("Final characters" G & H are the recommendation of ISO-2022). However, many existing files are in Big5 and there are more software (or fonts) supporting Big5 than CNS. To make HZ+S backward compatibile with Big5 (without much implementation effort), probably escape code for switch into Big5 is still recommended. May be keeping ~1{ and ~2{ for Big5 level1 and level2 coding is a good idea. [ Don't know if ISO-2022 has some unused "final characters" (like G & H) so that they can be employed to represent Big5 level1 and level2 coding. ~1{ and ~2{ can be replaced by some other appropriate "final characters". ] -- mcpong -- in GB2312-80:~{ÅÓÃñÖÎ~}, in Big5: Ãe¥Áªv From simpson@math.psu.edu Wed Apr 27 01:10:42 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA27483; Wed, 27 Apr 94 01:10:42 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id CAA03173; Wed, 27 Apr 1994 02:08:45 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 27 Apr 1994 02:08:44 -0400 Date: Wed, 27 Apr 1994 02:08:44 -0400 Message-Id: <199404270608.CAA09083@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: Re: proposals for 7-bit standard for mixed GB/Big5/ASCII In-Reply-To: <9404270353.AA26252@cs.ust.hk> References: <9404270353.AA26252@cs.ust.hk> Reply-To: simpson@math.psu.edu Status: O mcpong@cs.ust.hk (Dr. Man-Chi PONG) writes: > One point is that, though CNS & Big5 describes roughly the same set of > (traditional) Chinese characters, the code-points of many Chinese charcters > are different in CNS & Big5. I don't think CNS and Big5 are so different. Except for two Chinese characters, CNS contains exactly the same Chinese characters as Big5, and in the same order. Big5<->CNS conversion is fairly simple. Have a look at Ross's writeup on coding schemes at http://ifcss.org:8001/www/pub/software/info/cjk-codes/index.html > many existing files are in Big5 and there are more software (or fonts) > supporting Big5 than CNS. That's true. I don't know of any software that handles CNS directly. Big5<->CNS conversion routines are needed. These routines would be not much more complicated than GB<->HZ conversion routines. > To make HZ+S backward compatibile with Big5 (without much implementation > effort), probably escape code for switch into Big5 is still recommended. > May be keeping ~1{ and ~2{ for Big5 level1 and level2 coding is a good idea. You mean different escapes for Big5 and for CNS? I don't see why. HZ+S needs only CNS. The CNS<->Big5 algorithms will be simple. I have asked Ross for the mapping data. -- Steve From simpson@math.psu.edu Wed Apr 27 01:26:57 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA27900; Wed, 27 Apr 94 01:26:57 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id CAA03232; Wed, 27 Apr 1994 02:24:50 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 27 Apr 1994 02:24:49 -0400 Date: Wed, 27 Apr 1994 02:24:49 -0400 Message-Id: <199404270624.CAA09109@boole.math.psu.edu> To: yawei@ifcss.org (Ya-Gui Wei) Cc: soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu Subject: Re: HZ+S, a proposal to extend HZ to include both GB and Big5 In-Reply-To: <9404270211.AA20185@ifcss.org> References: <9404270211.AA20185@ifcss.org> Reply-To: simpson@math.psu.edu Status: O Ya-Gui Wei writes: > I'd love to post my HZ-Big5 proposal for discussion, but I am > still looking for the most recent version. Well, I hope it turns up. > I spend very little time on the net these days, so probably > won't be able to participate in the discussions in a major way. That is unfortunate. > (1) I'd think it may be a better idea to keep the escape > sequences to 2-bytes in length; That is probably a good idea. But before I change them again, let's gather a little more information on possible conflicts with Berkeley mail tilde escapes, etc. > (2) Could you post the conversion procedures between CNS and > Big5 (and cc me a copy?) I have asked Ross for the mapping data. The conversion should be very simple. > (3) I don't think it is neccessary to maintain a similarity > with ISO-2022-JUNET. Why resemble it without being > compatible? I agree. There is no point in resembling it if we are not compatible with it. Maybe we should drop the idea of Junet compatibility. CNS-11643 compatibility seems much more important, because CNS-11643 is the official ROC Taiwan national standard for 7-bit Big5. I'd still like to keep a degree of ISO-2022 compatibility. ISO-2022 is a family of multilingual coding schemes, Junet is just one scheme. > On the other hand, if ISO-2022-JUNET is a 7-bit standard (as you > told me), then why don't just copy the whole thing from it? Because it is much more complicated than what the Internet Chinese community needs. It includes not only Big5 and GB but also Japanese, Korean, ISO-8859-*, etc. Another problem is that it does not use HZ-style escape sequences. I'd like to do something based on HZ, because HZ is so popular in the Chinese Internet community. -- Steve From handa@etlken.etl.go.jp Wed Apr 27 03:00:38 1994 Received: from etlpost.etl.go.jp by ifcss.org (4.1/IFCSS-Mailer) id AA00512; Wed, 27 Apr 94 03:00:38 CDT Received: from etlpom.etl.go.jp by etlpost.etl.go.jp (5.67+1.6W/2.7W) id AA10026; Wed, 27 Apr 94 16:52:53 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA28560; Wed, 27 Apr 94 16:52:52 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA29829; Wed, 27 Apr 94 16:53:00 JST Date: Wed, 27 Apr 94 16:53:00 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9404270753.AA29829@etlken.etl.go.jp> To: butta1@crsa.bu.edu Cc: simpson@math.psu.edu, yawei@ifcss.org, soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu In-Reply-To: C I I Project's message of Wed, 27 Apr 1994 02:58:34 -0400 (EDT) Subject: Re: HZ+S, a proposal to extend HZ to include both GB and Big5 (fwd) Status: O Hi everyone, My name is Kenichi HANDA, one of writers of Mule (Multilingual Enhancement to GNU Emacs). I'm also a coauthor of RFC of ISO-2022-JP-2. I have just got a forwarded mail from which is concerning to extend the current Hz encoding method. In the mail, you refered ISO-2022-JUNET, but there's no such RFC, there exist instead ISO-2022-JP and ISO-2022-JP2. The former is currently used in Japanese news and mail. The latter is the extention to the former for more character sets. > > (2) Could you post the conversion procedures between CNS and > > Big5 (and cc me a copy?) > I have asked Ross for the mapping data. The conversion should be very > simple. Really? I hearded that some CNS characters are missing in Big5 and vice versa. > > (3) I don't think it is neccessary to maintain a similarity > > with ISO-2022-JUNET. Why resemble it without being > > compatible? > I agree. There is no point in resembling it if we are not compatible > with it. Maybe we should drop the idea of Junet compatibility. > CNS-11643 compatibility seems much more important, because CNS-11643 > is the official ROC Taiwan national standard for 7-bit Big5. Since CNS-11643 is registered in ISO, it is assigned an official for designation. Here's a list of : 94charsets: ASCII 'B' JISX0201-KANA 'I' JISX0201-ROMAN 'J' GB2312 'A' 96charsets: ISO8859-1 'A' (Latin1) ISO8859-2 'B' (Latin2) ... ISO8859-7 'F' (Greak) ... 94x94charsets GB2312 'A' JISX0208 'B' KSC5601 'C' JISX0212 'D' CNS11643 Set-1 'G' CNS11643 Set-2 'H' Since ISO-2022-JP-2 is designed as fully compatible with ISO-2022, it can easily be extened to handle CNS11643 also. > Because it is much more complicated than what the Internet Chinese > community needs. It includes not only Big5 and GB but also Japanese, > Korean, ISO-8859-*, etc. Another problem is that it does not use > HZ-style escape sequences. I'd like to do something based on HZ, > because HZ is so popular in the Chinese Internet community. The only reason why HZ is simple is that it is designed only for two character sets ASCII and GB2312. If you try to extend HZ for more character sets, I think it inevitably be more complicated. But, I also think that ISO-2022-JP-2 is too complicated even for multi character sets. The reason is that we should have made it compatible with current Korean encoding method and ISO2022's ridiculous restriction of prohibiting designating 96-charset to G0. My first idea for ISO-2022-JP-2 was to use only the following escape sequences ( is any , see the above list): ESC $ -- designate any 94x94charsets ESC , -- designate any 96charset ESC ( -- designate ASCII Don't you think it simple and powerful? In addition, while you don't use any 96charset, it's fully compatible with ISO2022. --- Ken'ichi HANDA handa@etl.go.jp From simpson@math.psu.edu Wed Apr 27 05:59:23 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA04623; Wed, 27 Apr 94 05:59:23 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id GAA04121; Wed, 27 Apr 1994 06:57:26 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 27 Apr 1994 06:57:25 -0400 Date: Wed, 27 Apr 1994 06:57:25 -0400 Message-Id: <199404271057.GAA09447@boole.math.psu.edu> To: mcevilly@netcom.com (Carlos McEvilly) Cc: soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu Subject: Re: proposals for 7-bit standard for mixed GB/Big5/ASCII In-Reply-To: <199404270655.XAA22290@netcom.com> References: <199404270608.CAA09083@boole.math.psu.edu> <199404270655.XAA22290@netcom.com> Reply-To: simpson@math.psu.edu Status: O Carlos McEvilly writes: > For what it's worth, ETen handles CNS directly if your ET batch file > contains: > ETCTL/C:SI1,1 Interesting! I didn't know that. Does this mean that ETen can directly understand 7-bit CNS, with each character represented by two bytes of printable ASCII corresponding to its 94x94 CNS code point? If so, that is _very_ good news, because that essentially means that ETen can already handle HZ+S. There will be no immediate need to deploy special HZ+S software, because ETen (the most popular Big5 software) can already do the job. -- Steve From simpson@math.psu.edu Wed Apr 27 06:17:27 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA04987; Wed, 27 Apr 94 06:17:27 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id HAA04228; Wed, 27 Apr 1994 07:15:25 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 27 Apr 1994 07:15:23 -0400 Date: Wed, 27 Apr 1994 07:15:23 -0400 Message-Id: <199404271115.HAA09469@boole.math.psu.edu> To: Jianqing Hu Cc: Xiaokun Zhu , soft-authors@ifcss.org Subject: Re: HZ+S, a proposal to extend HZ to include both GB and Big5 In-Reply-To: <9404270724.AA32029@iitmax.acc.iit.edu> References: <199404270624.CAA09109@boole.math.psu.edu> <9404270724.AA32029@iitmax.acc.iit.edu> Reply-To: simpson@math.psu.edu Status: O Jianqing Hu writes: > The current HZ two byte escape sequence already gives a lot of > troubles. Many paginator programs can't hadle it correctly. > So there are celvis, cless. A 3 byte escape for CNS will > makes the situation even worse. Thanks for that information. Unless there are objections, we will probably change to 2-byte escape sequences, probably ~G and ~H to switch into Level 1 and Level 2 respectively, as was suggested by Xiaokun Zhu. > If the level2 is rarely used and level1 needs less then 94X94, > can we limit level2 within level1 ecape ? > example ~G{some level1 characters ~H{some level2 characters~H} ~} What would be the advantage of this limitation? In the current proposal, we can use ~H{ to switch into Level 2 any time, not only within Level 1. This is more in the style of ISO-2022. Also, note that in the current proposal, we don't have any ~H}. Instead we use ~} to switch to ASCII, ~G{ to switch to Level 1. And this ~G{ will probably be replaced by ~G. -- Steve From simpson@math.psu.edu Wed Apr 27 07:09:30 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA06364; Wed, 27 Apr 94 07:09:30 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id IAA04460; Wed, 27 Apr 1994 08:07:10 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 27 Apr 1994 08:07:08 -0400 Date: Wed, 27 Apr 1994 08:07:08 -0400 Message-Id: <199404271207.IAA09533@boole.math.psu.edu> To: handa@etlken.etl.go.jp (Kenichi Handa) Cc: butta1@crsa.bu.edu, yawei@ifcss.org, rap@doc.imperial.ac.uk, soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu Subject: Re: HZ+S, a proposal to extend HZ to include both GB and Big5 (fwd) In-Reply-To: <9404270753.AA29829@etlken.etl.go.jp> References: <9404270753.AA29829@etlken.etl.go.jp> Reply-To: simpson@math.psu.edu Status: O Kenichi Handa writes: > My name is Kenichi HANDA, one of writers of Mule > (Multilingual Enhancement to GNU Emacs). I'm also a > coauthor of RFC of ISO-2022-JP-2. Dear Mr. Handa, Oh, good! Welcome! I'm very glad for your participation. I should also mention that Kenichi Handa is the leader of the Mule team. Mule is a wonderful piece of software. Thanks, Nelson, for bringing Mr. Handa into the discussion. > In the mail, you refered ISO-2022-JUNET, but there's no such > RFC, there exist instead ISO-2022-JP and ISO-2022-JP2. > The former is currently used in Japanese news and mail. The > latter is the extention to the former for more character > sets. By ISO-2022-JUNET I was referring to the *junet* scheme used in Mule. Sorry for the confusion. I'll stop talking about ISO-2022-JUNET and start talking about ISO-2022-JP-2. > I heard that some CNS characters are missing in Big5 and vice versa. According to Ross's information at http://ifcss.org:8001/www/pub/software/info/cjk-codes/index.html, the Chinese characters in CNS are exactly the same as the Chinese characters in Big5, except for only two redundant characters in Big5 Level 2. Ross says that apart from these two Chinese characters, the differences between CNS and Big5 are minor and are mostly in the arrangement of the numerals, Roman and Greek alphabets, special symbols, etc. But thank you for bringing this point up. We really need to get to the bottom of this. We need precise information. I have asked Ross for the mapping data between CNS and Big5. Ross, where are you? > Since CNS-11643 is registered in ISO, it is assigned an > official for designation. Here's a list of > : > 94charsets: > ASCII 'B' > JISX0201-KANA 'I' > JISX0201-ROMAN 'J' > GB2312 'A' > 96charsets: > ISO8859-1 'A' (Latin1) > ISO8859-2 'B' (Latin2) > ... > ISO8859-7 'F' (Greak) > ... > 94x94charsets > GB2312 'A' > JISX0208 'B' > KSC5601 'C' > JISX0212 'D' > CNS11643 Set-1 'G' > CNS11643 Set-2 'H' > > Since ISO-2022-JP-2 is designed as fully compatible with > ISO-2022, it can easily be extened to handle CNS11643 also. Yes. That's why we are using G and H as part of the escape sequences for CNS Level 1 and 2. We want to have a degree of ISO-2022 compatibility. > The only reason why HZ is simple is that it is designed only > for two character sets ASCII and GB2312. If you try to > extend HZ for more character sets, I think it inevitably be > more complicated. Yes. The HZ standard and the HZ+S proposal are certainly not perfect. The motivation for HZ+S that the Internet Chinese community needs a practical, simple standard for 7-bit mixed GB/Big5/ASCII. Since the HZ standard for 7-bit mixed GB/ASCII already exists (de facto) and is quite popular, we are aiming to be downward compatible with HZ while at the same time gently pulling HZ in the direction of ISO-2022. I grant that ISO-2022 is technically superior to the HZ+S proposal, especially because it accommodates 96 character ISO-8859-*, which HZ+S can never do. But compromises are sometimes necessary. At least we can take care to design HZ+S so that it is essentially equivalent to a subset of ISO-2022 (except for using different escape sequences). I would very much appreciate your insight as to how we can do this. Actually, your help is essential! > My first idea for ISO-2022-JP-2 was to use only the following > escape sequences ( is any , see the above list): > ESC $ -- designate any 94x94charsets > ESC , -- designate any 96charset > ESC ( -- designate ASCII > Don't you think it simple and powerful? In addition, while > you don't use any 96charset, it's fully compatible with > ISO2022. Yes, this seems like a beautiful way to do things. Why was this not made the basis for ISO-2022-JP-2? I guess because of the Korean problem that you mentioned. What a pity. Anyway, welcome aboard! Sincerely, -- Steve Stephen G. Simpson Department of Mathematics, Pennsylvania State University 333 McAllister Building, University Park, State College, PA 16802 Office phone +1 814 863-0775 Home phone +1 814 238-2274 E-mail simpson@math.psu.edu FAX +1 814 865-3735 World Wide Web http://www.math.psu.edu/simpson/ From simpson@math.psu.edu Wed Apr 27 07:35:22 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA06879; Wed, 27 Apr 94 07:35:22 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id IAA04576; Wed, 27 Apr 1994 08:33:11 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 27 Apr 1994 08:33:10 -0400 Date: Wed, 27 Apr 1994 08:33:10 -0400 Message-Id: <199404271233.IAA09590@boole.math.psu.edu> To: handa@etlken.etl.go.jp (Kenichi Handa) Cc: butta1@crsa.bu.edu, rap@doc.imperial.ac.uk, yawei@ifcss.org, soft-authors@ifcss.org Subject: Big5 in Mule In-Reply-To: <9404270753.AA29829@etlken.etl.go.jp> References: <9404270753.AA29829@etlken.etl.go.jp> Reply-To: simpson@math.psu.edu Status: O Dear Mr. Handa, For the sake of the HZ+S discussion, could you please clarify, exactly how does Mule handle Big5 and related standards? What final characters are used? What internal codes are used? What 7-bit representations are used in Mule *junet*? Is it exactly CNS, or is it something a little different? Your help is very much appreciated! -- Steve From handa@etlken.etl.go.jp Wed Apr 27 07:44:03 1994 Received: from etlpost.etl.go.jp by ifcss.org (4.1/IFCSS-Mailer) id AA07123; Wed, 27 Apr 94 07:44:03 CDT Received: from etlpom.etl.go.jp by etlpost.etl.go.jp (5.67+1.6W/2.7W) id AA12021; Wed, 27 Apr 94 21:41:09 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA01198; Wed, 27 Apr 94 21:41:09 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA29988; Wed, 27 Apr 94 21:41:17 JST Date: Wed, 27 Apr 94 21:41:17 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9404271241.AA29988@etlken.etl.go.jp> To: simpson@math.psu.edu Cc: butta1@crsa.bu.edu, yawei@ifcss.org, rap@doc.imperial.ac.uk, soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Wed, 27 Apr 1994 08:07:08 -0400 <199404271207.IAA09533@boole.math.psu.edu> Subject: Re: HZ+S, a proposal to extend HZ to include both GB and Big5 (fwd) Status: O On Wed, 27 Apr 1994 08:07:08 -0400, > Dear Mr. Handa, > Oh, good! Welcome! I'm very glad for your participation. I'm glad too to get such an prompt reply. I sent the previous mail also to , but it was rejected because I'm not registered in the list. Someone please add me to the list and forward this mail to the above address. > By ISO-2022-JUNET I was referring to the *junet* scheme used in Mule. Ah! I see. *junet* of Mule is the same as my initial idea about ISO-2022-JP-2 which I mentioned in the previsou mail. > > I heard that some CNS characters are missing in Big5 and vice versa. > According to Ross's information at > http://ifcss.org:8001/www/pub/software/info/cjk-codes/index.html, the > Chinese characters in CNS are exactly the same as the Chinese > characters in Big5, except for only two redundant characters in Big5 > Level 2. I didn't know that... Hmm, then mapping Big5 to CNS seems a good idea to pass Big5 through 7-bit environment. > The motivation for HZ+S that the Internet Chinese community needs a > practical, simple standard for 7-bit mixed GB/Big5/ASCII. Since the > HZ standard for 7-bit mixed GB/ASCII already exists (de facto) and is > quite popular, we are aiming to be downward compatible with HZ while > at the same time gently pulling HZ in the direction of ISO-2022. I agree. Treating '~{' and '~}' of HZ just as aliases for 'ESC $ A' and 'ESC ( B' of ISO2022 seems to be a reasonable solution. Handling 3-byte sequence (such as 'ESC $ A') is not that more difficult than handling 2-byte sequence. --- Ken'ichi HANDA handa@etl.go.jp From handa@etlken.etl.go.jp Wed Apr 27 07:50:29 1994 Received: from etlpost.etl.go.jp by ifcss.org (4.1/IFCSS-Mailer) id AA07344; Wed, 27 Apr 94 07:50:29 CDT Received: from etlpom.etl.go.jp by etlpost.etl.go.jp (5.67+1.6W/2.7W) id AA12076; Wed, 27 Apr 94 21:48:02 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA01283; Wed, 27 Apr 94 21:48:01 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA29992; Wed, 27 Apr 94 21:48:09 JST Date: Wed, 27 Apr 94 21:48:09 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9404271248.AA29992@etlken.etl.go.jp> To: simpson@math.psu.edu Cc: butta1@crsa.bu.edu, rap@doc.imperial.ac.uk, yawei@ifcss.org, soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Wed, 27 Apr 1994 08:33:10 -0400 <199404271233.IAA09590@boole.math.psu.edu> Subject: Re: Big5 in Mule Status: O On Wed, 27 Apr 1994 08:33:10 -0400, Stephen G Simpson said: > For the sake of the HZ+S discussion, could you please clarify, exactly > how does Mule handle Big5 and related standards? What final > characters are used? What internal codes are used? What 7-bit > representations are used in Mule *junet*? Is it exactly CNS, or is it > something a little different? It is written in mule/doc/coding.txt as below, but now I'm considering the way of treating Big5 just as the same character set of CNS but in different coding scheme. ------------------------------------------------------------ 9. Special treatment of Big5 As far as I know, there's several different codes called Big5. The most famous ones are Big5-ETen and Big5-HKU-form2. Since both of them use a code range 0xa140 - 0xfefe (in each row, columns (second byte) 0x7f - 0xa0 is skipped) and number of characters is more than 13000, it's impossible to treat each of them as a single character-set in the current Mule system. So, Mule treat them in a quite irregular manner as described below: (1) Mule does not treats them as a different character set, but as the same character set called Big5. Caution!! Big5 is a different character set from GB. (2) Mule divides Big5 into two sub-character-sets: 0xa140 - 0xc67e (Level 1) 0xc6a1 - 0xfefe (Level 2) and allocates two leading-chars lc-big5-1 and lc-big5-2 to them. (See character.txt) (3) Usually, each leading-char (or character-set) has unique character category. But lc-big5-1 and lc-big5-2 has the same character category of mnemonic 't'. So, regular expression "\\ct" matches any Big5 (Level 1 and Level 2) characters. (See syntax.txt) (4) If you specify ISO2022 type coding-system on output, Mule converts Big5 code using unofficial final-characters '0' (for Level 1) and '1' (for Level 2). (5) You can use either fonts of ETen or HKU for displaying Big5 code. Mule judges which font is used by examining existence of character whose code point is 0xC6A1. If it exists, the font is HKU, else the fonts is ETen. ------------------------------------------------------------ In (2), Each level of Big5 is mapped into the range of 0x2121-0x7F7F. --- Ken'ichi HANDA handa@etl.go.jp From rap@doc.ic.ac.uk Wed Apr 27 09:06:58 1994 Return-Path: Received: from santos.doc.ic.ac.uk by ifcss.org (4.1/IFCSS-Mailer) id AA09624; Wed, 27 Apr 94 09:06:58 CDT Received: by santos.doc.ic.ac.uk (Smail3.1.28.1 #7) id m0pwADs-00070bC; Wed, 27 Apr 94 15:04 BST Message-Id: Date: Wed, 27 Apr 94 15:04 BST From: Ross Paterson To: simpson@math.psu.edu Subject: CNS and Big5 Cc: handa@etlken.etl.go.jp, soft-authors@ifcss.org Status: RO Stephen G Simpson writes: > mcpong@cs.ust.hk (Dr. Man-Chi PONG) writes: > > One point is that, though CNS & Big5 describes roughly the same set of > > (traditional) Chinese characters, the code-points of many Chinese charcters > > are different in CNS & Big5. > > I don't think CNS and Big5 are so different. Except for two Chinese > characters, CNS contains exactly the same Chinese characters as Big5, > and in the same order. Big5<->CNS conversion is fairly simple. I've been looking at it, and it isn't as simple as I'd thought. There are some minor re-arrangements in the symbol and alphabetic part, but CNS sets out to correct the Big5 hanzi set: apart from deleting the above-mentioned 2 characters as duplicates, about 14 hanzi are relocated (presumably on the basis of corrected stroke counts), which changes lots of codes. I apologize for spreading misinformation on this point. So the mapping is approximately 1-1, but it will need a table to be done simply. That will be simple and fast, but big on data space (about 35k for the CNS->Big5 map). > > many existing files are in Big5 and there are more software (or fonts) > > supporting Big5 than CNS. Whatever format is used will require some conversion to and from Big5 (since Big5 isn't mail-safe) so it may as well be an existing standard. Someone raised the issue of the tildes conflicting with Berkeley Unix mail. Not a problem, I think, since no-one is going to type these sequences. The mail program would complain about "~{" too, but no-one types it. -- Ross From simpson@math.psu.edu Wed Apr 27 09:09:18 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA09681; Wed, 27 Apr 94 09:09:18 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id KAA05178; Wed, 27 Apr 1994 10:01:14 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 27 Apr 1994 10:01:12 -0400 Date: Wed, 27 Apr 1994 10:01:12 -0400 Message-Id: <199404271401.KAA09773@boole.math.psu.edu> To: handa@etlken.etl.go.jp (Kenichi Handa) Cc: ccnet-l@uga.cc.uga.edu, butta1@crsa.bu.edu, rap@doc.imperial.ac.uk, yawei@ifcss.org, soft-authors@ifcss.org Subject: Re: Big5 in Mule In-Reply-To: <9404271248.AA29992@etlken.etl.go.jp> References: <199404271233.IAA09590@boole.math.psu.edu> <9404271248.AA29992@etlken.etl.go.jp> Reply-To: simpson@math.psu.edu Status: RO Kenichi Handa writes: > > For the sake of the HZ+S discussion, could you please clarify, exactly > > how does Mule handle Big5 and related standards? What final > > It is written in mule/doc/coding.txt as below, but now I'm > considering the way of treating Big5 just as the same > character set of CNS but in different coding scheme. I propose that we cooperate to make Mule and HZ+S compatible in this respect. CNS seems like a good way to go, but let's get a little more information. If CNS turns out to be very different from the de facto Big5 that is popular in Taiwan, we may want to reconsider. Ross? -- Steve From mcevilly@netcom.com Wed Apr 27 11:05:59 1994 Return-Path: Received: from netcom.com (netcom8.netcom.com) by ifcss.org (4.1/IFCSS-Mailer) id AA13837; Wed, 27 Apr 94 11:05:59 CDT Received: by netcom.com (8.6.8.1/SMI-4.1/Netcom) id JAA25920; Wed, 27 Apr 1994 09:05:15 -0700 From: mcevilly@netcom.com (Carlos McEvilly) Message-Id: <199404271605.JAA25920@netcom.com> Subject: Re: proposals for 7-bit standard for mixed GB/Big5/ASCII To: simpson@math.psu.edu Date: Wed, 27 Apr 1994 09:05:14 -0700 (PDT) Cc: ccnet-l@uga.bitnet, soft-authors@ifcss.org In-Reply-To: <199404271057.GAA09447@boole.math.psu.edu> from "Stephen G Simpson" at Apr 27, 94 06:57:25 am X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 2372 Status: RO > Carlos McEvilly writes: > > For what it's worth, ETen handles CNS directly if your ET batch file > > contains: > > ETCTL/C:SI1,1 And now I have to eat my words. I mistakenly thought that the "tongyonghanzibiaozhunjiaohuanma," as it is referred to (in Chinese) in Huang Da-Yi's "Zhongwenzima" book was the same "tongyongma" that is referred to in the ETen documentation. Apparently not--I just did an experiment, creating some sample codepoints and changing my batch file. The experiment failed. The etctl/c:si1,1 line did have an effect; the internal code used is certainly not Big5, but it is also not CNS, at least not the CNS listed in the document I have (Zhongwenbiaozhunjiaohuanma, CNS 11643 x 5012, printed June 1992). > Interesting! I didn't know that. Does this mean that ETen can > directly understand 7-bit CNS, with each character represented by two > bytes of printable ASCII corresponding to its 94x94 CNS code point? > If so, that is _very_ good news, because that essentially means that > ETen can already handle HZ+S. There will be no immediate need to > deploy special HZ+S software, because ETen (the most popular Big5 > software) can already do the job. Sorry to get you so excited over nothing! Still, the above command might be of interest. The '1' in the etctl/c:si1,1 line represents 'Tongyongma,' whatever that is. The default is 0, for Big5; other values are: 2 Gonghui (TCA) 3 ETen 4 Wang-An 5 IBM 5550 6-13 reserved ... If we can create a table in the same format used by ETen, we could use one of the reserved values to activate CNS usage. The ETen company might even already have such a table--perhaps one of the netters in Taiwan could check on this? It might be the case that for ETen to make use of CNS, the high bit might have to be turned on in both bytes. This is not a large problem, since it would affect all bytes, not just some of them as in Big5. Another bit of information that could prove useful: The reason the value '1' appears twice in the etctl/c:si1,1 line is that the first value determines which coding is used for reading and writing files, while the second value determines which coding is used to write to the display and the printer. Carlos McEvilly | email for info on Bamboo Helper Chinese text mcevilly@netcom.com | tools, or ftp to ftp.netcom.com:/pub/mcevilly From CAI@neurophys.wisc.edu Wed Apr 27 12:11:33 1994 Return-Path: Received: from neurophys.wisc.edu (mve.neurophys.wisc.edu) by ifcss.org (4.1/IFCSS-Mailer) id AA16103; Wed, 27 Apr 94 12:11:33 CDT Received: from VMSmail by neurophys.wisc.edu; Wed, 27 Apr 94 12:08 CST Message-Id: <24042712081368@neurophys.wisc.edu> Date: Wed, 27 Apr 94 12:08 CST From: CAI@neurophys.wisc.edu Subject: Some considerations To: soft-authors@ifcss.org, yawei@ifcss.org, lee@simd.stanford.EDU X-Vms-To: IN%"soft-authors@ifcss.org", IN%"yawei@ifcss.org",IN%"lee@simd.stanford.edu" Status: RO Since I initiated this round of discussion, I feel I should say something at this stage. 1. Since CNS is quite different from Big5, I consider it to be another coding scheme. And since it is NOT widely used, we'd better deal with Big5 only at this stage. (I personally consider it not wise for the Taiwan government to create another standard, but this is another issue). If it turned out that CNS is accepted by the public, it is not too late to incooperate it at that time. But it is not wise for us to "promote" this standard. Think about this, everytime you get an article in CNS, you need to convert to Big5 in order to process it with some other software that doesn't support CNS. 2. Our main purpose is to put GB and Big5 together, as the title we are all using says, so I don't think we should care much more about Japanese, Koreans. In practice, rarely we need to deal with Japanese and Chinese together. Even if they do mixed up in one article, most end users simply will not keep the fonts to view/print all of them. So let's concentrate on GB and Big5 for now, while keep an eye on others (if we get it for free, go ahead, otherwise forget about it). 3. I personally prefer to keep the escape sequence as two-bytes, as Ya-Gui said in his previous message, although I am not against using three-bytes if it is justified. Why not using ~{ switch to GB ~} switch to ASCII ~~ character ~ ~\n line continuation ~[ switch to Big5 level one ~] switch to Big5 level two (or -( ) The most important thing after we set a standard is to implement it. We need Ya-Gui and others to modify the software to support the new standard. Cai From sava!zpy@cs.UMD.EDU Wed Apr 27 13:16:16 1994 Return-Path: Received: from mimsy.cs.UMD.EDU by ifcss.org (4.1/IFCSS-Mailer) id AA18552; Wed, 27 Apr 94 13:16:16 CDT Received: from sava.UUCP by mimsy.cs.UMD.EDU (8.6.5/UMIACS-0.9/04-05-88) id OAA02105; Wed, 27 Apr 1994 14:14:00 -0400 Received: from chengdu by sava (NX5.67e/NX3.0M) id AA17731; Wed, 27 Apr 94 11:06:47 -0400 From: Zhiping You Message-Id: <9404271506.AA17731@sava> Received: by chengdu (NX5.67e/NX3.0X) id AA02845; Wed, 27 Apr 94 11:07:19 -0400 Date: Wed, 27 Apr 94 11:07:19 -0400 Received: by NeXT.Mailer (1.95) Received: by NeXT Mailer (1.95) To: soft-authors@ifcss.org Subject: sign off Status: RO Can somebody sign me off this mailing list. Thanks. From handa@etlken.etl.go.jp Wed Apr 27 19:24:51 1994 Received: from etlpost.etl.go.jp by ifcss.org (4.1/IFCSS-Mailer) id AA02004; Wed, 27 Apr 94 19:24:51 CDT Received: from etlpom.etl.go.jp by etlpost.etl.go.jp (5.67+1.6W/2.7W) id AA14668; Thu, 28 Apr 94 09:22:31 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA05671; Thu, 28 Apr 94 09:22:30 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA00577; Thu, 28 Apr 94 09:22:38 JST Date: Thu, 28 Apr 94 09:22:38 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9404280022.AA00577@etlken.etl.go.jp> To: rap@doc.ic.ac.uk Cc: simpson@math.psu.edu, soft-authors@ifcss.org In-Reply-To: Ross Paterson's message of Wed, 27 Apr 94 15:04 BST Subject: Re: CNS and Big5 Status: RO On Wed, 27 Apr 94 15:04 BST, Ross Paterson said: > I've been looking at it, and it isn't as simple as I'd thought. There are > some minor re-arrangements in the symbol and alphabetic part, but CNS sets > out to correct the Big5 hanzi set: apart from deleting the above-mentioned > 2 characters as duplicates, about 14 hanzi are relocated (presumably > on the basis of corrected stroke counts), which changes lots of codes. > I apologize for spreading misinformation on this point. > So the mapping is approximately 1-1, but it will need a table to be > done simply. That will be simple and fast, but big on data space (about > 35k for the CNS->Big5 map). I found a conversion routine between CNS and Big5 in the distribution of Wnn. It does the conversion as follows: * most of characters are converted simply by shifting and adding/subtracting some value. * some specific characters are converted directly one by one. The file name is 'Wnn/etc/xutoj.c'. So, once we can make this kind of conversion routine (written in C) de facto standard, it's not that difficult to modify the current chinese softwares. --- Ken'ichi HANDA handa@etl.go.jp From simpson@math.psu.edu Wed Apr 27 19:56:58 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA03009; Wed, 27 Apr 94 19:56:58 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id UAA10122; Wed, 27 Apr 1994 20:54:20 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 27 Apr 1994 20:54:19 -0400 Date: Wed, 27 Apr 1994 20:54:19 -0400 Message-Id: <199404280054.UAA10954@boole.math.psu.edu> To: soft-authors@ifcss.org Cc: handa@etlken.etl.go.jp, yawei@ifcss.org Subject: CNS and Big5 In-Reply-To: References: Reply-To: simpson@math.psu.edu Status: RO Ross Paterson writes: > Stephen G Simpson writes: > > I don't think CNS and Big5 are so different. Except for two Chinese > > characters, CNS contains exactly the same Chinese characters as Big5, > > and in the same order. Big5<->CNS conversion is fairly simple. > > I've been looking at it, and it isn't as simple as I'd thought. There are > some minor re-arrangements in the symbol and alphabetic part, but CNS sets > out to correct the Big5 hanzi set: apart from deleting the above-mentioned > 2 characters as duplicates, about 14 hanzi are relocated (presumably > on the basis of corrected stroke counts), which changes lots of codes. > I apologize for spreading misinformation on this point. Apology accepted. I too am guilty of spreading misinformation. So, it appears that CNS is not so very close to Big5 after all. This means that we have a serious decision to make: Do we want the future HZ+ standard to use CNS or Big5? I am now leaning strongly toward Big5, simply because Big5 is extremely popular. The point is that HZ is successful because it is related to GB in a simple way. My feeling is that HZ+ in order to be successful must likewise be related to Big5 in a simple way. My earlier enthusiasm for CNS was based on wrong assumptions about its relationship to Big5. I now think that CNS is a standard that apparently nobody is using. I assume that everyone agrees with me. > Someone raised the issue of the tildes conflicting with Berkeley Unix mail. > Not a problem, I think, since no-one is going to type these sequences. > The mail program would complain about "~{" too, but no-one types it. OK, that is good to know. We don't have to worry about Berkeley mail tilde escapes. So, I now go back to something like my earlier proposal. But let me state it more precisely. HZ+S Version 0.3 By Big5 we mean the ETen implementation of Big5, chosen because Eten is the most popular Big5 software. ETen Big5 is described precisely in my document ifcss.org:/software/fonts/d/etfonts.txt. The purpose of HZ+S is to merge GB, Big5, and ASCII into a standard 7-bit representation for convenient e-mail transmission, etc. GB already has a standard 7-bit representation, implemented in HZ. In order to give Big5 a standard 7-bit representation, we divide Big5 into two blocks. Blocks 1 and 2 consist of the ranges A140 to C87F and C940 to FDFE respectively, where the second byte is always restricted to the ranges 40 to 7F and A1 to FE. Block 1 contains 5401 more frequently used Chinese characters plus numerals, Roman and Greek and Cyrillic and Bopomofo alphabets, symbols, numerals, Japanese Kana, etc. Block 2 contains less frequently used Chinese characters plus some additional symbols. We pack each of the Block 1 and 2 ranges in order into an initial segment of the range 2121 to 7E7E where both bytes are restricted to the range 21 to 7E. This packing is implemented by a simple macro. Thus each character in each Big5 block is mapped to a code consisting of two printable ASCII bytes. In addition to the the above, we need a way to distinguish between Big5 Block 1, Big5 Block 2, GB, and ASCII. We can do so by using four tilde escape sequences: ~{ to switch into GB (just as in the current HZ standard) ~} to switch into ASCII (just as in the current HZ standard) ~1 to switch into Big5 Block 1 (this is new) ~2 to switch into Big5 Block 2 (this is new) We call this proposal HZ+S version 0.3. (If it becomes widely accepted, we will call it HZ+.) HZ+S has several advantages: 1. HZ+S is a simple extension of the existing HZ standard. 2. Under HZ+S, the ~2 escape sequence would hardly ever be used, because it is needed only to switch into Big5 Block 2, the block containing less frequently used Chinese characters. Most of the time, we would only use ~1 to switch into Big5 Block 1, plus ~{ and ~} as in HZ. 3. HZ+S is somewhat similar to Ya-Gui Wei's unreleased HZ-2 proposal. I would encourage Ya-Gui to release his proposal for discussion. Comments? -- Steve From mcevilly@netcom.com Wed Apr 27 21:04:50 1994 Return-Path: Received: from netcom.com (netcom12.netcom.com) by ifcss.org (4.1/IFCSS-Mailer) id AA05028; Wed, 27 Apr 94 21:04:50 CDT Received: by netcom.com (8.6.8.1/SMI-4.1/Netcom) id TAA17183; Wed, 27 Apr 1994 19:04:11 -0700 From: mcevilly@netcom.com (Carlos McEvilly) Message-Id: <199404280204.TAA17183@netcom.com> Subject: Re: CNS and Big5 To: ccnet-l@uga.bitnet Date: Wed, 27 Apr 1994 19:04:10 -0700 (PDT) Cc: soft-authors@ifcss.org In-Reply-To: <199404280054.UAA10954@boole.math.psu.edu> from "Stephen G Simpson" at Apr 27, 94 08:54:19 pm X-Mailer: ELM [version 2.4 PL23] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 507 Status: RO > 1 and 2 ranges in order into an initial segment of the range 2121 to > 7E7E where both bytes are restricted to the range 21 to 7E. This > packing is implemented by a simple macro. Thus each character in > each Big5 block is mapped to a code consisting of two printable > ASCII bytes. Small problem: 7E is ~. Would 2121-7DFF be a large enough range? Carlos McEvilly | email for info on Bamboo Helper Chinese text mcevilly@netcom.com | tools, or ftp to ftp.netcom.com:/pub/mcevilly From yawei Wed Apr 27 22:48:32 1994 Return-Path: Received: by ifcss.org (4.1/IFCSS-Mailer) id AA08605; Wed, 27 Apr 94 22:48:32 CDT Date: Wed, 27 Apr 94 22:48:32 CDT From: yawei (Ya-Gui Wei) Message-Id: <9404280348.AA08605@ifcss.org> To: butta1@crsa.bu.edu, handa@etlken.etl.go.jp Subject: Proposed HZ-like 7-bit rendition of Big-5 character set Cc: alt.chinese.computing@usenet.ucs.indiana.edu, ccnet-l@uga.cc.uga.edu, lee@fritter.stanford.edu, simpson@math.psu.edu, soft-authors, yawei Status: RO Thanks to Fung Fung Lee for helping me recovering the following document. Heeding Steve's suggestion, I am posting it here for comment without further change. Nonetheless, there are a few points that I have discussed with Fung and others and would like present them here seperately: 1. The spec probably should be presented as a 7-bit rendition of Big-5 without any inherent connection to HZ, i.e., a developer should feel free to implement it seperately and independently of HZ. Therefore the mention of conversion to HZ in the last paragraph should be deleted. 2. The proposal that Big-5 escape sequences may occur within GB context (and vice versa) probably should also be deleted. 3. The name of the spec probably should change to "HZ-Big5" or something similar. 4. Some of the escape sequences proposed should probably be changed. In particular "~|" so that users don't inadvertently execute a Unix command when trying to send mail. 5. The escape mechanism should be kept compatible with HZ and leave room (there are plenty of room) for future expansion. BTW, CCNET-L has been rejecting my postings. I haven't been a reader for a while. If someone deems it proper to forward this message there, please feel free to do so. Ya-Gui Wei ------- Date: Sat, 3 Jul 1993 19:17:11 -0500 >From: "Ya-Gui Wei" To: lee@fritter.stanford.edu Subject: HZ-2 spec *HZ2.TXT* [This is an ASCII document.] A Proposal For Extending The "HZ" Specification (Draft 1.1b) May 13, 1993 Last Revised: July 2, 1993 The "HZ" representation (Fung Fung Lee, 1989) of GB2312 Chinese text has been implemented in various software packages under different platforms, and tested in the usenet newsgroup "alt.chinese.text" and various other forums with considerable success. This proposal seeks to expand the specification to _simultaneously_ accommodate other Hanzi coding standards under an umbrella scheme that is analogous to the original "HZ" scheme (the current proposal only seeks to include characters in the Big5 character set). ** ENCODING GB2312 ** ===================== The Encoding of GB2312 characters will be in accordance to the original HZ specification (hereafter, HZ-1). All provisions in HZ-1, except as otherwise amended below, will remain valid. This proposed extended HZ specification may be referred to as HZ-2. ** NEW ESCAPE SEQUENCES ** ========================== Two new tilde- escape sequences will be defined in HZ-2: ~| -- Escape into Big5 block 1; ~+ -- Escape into Big5 block 2; The above escape sequences may occur anywhere in the ASCII context. They may also occur within the Hanzi context, provided that the first byte of the escape sequence aligns with the first byte of a Hanzi character which would have existed in the position. The ~{ escape sequence defined in HZ-1 is extended so that it can also occur within the Hanzi context, subject to the alignment requirement described in the previous paragraph. Its definition of "Escape to 7-bit GB" remains unchanged. The ~} sequence, which only occurs within Hanzi context and must be aligned as described above, serves to escape back to ASCII from all Hanzi context. Improperly aligned escape sequences in the Hanzi context will not be interpreted as escape sequences, but rather, will be regarded as components of Hanzi characters. Examples of valid HZ-2 sequences follow: ~{UbJGR;>d;0!#~} => All characters from GB. ~|)})M.A.n~} => All characters from Big5. ~|Ps~{WVTuC4Dn#?~} => First character is Big5. ~{KDJ.SV3F~+!m~} => Last character is Big5 (block 2). ** ENCODING Big5 ** ==================== The Big5 character set is divided into two blocks, with block 1 consisting of the following: - the 471 symbols; - the 5401 frequently used characters; - the first 1099 infrequently used characters; - and the unused code space between frequently used and infrequently used characters. and block 2 consists of the remaining infrequently used characters and the unused code space that follows. Big5 characters in Block 1 are encoded, in a HZ-2 document, by a pair of printable ASCII characters as follows. Note that the encoded block 1 characters must always follow the ~| escape sequence. Let: b1 = 1st byte of Big5 input (range 0xA1-0xCF) b2 = 2nd byte of Big5 input (range 0x40-0x7E, 0xA1-0xFE) a1 = 1st byte of HZ-2 output (range 0x21-0x7E) a2 = 2nd byte of HZ-2 output (range 0x21-0x7E) Then, C code for Big5 -> HZ-2 Conversion: a1 = ((b1 - 0xA1) << 1) + (b2 >> 7) + 0x21; a2 = b2 & 0x7F; Backward Conversion: b1 = ((a1 - 0x21) >> 1) + 0xA1; b2 = a2 | (((a1 - 0x21) & 1) << 7); Characters in Big5 block 2 are encoded in a similar fashion as described below. Note that the encoded block 2 characters must always follow the ~+ escape sequence. Let: b1, b2, a1, a2 same as for block 1, except that b1 now ranges 0xD0-0xFE. Then, Big5->HZ-2 Conversion: a1 = ((b1 - 0xD0) << 1) + (b2 >> 7) + 0x21; a2 = b2 & 0x7F; Backward Conversion: b1 = ((a1 - 0x21) >> 1) + 0xD0; b2 = a2 | (((a1 - 0x21) & 1) << 7); In words, the high bit of the second byte is shifted into the low bit of the first byte, while the relevant bits in the first byte (after converted to base 0 ordinal number) are shifted to the left by 1 position. Adding back 0x21 converts it into printable ASCII. Finally, byte 2 can be converted to ASCII by simply stripping off the high bit. [Note: An alternative would be not shifting the bits in the first byte, while the 8th bit of the second byte is moved to the 7th bit of the first byte. This has the disadvantage of creating an discontinous range for the output, and the characters would not remain in the same order as in the Big5 table. Moreover, in some languages, shifting the high bit of byte 2 to the low bit of byte 1, and the shifting of the rest of byte 1, can all be accomplished by a single rotate instruction, while the alternative is less straight forward. In C, the numbers of operations for both schemes are the same.] ** CONSIDERATIONS IN CONVERSION ** ================================== A simple conversion application with the above algorithms can be very easily implemented. However, to maintain maximal compatibility with HZ-1, a Big5 -> HZ-2 conversion program is recommended to include a Big5->GB conversion step, such that all Big5 characters that can be reversibly converted to GB are converted to GB before the conversion to HZ-2. In this way, the resulting HZ-2 file can, in the most part, be read with existing HZ-1 compliant software except for the few characters that cannot be reversibly converted to GB. [EOF] From simpson@math.psu.edu Wed Apr 27 22:51:29 1994 Return-Path: Received: from cnd.org (cnd.cnd.org) by ifcss.org (4.1/IFCSS-Mailer) id AA08719; Wed, 27 Apr 94 22:51:29 CDT Received: from leibniz.math.psu.edu by cnd.org (4.1/4.7) id AA26964; Wed, 27 Apr 94 19:55:20 PDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id WAA10658; Wed, 27 Apr 1994 22:51:54 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 27 Apr 1994 22:51:53 -0400 Date: Wed, 27 Apr 1994 22:51:53 -0400 Message-Id: <199404280251.WAA11116@boole.math.psu.edu> To: soft-authors@ifcss.org Cc: handa@etlken.etl.go.jp, yawei@ifcss.org Subject: small corrections to HZ+S 0.3 Status: O > > 1 and 2 ranges in order into an initial segment of the range 2121 to > > 7E7E where both bytes are restricted to the range 21 to 7E. This > > packing is implemented by a simple macro. Thus each character in > > each Big5 block is mapped to a code consisting of two printable > > ASCII bytes. Carlos writes: > Small problem: 7E is ~. Would 2121-7DFF be a large enough range? Sorry, my mistake. Instead of 2121 to 7E7E, make it 2121 to 7D7E. The first byte is restricted to 21 to 7D and the second byte is restricted to 21 to 7E. So ~ is not allowed as the first byte of a character code. This range is large enough. Also, another small error. In describing HZ+S I said "merge" when I should have said "mix". I'll correct these mistakes in HZ+S 0.4. But let's get some more feedback first. Does anyone foresee problems in allowing ~ to occur as the second byte of a character code? I'm wondering whether software could get confused and think that such a ~ is the first byte of a ~1 or ~2 escape. I propose that we avoid this problem by insisting that ~1 and ~2 escapes must always be immediately preceded by some whitespace. Could somebody please post the official HZ spec? -- Steve From handa@etlken.etl.go.jp Wed Apr 27 22:57:25 1994 Received: from etlpost.etl.go.jp by ifcss.org (4.1/IFCSS-Mailer) id AA08985; Wed, 27 Apr 94 22:57:25 CDT Received: from etlpom.etl.go.jp by etlpost.etl.go.jp (5.67+1.6W/2.7W) id AA16690; Thu, 28 Apr 94 12:54:58 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA07918; Thu, 28 Apr 94 12:54:57 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA00806; Thu, 28 Apr 94 12:55:05 JST Date: Thu, 28 Apr 94 12:55:05 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9404280355.AA00806@etlken.etl.go.jp> To: simpson@math.psu.edu Cc: soft-authors@ifcss.org, yawei@ifcss.org In-Reply-To: Stephen G Simpson's message of Wed, 27 Apr 1994 22:51:53 -0400 <199404280251.WAA11116@boole.math.psu.edu> Subject: Re: small corrections to HZ+S 0.3 Status: O On Wed, 27 Apr 1994 22:51:53 -0400, Stephen G Simpson said: > Does anyone foresee problems in allowing ~ to occur as the second byte > of a character code? I'm wondering whether software could get > confused and think that such a ~ is the first byte of a ~1 or ~2 > escape. I propose that we avoid this problem by insisting that ~1 and > ~2 escapes must always be immediately preceded by some whitespace. I think ~1 and ~2 are not appropriate sequences because '~' is used as an operator of C in conjunction with expression. Within C program, we may encounter such sequences. '~(' is also not appropriate by the same reason. How about '~[', '~]'? But, anyway, I still believe that mapping BIG5 to CNS is better even though: > out to correct the Big5 hanzi set: apart from deleting the above-mentioned > 2 characters as duplicates These two missing charcaters are (from Wnn/etc/xutoj.c): #define BIG5_MU 0xc94a /* MU on Tsang Jye */ #define BIG5_GRHNE 0xddfc /* GRHNE on Tsang Jye */ If these are really duplication of the other characters, we can map them to the other one on BIG5->CNS conversion. Then, of course, BIG5->CNS->BIG5 can't reproduce the original one if the original has the characters above, does it really a serious problem? As I wrote before, mapping of BIG5<->CNS doesn't require a table for whole characters. --- Ken'ichi HANDA handa@etl.go.jp From handa@etlken.etl.go.jp Thu Apr 28 05:46:22 1994 Received: from etlpost.etl.go.jp by ifcss.org (4.1/IFCSS-Mailer) id AA20035; Thu, 28 Apr 94 05:46:22 CDT Received: from etlpom.etl.go.jp by etlpost.etl.go.jp (5.67+1.6W/2.7W) id AA20152; Thu, 28 Apr 94 19:43:49 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA13239; Thu, 28 Apr 94 19:43:48 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA01058; Thu, 28 Apr 94 19:43:55 JST Date: Thu, 28 Apr 94 19:43:55 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9404281043.AA01058@etlken.etl.go.jp> To: yawei@ifcss.org Cc: butta1@crsa.bu.edu, alt.chinese.computing@usenet.ucs.indiana.edu, ccnet-l@uga.cc.uga.edu, lee@fritter.stanford.edu, simpson@math.psu.edu, soft-authors@ifcss.org, yawei@ifcss.org In-Reply-To: Ya-Gui Wei's message of Wed, 27 Apr 94 22:48:32 CDT <9404280348.AA08605@ifcss.org> Subject: Two kinds of Big5, ETen and HKU? Status: O Could someone tell me which Big5 is now standard, ETen or HKU? Long ago, I read an article in alt.chinese.text that ETen is the correct one and HKU should not be used. But, I just found that `big5' table in `csmaps' (which give a map between many CJK character sets and Unicode) is based on HKU. You can get the map from: METIS.COM [140.186.33.40]: /pub/csmaps Map of CNS->Unicode is also in the directory. The difference of HKU and ETen is that HKU starts Level2 from 0xC740 whereas ETen starts Level2 from 0xC940. --- Ken'ichi HANDA handa@etl.go.jp From rap@doc.ic.ac.uk Thu Apr 28 07:05:08 1994 Return-Path: Received: from cfast.doc.ic.ac.uk by ifcss.org (4.1/IFCSS-Mailer) id AA21758; Thu, 28 Apr 94 07:05:08 CDT Received: by cfast.doc.ic.ac.uk (Smail3.1.28.1 #7) id m0pwUnQ-0000OsC; Thu, 28 Apr 94 13:02 BST Message-Id: Date: Thu, 28 Apr 94 13:02 BST From: Ross Paterson To: handa@etlken.etl.go.jp Subject: Re: Two kinds of Big5, ETen and HKU? Cc: soft-authors@ifcss.org Status: O handa@etlken.etl.go.jp (Kenichi Handa) writes: > Long ago, I read an article in alt.chinese.text that ETen is > the correct one and HKU should not be used. > > But, I just found that `big5' table in `csmaps' (which give > a map between many CJK character sets and Unicode) is based > on HKU. You can get the map from: > METIS.COM [140.186.33.40]: /pub/csmaps > Map of CNS->Unicode is also in the directory. These are working drafts, I think. Final tables are placed in ftp://unicode.org/pub/MappingTables/EastAsia_maps/ Checking again, it seems that the 'big5' table doesn't reflect the HKU version: the space between the 2 hanzi levels is used for kana, Cyrillic and enclosed digits. -- Ross From simpson@math.psu.edu Thu Apr 28 08:37:46 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA23976; Thu, 28 Apr 94 08:37:46 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id JAA13526; Thu, 28 Apr 1994 09:35:44 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 28 Apr 1994 09:35:42 -0400 Date: Thu, 28 Apr 1994 09:35:42 -0400 Message-Id: <199404281335.JAA15569@boole.math.psu.edu> To: handa@etlken.etl.go.jp (Kenichi Handa) Cc: yawei@ifcss.org, butta1@crsa.bu.edu, ccnet-l@uga.cc.uga.edu, lee@fritter.stanford.edu, soft-authors@ifcss.org Subject: Two kinds of Big5, ETen and HKU? In-Reply-To: <9404281043.AA01058@etlken.etl.go.jp> References: <9404280348.AA08605@ifcss.org> <9404281043.AA01058@etlken.etl.go.jp> Reply-To: simpson@math.psu.edu Status: O Kenichi Handa writes: > The difference of HKU and ETen is that HKU starts Level2 > from 0xC740 whereas ETen starts Level2 from 0xC940. In my HZ+S 0.3 posting, I chose C940 as the starting point of Block 2, instead of C740 or C6A1. There were two reasons for this choice. First, I wanted to stay compatible with ETen, the most popular Big5 software, and in ETen the 7652 less frequently used Chinese characters start at C940. Another reason, which I didn't state previously, is that in this way HZ+S can accommodate both ETen Big5 and HKU Big5. The point is that HZ+S doesn't care whether you are using HKU Big5 or ETen Big5, because in both cases the less frequently used Chinese characters start at 2121 of HZ+S Block 2. In this way HZ+S can perhaps heal or unify the split between ETen Big5 and HKU Big5. -- Steve From simpson@math.psu.edu Thu Apr 28 08:50:27 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA24316; Thu, 28 Apr 94 08:50:27 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id JAA13592; Thu, 28 Apr 1994 09:48:23 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 28 Apr 1994 09:48:22 -0400 Date: Thu, 28 Apr 1994 09:48:22 -0400 Message-Id: <199404281348.JAA15584@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu Cc: soft-authors@ifcss.org Subject: HZ+S proposal, version 0.4 (essentially the same as 0.3) Reply-To: simpson@math.psu.edu Status: O It appears that CNS is not so very close to Big5 after all. This means that we have a serious decision to make: Do we want the future HZ+S or HZ+ standard to use CNS or Big5? I am now leaning strongly toward Big5, simply because Big5 is extremely popular. The point is that HZ is successful because it is related to GB in a simple way. My feeling is that HZ+S in order to be successful must likewise be related to Big5 in a simple way. Ross wrote: > Someone raised the issue of the tildes conflicting with Berkeley Unix mail. > Not a problem, I think, since no-one is going to type these sequences. > The mail program would complain about "~{" too, but no-one types it. That is good to know. We don't have to worry about conflict with Berkeley mail tilde escapes. A number of people have expressed a preference for two-byte tilde escapes, so let's move in that direction. So, I now go back to something similar to my original HZ+S proposal. But let me state it more precisely. HZ+S Version 0.4 By Big5 we mean the ETen implementation of Big5, chosen because Eten is the most popular Big5 software. ETen Big5 is described precisely in my document ifcss.org:/software/fonts/d/etfonts.txt. The purpose of HZ+S is to provide a 7-bit representation of mixed GB, Big5, and ASCII for convenient e-mail transmission, etc. GB already has a standard 7-bit representation, implemented in HZ. In order to give Big5 a standard 7-bit representation, we partition the Big5 code range into two blocks. Blocks 1 and 2 consist of the ranges A140 to C87F and C940 to FDFE respectively, where the second byte is restricted to the ranges 40 to 7F and A1 to FE. Block 1 contains 5401 more frequently used Chinese characters plus numerals, Roman and Greek and Cyrillic and Bopomofo alphabets, symbols, numerals, Japanese Kana, etc. Block 2 contains less frequently used Chinese characters plus some additional symbols. We pack each of the Block 1 and 2 ranges in order into an initial segment of the range 2121 to 7D7E where both bytes are restricted to the range 21 to 7E. This packing is implemented by a simple macro. Thus each character in each Big5 block is mapped to a code consisting of two printable ASCII bytes. In addition to the the above, we need a way to distinguish between Big5 Block 1, Big5 Block 2, GB, and ASCII. We can do so by using four tilde escape sequences: ~{ to switch into GB (just as in the current HZ standard) ~} to switch into ASCII (just as in the current HZ standard) ~1 to switch into Big5 Block 1 (this is new) ~2 to switch into Big5 Block 2 (this is new) We call this proposal HZ+S version 0.3. (If it becomes widely accepted, we will call it HZ+.) HZ+S has several advantages: 1. HZ+S is a simple extension of the existing HZ standard. 2. Under HZ+S, the ~2 escape sequence would hardly ever be used, because it is needed only to switch into Big5 Block 2, the block containing less frequently used Chinese characters. Most of the time, we would only use ~1 to switch into Big5 Block 1, plus ~{ and ~} as in HZ. 3. HZ+S is somewhat similar to Ya-Gui Wei's unreleased HZ-2 proposal. I would encourage Ya-Gui to release his proposal for discussion. Comments? -- Steve Stephen G. Simpson Department of Mathematics, Pennsylvania State University 333 McAllister Building, University Park, State College, PA 16802 Office phone +1 814 863-0775 Home phone +1 814 238-2274 E-mail simpson@math.psu.edu FAX +1 814 865-3735 World Wide Web http://www.math.psu.edu/simpson/ From Ricky.Yeung@Eng.Sun.COM Thu Apr 28 14:19:48 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA05817; Thu, 28 Apr 94 14:19:48 CDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA23840; Thu, 28 Apr 94 12:13:59 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA25124; Thu, 28 Apr 94 12:12:49 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA18473; Thu, 28 Apr 1994 12:13:14 +0800 Date: Thu, 28 Apr 1994 12:13:14 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9404281913.AA18473@hk.Eng.Sun.COM> To: handa@etlken.etl.go.jp Cc: yawei@ifcss.org, butta1@crsa.bu.edu, alt.chinese.computing@usenet.ucs.indiana.edu, ccnet-l@uga.cc.uga.edu, lee@fritter.stanford.edu, simpson@math.psu.edu, soft-authors@ifcss.org, yawei@ifcss.org In-Reply-To: Kenichi Handa's message of Thu, 28 Apr 94 19:43:55 JST <9404281043.AA01058@etlken.etl.go.jp> Subject: Is there really a Big5 Coding Scheme released by HKU? (Re: Two kinds of Big5, ETen and HKU?) Content-Length: 1546 Status: O >Could someone tell me which Big5 is now standard, ETen or >HKU? >Long ago, I read an article in alt.chinese.text that ETen is >the correct one and HKU should not be used. Let me clarify the so-called "HKU standard" once more. It's a perfect case of ~{RT6o4+6o~},~{O07GJ$JG~}. Maybe if I tell the entire story, it would settle that once for all. It went back to the June 4th period in 1989. One of the student societies of HKU was feeding June 4th related news in Big5 code to the outside world. A HKU student released a dos viewer program to read those news. He bundled a bitmap font with his program and hardwired the Big5 code range information in the code. I was then a graduate student, and was requested to put an earlier version of the font on my departmental machine for ftp. I examined the font and discovered its original source. I raised the question of its legitimacy and refused to archive it on my site. Later on, another version of the font was created and achieved in other sites. Afterward, someone converted the bitmap file to BDF (originally intended to be used with my xhzview program) using the incorrect code range information. Many software authors henceforth thought that it's a new Big5 standard by "HKU", and started "supporting" it. This is NOT a Big5 coding scheme released by HKU. Let me appeal to the Chinese software authors, PLEASE DON'T support this so-called "HKU" standard. It's too expensive to let this mistake propagate. Let's stop it now before this wrong becomes "standard". -Ricky From simpson@math.psu.edu Thu Apr 28 18:17:20 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA14758; Thu, 28 Apr 94 18:17:20 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id TAA17896; Thu, 28 Apr 1994 19:11:12 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 28 Apr 1994 19:11:10 -0400 Date: Thu, 28 Apr 1994 19:11:10 -0400 Message-Id: <199404282311.TAA16305@boole.math.psu.edu> To: "Ning Mosberger-Tang" Cc: shann@WANGWEI.MATH.NCU.EDU.TW, pinghua@econ.berkeley.edu, hch@CCCA.NCTU.edu.tw, ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org, yawei@ifcss.org Subject: Re: GB/Big5 problem in chinese.* In-Reply-To: <199404282139.AA11683@optima.cs.arizona.edu> References: <199404281908.PAA15999@boole.math.psu.edu> <199404282139.AA11683@optima.cs.arizona.edu> Reply-To: simpson@math.psu.edu Status: O > Remember that they won't be able to post HZ+S big5 articles before > on-line/off-line convertors are written according to HZ+S standard. > When such a convertor exists, there's good reason to expect that > a way to read HZ+S mixed gb/big5 articles exists as well or will soon > be developed. Based on the already existing GB<->HZ converter, it will be rather easy to write a GB<->HZ+S converter and a Big5<->HZ+S converter. However, if a user has both of these converters and no GB software, he will still be unable to read GB articles. And if a user has all these converters but no Big5 software, he will still be unable to read Big5 articles. > My thinking was that software which could read mixed HZ+S code > would soon catch up, provided that your proposals gets widely > accepted. Before that, people could use a subset of your HZ+S > (namely HZ code) to communicate. The HZ subset consists of ~{ and ~}. The Big5 subset consists of ~1 and ~2 and ~}. People could use the HZ subset to communicate in HZ, or they could use the Big5 subset to communicate in Big5. The HZ users would use their favorite HZ software (say ZWDOS), or their favorite GB software (say BYX or NJStar) plus the GB<->HZ+S converter. The Big5 users would use their favorite Big5 software (say ETen) plus the Big5<->HZ+S converter. But, I am beginning to understand your point. You are saying that you could throw newsgroups open to HZ+S postings and then wait for the newsreading software to catch up. There would be a period of chaos, but everybody would be able to read SOME of the articles using essentially only their existing software (be it GB/HZ software or Big5 software). Those who want to read ALL of the articles would have to upgrade their software. The upgraded software would incorporate both GB and Big5 display (the best option), or GB display and Big5->GB conversion, or Big5 display and GB->Big5 conversion. Eventually everybody who cares would get some kind of appropriate software. Great! This is interesting. It never occurred to me that HZ+S might be useful as a newsgroup standard. I must say that the idea of a mixed GB/Big5 newsgroup seems somewhat alarming, but it is starting to make sense to me. Does anybody else besides Ning think that this might be realistic? Best wishes, -- Steve From handa@etlken.etl.go.jp Thu Apr 28 19:38:50 1994 Received: from etlpost.etl.go.jp by ifcss.org (4.1/IFCSS-Mailer) id AA17232; Thu, 28 Apr 94 19:38:50 CDT Received: from etlpom.etl.go.jp by etlpost.etl.go.jp (5.67+1.6W/2.7W) id AA23169; Fri, 29 Apr 94 09:36:36 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA17015; Fri, 29 Apr 94 09:36:35 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA01283; Fri, 29 Apr 94 09:36:43 JST Date: Fri, 29 Apr 94 09:36:43 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9404290036.AA01283@etlken.etl.go.jp> To: rap@doc.ic.ac.uk Cc: soft-authors@ifcss.org In-Reply-To: Ross Paterson's message of Thu, 28 Apr 94 13:02 BST Subject: Re: Two kinds of Big5, ETen and HKU? Status: O On Thu, 28 Apr 94 13:02 BST, Ross Paterson said: > These are working drafts, I think. Final tables are placed in > ftp://unicode.org/pub/MappingTables/EastAsia_maps/ > Checking again, it seems that the 'big5' table doesn't reflect the HKU > version: the space between the 2 hanzi levels is used for kana, Cyrillic > and enclosed digits. Thank you for the info. I checked it. It is surely based on BIG5 ETen. BTW, I now have taipei15 and taipei24 as BIG5 fonts, but both of them don't have any character in 0xC6A1..0xC8FF (kana, cyrillic, etc...). Does anyone have a Big5 font which consists of those characters? --- Ken'ichi HANDA handa@etl.go.jp From simpson@math.psu.edu Thu Apr 28 19:42:58 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA17297; Thu, 28 Apr 94 19:42:58 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id UAA18284; Thu, 28 Apr 1994 20:38:55 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 28 Apr 1994 20:38:54 -0400 Date: Thu, 28 Apr 1994 20:38:54 -0400 Message-Id: <199404290038.UAA16428@boole.math.psu.edu> To: Jing-Shin Chang Cc: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: CNS converter. Re: 7-bit clean big5 In-Reply-To: <199404282339.TAA18033@leibniz.math.psu.edu> References: <199404282339.TAA18033@leibniz.math.psu.edu> Reply-To: simpson@math.psu.edu Status: RO Jing-Shin Chang writes: > Dear CCNET netters, > You don't need to invent another wheel when it is available!! > Z@D>]BDbFnGB]_D!T6oGDMD' Welcome to the discussion. We need information from people who are familiar with CNS. > The 3rd line of this email is an encoded CNS message, which > is the chinese translation of the 2nd line. This encoding > scheme is widely used in the mail and news systems in the > SEEDNET of the ROC. And it is probably what you are seeking > for as a 7-bit clean code. > (The SeedNet is the *.com.tw, *.org.tw and *.net.tw parts of the > Internet. However, only mail access is available due to some > legal problems in the ROC telecommunication laws.) > There are also free CNS fonts made available by the ROC government > on Jan. (They are 24x24 and 40x40 fonts though. This sounds good. Please tell us more. Is this form of CNS used anywhere else in addition to e-mail and news on SeedNet? What about *.edu.tw? What about the world outside the Internet -- commercial use, word processing? Where is the software that can handle CNS in this form? What is "Jan"? > I am strongly in favor of using the CNS code as the exchange > code for the chinese.* newsgroups and use plug-in conversion > programs, as mentioned by one of Ning's posting, between > the CNS and the reader's favorite coding system, so as to > hide the readers from the exchange code. I wonder how the HZ users will like this proposal? :-) -- Steve Stephen G. Simpson Department of Mathematics, Pennsylvania State University 333 McAllister Building, University Park, State College, PA 16802 Office phone +1 814 863-0775 Home phone +1 814 238-2274 E-mail simpson@math.psu.edu FAX +1 814 865-3735 World Wide Web http://www.math.psu.edu/simpson/ From simpson@math.psu.edu Thu Apr 28 21:01:42 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA20281; Thu, 28 Apr 94 21:01:42 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id VAA18673; Thu, 28 Apr 1994 21:59:45 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 28 Apr 1994 21:59:43 -0400 Date: Thu, 28 Apr 1994 21:59:43 -0400 Message-Id: <199404290159.VAA16534@boole.math.psu.edu> To: handa@etlken.etl.go.jp (Kenichi Handa) Cc: soft-authors@ifcss.org Subject: Big5 fonts In-Reply-To: <9404290036.AA01283@etlken.etl.go.jp> References: <9404290036.AA01283@etlken.etl.go.jp> Reply-To: simpson@math.psu.edu Status: O Kenichi Handa writes: > BTW, I now have taipei15 and taipei24 as BIG5 fonts, but > both of them don't have any character in 0xC6A1..0xC8FF > (kana, cyrillic, etc...). Does anyone have a Big5 font > which consists of those characters? The KC fonts appear to contain virtually everything that is in ETen fonts, including Kana, Cyrillic, etc. The advantage of the KC fonts is that they are free. The disadvantage is that the Kana, etc. is not always in the same locations as in the ET fonts. To partially overcome this disadvantage, I created an HBF header file which makes the 24x24 KC Kai and Ming fonts look almost exactly like the 24x24 ETen Kai and Ming fonts. These header files are available at /ifcss.org:/software/fonts/HBF: -rw-rw-r-- 1 109 2676 Nov 8 07:59 etkck24.hbf -rw-rw-r-- 1 109 2678 Nov 8 07:59 etkcm24.hbf and the necessary bitmap font files are at /ifcss.org:/software/fonts: -rw-rw-r-- 1 109 1005376 Aug 28 1993 kcchin24.f00 -rw-rw-r-- 1 109 1005376 Aug 28 1993 kcchin24.f02 By running hbftobdf on these HBF fonts, you can create BDF fonts which are almost identical to the ETen 24x24 Kai and Ming fonts. Hbftobdf is at /ifcss.org:/software/fonts/HBF/c: -rw-rw-r-- 1 120 11325 Oct 18 1993 hbf.tar.gz Sorry, I can't help you with the 15x15 ETen fonts. -- Steve Stephen G. Simpson Department of Mathematics, Pennsylvania State University 333 McAllister Building, University Park, State College, PA 16802 Office phone +1 814 863-0775 Home phone +1 814 238-2274 E-mail simpson@math.psu.edu FAX +1 814 865-3735 World Wide Web http://www.math.psu.edu/simpson/ From simpson@math.psu.edu Fri Apr 29 22:01:06 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA13429; Fri, 29 Apr 94 22:01:06 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id WAA28440; Fri, 29 Apr 1994 22:59:07 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 29 Apr 1994 22:59:06 -0400 Date: Fri, 29 Apr 1994 22:59:06 -0400 Message-Id: <199404300259.WAA18754@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: HZ+S <-> Big5 routines Reply-To: simpson@math.psu.edu Status: RO Based on Fung's HZ <-> GB programs, I hacked up some HZ+S <-> Big5 programs. Unfortunately I have no time this weekend to test and debug them. In case anybody wants to look at them, they are at math.psu.edu, /pub/simpson/chinese/hzps. -- Steve From cyl Sat Apr 30 12:32:20 1994 Return-Path: Received: by ifcss.org (4.1/IFCSS-Mailer) id AA08436; Sat, 30 Apr 94 12:32:20 CDT Date: Sat, 30 Apr 1994 12:32:19 -0500 (CDT) From: Nelson CHIN Subject: Re: iii conversion code, Big-5 <--> 7-bit CNS To: Stephen G Simpson Cc: Software Authors , shin@hermes.ee.nthu.edu.tw In-Reply-To: <199404301415.KAA19935@boole.math.psu.edu> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Status: RO On Sat, 30 Apr 1994, Stephen G Simpson wrote: > Would you like me to upload this to ifcss.org? The gzipped tar file > is 200 K. Which would you prefer: gzipped tar, compressed tar, zip, > or what? I'm not sure where you would want to store it. There is C > source code and a DOS executable. The source compiles with no problem > on a Sparcstation. you can gzip tar the src, and i'll put it in software/unix/convert there's already a dos zip package of chcode, in software/dos/convert, i'm not sure if they're the same.. you can go and check. > > -- Steve > > From: shin@hermes.ee.nthu.edu.tw (Jing-Shin Chang) > To: simpson@math.psu.edu > Subject: Re: CNS converter. Re: 7-bit clean big5 > Date: Sun, 1 May 1994 09:37:03 +0800 > > How can I use this stuff to convert between Big5 and 7-bit CNS? > > You can type: > > chcode B big5_file P cns-p_file > > with the distributed version. The B (input) and P (output) code > designator characters can be replaced by other capital letters for > other codes: > > I: IBM5550, E: EUC, B: Big5, N: CNS (internal, *), > P: CNS (7-bit version), H: IBM Host codes, > T: TCA (Taipei Computer Association recommended code) > L: Telgram code > > You can convert files in the reverse direction like: > > chcode P cns-p_file B big5_file. > > This distribution is not designed as a filter to accept character > streams from stdin and to stdout. Therefore, it is awkard as a plug-in > module for converting betwee the Taiwanese codes. I had promised > Nelson of IFCSS to upload a modified version of my own in a few days, > which could be used as a filter or as a stand alone converter. My revision > had provided interfaces like this: > > ccf -bp big5_file cns-p_file > ccf -bp < big5_file > cns-p_file > cat big5_file (or other stdin stream) | ccf -bp | more > > or something like that. I used to use my revison in reading the seednet > postings, which use CNS-P codes, by using the '| ccf -pb |more' pipeline > commands together with my 'rn' for my Big5 terminal. And I am developing > a (unfully tested) on-line conversion program, which turns a chinese > terminal into other different chinese modes (much like the 'hztty' > program.) But, since my time is tight for my PhD program, such a work > can be done only at my leisure time. So I won't announce this program > until it is fully tested. how about merging this on-line conversion program with hztty, and updating it to include the HZ+S, when it's standardized. nelson From simpson@math.psu.edu Sat Apr 30 12:38:51 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA08786; Sat, 30 Apr 94 12:38:51 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id NAA01594; Sat, 30 Apr 1994 13:36:48 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Sat, 30 Apr 1994 13:36:47 -0400 Date: Sat, 30 Apr 1994 13:36:47 -0400 Message-Id: <199404301736.NAA20397@boole.math.psu.edu> To: soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu Subject: HZ+S <-> Big5 should be OK now Reply-To: simpson@math.psu.edu Status: RO There was a bad bug in the HZ+S <-> Big5 programs. I think I've fixed it now. I've also cleaned up the code a little bit. The new version, 0.51, is at ftp.math.psu.edu, /pub/chinese/hzps. Please test it if you have time. If you can, try it on files that have ASCII and Big5 block 1 and Big5 block 2 all mixed on the same line, different lines, etc. Also, try it on files with lines of various lengths including some very long lines. Let me know what happens. -- Steve From simpson@math.psu.edu Sat Apr 30 13:00:33 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA09543; Sat, 30 Apr 94 13:00:33 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id NAA01712; Sat, 30 Apr 1994 13:58:35 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Sat, 30 Apr 1994 13:58:34 -0400 Date: Sat, 30 Apr 1994 13:58:34 -0400 Message-Id: <199404301758.NAA20443@boole.math.psu.edu> To: Nelson CHIN Cc: ccnet-l@uga.cc.uga.edu, Software Authors , shin@hermes.ee.nthu.edu.tw Subject: III chcode, Big-5 <--> 7-bit CNS In-Reply-To: References: <199404301415.KAA19935@boole.math.psu.edu> Reply-To: simpson@math.psu.edu Status: RO I wrote: > Would you like me to upload this to ifcss.org? Nelson wrote: > how about merging this on-line conversion program with hztty, and > updating it to include the HZ+S, when it's standardized. I don't know who is maintaining hztty. I also don't know who is maintaining the III chcode code conversion package, which seems like a pretty big project. So I would rather not try to merge these two packages now. If the maintainers of hztty or III chcode want to incorporate HZ+S at some point in the future, I'd be happy to cooperate with them. For now, I would prefer to upload III chcode in its present form to ifcss.org. The reason is that some people have expressed interest in 7-bit CNS for use in mail, newsgroups, etc. If the III chcode CNS <-> Big5 conversion routines work well and can easily be incorporated into other software, there may be no need for HZ+S. So, let's make the routines available and let people play with them. -- Steve From cyl Sat Apr 30 13:30:15 1994 Return-Path: Received: by ifcss.org (4.1/IFCSS-Mailer) id AA10585; Sat, 30 Apr 94 13:30:15 CDT Date: Sat, 30 Apr 1994 13:30:14 -0500 (CDT) From: Nelson CHIN Subject: Re: III chcode, Big-5 <--> 7-bit CNS To: Stephen G Simpson Cc: Software Authors , alt.chinese.computing@usenet.ucs.indiana.edu, shin@hermes.ee.nthu.edu.tw In-Reply-To: <199404301758.NAA20443@boole.math.psu.edu> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Status: RO On Sat, 30 Apr 1994, Stephen G Simpson wrote: > 7-bit CNS for use in mail, newsgroups, etc. If the III chcode CNS <-> > Big5 conversion routines work well and can easily be incorporated into > other software, there may be no need for HZ+S. So, let's make the > routines available and let people play with them. > > -- Steve it's true if HZ+S is just to provide 7-8 bit conversion of just big5 text w/o gb code , but i see hz+s as a good foundation for mixed code text, and when softwares can handle mix code, then they can also keep up with the internationalisation trend. nelson