From shin@hermes.ee.nthu.edu.tw Wed Jun 1 00:43:39 1994 Return-Path: Received: from nthu.edu.tw by ifcss.org (4.1/IFCSS-Mailer) id AA26128; Wed, 1 Jun 94 00:43:39 CDT Received: from hermes.ee.nthu.edu.tw ([140.114.28.27]) by nthu.edu.tw with SMTP id <181936(1)>; Wed, 1 Jun 1994 13:47:20 +0800 Received: by hermes.ee.nthu.edu.tw (4.1/SMI-4.0/shin-1.0) id AA02319; Wed, 1 Jun 94 13:39:30 CST Date: Thu, 2 Jun 1994 03:39:30 +0800 From: shin@hermes.ee.nthu.edu.tw (Jing-Shin Chang) Message-Id: <9406010539.AA02319@hermes.ee.nthu.edu.tw> To: ccnet-l@uga.uga.edu, soft-authors@ifcss.org Subject: Logic Reasoning (was: re: Don't quote selectively) Status: RO I hope that Steve could answer the following questions, then I will understand whether the problem I raised was fully understood, or there is a misunderstanding due to language barrier (or simply because some illogical logic reasoning). (Answers from other netters are also welcome including falmes. That will help me to know whether I should post it in HZ(+). -^^-) 1. The following posting is by John entitled "Sonnet" to chinese.test. It was encoded with HZ-S. How many lines are there which contain interruptting ~2 characters in this 14-line poem? (including interrupting ~1 chars if a line begins with a ~2 char.) ~1YC5BVwVw'y~2.%~10M~} ~2-4Ec~1FG(n(,5>*u~} ~11gL"*]U]1C~2^w~1Kq~} ~1>| (B5E2 == HZ-S) > (HZ-2 == B5E3) (>: is more frequent than, ==: is equally frequent as) In other words, for ANY big5 text, B5E1 will contain more ~2 chars than the others, B5E2 and HZ-S will have the same number of ~2 chars, but they contain more ~2 chars than the other two schemes (namely HZ-2 and B5E3). HZ-2 and B5E3 contain the least number of ~2 chars among all. And they both contain the same number of ~2 chars. (What a lengthy description, hope that is clear enough. -^^-) 3. If your answer to 2. is NO. Can you explain it? (Preferably with mathematics or some kind of formal logic.) (If you don't agree with that, I don't know what can be said ...) 4. If someone reported that, based on HZ-2 or B5E3, there are 6% to 10% lines that contain interrupting ~2 characters, do you think HZ-S will have a fewer number of such lines than this 6% (or 10%) figure. (In your previous postings, your answer seems to be YES, implicitly. Could you answer this more explicitly?) In addition, do you think all the other HZ+ proposals (excluding HZ-2) will have a fewer number of ~2 chars than this figure. (We know that B5E1 has much more, and this is what you tried to emphasized. But the problem I raised does not depend on this, do you agree? I mean that all the other proposals available, excluding HZ-2, will NOT have a less number of ~2 chars, no matter how they defined their ~1 and ~2 blocks, than this figure. Do you agree?) 5. Did you recognize that I NOT ONLY test my Betty with B5E1 but also with B5E3? (I think you at least know that I had tested the files with B5E3.) (Also, B5E2, which has as many switching as HZ-S, is also tested as indicated in your previous posting which recite my first posting on this problem.) 6. Did you know that the 6% and 10% figures were based on tests over 2M bytes of big5 files (1M+ from HXWZ and 1M+ from Professor Shann's collection on Chinese literatures, respectively)? Did you recognize that the figures are NOT based on B5E1? 7. How many tests had you done on the problem I raised? What are the sample sizes? or you don't think it requires any testing? 8. Did you know that I was concerning about on-line reading and conversion for news postings, NOT on downloading files and converting them off-line? (Under such circumstances, I don't think any current implementation of HZ+ filters are not hiding those esc. chars. used by HZ+.) 9. Did you know (or ever make any reasonable guesses on) the technical problems why a good general purpose viewer (or code converter) like hztty do not strip the escape chars used in HZ when reading HZ newsgroup? 10. Did you think that I was rasing the problem to advocate, explicitly or implicitly, my B5E3 over your HZ-S? (If your answer is YES. I am sorry to say that my answer is NO.) (In fact, when the problem was posted, B5E3 is still not announced.) 11. What is a good news reader or mail reader you thought of when the HZ+ standard is announced as a formal standard for the chinese.* newsgroup and related mailing list? Note: In Big5 user's daily environment, they don't need to prepare a file off-line, convert it with a filter, then post it to the newsgroup. They just post it or follow-up directly using non-complicated (non-multi-lingual) news readers and an ordinary editor (e.g., the default "vi" editor available in any Unix system). They don't need to download a posting and convert it with a filter to remove those esc. sequences. They don't see the escape chars within their text even with a less complicated news reader. If we are to provide HZ+-aware news reader that's OUR problems to consider all these. 12. Based on your familiarity with ETen, KC, Mule and cxterm, did you think they are useful for resolving the problem I raised? Did you think other news readers are equally easy for resolving the problem? Is there any other news readers other than the multi-lingual Mule which provides an easy solution? Finally, I would like to withdraw all my Big5/7 proposals, so that our focus is not limited to the encoding schemes. I would be happy to expect that there be soon available popular news readers that are HZ+-aware (and Big5-Users-aware). I would also be appreciate to those who are familiar with Big5 users' envirnments if they could make their solutions widely available. But a final word of reminding: before putting any HZ+ standard to the chinese.*, please take care of the multi-lingual nature of the HZ+ and the extra problems (if you think that they exist) introduced by the 2-block encoding of Big5. Be sure that you are careful enough in considering every step of the interactions between the users (especially the Big5 users) and the news readers or mail readers. Simulate a HZ+ user's environment carefully before one post his/her suggestions or solutions. We need empirical works, not heuristics. I would be happy to be an ordinary big5 user, and I will choose the most favorable one, be it tw.chinese.* or HZ+-based chinese.*. If tw.chinese.* turn out to be my favorite, I hope that tw.chinese.* will not have much problem in mirrowing the HZ+-based newsgroups, which is a little more complicated due to its multi-lingual natural. I had emphasized a little from a Big5 user's point of view. But I think the HZ users also have to change, to some extent, their software environment in order to read the HZ+ postings (esp. when a mixed Big5/HZ article comes to your terminal) if it is accepted as the formal spec. So just let us focus on OUR problems. -^^- From shin@hermes.ee.nthu.edu.tw Wed Jun 1 00:52:50 1994 Return-Path: Received: from nthu.edu.tw by ifcss.org (4.1/IFCSS-Mailer) id AA26397; Wed, 1 Jun 94 00:52:50 CDT Received: from hermes.ee.nthu.edu.tw ([140.114.28.27]) by nthu.edu.tw with SMTP id <181936(2)>; Wed, 1 Jun 1994 13:56:28 +0800 Received: by hermes.ee.nthu.edu.tw (4.1/SMI-4.0/shin-1.0) id AA02364; Wed, 1 Jun 94 13:49:10 CST Date: Thu, 2 Jun 1994 03:49:10 +0800 From: shin@hermes.ee.nthu.edu.tw (Jing-Shin Chang) Message-Id: <9406010549.AA02364@hermes.ee.nthu.edu.tw> To: soft-authors@ifcss.org Subject: Logic Reasoning (was: re: Don't quote selectively) Status: O I hope that Steve could answer the following questions, then I will understand whether the problem I raised was fully understood, or there is a misunderstanding due to language barrier (or simply because some illogical logic reasoning). (Answers from other netters are also welcome including falmes. That will help me to know whether I should post it in HZ(+). -^^-) 1. The following posting is by John entitled "Sonnet" to chinese.test. It was encoded with HZ-S. How many lines are there which contain interruptting ~2 characters in this 14-line poem? (including interrupting ~1 chars if a line begins with a ~2 char.) ~1YC5BVwVw'y~2.%~10M~} ~2-4Ec~1FG(n(,5>*u~} ~11gL"*]U]1C~2^w~1Kq~} ~1>| (B5E2 == HZ-S) > (HZ-2 == B5E3) (>: is more frequent than, ==: is equally frequent as) In other words, for ANY big5 text, B5E1 will contain more ~2 chars than the others, B5E2 and HZ-S will have the same number of ~2 chars, but they contain more ~2 chars than the other two schemes (namely HZ-2 and B5E3). HZ-2 and B5E3 contain the least number of ~2 chars among all. And they both contain the same number of ~2 chars. (What a lengthy description, hope that is clear enough. -^^-) 3. If your answer to 2. is NO. Can you explain it? (Preferably with mathematics or some kind of formal logic.) (If you don't agree with that, I don't know what can be said ...) 4. If someone reported that, based on HZ-2 or B5E3, there are 6% to 10% lines that contain interrupting ~2 characters, do you think HZ-S will have a fewer number of such lines than this 6% (or 10%) figure. (In your previous postings, your answer seems to be YES, implicitly. Could you answer this more explicitly?) In addition, do you think all the other HZ+ proposals (excluding HZ-2) will have a fewer number of ~2 chars than this figure. (We know that B5E1 has much more, and this is what you tried to emphasized. But the problem I raised does not depend on this, do you agree? I mean that all the other proposals available, excluding HZ-2, will NOT have a less number of ~2 chars, no matter how they defined their ~1 and ~2 blocks, than this figure. Do you agree?) 5. Did you recognize that I NOT ONLY test my Betty with B5E1 but also with B5E3? (I think you at least know that I had tested the files with B5E3.) (Also, B5E2, which has as many switching as HZ-S, is also tested as indicated in your previous posting which recite my first posting on this problem.) 6. Did you know that the 6% and 10% figures were based on tests over 2M bytes of big5 files (1M+ from HXWZ and 1M+ from Professor Shann's collection on Chinese literatures, respectively)? Did you recognize that the figures are NOT based on B5E1? 7. How many tests had you done on the problem I raised? What are the sample sizes? or you don't think it requires any testing? 8. Did you know that I was concerning about on-line reading and conversion for news postings, NOT on downloading files and converting them off-line? (Under such circumstances, I don't think any current implementation of HZ+ filters are not hiding those esc. chars. used by HZ+.) 9. Did you know (or ever make any reasonable guesses on) the technical problems why a good general purpose viewer (or code converter) like hztty do not strip the escape chars used in HZ when reading HZ newsgroup? 10. Did you think that I was rasing the problem to advocate, explicitly or implicitly, my B5E3 over your HZ-S? (If your answer is YES. I am sorry to say that my answer is NO.) (In fact, when the problem was posted, B5E3 is still not announced.) 11. What is a good news reader or mail reader you thought of when the HZ+ standard is announced as a formal standard for the chinese.* newsgroup and related mailing list? Note: In Big5 user's daily environment, they don't need to prepare a file off-line, convert it with a filter, then post it to the newsgroup. They just post it or follow-up directly using non-complicated (non-multi-lingual) news readers and an ordinary editor (e.g., the default "vi" editor available in any Unix system). They don't need to download a posting and convert it with a filter to remove those esc. sequences. They don't see the escape chars within their text even with a less complicated news reader. If we are to provide HZ+-aware news reader that's OUR problems to consider all these. 12. Based on your familiarity with ETen, KC, Mule and cxterm, did you think they are useful for resolving the problem I raised? Did you think other news readers are equally easy for resolving the problem? Is there any other news readers other than the multi-lingual Mule which provides an easy solution? Finally, I would like to withdraw all my Big5/7 proposals, so that our focus is not limited to the encoding schemes. I would be happy to expect that there be soon available popular news readers that are HZ+-aware (and Big5-Users-aware). I would also be appreciate to those who are familiar with Big5 users' envirnments if they could make their solutions widely available. But a final word of reminding: before putting any HZ+ standard to the chinese.*, please take care of the multi-lingual nature of the HZ+ and the extra problems (if you think that they exist) introduced by the 2-block encoding of Big5. Be sure that you are careful enough in considering every step of the interactions between the users (especially the Big5 users) and the news readers or mail readers. Simulate a HZ+ user's environment carefully before one post his/her suggestions or solutions. We need empirical works, not heuristics. I would be happy to be an ordinary big5 user, and I will choose the most favorable one, be it tw.chinese.* or HZ+-based chinese.*. If tw.chinese.* turn out to be my favorite, I hope that tw.chinese.* will not have much problem in mirrowing the HZ+-based newsgroups, which is a little more complicated due to its multi-lingual natural. I had emphasized a little from a Big5 user's point of view. But I think the HZ users also have to change, to some extent, their software environment in order to read the HZ+ postings (esp. when a mixed Big5/HZ article comes to your terminal) if it is accepted as the formal spec. So just let us focus on OUR problems. -^^- From shin@hermes.ee.nthu.edu.tw Wed Jun 1 05:18:08 1994 Return-Path: Received: from nthu.edu.tw by ifcss.org (4.1/IFCSS-Mailer) id AA03275; Wed, 1 Jun 94 05:18:08 CDT Received: from hermes.ee.nthu.edu.tw ([140.114.28.27]) by nthu.edu.tw with SMTP id <181947(1)>; Wed, 1 Jun 1994 18:21:44 +0800 Received: by hermes.ee.nthu.edu.tw (4.1/SMI-4.0/shin-1.0) id AA02520; Wed, 1 Jun 94 18:14:12 CST Date: Thu, 2 Jun 1994 08:14:12 +0800 From: shin@hermes.ee.nthu.edu.tw (Jing-Shin Chang) Message-Id: <9406011014.AA02520@hermes.ee.nthu.edu.tw> To: ccnet-l%uga.BITNET@psuvm.psu.edu, chinesenet@uclink.berkeley.edu, simpson@math.psu.edu, soft-authors@ifcss.org Subject: Re: Don't quote selectively Status: O I hope that I don't need to post any such kind of posting in response to such logical reasoning and partial quotation anymore. -^^- |> |> I was indeed quoting Jing-Shin selectively, but not in an improper |> manner. |> |> Let me remind everybody of Jing-Shin's original posting on this topic, |> entitled "ONE problem with HZ+". Jing-Shin wrote: |> |> > Just to mention a problem and this problem could be a serious one: |> > When I tried a recent revision of my on line code converter Betty |> > (BE51/2-tty, something like hztty) for viewing HZ+ documents. I |> > found that Big5 users may find that their sentences are divided |> > seriously by those ~1's and ~2's, NOT JUST AT THE BEGIN/END OF A |> > LINE OR WHEN ENGLISH CHARS APPEARS. You may therefore find |> > something like HWA~2XIA~1WEN~2JAI in your news or mail |> > postings. When the Big5 part are mixed with HZ, the situation could |> > be worse. |> |> I think it is important to point out that this so-called problem with |> HZ+ is only a problem for Jing-Shin. It will not be a problem for |> anybody else. |> |> The reason that Jing-Shin has a problem is that, when Jing-Shin |> discovered his alleged problem, he was using his own peculiar 7-bit This is not an alleged problem. It was a simulation result. Did you ever think of such simulation before put something to the users? |> Big5 mapping scheme, which is extremely poor, because it puts many |> very common Chinese characters into block 2, thus necessitating the |> use of ~2 in the middle of almost every line of Big5 text. Other |> mapping schemes do not have this property. As indicated in the above recitation, the original test is not only based on B5E1 but also on B5E2. And in later tests, I was based on B5E3, which has the least switching frequency as HZ-2 does. If you are saying that B5E2, which has the same switching frequency as HZ-S, is pecurior in this aspect. I think you had partially recognized that HZ-S also suffer from the same problem. So just don't said that it is unique to B5E1 and make the main points confused. My point is clear: it is resolvable and better resolve it within the news readers. But don't use the wrong points. I think you are expert in mathematics and logic, you should not quote it this way under normal situation. |> |> What Jing-Shin is calling a problem with HZ+ will only be a problem if |> we decide to base HZ+ on Jing-Shin's mapping scheme. Jing-Shin's Which scheme? B5E1 ? B5E2 ? B5E3 ? Be clear and don't confuse others with a vary ambiguous vocabulary. And do not use ambiguous words in an attempt to remove other's focus. People who knows the various schemes also knows which could introduce more problem, but the others may be confused with such irresponsible words. If you said that B5E1 is worst in this regard, that's OK. But this statement does not deny the existence of the problem when using the other schemes since my reports are not entirely based on this scheme. (I hate to use so many words instead of simple logic in front of a mathematician.) |> so-called problem with HZ+ will *not* occur if we use my HZ+S mapping |> scheme, or almost any other mapping scheme that anyone can think of. That's a funny logic. B5E2 is as good as HZ+S in this aspect, and HZ-2 and B5E3 is better than HZ+S in this aspect. What are the other mapping schemes other can think of. Don't pretend that HZ+S does not suffer from this problem. My reports just gave a lower bound. |> |> Recently I summarized this discussion as follows: |> |> > ++ Jing-Shin's so-called problem with HZ+ was a consequence of the fact |> > ++ that he was looking only at his own mapping scheme, B5E1. **** |> |> Jing-Shin objected to this summary, but I think this summary was fair. I object to this summary because you are repeatedly saying that problem is alleged because I was using *B5E1* for the simulation. And after the last posting, in which I clearly indicated that my reports are based on B5E3, you still quote selectively this statement and make your conclusion. I don't think you need to do so and should stop to quote other's posting in this way. |> |> > I was once asked to get familiar with the HZ user's environment. Now, |> > can I asked people to respect Big5 Users' environments in the same |> > manner? |> |> Here Jing-Shin is making a good point. Discussions on HZ+ should be |> based on some understanding of both HZ/GB and Big5 user environments. |> |> I myself am familiar with four Big5 environments: ETen, KC, Mule, and |> cxterm. Based on my familiarity with these environments, I think HZ+ |> would be very suitable for the chinese.* newsgroups and for any other |> situation where 7-bit Big5 is needed, for instance international Big5 |> e-mail. Mule is probably suitable in this regard due to its multilingual natual. However, it is a big jiant (15M uncompressed, excluding patches, mule-1.1). Is it also accessible to non-workstation users? In addition, can you explain why ETen, KC, cxterm is also suitable? Do you mean that people were using Eten and KC for HZ newsgroup (e.g., ACT) reading and posting? (not to mention HZ+.) Cxterm + hztty do allow users to view HZ articles. Hztty is a good tool, but for some reasons, it does not strip those esc. sequences. Under such circumstances, as a big5 user, do you think you prefer to read ACT than ACT.big5. |> |> -- Steve |> From rap@doc.ic.ac.uk Wed Jun 1 05:41:11 1994 Return-Path: Received: from santos.doc.ic.ac.uk by ifcss.org (4.1/IFCSS-Mailer) id AA03732; Wed, 1 Jun 94 05:41:11 CDT Received: by santos.doc.ic.ac.uk (Smail3.1.28.1 #7) id m0q8nhP-00070hC; Wed, 1 Jun 94 11:39 BST Message-Id: Date: Wed, 1 Jun 94 11:39 BST From: Ross Paterson To: soft-authors@ifcss.org Subject: conversion between ISO-2022 and UTF Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Status: O Just in case anyone is interested, I've placed in ftp://ifcss.org/software/unix/convert/utf.tar.gz a program to convert from an ISO-2022 stream to UTF (the encoding of Unicode/ISO-10646 that it seems will be used under Unix), and a few related utilities. None of this will be much use unless you have an environment that can display UTF, such as 9term (ftp://ftp.cs.su.oz.au/matty/unicode/). -- Ross Paterson Department of Computing, Imperial College, London SW7 From jiang Wed Jun 1 10:26:09 1994 Return-Path: Received: by ifcss.org (4.1/IFCSS-Mailer) id AA12679; Wed, 1 Jun 94 10:26:09 CDT Date: Wed, 1 Jun 94 10:26:09 CDT From: jiang (Yuan Jiang) Message-Id: <9406011526.AA12679@ifcss.org> To: simpson@math.psu.edu, soft-authors Subject: Re: Reminder: last chance to vote on CNS Status: O >Yuan Jiang writes: > > The issue has always been which 7-bit Big5 representation to use 7-bit CNS is already 7 bit. It nenver needs HZ+'s support. Why do we need to raise a non-issue to debate and vote? Are we playing games? > > for a HZ+ standard. > >I disagree. That is one issue, but it is not the only issue. Another >issue is whether HZ+ shall support CNS. > >-- Steve > From butta1@crsa.bu.edu Wed Jun 1 11:28:27 1994 Return-Path: Received: from crsa.bu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA15148; Wed, 1 Jun 94 11:28:27 CDT Received: from ithake.bu.edu by crsa.bu.edu (8.6.4/Spike-2.1) id MAA24636; Wed, 1 Jun 1994 12:22:20 -0400 Received: from localhost by ithake.bu.edu (8.6.4/Spike-2.1) id MAA21418; Wed, 1 Jun 1994 12:21:43 -0400 Date: Wed, 1 Jun 1994 12:21:41 -0400 (EDT) From: C I I Project Subject: Re: Reminder: last chance to vote on CNS To: soft-authors@ifcss.org In-Reply-To: <9406011526.AA12679@ifcss.org> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Status: O On Wed, 1 Jun 1994, Yuan Jiang wrote: > >Yuan Jiang writes: > 7-bit CNS is already 7 bit. It nenver needs HZ+'s support. > Why do we need to raise a non-issue to debate and vote? Are we > playing games? 7-bit CNS as implemented in SEEDnet is not an issue. But if we are to mixed in with HZ, then that's where HZ+ comes in, as mixing 7-bit CNS into HZ text would require escape sequences to distinguish between GB and CNS code. Nelson From simpson@math.psu.edu Wed Jun 1 17:46:39 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA05069; Wed, 1 Jun 94 17:46:39 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id SAA02197; Wed, 1 Jun 1994 18:44:22 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 1 Jun 1994 18:44:20 -0400 Date: Wed, 1 Jun 1994 18:44:20 -0400 Message-Id: <199406012244.SAA02962@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, chinesenet@uclink.berkeley.edu, soft-authors@ifcss.org Subject: HZ+S upgrade, new features Reply-To: simpson@math.psu.edu Status: O HZ+ is a proposed 7-bit standard for mixed 7-bit Big5, GB, and ASCII. It is intended for use in international e-mail and news postings. HZ+S is my specific proposal for HZ+. I have just made a new version of HZ+S available by anonymous FTP. It is at ftp.math.psu.edu, /pub/simpson/chinese/hzp.tar.Z. New features: 1. Faster HZ+S <-> Big5 conversion. Multiplication and division are no longer used. With this speedup, HZ+S is no longer subject to criticism on grounds of "inefficiency". 2. Big5 escape sequences have been changed to ~> and ~< , my personal preference, rather than ~1 and ~2. This will probably change again in the future. Current possibilities that I am aware of are: ~1 and ~2, ~[ and ~], ~> and ~<, ~y and ~z. Any other suggestions? 3. Revised README and HZ+S specification document. 4. Optional CNS support. (Although CNS will not be part of the HZ+ standard any time soon, I am providing some experimental software to show how HZ+ might support CNS at some time in the future.) -- Steve Stephen G. Simpson Department of Mathematics, Pennsylvania State University 333 McAllister Building, University Park, State College, PA 16802 Office phone +1 814 863-0775 Home phone +1 814 238-2274 E-mail simpson@math.psu.edu FAX +1 814 865-3735 World Wide Web http://www.math.psu.edu/simpson/ From cyl Wed Jun 1 17:55:01 1994 Return-Path: Received: by ifcss.org (4.1/IFCSS-Mailer) id AA05313; Wed, 1 Jun 94 17:55:01 CDT Date: Wed, 1 Jun 1994 17:54:59 -0500 (CDT) From: Nelson CHIN Subject: let's move on w/vote for HZ+; MacblueTelnet asks for update To: soft-authors@ifcss.org Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Status: O it's obvious from those who voted that we should put CNS issue aside for now. So let's make a vote on HZ+ and present something to the chinese.* committee. Nelson ---------- Forwarded message ---------- Date: Thu, 2 Jun 1994 06:02:30 +0800 (CST) >From: Steve Chiu Oh, the PowerPC port of MacBlueTelnet is almost ready. I should release it in a few days. And I wonder if chinese.* reach the agreement to use any HS+? standard ? I would like to include it in the next release, because I am afraid I might not have chance to implement that since I might go to army soon (not sure, anyway) macblue From Ricky.Yeung@Eng.Sun.COM Thu Jun 2 03:52:20 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA25894; Thu, 2 Jun 94 03:52:20 CDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA00455; Thu, 2 Jun 94 01:50:18 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA12075; Thu, 2 Jun 94 01:51:34 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA21788; Thu, 2 Jun 1994 01:51:34 +0800 Date: Thu, 2 Jun 1994 01:51:34 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406020851.AA21788@hk.Eng.Sun.COM> To: soft-authors@ifcss.org Subject: Maximal ASCII transparency and a proposed enhencement Content-Length: 2172 Status: O The various HZ+ proposals don't seem to take into consideration of the principle of "maximal ASCII transparency." Let me define what I meant by that. Assuming I have an ASCII ONLY file f, and there is a (HZ+->Big5) converter program p. Maximum ASCII transparency means that when applying p to file f, the resulting output f' should be as close as to f as possible. Otherwise information would be lost. Obviously 100% transparency is not possible if we use some escape sequence to switch into non-ASCII code. What we want is to be as transparent as possible. Let's look at the HZ+S proposals that use ~1 or ~ numerals. If a piece of ASCII text contains the ~1 or ~2, it will be interpreted as 7-bit Big5 even though ~1 would really mean "not one..." in a C/C++ program and also occasionally in regular text when you talk about math. Proposal HZ+L uses ~y and ~z with more thoughtful reasons (see Dr. Fung Fung Lee's message). Unfortunately its ASCII transparency is again not so good. For example, the decoder could misinterpret a C++ destructor that starts with y or z to be 7-bit Big5. The ~ is so commonly used in various languages (for examples, C/C++, Latex, math, etc.) that you really don't want to further overload it. To maximize ASCII transparency, I hereby propose we use the ~{ as the sole escape sequence. For example, if we want to go with, say HZ+L, let's do ~{~ybcdefg~zhihk~} instead of ~ybcdefg~}~zhihk~} or ~ybcdefg~zhihk~} Notice that the ~{ sequence is already in use by HZ, so we're not really sacraficing more than we have now for having HZ. The only disadvantage that I can think of is the extra two bytes, but ASCII transparency is so important that I think they are worth every bit. [[ There are some other good reasons for choosing ~{ as the sole escape sequence. Maximal ASCII transparency is definitely the most important one. ]] ==================================================================== Ricky Yeung, PhD SunSoft, Inc. 2550 Garcia Ave. MTV10-228 Mountain View, CA 94043 Disclaimer: All statements and opinions expressed here are my own. They have no connection with my empolyer. ~{QnUq8;~} From deng@bunny.rhic.bnl.gov Thu Jun 2 08:26:21 1994 Return-Path: Received: from bunny.rhic.bnl.gov.zoo by ifcss.org (4.1/IFCSS-Mailer) id AA02868; Thu, 2 Jun 94 08:26:21 CDT Received: by bunny.rhic.bnl.gov.zoo (4.1/SMI-4.1) id AA20337; Thu, 2 Jun 94 09:25:53 EDT Message-Id: <9406021325.AA20337@bunny.rhic.bnl.gov.zoo> X-Phone: (516) 282-2197 (o), 266-6241 (h) X-Fax: (516) 282-2588 To: simpson@math.psu.edu Cc: ccnet-l@uga.cc.uga.edu, chinesenet@uclink.berkeley.edu, soft-authors@ifcss.org Subject: Re: HZ+S upgrade, new features In-Reply-To: Your message of "Wed, 01 Jun 94 18:44:20 EDT." <199406012244.SAA02962@boole.math.psu.edu> Date: Thu, 02 Jun 94 09:25:52 -0400 From: D-P Deng Status: O Has anyone written "hztty"-like hzpstty for cxterm? Since chinese.* is going to be created in a matter of days, it'd be great to have one. From simpson@math.psu.edu Thu Jun 2 10:18:26 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA07399; Thu, 2 Jun 94 10:18:26 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id LAA10009; Thu, 2 Jun 1994 11:16:03 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 2 Jun 1994 11:16:02 -0400 Date: Thu, 2 Jun 1994 11:16:02 -0400 Message-Id: <199406021516.LAA08180@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org, chinesenet@uclink.berkeley.edu Subject: how to use my HZ+S software In-Reply-To: References: Reply-To: simpson@math.psu.edu Status: O > Hi, there. I downloaded the hzp file and compiled it onto my UNIX > system. However, I could not use it to convert the file. Why? Pls > give me some instruction on this such as 1) Do I need some special > font files along with hzp; 2)what is the sytax for use these excutable > files, like hzp2gb < in file > out file, etc. Thanks in advance. Hi, Thanks for your interest in HZ+S. (0) I will place a copy of this message with my other HZ+S material at ftp.math.psu.edu, /pub/simpson/chinese/hzp/. (1) You don't need any special font files. My programs should work with your existing GB and/or Big5 software. Please let me know of any problems that you experience. (2) All of these programs are filters. The filter named aaa2bbb converts AAA data into BBB data. For instance big2hzp < file.big > file.hzp converts a Big5 file to an HZ+ file, and cat file.big | big2hzp | hzp2big should be equivalent to cat file.big. (3) If you have a mixed HZ+ file, you can convert it to Big5 or GB (your choice) by doing something like cat file.hzp | hzp2gb | gb2big | hzp2big -8 > file.big or cat file.hzp | hzp2big | big2gb | hzp2gb -8 > file.gb assuming you have Big5<->GB filters. The -8 option tells hzp2gb and hzp2big to pass 8-bit data. Big5<->GB filters are available for anonymous FTP at ifcss.org, /software/unix/convert/hc-30.tar.gz. (4) You can always use the -h option to get help. For instance, hzp2big -h displays help on hzp2big. -- Steve Stephen G. Simpson Department of Mathematics, Pennsylvania State University 333 McAllister Building, University Park, State College, PA 16802 Office phone +1 814 863-0775 Home phone +1 814 238-2274 E-mail simpson@math.psu.edu FAX +1 814 865-3735 World Wide Web http://www.math.psu.edu/simpson/ From simpson@math.psu.edu Thu Jun 2 11:11:54 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA10122; Thu, 2 Jun 94 11:11:54 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id MAA10319; Thu, 2 Jun 1994 12:09:52 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 2 Jun 1994 12:09:50 -0400 Date: Thu, 2 Jun 1994 12:09:50 -0400 Message-Id: <199406021609.MAA08258@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: proposed changes in HZ+ In-Reply-To: <9406020851.AA21788@hk.Eng.Sun.COM> References: <9406020851.AA21788@hk.Eng.Sun.COM> Reply-To: simpson@math.psu.edu Status: O Ricky Yeung writes: > To maximize ASCII transparency, I hereby propose we use the ~{ as the > sole escape sequence. For example, if we want to go with, say HZ+L, > let's do > > ~{~ybcdefg~zhihk~} > > instead of > > ~ybcdefg~}~zhihk~} or ~ybcdefg~zhihk~} This is something like the reasoning behind my original HZ+ proposal a long time ago, which used ~1{ and ~2{. At that time, a lot of people expressed the opinion that all the escape sequences should be only two bytes in length, so I changed them to ~1 and ~2 and that is what they have been until now. My considered opinion is that the existing ~~ escape sequence for ~ provides an adequate amount of ASCII compatibility. HZ and HZ+ will never provide perfect ASCII transparency, and I'm far from convinced that maximal ASCII transparency is worth the additional delay and the confusion. I'm not even convinced that Ricky's proposal provides maximal ASCII transparency in the sense that he defined it, let alone other senses that may be relevant. In addition, there are also some vocal people who seem prepared to argue endlessly that it may be important to choose an HZ+ mapping scheme that minimizes the number of bytes used for escape sequences in large text corpuses, regardless of all other considerations. In view of the above, I'm reluctant to get into what is certain to be a long and complicated debate about escape sequences. As Nelson and I have pointed out, there is some urgency about reaching an HZ+ consensus. Instead of making more changes in HZ+, I prefer to work toward reaching a consensus as soon as possible. The best HZ+ is the one that is simplest and cleanest while at the same time adequate for the intended purpose, namely 7-bit e-mail and news. More than anything else, the success of HZ+ will depend on our willingness and ability to reach a consensus and then to promote HZ+ by using it and telling people about it and writing software that supports it. -- Steve From simpson@math.psu.edu Thu Jun 2 11:46:09 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA11781; Thu, 2 Jun 94 11:46:09 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id MAA10516; Thu, 2 Jun 1994 12:43:53 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 2 Jun 1994 12:43:51 -0400 Date: Thu, 2 Jun 1994 12:43:51 -0400 Message-Id: <199406021643.MAA08333@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, chinesenet@uclink.berkeley.edu, soft-authors@ifcss.org Cc: ygz@cs.purdue.edu Subject: hztty support for HZ+ ? In-Reply-To: <9406021325.AA20337@bunny.rhic.bnl.gov.zoo> References: <199406012244.SAA02962@boole.math.psu.edu> <9406021325.AA20337@bunny.rhic.bnl.gov.zoo> Reply-To: simpson@math.psu.edu Status: O D-P Deng writes: > Has anyone written "hztty"-like hzpstty for cxterm? Since chinese.* > is going to be created in a matter of days, it'd be great to have > one. Using my HZ+S filters, I'm sure it will be possible to extend hztty to support HZ+S. My guess is that Yongguang Zhang, the author of cxterm and hztty, is very busy upgrading cxterm for X11R6. So far as hztty goes, he is probably waiting for us to reach a consensus on HZ+. This is another reason why we should work hard to reach a consensus as soon as possible. A suggestion for Yongguang. Why not write hztty to use an external filter that can be specified on the hztty command line? If the technical difficulties are not too great, this approach has the advantage of flexibility and you could do it now, before an HZ+ consensus is reached. Then hztty would be ready for HZ+ as soon as there is a consensus. -- Steve From simpson@math.psu.edu Thu Jun 2 15:48:55 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA23254; Thu, 2 Jun 94 15:48:55 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id QAA12001; Thu, 2 Jun 1994 16:46:20 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 2 Jun 1994 16:46:18 -0400 Date: Thu, 2 Jun 1994 16:46:18 -0400 Message-Id: <199406022046.QAA08739@boole.math.psu.edu> To: Jing-Shin Chang Cc: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: answers to all of Jing-Shin's questions In-Reply-To: <199406010551.BAA26319@leibniz.math.psu.edu> Reply-To: simpson@math.psu.edu Status: O Jing-Shin Chang writes: > I hope that Steve could answer the following questions, then I will > understand whether the problem I raised was fully understood, or there > is a misunderstanding due to language barrier (or simply because some > illogical logic reasoning). > ... Just so that Jing-Shin won't feel that he and his mapping schemes are being treated unfairly, I'll answer all of his many questions. However, let me say at the outset that I don't think any of these questions are important or relevant to the problem of defining HZ+ or reaching consensus on HZ+. > 1. The following posting is by John entitled "Sonnet" to chinese.test. > It was encoded with HZ-S. How many lines are there which contain > interruptting ~2 characters in this 14-line poem? > (including interrupting ~1 chars if a line begins with a ~2 char.) > > ~1YC5BVwVw'y~2.%~10M~} > ~2-4Ec~1FG(n(,5>*u~} > ~11gL"*]U]1C~2^w~1Kq~} > ~1>| > ~1Ih1e^C^C1\+W5z~} > ~1H5H53\Uc1C:g;o~} > ~1?#~2/=~10D'@F<3}?v~} > ~16.W(FrRz(nE/IS~} > ~1[D~2cK~10fFv,[8e:E~} > ~1XWH|PB]E']~} > > ~1F ~2<$~15B+81\K*)r*u~} > ~1 ~1*]X-5H1C7&G`(~~} I count 7. > 2. Can you agree that the frequency of switching has the following > order? > > B5E1 > (B5E2 == HZ-S) > (HZ-2 == B5E3) > > (>: is more frequent than, ==: is equally frequent as) > > In other words, for ANY big5 text, B5E1 will contain more ~2 chars > than the others, B5E2 and HZ-S will have the same number of ~2 chars, > but they contain more ~2 chars than the other two schemes (namely HZ-2 > and B5E3). HZ-2 and B5E3 contain the least number of ~2 chars among all. > And they both contain the same number of ~2 chars. > (What a lengthy description, hope that is clear enough. -^^-) No, I disagree. It depends on which Big5 text we look at. By selecting our corpus of Big5 text, we can get any result we want. Furthermore, such comparisons are pointless unless you quantify them. The really important point is that your B5E1 mapping scheme will put many ~2's in almost every line of normal Big5 text. by contrast, the mapping schemes proposed by other people will put ~2 in very few lines, namely only when a very rare Chinese character is encountered. > 3. If your answer to 2. is NO. Can you explain it? (Preferably with > mathematics or some kind of formal logic.) > > (If you don't agree with that, I don't know what can be said ...) I don't agree with what you said in 2. My explanation is above. > 4. If someone reported that, based on HZ-2 or B5E3, there are 6% to > 10% lines that contain interrupting ~2 characters, do you think > HZ-S will have a fewer number of such lines than this 6% (or 10%) > figure. > > (In your previous postings, your answer seems to be YES, implicitly. > Could you answer this more explicitly?) I can't comment on somebody else's tests or what the results would be if those tests were done differently. My tests of HZ+S on modern text give between 0 and 1/2 percent. On a file of ancient Chinese poems, the figure is about 2 percent. These are percentages of lines containing ~2's, not percentages of characters. > In addition, do you think all the other HZ+ proposals (excluding HZ-2) > will have a fewer number of ~2 chars than this figure. I have not tested them. I think that all of the mapping schemes will insert about the same number of ~2's as HZ+S, namely less than 1/10 of one percent measured as a percentage of file size, on normal text. The only exception is your is B5E1 proposal, where ~2's will add something like 50 percent to the file size. This is one of the reasons why I think your B5E1 proposal is a bad idea. > (We know that B5E1 has much more, and this is what you tried to > emphasized. But the problem I raised does not depend on this, do you > agree? No, I don't agree. The problem that you raised does indeed depend on this. The problem that you raised is only a problem for your mapping scheme, BE51. It is not a problem for anybody else's mapping scheme. > I mean that all the other proposals available, excluding HZ-2, > will NOT have a less number of ~2 chars, no matter how they defined > their ~1 and ~2 blocks, than this figure. Do you agree?) I can't understand this question. > 5. Did you recognize that I NOT ONLY test my Betty with B5E1 but also > with B5E3? No, I didn't recognize that. Also, I didn't care about that. > (I think you at least know that I had tested the files with B5E3.) > (Also, B5E2, which has as many switching as HZ-S, is also tested > as indicated in your previous posting which recite my first posting > on this problem.) I don't think this was clearly indicated by you. But again, I don't really care about it. For me, the main point was that you told everybody on ccnet and soft-authors that there was a serious problem with HZ+. I think it is worth pointing out that your alleged problem with HZ+ is only a problem with your mapping scheme, not with HZ+. If I had not pointed this out, then some people might have gotten the impression that you had discovered a genuine problem with HZ+. Some people may still have this impression. I wish you would take action to correct this impression. But I don't really expect you to take such action. I don't know where you stand on HZ+, but I am trying to build a consensus in favor of HZ+. Your phony allegations struck me as quite serious and worth taking the time to refute carefully and clearly. > 6. Did you know that the 6% and 10% figures were based on tests over 2M > bytes of big5 files (1M+ from HXWZ and 1M+ from Professor Shann's > collection on Chinese literatures, respectively)? No, I didn't know that, nor did I care about that. > Did you recognize that the figures are NOT based on B5E1? Yes. The figure for B5E1 would have to be almost 100 percent, not 6 percent. > 7. How many tests had you done on the problem I raised? What are the > sample sizes? or you don't think it requires any testing? I have done a little testing. My test files are in the 10-20 K range. I don't think it needs more testing than this. > 8. Did you know that I was concerning about on-line reading and conversion > for news postings, NOT on downloading files and converting them > off-line? Yes, I know that. But I don't consider it important. To me, the main point was your assertion that all HZ+ files would be full of ~2's, when in fact that is not the case. > 9. Did you know (or ever make any reasonable guesses on) the technical > problems why a good general purpose viewer (or code converter) > like hztty do not strip the escape chars used in HZ when reading > HZ newsgroup? I have thought some about how to write viewers and code converters for HZ+. There are tradeoffs involving different methods of handling escape sequences. I don't think there are any technical problems here that cannot be overcome. I think the burden here will fall on the people who write Chinese software. Your attempt to discuss these tradoffs has been spoiled by your allegation of a so-called problem with HZ+, where in fact no problem exists. The problem that you allege is a problem with your mapping scheme, not a problem with HZ+. > 10. Did you think that I was rasing the problem to advocate, explicitly > or implicitly, my B5E3 over your HZ-S? > (If your answer is YES. I am sorry to say that my answer is NO.) > (In fact, when the problem was posted, B5E3 is still not > announced.) I have no idea why you are raising this fake problem. Nevertheless, I think it is very important for everybody to recognize that this problem is a fake problem, not a real problem. > 11. What is a good news reader or mail reader you thought of when the HZ+ > standard is announced as a formal standard for the chinese.* > newsgroup and related mailing list? I think that any existing news and mail software will work with HZ+. I don't think that the announcement of the HZ+ standard should be accompanied by a recommendation of any particular software, except sample HZ+ implementations illustrating the HZ+ concept. If mail and news readers become HZ+ aware, that should be announced when it happens. Right now, no mail and news readers are HZ+ aware, because we have not yet reached a consensus on what HZ+ is. > Note: In Big5 user's daily environment, they don't need to prepare > a file off-line, convert it with a filter, then post it > to the newsgroup. They just post it or follow-up directly using > non-complicated (non-multi-lingual) news readers and an ordinary > editor (e.g., the default "vi" editor available in any Unix system). > They don't need to download a posting and convert it with a filter > to remove those esc. sequences. They don't see the escape chars > within their text even with a less complicated news reader. If we > are to provide HZ+-aware news reader that's OUR problems to consider > all these. > > 12. Based on your familiarity with ETen, KC, Mule and cxterm, did you > think they are useful for resolving the problem I raised? They are not useful for solving your problem with your mapping scheme (what you described as a problem with HZ+, but what is in fact a problem with your mapping scheme). ETen, KC, Mule, and cxterm will be very useful in all aspects of HZ+ implementation. > Did you think other news readers are equally easy for resolving > the problem? Is there any other news readers other > than the multi-lingual Mule which provides an easy solution? GNUS will provide an easy solution in any Chinese environment where Emacs runs. It's too early to say which solution will be easier or better. -- Steve From simpson@math.psu.edu Thu Jun 2 15:58:25 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA23686; Thu, 2 Jun 94 15:58:25 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id QAA12086; Thu, 2 Jun 1994 16:56:08 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 2 Jun 1994 16:56:06 -0400 Date: Thu, 2 Jun 1994 16:56:06 -0400 Message-Id: <199406022056.QAA08757@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: reply to Jing-Shin's positive points Reply-To: simpson@math.psu.edu Status: O Jing-Shin writes: > Finally, I would like to withdraw all my Big5/7 proposals, so that > our focus is not limited to the encoding schemes. OK, thank you. That will simplify our voting. > I would be happy to expect that there be soon available popular > news readers that are HZ+-aware (and Big5-Users-aware). > I would also be appreciate to those who are familiar with Big5 users' > envirnments if they could make their solutions widely available. I hope so too. > But a final word of reminding: before putting any HZ+ standard to the > chinese.*, please take care of the multi-lingual nature of the HZ+ > and the extra problems (if you think that they exist) introduced by > the 2-block encoding of Big5. > > Be sure that you are careful enough in considering every step of the > interactions between the users (especially the Big5 users) and the news > readers or mail readers. > > Simulate a HZ+ user's environment carefully before one post his/her > suggestions or solutions. We need empirical works, not heuristics. I agree completely. We will need HZ+ implementations which take account of the structure of Big5 and the different environments where people like to use e-mail and news. > I would be happy to be an ordinary big5 user, and I will choose the most > favorable one, be it tw.chinese.* or HZ+-based chinese.*. If tw.chinese.* > turn out to be my favorite, I hope that tw.chinese.* will not have > much problem in mirrowing the HZ+-based newsgroups, which is a little > more complicated due to its multi-lingual natural. It may turn out that 8-bit Big5 mirrors will be convenient for many or most users in Taiwan. However, this solution will not be available to everybody. The reason is that not everybody's news feed is 8-bit clean. > I had emphasized a little from a Big5 user's point of view. But I think > the HZ users also have to change, to some extent, their software > environment in order to read the HZ+ postings (esp. when a mixed Big5/HZ > article comes to your terminal) if it is accepted as the formal spec. Absolutely correct. Big5 users should not feel that HZ+ was created only to make trouble for them. HZ+ will make trouble for HZ users also. However, I hope the payoff will be worth the trouble. The payoff will be better communication in Chinese on the Internet. -- Steve From simpson@math.psu.edu Thu Jun 2 16:12:57 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA24358; Thu, 2 Jun 94 16:12:57 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id RAA12194; Thu, 2 Jun 1994 17:10:55 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 2 Jun 1994 17:10:53 -0400 Date: Thu, 2 Jun 1994 17:10:53 -0400 Message-Id: <199406022110.RAA08789@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: results of voting on whether HZ+ will support CNS Reply-To: simpson@math.psu.edu Status: RO I called for a vote in soft-authors@ifcss.org on whether HZ+ will support CNS. The options were: A. Add CNS support, in addition to the existing Big5 support. B. Drop Big5 support. Support CNS instead. C. Postpone a decision on CNS for the time being. The voting is now finished. The list of votes is as follows: "Simon Chow ~{V\9zP[~}" C Jianqing Hu C mcpong@cs.ust.hk (Dr. Man-Chi PONG) C Bing Ran/~{H=1x~} C mcevilly@netcom.com (Carlos McEvilly) C lee@simd.stanford.edu (Fung Fung Lee) C Xiaokun Zhu C CAI@neurophys.wisc.edu C Edmund Lai B Nelson CHIN C mleisher@crl.nmsu.edu (Mark Leisher) C Stephen G Simpson C The tally is 0 votes for A, 1 vote for B, 11 votes for C. Option C carries. This represents a decision that HZ+ will not support CNS at the present time. This decision does not preclude adding CNS support at some future time. I would like to thank everyone who voted. By voting, you helped to advance the process of reaching a consensus on HZ+. -- Steve From czha@CATI.CSUFresno.EDU Thu Jun 2 16:45:20 1994 Return-Path: Received: from CATI.CSUFresno.EDU (caticsuf.cati.csufresno.edu) by ifcss.org (4.1/IFCSS-Mailer) id AA25817; Thu, 2 Jun 94 16:45:20 CDT Received: (czha@localhost) by CATI.CSUFresno.EDU (8.6.8.1/8.6.5) id OAA13393; Thu, 2 Jun 1994 14:42:47 -0700 Date: Thu, 2 Jun 1994 14:42:47 -0700 From: Charlie Zha Message-Id: <199406022142.OAA13393@CATI.CSUFresno.EDU> X-Mailer: idsmail 4.0 To: @UGA.CC.UGA.EDU:owner-ccnet-l@UGA.CC.UGA.EDU Subject: Re: CXTERM for Other OS? Cc: soft-authors@ifcss.org Status: O Wu, it's you again. I totally agree with you that terminal emulator is a better way to go then patches. Regarding Chinese programming tools, I think you can use all Windows programming tools under Cstar or Twinbridge. Actually, I know Twinbridge team (which Cstar author used to belongs to) is using Borland C++, Visual C++, and Visual Basic under TwinBridge to develope their own stuff. In the three areas that you identified: 1) Compiler related tools 2) terminal emulation 3) appilications we can see some efforts in all areas. What I feel is that we need a team to integrate some of those efforts. For the most part, development may not be necessary. If we can come up with a suite of Chinese-capable applications, that will be great. I personally like to see the team effort starts on Linux platform first, while porting the application suite to other platform later. The reason is that Linux is a stable, powerful, 32-bit, free OS. DOS is too limiting. Windows is too overwhelming. Mac is of limited market share. This is though my own personal opinion, I don't want to debate with anyone about it. Anyway, let's see what can we do. Regards. Charlie Zha, a Far-Sider From cyl Thu Jun 2 17:31:49 1994 Return-Path: Received: by ifcss.org (4.1/IFCSS-Mailer) id AA27772; Thu, 2 Jun 94 17:31:49 CDT Date: Thu, 2 Jun 1994 17:31:49 -0500 (CDT) From: Nelson CHIN Subject: Re: CXTERM for Other OS? To: Charlie Zha Cc: owner-ccnet-l%UGA.CC.UGA.EDU@UGA.CC.UGA.EDU, soft-authors@ifcss.org In-Reply-To: <199406022142.OAA13393@CATI.CSUFresno.EDU> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Status: O On Thu, 2 Jun 1994, Charlie Zha wrote: > I personally like to see the team effort starts on Linux platform first, while > porting the application suite to other platform later. The reason is that > Linux is a stable, powerful, 32-bit, free OS. DOS is too limiting. Windows is > too overwhelming. Mac is of limited market share. This is though my own > personal opinion, I don't want to debate with anyone about it. > > Anyway, let's see what can we do. if you haven't already i suggest anyone interested in extending Linux to Chinese to sign up on the linux internationlization list, at li-international-request@orac.iinet.com.au with body message containing subscribe li-international they are currently talking about subdividing the list into different countries, so if enough interested, a list may created for the chinese development team. Nelson From jiang Thu Jun 2 20:53:06 1994 Return-Path: Received: by ifcss.org (4.1/IFCSS-Mailer) id AA04738; Thu, 2 Jun 94 20:53:06 CDT Date: Thu, 2 Jun 94 20:53:06 CDT From: jiang (Yuan Jiang) Message-Id: <9406030153.AA04738@ifcss.org> To: deng@bunny.rhic.bnl.gov Subject: Re: HZ+S upgrade, new features Cc: ccnet-l@uga.cc.uga.edu, chinesenet@uclink.berkeley.edu, soft-authors Status: O Jing-Shin seems to have "hztty"-like program supporting his HZ+BEU schemes. I hopw he can share his program with chinese.* readers. From jiang Thu Jun 2 22:31:47 1994 Return-Path: Received: by ifcss.org (4.1/IFCSS-Mailer) id AA09340; Thu, 2 Jun 94 22:31:47 CDT Date: Thu, 2 Jun 94 22:31:47 CDT From: jiang (Yuan Jiang) Message-Id: <9406030331.AA09340@ifcss.org> Subject: 7-bit Big5 representations Cc: ccnet-l@uga.cc.uga.edu, chinesenet@uclink.berkeley.edu, soft-authors Status: O Several 7-bit Big5 representations for HZ+ have been proposed. 1. HZ+S 2. HZ-2 3. B5E1 4. B5E2 5. B5E3 6. 7-bit CNS They all have their merits and drawbacks. I don't have strong opinions of either of them. However, I think the debate atmosphere has become unhealthy. I don't believe anything healthy can come out of this muddy atmosphere, and I will be ashamed of myself if anything unhealthy comes out. If there is a call for vote, I will boycot such a vote and will call others to boycot it. I will submit the number of people who boycot the vote to chinese.* committee for reference. I am on chinese.* committee too. I believe chinese.* desperately needs a 7-bit Big5 representation too. But I don't beleive soft-authors should shuff anything unhealthy to the end users. I believe this matter is best settled by the ultimate jurors, the end users. Let all schemes be test over chinese.test and let the users tell us which one benefit them most. A standard might be created earlier by voting over soft-authors, it may not benefit the users or take a long time to benefit them (in the case of Unicode, it takes forever to have the users benefited). On the other hand, it may take longer for the users to pick one for a standard, but by the time a standard is settled, the users will have already be benefited. Yuan From simpson@math.psu.edu Fri Jun 3 16:33:53 1994 Return-Path: Received: from cnd.org (cnd.cnd.org) by ifcss.org (4.1/IFCSS-Mailer) id AA02527; Fri, 3 Jun 94 16:33:53 CDT Received: from leibniz.math.psu.edu by cnd.org (4.1/4.7) id AA21987; Fri, 3 Jun 94 12:31:44 PDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id PAA20666; Fri, 3 Jun 1994 15:31:01 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 3 Jun 1994 15:31:00 -0400 Date: Fri, 3 Jun 1994 15:31:00 -0400 Message-Id: <199406031931.PAA10675@boole.math.psu.edu> To: chinesenet@uclink.berkeley.edu, soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu Subject: Yuan Jiang's boycott Reply-To: simpson@math.psu.edu Status: O Yuan Jiang wrote: > If there is a call for vote, I will boycot such a vote and will > call others to boycot it. I will submit the number of people who > boycot the vote to chinese.* committee for reference. This is rather disturbing. I would urge Yuan Jiang and everybody else not to boycott votes in soft-authors@ifcss.org. Please remember that the purpose of the upcoming vote will be to advance a consensus on a communication standard, HZ+. By the nature of communication, any communication standard requires a consensus. In the case of HZ+, a consensus is urgently needed in view of the impending creation of the chinese.* newsgroups. We want to be able to say that HZ+ has the backing of a knowledgeable group of Chinese software users and developers. This kind of leadership is important for the success of HZ+. > I believe chinese.* desperately needs a 7-bit Big5 representation > too. But I don't beleive soft-authors should shuff anything > unhealthy to the end users. Soft-authors cannot force anybody to use HZ+. The ultimate test of HZ+ will be its practicality in applications. If HZ+ is a poor standard, nobody will use it and it will die, no matter who endorses it. The role of soft-authors will be to suggest and encourage the acceptance of HZ+, not to force it. > I believe this matter is best settled by the ultimate jurors, the > end users. Let all schemes be test over chinese.test and let the > users tell us which one benefit them most. Yes. The end users are the ultimate jurors. However, soft-authors can also play a valuable role. Somebody has to exercise some leadership, and soft-authors is a logical place to look for leadership on matters such as HZ+. > A standard might be created earlier by voting over soft-authors, it > may not benefit the users or take a long time to benefit them (in > the case of Unicode, it takes forever to have the users benefited). > On the other hand, it may take longer for the users to pick one for > a standard, but by the time a standard is settled, the users will > have already be benefited. Here Yuan is presenting a false alternative: Either soft-authors create a standard by voting, or users create a standard by means of a testing process, but not both. In my view, the best way is a combination of these two approaches. Soft-authors can exercise leadership and endorse an HZ+ standard. Users can test HZ+ and competing schemes on chinese.test and elsewhere. Feedback from the testing process may lead to improvements in HZ+, or it may lead to the abandonment of HZ+ in favor of some other standard. In any case, the result will be improved Chinese-language communication on the Internet. -- Steve Stephen G. Simpson Department of Mathematics, Pennsylvania State University 333 McAllister Building, University Park, State College, PA 16802 Office phone +1 814 863-0775 Home phone +1 814 238-2274 E-mail simpson@math.psu.edu FAX +1 814 865-3735 World Wide Web http://www.math.psu.edu/simpson/ From Ricky.Yeung@Eng.Sun.COM Fri Jun 3 17:33:24 1994 Return-Path: Received: from cnd.org (cnd.cnd.org) by ifcss.org (4.1/IFCSS-Mailer) id AA05669; Fri, 3 Jun 94 17:33:24 CDT Received: from Sun.COM by cnd.org (4.1/4.7) id AA15211; Fri, 3 Jun 94 10:46:45 PDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA29506; Fri, 3 Jun 94 10:45:59 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA21459; Fri, 3 Jun 94 10:47:22 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA29348; Fri, 3 Jun 1994 10:47:03 +0800 Date: Fri, 3 Jun 1994 10:47:03 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406031747.AA29348@hk.Eng.Sun.COM> To: soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Thu, 2 Jun 1994 12:09:50 -0400 <199406021609.MAA08258@boole.math.psu.edu> Subject: Re: proposed changes in HZ+ Content-Length: 4049 Status: O > Ricky Yeung writes: > > To maximize ASCII transparency, I hereby propose we use the ~{ as the > > sole escape sequence. For example, if we want to go with, say HZ+L, > > let's do > > > > ~{~ybcdefg~zhihk~} > > > > instead of > > > > ~ybcdefg~}~zhihk~} or ~ybcdefg~zhihk~} > > This is something like the reasoning behind my original HZ+ proposal a > long time ago, which used ~1{ and ~2{. At that time, a lot of people > expressed the opinion that all the escape sequences should be only two > bytes in length, so I changed them to ~1 and ~2 and that is what they > have been until now. Yes, we should keep the escape sequence length short. Notice that what I'm proposing is two extra bytes per switching into HZ mode, while the above is three bytes per going into a character section. More importantly, your suggestion of using ~1{ and ~2}, like the others, decreases ASCII transparency. > My considered opinion is that the existing ~~ escape sequence for ~ > provides an adequate amount of ASCII compatibility. HZ and HZ+ will > never provide perfect ASCII transparency, and I'm far from convinced > that maximal ASCII transparency is worth the additional delay and the Not until you use "more/type" with "hztty/zwdos" to view your favorite C/C++, LaTex source files, and suddenly see unintended Chinese characters pop up. > confusion. I'm not even convinced that Ricky's proposal provides > maximal ASCII transparency in the sense that he defined it, let alone > other senses that may be relevant. It's better than the other HZ proposals in terms of ASCII transparency, don't you agree? BTW, I didn't claim it's optimal but I think it's no worse than HZ and better than other HZ+ proposals. Introducing new ~ escape sequence decreases ASCII transparency. HZ already introduced (basically) one non-transparency (lack of a better term) without much choice. I believe the best way would be not to further erode ASCII transparency but leveraging on the exisitng HZ sequence. Moreover, I think the originator of HZ intentionally left the other ~ pairs undefined mainly for the sake of extensibility. With my proposal, I think we can close the loop by defining the HZ syntax in such a way that every ~i is defined to map to ~i, where i can be any printable ASCII except ~, {, }, and newline. Switching to BIG5, GB, or other character code scheme (future extensibility) could then be localized inside the HZ mode, and ~{ is the only way to enter into the HZ mode. > In addition, there are also some vocal people who seem prepared to > argue endlessly that it may be important to choose an HZ+ mapping > scheme that minimizes the number of bytes used for escape sequences in > large text corpuses, regardless of all other considerations. > > In view of the above, I'm reluctant to get into what is certain to be > a long and complicated debate about escape sequences. As Nelson and I > have pointed out, there is some urgency about reaching an HZ+ > consensus. Instead of making more changes in HZ+, I prefer to work > toward reaching a consensus as soon as possible. > > The best HZ+ is the one that is simplest and cleanest while at the > same time adequate for the intended purpose, namely 7-bit e-mail and > news. More than anything else, the success of HZ+ will depend on our > willingness and ability to reach a consensus and then to promote HZ+ > by using it and telling people about it and writing software that > supports it. I don't believe my proposal is more complicated and less clean; in fact, it would even make life simpler for future extensibility. The reason I posted my previous message is that I see a technical (not political) problem, namely the maximal ASCII transparency issue, with the current proposals. And I'm offering an enhencement solution that would maximize ASCII transparency within the HZ framework. -Ricky From Ricky.Yeung@Eng.Sun.COM Sat Jun 4 10:24:46 1994 Return-Path: Received: from cnd.org (cnd.cnd.org) by ifcss.org (4.1/IFCSS-Mailer) id AA05669; Fri, 3 Jun 94 17:33:24 CDT Received: from Sun.COM by cnd.org (4.1/4.7) id AA15211; Fri, 3 Jun 94 10:46:45 PDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA29506; Fri, 3 Jun 94 10:45:59 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA21459; Fri, 3 Jun 94 10:47:22 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA29348; Fri, 3 Jun 1994 10:47:03 +0800 Date: Fri, 3 Jun 1994 10:47:03 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406031747.AA29348@hk.Eng.Sun.COM> To: soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Thu, 2 Jun 1994 12:09:50 -0400 <199406021609.MAA08258@boole.math.psu.edu> Subject: Re: proposed changes in HZ+ Content-Length: 4049 Status: O > Ricky Yeung writes: > > To maximize ASCII transparency, I hereby propose we use the ~{ as the > > sole escape sequence. For example, if we want to go with, say HZ+L, > > let's do > > > > ~{~ybcdefg~zhihk~} > > > > instead of > > > > ~ybcdefg~}~zhihk~} or ~ybcdefg~zhihk~} > > This is something like the reasoning behind my original HZ+ proposal a > long time ago, which used ~1{ and ~2{. At that time, a lot of people > expressed the opinion that all the escape sequences should be only two > bytes in length, so I changed them to ~1 and ~2 and that is what they > have been until now. Yes, we should keep the escape sequence length short. Notice that what I'm proposing is two extra bytes per switching into HZ mode, while the above is three bytes per going into a character section. More importantly, your suggestion of using ~1{ and ~2}, like the others, decreases ASCII transparency. > My considered opinion is that the existing ~~ escape sequence for ~ > provides an adequate amount of ASCII compatibility. HZ and HZ+ will > never provide perfect ASCII transparency, and I'm far from convinced > that maximal ASCII transparency is worth the additional delay and the Not until you use "more/type" with "hztty/zwdos" to view your favorite C/C++, LaTex source files, and suddenly see unintended Chinese characters pop up. > confusion. I'm not even convinced that Ricky's proposal provides > maximal ASCII transparency in the sense that he defined it, let alone > other senses that may be relevant. It's better than the other HZ proposals in terms of ASCII transparency, don't you agree? BTW, I didn't claim it's optimal but I think it's no worse than HZ and better than other HZ+ proposals. Introducing new ~ escape sequence decreases ASCII transparency. HZ already introduced (basically) one non-transparency (lack of a better term) without much choice. I believe the best way would be not to further erode ASCII transparency but leveraging on the exisitng HZ sequence. Moreover, I think the originator of HZ intentionally left the other ~ pairs undefined mainly for the sake of extensibility. With my proposal, I think we can close the loop by defining the HZ syntax in such a way that every ~i is defined to map to ~i, where i can be any printable ASCII except ~, {, }, and newline. Switching to BIG5, GB, or other character code scheme (future extensibility) could then be localized inside the HZ mode, and ~{ is the only way to enter into the HZ mode. > In addition, there are also some vocal people who seem prepared to > argue endlessly that it may be important to choose an HZ+ mapping > scheme that minimizes the number of bytes used for escape sequences in > large text corpuses, regardless of all other considerations. > > In view of the above, I'm reluctant to get into what is certain to be > a long and complicated debate about escape sequences. As Nelson and I > have pointed out, there is some urgency about reaching an HZ+ > consensus. Instead of making more changes in HZ+, I prefer to work > toward reaching a consensus as soon as possible. > > The best HZ+ is the one that is simplest and cleanest while at the > same time adequate for the intended purpose, namely 7-bit e-mail and > news. More than anything else, the success of HZ+ will depend on our > willingness and ability to reach a consensus and then to promote HZ+ > by using it and telling people about it and writing software that > supports it. I don't believe my proposal is more complicated and less clean; in fact, it would even make life simpler for future extensibility. The reason I posted my previous message is that I see a technical (not political) problem, namely the maximal ASCII transparency issue, with the current proposals. And I'm offering an enhencement solution that would maximize ASCII transparency within the HZ framework. -Ricky From A7621GAC@AWIUNI11.EDVZ.UNIVIE.AC.AT Sat Jun 4 11:32:03 1994 Return-Path: Received: from cnd.org (cnd.cnd.org) by ifcss.org (4.1/IFCSS-Mailer) id AA02869; Sat, 4 Jun 94 11:32:03 CDT Received: from AWIUNI11.EDVZ.UniVie.AC.AT (helios.edvz.univie.ac.at) by cnd.org (4.1/4.7) id AA27633; Sat, 4 Jun 94 08:46:29 PDT Message-Id: <9406041546.AA27633@cnd.org> Received: from AWIUNI11.EDVZ.UNIVIE.AC.AT by AWIUNI11.EDVZ.UniVie.AC.AT (IBM VM SMTP V2R2) with BSMTP id 6317; Sat, 04 Jun 94 17:42:22 MEZ Received: from AWIUNI11.EDVZ.UNIVIE.AC.AT (NJE origin A7621GAC@AWIUNI11) by AWIUNI11.EDVZ.UNIVIE.AC.AT (LMail V1.2a/1.8a) with BSMTP id 5941; Sat, 4 Jun 1994 17:42:16 +0100 Date: Sat, 04 Jun 94 16:48:04 MEZ From: Werner Lemberg Subject: HBF standard reexamined To: soft-authors@ifcss.org Status: O Let's talk about something different than HZ+ :-) While implementing a program which uses the HBF-API, I found some problems in the HBF Standard 1.0 and want to add some suggestions also: o the standard lacks a detailed description of the relation between HBF_BYTE_2_RANGE and HBF_CODE_RANGE. At the beginning of the document (section 2.1) it is vaguely mentioned the term "linearized into the byte sequence", but I think a small diagram could help enormously: e.g. for Big 5: the glyphs in the bitmap file SPCFONT.24 are linearly ordered starting at position 0 in the following manner: 0xA140 0xA141 ... 0xA17E 0xA1A1 0xA1A2 ... 0xA1FE 0xA240 ... etc. This would clarify that HBF_BYTE_2_RANGE not only indicates allowed code values, but that there are no gaps in the bitmap font. o it would be nice to have error codes in the function HBF_GetBitmap() like this (think of this as a draft for version 1.1 :-) HBF_FILE_ERROR can't open font file for this code HBF_INVALID_HANDLE HBF_INVALID_CODE_RANGE HBF_INVALID_BYTE_2_RANGE o a simple function HBF_CheckCode() to check if a specified code is valid or not (with similar error codes to HBF_GetBitmap() ) o HBF should allow JIS xxxx, KS for valid HBF_CODE_SCHEME. IMHO the format of the bitmap fonts are identical o It is not explained exactly how to handle multiple entries of the same keyword like HBF_CODE_RANGE if using the function HBF_GetProperty(). I suggest to change this API-function: int HBF_GetProperty( OUT char * property IN HBF_Handle handle, IN char * propertyName ); with some error codes: NOT_COMPLETED there are still some entries INVALID_PROPERTY etc. o It should be mentioned somewhere that the COMMENT property can't be accessed with HBF_GetProperty(). IMHO this would be senseless. Comments ? Werner From Ricky.Yeung@Eng.Sun.COM Sat Jun 4 12:32:23 1994 Return-Path: Received: from cnd.org (cnd.cnd.org) by ifcss.org (4.1/IFCSS-Mailer) id AA04897; Sat, 4 Jun 94 12:32:23 CDT Received: from Sun.COM by cnd.org (4.1/4.7) id AA01440; Fri, 3 Jun 94 16:20:57 PDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA16131; Fri, 3 Jun 94 16:20:20 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA00417; Fri, 3 Jun 94 16:21:52 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA02443; Fri, 3 Jun 1994 16:21:35 +0800 Date: Fri, 3 Jun 1994 16:21:35 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406032321.AA02443@hk.Eng.Sun.COM> To: soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Thu, 2 Jun 1994 12:09:50 -0400 <199406021609.MAA08258@boole.math.psu.edu> Subject: Re: proposed changes in HZ+ Content-Length: 4189 Status: O Please excuse me if you see this twice. It's been a day since I sent the first one in the morning. It probably got lost somehow. -Ricky > Ricky Yeung writes: > > To maximize ASCII transparency, I hereby propose we use the ~{ as the > > sole escape sequence. For example, if we want to go with, say HZ+L, > > let's do > > > > ~{~ybcdefg~zhihk~} > > > > instead of > > > > ~ybcdefg~}~zhihk~} or ~ybcdefg~zhihk~} > > This is something like the reasoning behind my original HZ+ proposal a > long time ago, which used ~1{ and ~2{. At that time, a lot of people > expressed the opinion that all the escape sequences should be only two > bytes in length, so I changed them to ~1 and ~2 and that is what they > have been until now. Yes, we should keep the escape sequence length short. Notice that what I'm proposing is two extra bytes per switching into HZ mode, while the above is three bytes per going into a character section. More importantly, your suggestion of using ~1{ and ~2}, like the others, decreases ASCII transparency. > My considered opinion is that the existing ~~ escape sequence for ~ > provides an adequate amount of ASCII compatibility. HZ and HZ+ will > never provide perfect ASCII transparency, and I'm far from convinced > that maximal ASCII transparency is worth the additional delay and the Not until you use "more/type" with "hztty/zwdos" to view your favorite C/C++, LaTex source files, and suddenly see unintended Chinese characters pop up. > confusion. I'm not even convinced that Ricky's proposal provides > maximal ASCII transparency in the sense that he defined it, let alone > other senses that may be relevant. It's better than the other HZ proposals in terms of ASCII transparency, don't you agree? BTW, I didn't claim it's optimal but I think it's no worse than HZ and better than other HZ+ proposals. Introducing new ~ escape sequence decreases ASCII transparency. HZ already introduced (basically) one non-transparency (lack of a better term) without much choice. I believe the best way would be not to further erode ASCII transparency but leveraging on the exisitng HZ sequence. Moreover, I think the originator of HZ intentionally left the other ~ pairs undefined mainly for the sake of extensibility. With my proposal, I think we can close the loop by defining the HZ syntax in such a way that every ~i is defined to map to ~i, where i can be any printable ASCII except ~, {, }, and newline. Switching to BIG5, GB, or other character code scheme (future extensibility) could then be localized inside the HZ mode, and ~{ is the only way to enter into the HZ mode. > In addition, there are also some vocal people who seem prepared to > argue endlessly that it may be important to choose an HZ+ mapping > scheme that minimizes the number of bytes used for escape sequences in > large text corpuses, regardless of all other considerations. > > In view of the above, I'm reluctant to get into what is certain to be > a long and complicated debate about escape sequences. As Nelson and I > have pointed out, there is some urgency about reaching an HZ+ > consensus. Instead of making more changes in HZ+, I prefer to work > toward reaching a consensus as soon as possible. > > The best HZ+ is the one that is simplest and cleanest while at the > same time adequate for the intended purpose, namely 7-bit e-mail and > news. More than anything else, the success of HZ+ will depend on our > willingness and ability to reach a consensus and then to promote HZ+ > by using it and telling people about it and writing software that > supports it. I don't believe my proposal is more complicated and less clean; in fact, it would even make life simpler for future extensibility. The reason I posted my previous message is that I see a technical (not political) problem, namely the maximal ASCII transparency issue, with the current proposals. And I'm offering an enhencement solution that would maximize ASCII transparency within the HZ framework. -Ricky From @UTARLVM1.UTA.EDU:simpson@math.psu.edu Sat Jun 4 12:59:31 1994 Return-Path: <@UTARLVM1.UTA.EDU:simpson@math.psu.edu> Received: from by ifcss.org (4.1/IFCSS-Mailer) id AB06285; Sat, 4 Jun 94 12:59:31 CDT Received: from leibniz.math.psu.edu by UTARLVM1.UTA.EDU (IBM VM SMTP V2R2) with TCP; Fri, 03 Jun 94 14:48:42 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id PAA20787; Fri, 3 Jun 1994 15:48:45 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 3 Jun 1994 15:48:43 -0400 Date: Fri, 3 Jun 1994 15:48:43 -0400 Message-Id: <199406031948.PAA10701@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: are we ready for a vote on HZ+ ? In-Reply-To: <9406030331.AA09340@ifcss.org> References: <9406030331.AA09340@ifcss.org> Reply-To: simpson@math.psu.edu Status: O Yuan Jiang wrote: > Several 7-bit Big5 representations for HZ+ have been proposed. > 1. HZ+S > 2. HZ-2 > 3. B5E1 > 4. B5E2 > 5. B5E3 > 6. 7-bit CNS Recently this complicated situation became much simpler. The outcome of our recent vote was a decision to drop consideration of CNS for the time being. Proposal 6 is therefore not an issue right now. Moreover, Jing-Shin Chang has withdrawn proposals 3, 4, and 5. So we are now down to two proposals, my HZ+S and Ya-Gui's HZ-2. In view of the impending creation of the chinese.* newsgroups and other considerations, I think it is important to reach a consensus on HZ+ as soon as possible. I therefore want to call for a vote soon. The purpose of this vote would be to answer the question, which 7-bit Big5 mapping scheme shall HZ+ use? Before we vote, does anyone feel that there are further issues in need of discussion? One possible issue is escape sequences. For a long time HZ+S was using ~1 and ~2 to switch into Big5 blocks 1 and 2 respectively. Yidao Cai suggested ~[ and ~]. Fung-Fung Lee gave some good technical arguments for ~y and ~z. Recently I started using ~> and ~< (suggesting "more" and "less"), but I am willing to accept any of these choices for the sake of reaching a consensus. I don't consider the choice of escape sequences very important in itself. I'm hoping we can decide it as a separate issue. There is also a somewhat different suggestion from Ricky Yeung, namely to use something like ~{~y and ~{~z. I'm not sure whether this will lead to a formal proposal. If it does, then I'm not sure how to handle it. Another technical issue that concerns me is the following, from Ya-Gui's HZ-2 proposal. Somehow I didn't pay much attention to this before, but now I am finding it awkward. > to maintain maximal compatibility with HZ-1, a Big5 -> HZ-2 > conversion program is recommended to include a Big5->GB conversion > step, such that all Big5 characters that can be reversibly > converted to GB are converted to GB before the conversion to HZ-2. This idea strikes me as somewhat questionable and not quite fair to Big5. I'm sorry to bring this up now -- I wish I had brought it up earlier. Perhaps Ya-Gui can clarify? -- Steve From @UTARLVM1.UTA.EDU:simpson@math.psu.edu Sat Jun 4 13:32:23 1994 Return-Path: <@UTARLVM1.UTA.EDU:simpson@math.psu.edu> Received: from by ifcss.org (4.1/IFCSS-Mailer) id AC07214; Sat, 4 Jun 94 13:32:23 CDT Received: from leibniz.math.psu.edu by UTARLVM1.UTA.EDU (IBM VM SMTP V2R2) with TCP; Fri, 03 Jun 94 18:07:08 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id TAA22199; Fri, 3 Jun 1994 19:06:58 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 3 Jun 1994 19:06:56 -0400 Date: Fri, 3 Jun 1994 19:06:56 -0400 Message-Id: <199406032306.TAA11091@boole.math.psu.edu> To: chinesenet@uclink.berkeley.edu, ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: Hey, wait a minute! Reply-To: simpson@math.psu.edu Status: O Jeffrey Chen writes: > Steve is not a big5 user in my opinion becuase he is not even a > proficient CHINESE user. Jeffrey is right. I am not very proficient in the Chinese language. Call me a *student* of the language. But with more than 13000 Big5 characters, can anybody here claim to be anything else? :-) Furthermore, I have to admit that I am not a *typical* Big5 user. But I insist that I am a Big5 user in some sense. I even wrote some simple Big5 software: Flash, CChelp, and my Poor Man's Chinese TeX support for Big5. (I wrote Flash and CChelp specifically to help me learn Chinese characters.) > powerful US self-proclaimed Chinese experts I don't claim to be a Chinese expert. I know a few things about Big5 and Chinese software, that's all. Admittedly my position here is somewhat strange, but I have long felt a need for some kind of 7-bit Big5, and the chinese.* hierarchy is all the more reason to try to bring this about. I'd have been delighted if someone else had taken the leadership role, but nobody did, so I stepped in. I think the real problem has been lack of perspective and lack of leadership. People in Taiwan have a completely 8-bit clean environment, so they see no need for 7-bit Chinese. Outside Taiwan, the Chinese programmers are mostly mainlanders who grew up with GB, not Big5. So 7-bit Big5 fell through the cracks. In the meantime, there is no good way for a Big5 user in the US to send Chinese e-mail to Taiwan or to read tw.* newsgroups. Don't you find this situation somewhat ironical? I'm trying to bring about better communication among GB and Big5 users world-wide. Anybody who would like to help is welcome to join in. But please don't accuse me of US-American imperialism. :-) > CNS? "Don't muddy the water!" HZ-2? "Your experiment is flawed!" > When the argument is crumbling? "Let's call for a vote.. amoung > ourselves, of course." I am not trying to impose anything on anybody. I have tried to encourage a full discussion of all the issues related to CNS, alternative mapping schemes, etc. I'd be delighted if more Big5 programmers would speak up. But let's remember that chinese.* and the need for 7-bit Big5 are international issues, not Taiwanese issues. -- Steve From eric@coeus.ucsd.edu Sat Jun 4 15:44:16 1994 Return-Path: Received: from ucsd.edu by ifcss.org (4.1/IFCSS-Mailer) id AA12191; Sat, 4 Jun 94 15:44:16 CDT Received: from coeus.ucsd.edu by ucsd.edu; id NAA05625 sendmail 8.6.9/UCSD-2.2-sun via SMTP Sat, 4 Jun 1994 13:40:44 -0700 Received: by coeus.ucsd.edu (AIX 3.2/UCB 5.64/4.03) id AA24895; Sat, 4 Jun 1994 13:41:12 -0700 Date: Sat, 4 Jun 1994 13:41:12 -0700 From: eric@coeus.ucsd.edu (Bo Yang) Message-Id: <9406042041.AA24895@coeus.ucsd.edu> To: BGQI@medinfo.rochester.edu, CHLX%HEAL2@scs.slac.stanford.edu, Fshen@huhept.harvard.edu, Gao@ucc.uwindsor.ca, H9390030@hkucc.hku.hk, JLIN@uars.sunysb.edu, KELIN@ucrph0.ucr.edu, XDAI@orange.cc.utexas.edu, Xiaorong@ccwf.cc.utexas.edu, YLU@utphya.phya.utoledo.edu, Zhou@pupgg.princeton.edu, bshi@utphya.phya.utoledo.edu, eric@coeus.ucsd.edu, ghyuan@wam.umd.edu, helent@athena.mit.edu, huang@bakerloo.phy.tulane.edu, jinsheng@merle.acns.nwu.edu, jun_yang@mail.rad.ucla.edu, jxiong@physics.rutgers.edu, liang@eng.umd.edu, lit@ctrvax.vanderbilt.edu, lu@csem.lbl.gov, mujia@brahms.udel.edu, nanlu@eng.umd.edu, qixx0005@gold.tc.umn.edu, ren@cptca.neep.wisc.edu, sgao@physics.rutgers.edu, shao@gamma.is.tcu.edu, wang@tristan.tn.cornell.edu, yawang@lynx.dac.neu.edu, yl96@cunixd.cc.columbia.edu, yunzh@sdphu1.ucsd.edu, zhanxu@bepc2.ihep.ac.cn, zsdu@physics.berkeley.edu Subject: A Song Cc: cssa@ucsd.edu, soft-authors@ifcss.org Status: O Fragile --- from Sting's old album "...Nothing Like The Sun" If blood will flow when flesh and steel are one Drying in the colour of the evening sun, Tomorrow's rain will wash the stains away But something in our minds will always stay Perhaps this final act was meant To clinch a lifetime's argument That nothing comes from violence and nothing ever could For all those born beneath an angry star Lest we forget how fragile we are On and on the rain will fall Like tears from the star like tears from the star On and on the rain will say How fragile we are how fragile we are On and on the rain will fall Like tears from the star like tears from the star On and on the rain will say How fragile we are how fragile we are From jiang Sun Jun 5 15:34:38 1994 Return-Path: Received: by ifcss.org (4.1/IFCSS-Mailer) id AA24468; Sun, 5 Jun 94 15:34:38 CDT Date: Sun, 5 Jun 94 15:34:38 CDT From: jiang (Yuan Jiang) Message-Id: <9406052034.AA24468@ifcss.org> To: simpson@math.psu.edu, soft-authors Subject: Re: are we ready for a vote on HZ+ ? Status: O Steve Simpson writes >Recently this complicated situation became much simpler. The outcome >of our recent vote was a decision to drop consideration of CNS for the >time being. Proposal 6 is therefore not an issue right now. Ha. You acknowledged a few days ago that 1) whether to support CNS 2) whether to use 7-bit CNS to represent Big5 are two separate issues and would be voted separately. Now you tell people what they voted on wasn't 1), but 2) instead. You can switch the content and call white black as your wish. How convenient. I hope you are all ashamed of going along with such a game that is beyond fair technical competition. Jing-Shin could stand the heat, the twist of logic and false statements, and dropped out of the competition. What's next? Hire Toniya Hardin's body guard? >Ya-Gui's HZ-2 proposal. Somehow I didn't pay much attention to this >before, but now I am finding it awkward. From rap@doc.ic.ac.uk Mon Jun 6 05:13:39 1994 Return-Path: Received: from santos.doc.ic.ac.uk by ifcss.org (4.1/IFCSS-Mailer) id AA17959; Mon, 6 Jun 94 05:13:39 CDT Received: by santos.doc.ic.ac.uk (Smail3.1.28.1 #7) id m0qAbeY-00070iC; Mon, 6 Jun 94 11:11 BST Message-Id: Date: Mon, 6 Jun 94 11:11 BST From: Ross Paterson To: soft-authors@ifcss.org Subject: Re: HBF standard reexamined Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Status: O Werner Lemberg writes: > While implementing a program which uses the HBF-API, I found some problems > in the HBF Standard 1.0 and want to add some suggestions also: [documentation suggestion deleted] > o it would be nice to have error codes in the function HBF_GetBitmap() > like this (think of this as a draft for version 1.1 :-) > HBF_FILE_ERROR can't open font file for this code > HBF_INVALID_HANDLE > HBF_INVALID_CODE_RANGE > HBF_INVALID_BYTE_2_RANGE Not too difficult. > o a simple function HBF_CheckCode() to check if a specified code > is valid or not (with similar error codes to HBF_GetBitmap() ) How do you intend to use this? > o HBF should allow JIS xxxx, KS for valid HBF_CODE_SCHEME. IMHO the > format of the bitmap fonts are identical Yes -- they were designed to be. And I guess we should be using GL encodings for all these codesets. > o It is not explained exactly how to handle multiple entries of the > same keyword like HBF_CODE_RANGE if using the function > HBF_GetProperty(). I suggest to change this API-function: > int HBF_GetProperty( > OUT char * property > IN HBF_Handle handle, > IN char * propertyName ); How about just saying that the following properties are inaccessible: HBF_START_BYTE_2_RANGES HBF_BYTE_2_RANGE HBF_START_CODE_RANGES HBF_CODE_RANGE > o It should be mentioned somewhere that the COMMENT property can't > be accessed with HBF_GetProperty(). IMHO this would be senseless. Sounds reasonable. From czha@CATI.CSUFresno.EDU Mon Jun 6 12:56:02 1994 Return-Path: Received: from CATI.CSUFresno.EDU (caticsuf.cati.csufresno.edu) by ifcss.org (4.1/IFCSS-Mailer) id AA06676; Mon, 6 Jun 94 12:56:02 CDT Received: (czha@localhost) by CATI.CSUFresno.EDU (8.6.8.1/8.6.5) id KAA19781 for soft-authors@ifcss.org; Mon, 6 Jun 1994 10:53:24 -0700 Date: Mon, 6 Jun 1994 10:53:24 -0700 From: Charlie Zha Message-Id: <199406061753.KAA19781@CATI.CSUFresno.EDU> X-Mailer: idsmail 4.0 To: soft-authors@ifcss.org Subject: Help me with cxterm on Linux Status: O Could someone kindly help me to set up cxterm on Linux ? I got the linux box running fine along with cxterm. The only trouble is only and keys works, i.e., there are only ASCII and Chinese internal codes. According to the accompanying FAQ, it is because of missing dictionaries (input methods). I got cxterm.nf.*.dict.tar.gz from Walnut Creek Linux Toolkit CD-ROM. The dictionaries are in *.citnf format. I think the correct format should be *.cit. How can I convert them ? After I started X-windows and entered "cxterm &", machine sometimes hung up. What did I do wrong ? Or is cxterm not very stable ? Finally, has anyone tried to run Tcl/Tk, or XF with cxterm before ? Thanks in anticipation for your help. Charlie Zha From simpson@math.psu.edu Mon Jun 6 13:30:46 1994 Return-Path: Received: from cnd.org (cnd.cnd.org) by ifcss.org (4.1/IFCSS-Mailer) id AA07685; Mon, 6 Jun 94 13:30:46 CDT Received: from leibniz.math.psu.edu by cnd.org (4.1/4.7) id AA06664; Mon, 6 Jun 94 11:29:11 PDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id OAA17223; Mon, 6 Jun 1994 14:27:02 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Mon, 6 Jun 1994 14:27:00 -0400 Date: Mon, 6 Jun 1994 14:27:00 -0400 Message-Id: <199406061827.OAA21022@boole.math.psu.edu> To: jiang@ifcss.org (Yuan Jiang) Cc: soft-authors@ifcss.org Subject: Re: are we ready for a vote on HZ+ ? In-Reply-To: <9406052034.AA24468@ifcss.org> References: <9406052034.AA24468@ifcss.org> Reply-To: simpson@math.psu.edu Status: O > Ha. You acknowledged a few days ago that > 1) whether to support CNS > 2) whether to use 7-bit CNS to represent Big5 > are two separate issues and would be voted separately. I truly don't know what you are talking about. I never said that. My intention all along was to separate CNS from the mapping scheme issue. I stated this very clearly. I tried to make it very clear that a vote for option C meant a vote to drop consideration of CNS for the time being, so far as HZ+ is concerned. This includes both of items 1) and 2). I thought I made this perfectly clear to everyone. I am trying to be fair. Does anyone besides Yuan Jiang feel that this was not made clear? Does anyone want to change their vote? -- Steve From handa@etlken.etl.go.jp Mon Jun 6 21:26:56 1994 Received: from etlpost (etlpost.etl.go.jp) by ifcss.org (4.1/IFCSS-Mailer) id AA26186; Mon, 6 Jun 94 21:26:56 CDT Received: from etlpom.etl.go.jp by etlpost (5.67+1.6W/2.7W) id AA21386; Tue, 7 Jun 94 11:11:25 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA11613; Tue, 7 Jun 94 11:11:15 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA21193; Tue, 7 Jun 94 11:11:23 JST Date: Tue, 7 Jun 94 11:11:23 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9406070211.AA21193@etlken.etl.go.jp> To: mule@etl.go.jp Reply-To: mule@etl.go.jp Cc: ccnet%uga.BITNET@psuvm.psu.edu, soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Tue, 31 May 1994 12:36:11 -0400 <199405311636.MAA29487@boole.math.psu.edu> Subject: Re: Big5, CNS, Mule, and HZ+ Status: O Sorry for the late response on this thread. On Tue, 31 May 1994 12:36:11 -0400, Stephen G Simpson said: > I am willing to stipulate that the 13051 standard CNS Chinese > characters comprise most of the Chinese characters in Big5. However, > Big5 contains a number of additional Chinese characters, also various > sets of numerals, alphabets, punctuation, and special symbols, not to > mention vendor-defined codes, characters, and symbols. It may be that > CNS contains some of these Big5 extras, but CNS certainly does not > contain all of them. I have just get CNS fonts of Plain1&2 and checked the contents against taipei15.bdf font. CNS11643 BIG5 ------------------------------------------------------------ standard 13051 chars 13053 chars Chinese chars 4421-7d4b(plain1) a440-c67e 2121-7244(plain2) c940-f9d5 other symbols 408 chars 408 chars 2121-2570(plain1) a140-a3bf circled number, 30 chars none (1)..., i,ii... 2621-263e(plain1) radicals of 213 chars none Chinese chars 2721-2939(plain1) complicated 33 chars none symbols 4221-4241(plain1) ------------------------------------------------------------ `other symbols' are identical in both fonts. In `standard Chinese Chars', BIG5 has two more chars than CNS, those are: c94a: /* MU on Tsang Jye */ -- looks like arithmetic `pi' ddfc: /* GRHNE on Tsang Jye */ -- looks like below: * **** ******* * * * * * * ***** * *** ******** * * ********** ***** * * ***** * * * * ** ***** *** **** My guess is that designers of CNS removed those chars because of some reasonable reason such as: o those are incorrect charactes. o those are not used now. o those are duplicate of some other charactes. but, of course, I'm Japanese and have no right to judge it. ISO10646 maps those to FA0C and FA0D (within a private use area). Are you really using them? If not, I still think CNS is a superset of Big5. Even in the case that someone really want to send those characters, we can have some agreement on mapping them to CNS's not-used area (e.g. 7245 and 7246). > No. The Mule Big5 situation is somewhat complicated. The current > version of Mule supports Big5 in a way that is very similar to the way > that HZ+S supports it, but that may change in future versions. I have > not been able to find out exactly how Mr Handa proposes to support > Big5 in future versions of Mule, but he has assured me that all future > versions of Mule will support Big5 in some fashion, and I believe him. Sure, I don't drop Big5 support of course. > One point to keep in mind is that Mule's and Mr Handa's idea of Big5 > support may not be the same as HZ+'s idea of Big5 support. You have > to remember that Mule is a very complicated, multilingual application. > Mule has its own support for Roman, Greek, Russian, Japanese Kana, > etc. etc. Therefore in a sense (at least for certain applications), > Mule may have no need for any part of Big5 outside of the Big5 Chinese > characters. The reason why I want to change the way of Big5 supporting is just because I want to delete all adhoc code (in C files) existing just for Big5. > When Mr Handa says that he plans to support Big5, he may be referring > only to the 13051 standard CNS Chinese characters. This may be fine > for Mule, but it would be a mistake for HZ+, in my opinion. No, no, supporing Big5 means supporing full Big5 characters. > CNS may be a good idea for Mule, but it is not a good idea for HZ+. I > have two reasons for saying this. (1) Mule is an ISO-2022 > application, and ISO-2022 recognizes CNS. HZ+ is not suffering from > this constraint. (2) HZ+ may have some need for the multilingual > features of Big5. Mule can perhaps afford to ignore these features, > since Mule has its own extensive multilingual support. How outernal code is encoded and how Mule's internal code is encoded is independent. Mule can read/write any outernal encoding. So, even if you chose HZ+ encoding instead of CNS-based method, Mule can easily support it. --- Ken'ichi HANDA handa@etl.go.jp From shann@wangwei.math.ncu.edu.tw Tue Jun 7 05:11:21 1994 Return-Path: Received: from wangwei.math.ncu.edu.tw by ifcss.org (4.1/IFCSS-Mailer) id AA09927; Tue, 7 Jun 94 05:11:21 CDT Received: from qingzhao.math.ncu.edu.tw by wangwei.math.ncu.edu.tw (4.1/SMI-4.1) id AA07316; Tue, 7 Jun 94 18:07:10 CST Date: Tue, 7 Jun 94 18:07:10 CST From: shann@wangwei.math.ncu.edu.tw (Shann Wei-Chang) Message-Id: <9406071007.AA07316@wangwei.math.ncu.edu.tw> To: CCNET-L@uga.cc.uga.edu, soft-authors@ifcss.org Subject: More on CNS Status: O handa@etlken.etl.go.jp (Kenichi Handa) wrote: > I have just get CNS fonts of Plain1&2 and checked the > contents against taipei15.bdf font. > > > CNS11643 BIG5 > ------------------------------------------------------------ > standard 13051 chars 13053 chars > Chinese chars 4421-7d4b(plain1) a440-c67e > 2121-7244(plain2) c940-f9d5 > > other symbols 408 chars 408 chars > 2121-2570(plain1) a140-a3bf > > circled number, 30 chars none > (1)..., i,ii... 2621-263e(plain1) > > radicals of 213 chars none > Chinese chars 2721-2939(plain1) > > complicated 33 chars none > symbols 4221-4241(plain1) > ------------------------------------------------------------ > > `other symbols' are identical in both fonts. > > In `standard Chinese Chars', BIG5 has two more chars than > CNS, those are: > c94a: /* MU on Tsang Jye */ -- looks like arithmetic `pi' > ddfc: /* GRHNE on Tsang Jye */ -- looks like below: > > ........... > > My guess is that designers of CNS removed those chars > because of some reasonable reason such as: > o those are incorrect charactes. > o those are not used now. > o those are duplicate of some other charactes. > but, of course, I'm Japanese and have no right to judge it. Many thanks to Dr Handa, I have further comments. Topologically (so to speaking), I cannot distinguish the glyphs pointed by c94a and a461 (wu4). On the biggest dictionary that I have (**Please see appendix), I cannot find a character looks more like c94a than a461. So I support Handa's guess, c94a is either incorrect or duplicated. The glyph pointed by ddfc is, to my judgement, identical with that by dcd1 (huo4 on my input table, but hu4 on my dictionary: "the sound of throw up"). So I think ddfc is duplicated. This character is on Plain 2 of CNS, code 4176. Furthermore, although Plain 1 of CNS has the same set of characters as the frequently used characters in Big-5, CNS has corrected some mistakes: (1) ACFE should be put after AFCF, wrong number of strokes. (2) BE52 should be put after BBC7, wrong number of strokes. (3) C2CB should be put after C1AA, wrong number of strokes. (4) C3B9 should be switched with C3BA, wrong radicals. (5) C456 should be put after C360, wrong number of strokes. I talked with Chang Jing-Shin on the phone. He told me there are 8 or 9 other mistakes found on Plain 2 and less-frequently used characters in Big-5. Besides the optional Asian language kernel used on IBM's and SUN's workstations, VMS also supports CNS. On PC with ET, I can switch to CNS internal code (set high bits of CNS to 1) by ETCTL/C:SI1. I saw some CNS services in National Chung-Hsing (Zhong Xing) Univ. The conversion program between Big-5 and first two plains of CNS is written, tested and used for some months. I distrusted CNS code before. After a few days of study and talking with people, I have changed my mind. Now I would support CNS as an exchange code, if not an internal code yet. Jing-Shin said he can modify Simpson's conversion program a little bit to let it convert Big-5 to CNS. Even if he does not have time to do it, there are already conversion programs. For instance chcode-1.4. ** Appendix The biggest dictionary I have is the Xin Ci2 Dian3 (New Dictionary) published by San-Min (Three People) in 1989. It defines 16862 characters, 43140 phrases, totally 2604 pages. The chief editor was Dr. Liu2 Zhen4 Qiang2. It is a work done by 140 scholars in four years. This new dictionary is up to date in many aspects. From simpson@math.psu.edu Tue Jun 7 05:32:02 1994 Return-Path: Received: from cnd.org (cnd.cnd.org) by ifcss.org (4.1/IFCSS-Mailer) id AA10333; Tue, 7 Jun 94 05:32:02 CDT Received: from leibniz.math.psu.edu by cnd.org (4.1/4.7) id AA01883; Tue, 7 Jun 94 02:39:26 PDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id FAA25044; Tue, 7 Jun 1994 05:37:06 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Tue, 7 Jun 1994 05:37:05 -0400 Date: Tue, 7 Jun 1994 05:37:05 -0400 Message-Id: <199406070937.FAA22405@boole.math.psu.edu> To: mule@etl.go.jp Cc: ccnet%uga.BITNET@psuvm.psu.edu, soft-authors@ifcss.org Subject: Re: Big5, CNS, Mule, and HZ+ In-Reply-To: <9406070211.AA21193@etlken.etl.go.jp> References: <199405311636.MAA29487@boole.math.psu.edu> <9406070211.AA21193@etlken.etl.go.jp> Reply-To: simpson@math.psu.edu Status: O Kenichi Handa writes: > I have just get CNS fonts of Plain1&2 and checked the > contents against taipei15.bdf font. What is taipei15.bdf? From your description, taipei15.bdf apparently doesn't include some things that are present in ETen Big5, the most popular implementation of Big5. For instance, ETen Big5 has some Chinese characters above f9d5. Also, ETen Big5 includes the Russian alphabet and Japanese kana. From your description of taipei15.bdf, it appears that taipei15.bdf does not include these things. I don't know whether CNS-11643 includes them. I suspect it doesn't. A detailed description of ETen Big5 is in my document etfonts.doc, available for anonymous FTP at ftp.math.psu.edu, /pub/simpson/chinese/hzp/. > In `standard Chinese Chars', BIG5 has two more chars than > CNS, those are: > c94a: /* MU on Tsang Jye */ -- looks like arithmetic `pi' > ddfc: /* GRHNE on Tsang Jye */ -- looks like below: I'm not terribly worried about these two characters, because some people have said that they are duplicates of other characters. However, the existence of these two characters seems to imply that Big5->CNS translation cannot be 100 percent invertible. > Are you really using them? If not, I still think CNS is a > superset of Big5. I wouldn't draw this conclusion. As I pointed out above, there are still questions about Japanese kana, the Russian alphabet, and Chinese characters and graphical symbols above f9d5. There may be other questions. To conclude that CNS-11643 is a superset of Big5, you would have to compare CNS-11643 not only with taipei15.bdf but also with ETen Big5 and possibly other Big5 implementations. > No, no, supporing Big5 means supporing full Big5 characters. I'm glad to hear that. But how exactly do you define "full Big5 characters"? I don't think Big5 has been standardized to such an extent. Perhaps it can never be standardized. Big5 is a popular standard, not an official standard. In short, I still have some questions about whether Mule will really be able to provide excellent Big5 support by means of Big5<->CNS translation. -- Steve From ZHANGG@utkvx.utk.edu Tue Jun 7 10:11:42 1994 Return-Path: Received: from UTKVX2.UTK.EDU by ifcss.org (4.1/IFCSS-Mailer) id AA17210; Tue, 7 Jun 94 10:11:42 CDT Received: from utkvx.utk.edu by utkvx.utk.edu (PMDF V4.3-7 #6815) id <01HD9B8FD4UO8Y6X0Y@utkvx.utk.edu>; Tue, 7 Jun 1994 11:09:35 EDT Date: Tue, 07 Jun 1994 11:09:35 -0400 (EDT) From: Gang Zhang Subject: Question To: soft-authors@ifcss.org Message-Id: <01HD9B8FDXSI8Y6X0Y@utkvx.utk.edu> X-Vms-To: IN%"soft-authors@ifcss.org" Mime-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-Transfer-Encoding: 7BIT Status: O Hello! This is the first time I try to write a Chinese application. I have download guobiao.gb from ifcss.org. But I don't know where to get a font file that has the same charactor order as the guobiao table. May somebody help me about that? Thank you! Gang From cyl Tue Jun 7 10:46:57 1994 Return-Path: Received: by ifcss.org (4.1/IFCSS-Mailer) id AA18747; Tue, 7 Jun 94 10:46:57 CDT Date: Tue, 7 Jun 1994 10:46:56 -0500 (CDT) From: Nelson CHIN Subject: Japanese kana's use in ETen Big5 To: handa@etl.go.jp Cc: soft-authors@ifcss.org Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Status: O FYI, Japanese kana in ETen Big5 is extensively used, esp in taiwan's BIG5 mirrors of the japanese fj.* hierarchy. You can check it out by setting your NNTPSERVER aidebbs.edu.tw and look at the articles in csie.bbs.fj.* and tw.bbs.lang.japanese which is a taiwan group about japanese language. Also, many taiwan universities have license to use ETen fonts/software within their campus, and thus do use BDFs of ETen fonts. The current way of creating a bdf, is first to get the bitmaps from the ETen pc software, and then get the HBF header files from ifcss.org:/software/fonts/big5/hbf/eb5* and the hbftobdf utility in ifcss.org:/software/fonts/utils/hbf.tar.gz Before the advent of the hbf standard, leisher@crl.nmsu.edu provided some utilities to convert the bitmaps to BDF for the taiwan universities. Some of the resultant BDFs are available at ifcss.org:/software/fonts/big5/bdf/taipei*24.bdf.gz, but had the ETen specific stuff removed, i guess dues to copyright issues. for those who have no license, a work around would be to get the KC fonts and ETen implementation of their respective HBF files from ifcss.org:/software/fonts/big5/hbf/ Nelson From ygz@cs.purdue.edu Tue Jun 7 16:40:39 1994 Return-Path: Received: from arthur.cs.purdue.edu by ifcss.org (4.1/IFCSS-Mailer) id AA03109; Tue, 7 Jun 94 16:40:39 CDT Received: from ector.cs.purdue.edu (root@ector.cs.purdue.edu [128.10.2.10]) by arthur.cs.purdue.edu (8.6.4/PURDUE_CS-1.3) with ESMTP id ; Tue, 7 Jun 1994 16:38:36 -0500 Received: from localhost (ygz@localhost [127.0.0.1]) by ector.cs.purdue.edu (8.6.4/PURDUE_CS-1.3) with SMTP id ; Tue, 7 Jun 1994 16:38:34 -0500 Message-Id: <199406072138.QAA13442@ector.cs.purdue.edu> To: Multiple recipients of list CCNET-L <@VM.CC.PURDUE.EDU:CCNET-L@UGA>, soft-authors@ifcss.org Subject: X11R6 CXTERM (preview) Date: Tue, 07 Jun 1994 16:38:32 -0500 From: ygz@cs.purdue.edu (Yongguang Zhang) Status: O I have sent it to X Consortium. For those who have X11R6 installed, if you want to try it, please let me know. This version only works in X11R6, NOT under X11R4, R5, or OpenWindow. (It is time to bug your system manager for upgrading your X Window. :-) It now supports phrase input, phrases in .tit, wildcard and wildchar (like * and ? you use to match Unix files), association input, etc. (Pretty much the similiar things in cxterm NewFace.) We have decided to adopt ETen Big5 as the reference implementation for Big5 encoding in the new cxterm. I'll convert all the Big5 text in the Purdue ftp site soon. X11R6 cxterm also supports JIS and KS. I will add some new features and write backward patches for X11R4 and R5, before I make a formal release. So let me know your comments and suggestion. Thanks, --ygz P.S.: On some other things, a better GB to Big5 conversion program is available in ftp.cs.purdue.edu: pub/ygz/utils/wordg2b.tar.Z. It is based on words instead of single characters. In case you like technical terms: it uses a word segmentation technique based on forward and backward maximum matching of occurrence probabilities. From simpson@math.psu.edu Tue Jun 7 23:08:15 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA17977; Tue, 7 Jun 94 23:08:15 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id AAA03480; Wed, 8 Jun 1994 00:06:07 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 8 Jun 1994 00:06:05 -0400 Date: Wed, 8 Jun 1994 00:06:05 -0400 Message-Id: <199406080406.AAA24034@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: Re: proposed changes in HZ+ In-Reply-To: <9406031747.AA29348@hk.Eng.Sun.COM> References: <199406021609.MAA08258@boole.math.psu.edu> <9406031747.AA29348@hk.Eng.Sun.COM> Reply-To: simpson@math.psu.edu Status: O Ricky Yeung writes: > It's better than the other HZ proposals in terms of ASCII transparency, > don't you agree? BTW, I didn't claim it's optimal but I think it's > no worse than HZ and better than other HZ+ proposals. I agree. Your modification offers more ASCII transparency than the existing HZ+ proposals. I'm not sure that your modification is optimal in this respect, but it is an improvement. > Introducing new ~ escape sequence decreases ASCII transparency. HZ > already introduced (basically) one non-transparency (lack of a better > term) without much choice. I believe the best way would be not to > further erode ASCII transparency but leveraging on the exisitng HZ > sequence. Yes, Ricky's modification does have this technical advantage. The only disadvantage that I see is that the total number of bytes used in escape sequences is increased considerably. > I don't believe my proposal is more complicated and less clean; in fact, > it would even make life simpler for future extensibility. The reason I > posted my previous message is that I see a technical (not political) > problem, namely the maximal ASCII transparency issue, with the current > proposals. And I'm offering an enhencement solution that would maximize > ASCII transparency within the HZ framework. >From a technical point of view, I think Ricky may have a good point. Politically, I don't want to hold up HZ+ any longer than necessary. How about the following way of dealing with this? We could go ahead and vote on the mapping scheme issue, deciding between HZ+S and HZ-2. Then we vote on the escape sequences, i.e. whether to use ~1 and ~2, ~y and ~z, etc. At that point we have a well-defined HZ+ standard. Then we vote on whether to modify the standard in the way that Ricky proposed. Would this procedure be agreeable to everybody? -- Steve From simpson@math.psu.edu Tue Jun 7 23:36:24 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA18837; Tue, 7 Jun 94 23:36:24 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id AAA03628; Wed, 8 Jun 1994 00:33:30 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 8 Jun 1994 00:33:27 -0400 Date: Wed, 8 Jun 1994 00:33:27 -0400 Message-Id: <199406080433.AAA24078@boole.math.psu.edu> To: chinesenet@uclink.berkeley.edu, ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: response to a good-natured flame In-Reply-To: <9406042236.aa01312@dispatch.demon.co.uk> References: <9406042236.aa01312@dispatch.demon.co.uk> Reply-To: simpson@math.psu.edu Status: O John Delacour writes: > Steve Simpson has written the code for hzp and I was delighted that > he did and have followed the discussion of it very closely, as you > all know. But why on earth has Steve not produced something for > the end user based on his code?? In part the answer is, because I'm not a hotshot programmer. But aren't my filters good enough? I admit that they are not terribly user friendly, but they do give DOS and Unix users a simple way to convert files from Big5 to HZ+ and back. Posting a Big5 message requires a three step process: save the message in a file, convert the file to HZ+, post the converted file. My feeling was that this is awkward but good enough as a stopgap, until software becomes HZ+ aware. Perhaps your question is really Mac-specific. I don't know anything about MacOS, but from what you were saying earlier, I think that perhaps the Unix/DOS concept of a filter (from standard input to standard output) does not apply to Macs, and perhaps this is why my filter programs are particularly awkward to use on Macs. I'm not sure how to remedy this. We need a Mac programmer! It would be possible to rewrite my programs so that they would convert files rather than standard input and output? I.e. the Unix or DOS syntax would be something like big2hzp infile outfile instead of big2hzp < infile > outfile Would this help? I really don't know anything about Mac OS. Does it have a well defined concept of a file and file conversion? > You have said, Steve, that it would be no problem to knock up some > LISP, or whatever, but you haven't done it. Too busy responding to flames. :-) > In the mean time I've got HZ added to two Mac programmmes written > by authors with no knowledge of Chinese at all and no interest in > China. So if you are convinced that HZ+S is a good thing (which I > think it is) then get on and produce something for the end user, > otherwise it'll die in the nest. Adding HZ+ support to an existing program should be no harder than adding HZ support. I think the real problem is that we have not yet agreed on an HZ+ standard. Once we agree on a standard, we can start asking programmers to support it in their existing telecommunications programs. Until then, we have to rely on standalone programs such as my Big5<->HZ+ filters. > I asked you last week if you didn't know a Mac programmer that > could compile your code and you answered a simple no, so it's me > that has to go out trying to get someone to manufacture your > brainwave for the Mac, when you should be out there yourself > knocking on doors! I am reluctant to ask programmers to support HZ+ until we get a consensus on what HZ+ is. > Forget the likes of Jeffrey Chen -- I can't imagine why you replied > to him so politely; he's obviously a maniac. Well, there's a lot of hostility there all right, but in the past he did raise some good points about Big5 environments. -- Steve From simpson@math.psu.edu Wed Jun 8 00:11:59 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA20658; Wed, 8 Jun 94 00:11:59 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id BAA03864; Wed, 8 Jun 1994 01:09:33 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 8 Jun 1994 01:09:30 -0400 Date: Wed, 8 Jun 1994 01:09:30 -0400 Message-Id: <199406080509.BAA24139@boole.math.psu.edu> To: mule@etl.go.jp Cc: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: Re: Big5, CNS, Mule, and HZ+ In-Reply-To: <9406071252.AA21632@etlken.etl.go.jp> References: <199406070937.FAA22405@boole.math.psu.edu> <9406071252.AA21632@etlken.etl.go.jp> Reply-To: simpson@math.psu.edu Status: O Kenichi Handa writes: > Do you know who is maintaining/correcting/updating Big5? I don't think that there is any official Big5 standards body. Big5 is a popular standard, not an official standard. > In your document `etfonts.doc' (I just ftped it), you > mention "ETEN Version 2.00.03". Who is deciding this > version number? This is just the version number of the ETen version that I had at the time. The current ETen version number is higher, maybe 3.2 or something like that. > They surely contains (above f9d6) 7 Chinese chars, 33 > graphic chars, and 2 funny black boxes at fa0e and fa40. > These are apparently added later. But they still don't have > any characters in the range c6a1-c8d3. In ETen Big5, the c6a1 - c8fe range contains some interesting things, including Japanese kana and the Russian alphabet. > Non-chinese characters are not included in CNS-11643. If this is correct, then it is a significant obstacle to Big5 <-> CNS conversion. ETen Big5 definitely does include a lot of non-Chinese characters. Among these are Japanese kana, the Russian alphabet, etc. > If those are duplicates of others, it measn that "100 percent > invertible" is not that important. 100 percent invertibility is not as important as some other things, but it would still be desirable. > Actually these are the problem if again these are really > being used. Certainly these Big5 features are being used. I am referring to Chinese characters above F9D6, also Japanese kana, Russian alphabet, etc. ETen Big5 includes these features because users like them. > But, if ETen is not the sole Big5 implementation, how you > can conclude that ETen is the most popular one? I don't have user statistics, but ETen is a very, very popular Chinese environment for MS-DOS and I am sure that millions of people in Taiwan and elsewhere use it every day. People in Taiwan have repeatedly assured me that ETen is by far the most popular Chinese environment, maybe 90 percent of the market or more. ETen is also very popular in Hong Kong and the US. > I think there're many taipei16/24 users because they are provided > by the very famous FTP sites (ifcss.org). Not nearly as many as ETen Big5 users. With ETen we are talking about orders of magnitude, factors of 10 or 100 times as many users. > I didn't know that there're many implementations of Big5 > other than ETen and even didn't know that ETen has several > versions. (I now know that HKU version is just a mistake). ETen is a software package. Like all popular software packages, it is updated and revised from time to time. These revisions are called versions. It is just like SunOS 4.1.3 or MS-DOS 6.0. There are other Big5 software packages that compete with ETen, and probably each of them has a slightly different version of Big5. But ETen is by far the most popular Chinese software in Taiwan, and the most popular Big5 software anywhere. > If Big5 can never be standardized, don't you feel uneasy > about using such a thing as a very important base of > communication? I don't really feel uneasy about using Big5. Millions of people use it every day, with great success. It would be nice if Big5 were an official standard, like ASCII or GB. Unfortunately, this is not the case. We must adapt to the real world, not to the world of our dreams and wishes. > Just maping chars not in CNS to code spaces which are not used by > CNS and vice versa as I wrote before. If this private mapping get > popular outside of Mule, the load of Mule gets light. I don't think a private mapping is the way to go. I think we should use mappings that are standard and public. If we can't use a standard mapping, then we should use a mapping that is very simple and 100 percent invertible. That is the approach that you followed in Mule 1.0 and that I followed in HZ+S, in our common approach to Big5. > I think limiting Big5 as a subset of CNS and using other character > sets (e.g. JISX0208 and ISO8859-5 (cyrillic)) mixtured with CNS is > better. People using non-standard graphic characters are difficult > to shift to standard world of ISO2022 or ISO10646 (including > Unicode). Unfortunately, millions of Big5 users do not agree with you. I want HZ+ to support Big5 as it is, not as we would like it to be. -- Steve From simpson@math.psu.edu Wed Jun 8 00:43:39 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA22309; Wed, 8 Jun 94 00:43:39 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id BAA04002; Wed, 8 Jun 1994 01:41:28 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 8 Jun 1994 01:41:26 -0400 Date: Wed, 8 Jun 1994 01:41:26 -0400 Message-Id: <199406080541.BAA24213@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: Re: how to use my HZ+S software In-Reply-To: <2sllfn$nu8@ifcss.org> References: <199406021518.KAA27196@news.cs.utexas.edu> <2sllfn$nu8@ifcss.org> Reply-To: simpson@math.psu.edu Status: O Yilong Chen writes: > : 3. Advantages of HZ+. > i prefer to call this Features of HZ+, since some of the features may not > be advantages to some users/developers, and also as the number of > competing hz+ schemes has been narrowed down dramatically. > Most stated advantages are not as a comparison to alternative hz+ > schemes, as most schemes have been withdrawn from the competition. OK, OK. I have changed hzp.doc to say "features" instead of "advantages". But I still think that all of these features are beneficial features, not just features. :-) -- Steve From simpson@math.psu.edu Wed Jun 8 08:43:45 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA04370; Wed, 8 Jun 94 08:43:45 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id JAA05758; Wed, 8 Jun 1994 09:41:37 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 8 Jun 1994 09:41:36 -0400 Date: Wed, 8 Jun 1994 09:41:36 -0400 Message-Id: <199406081341.JAA24990@boole.math.psu.edu> To: soft-authors@ifcss.org, chinesenet@uclink.berkeley.edu, ccnet-l@uga.cc.uga.edu Subject: Re: response to a good-natured flame In-Reply-To: <9406080719.aa07727@dispatch.demon.co.uk> References: <9406080719.aa07727@dispatch.demon.co.uk> Reply-To: simpson@math.psu.edu Status: O John Delacour writes: > >>How easy was it to include the code? I ask because the > >>argument is still going on about the conversion from Big5 > >>to 7-bit ("HZ+"). It will probably be a week or two before > >>HZ+ is accepted, so there's no point yet in including it in > >>any software, but it would be interesting to know if there's > >>any big problem. A week or two? There are still three votes to be taken, so it could take considerably longer. I was hoping to reach a quick consensus to the effect that HZ+ equals HZ+S, but people keep proposing changes. Once soft-authors reaches a consensus on the definition of HZ+, I'm sure that authors of telecommunications packages will find it very easy to add HZ+ support. But first, we need that consensus. > I'll see what I can do to get a basic Mac application for HZ+S and one for > B5BE and make them available but it might be a few days. As to the code > for the third proposal (HZ-2) I have no idea where this is. Did I miss > something? Jing-Shin has withdrawn B5E1 and B5E2 and B5E3, but if you are still interested, his information may still be available at his FTP site. To the best of my knowledge, nobody has written any code for HZ-2, which exists only as a proposal document. Thanks very much for your good work in promoting HZ and HZ+ on the Mac. -- Steve From simpson@math.psu.edu Wed Jun 8 12:35:10 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA12682; Wed, 8 Jun 94 12:35:10 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id NAA07228; Wed, 8 Jun 1994 13:33:05 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 8 Jun 1994 13:33:03 -0400 Date: Wed, 8 Jun 1994 13:33:03 -0400 Message-Id: <199406081733.NAA25473@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: comments on Ricky Yeung's MAT proposal Reply-To: simpson@math.psu.edu Status: O Thinking more about Ricky Yeung's MAT proposal (MAT = maximal ascii transparency), I can see some possible objections to it. Objection 1 Consider what would happen in a fairly typical case, when you convert a Big5 file. Let's say your Big5 file looks like this: 8-bit Big5 data line one 8-bit Big5 data line two 8-bit Big5 data line three Under HZ+S this would convert to: ~>7-bit Big5 data line one~} ~>7-bit Big5 data line two~} ~>7-bit Big5 data line three~} Under HZ+S as modified by Ricky's proposal, this would become: ~{~>7-bit Big5 data line one~} ~{~>7-bit Big5 data line two~} ~{~>7-bit Big5 data line three~} I don't see that this is an improvement. It is longer, and the escape sequences are just as likely to be misinterpreted, if not more so. Admittedly this is a rather small, unimportant matter. But if we are going to have to spend time debating Ricky's proposal, we may as well begin now. Objection 2 Another objection to Ricky's proposal is that, politically, it isn't fair to Big5. For GB users there is no objection, because ~{ is used to switch into 7-bit GB mode, just as under the HZ standard. But for Big5 users, it is unfair because whenever they want to switch into Big5 mode, they are forced to switch to GB mode first. This is politically objectionable. We want to treat GB and Big5 equally and fairly. Big5 users who don't care about GB should be able to use HZ+ for e-mail etc. without having to get involved with GB, just as GB users don't have to be concerned with Big5. -- Steve From simpson@math.psu.edu Wed Jun 8 16:22:33 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA21863; Wed, 8 Jun 94 16:22:33 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id RAA08792; Wed, 8 Jun 1994 17:20:19 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 8 Jun 1994 17:20:17 -0400 Date: Wed, 8 Jun 1994 17:20:17 -0400 Message-Id: <199406082120.RAA25837@boole.math.psu.edu> To: mule@etl.go.jp Cc: ccnet%uga.BITNET@psuvm.psu.edu, soft-authors@ifcss.org Subject: Re: Big5, CNS, Mule, and HZ+ In-Reply-To: <9406070211.AA21193@etlken.etl.go.jp> References: <199405311636.MAA29487@boole.math.psu.edu> <9406070211.AA21193@etlken.etl.go.jp> Reply-To: simpson@math.psu.edu Status: O About Mule's support for Big5, Kenichi Handa writes: > The reason why I want to change the way of Big5 supporting > is just because I want to delete all adhoc code (in C files) > existing just for Big5. I don't understand how you plan to accomplish this. My impression is that the ad hoc Big5 code is there because Big5's character set is too big to fit into one 94x94 code space: 13000 > 94 * 94 Therefore, unlike GB or JIS, Big5 needs two 94x94 blocks and therefore two ISO-2022 escape sequences. This implies that, for example, Mule's regular expression for a Big5 character has to match both blocks, thus necessitating special C code. I don't see how you can avoid this necessity, so long as you stay within the ISO-2022 framework. CNS is subject to the same problem. Another remark about Big5 versus CNS. It's true that CNS is registered as an official ISO-2022 character set, while Big5 is not. However, as Ross Paterson has pointed out, ISO-2022 reserves certain escape sequences ("final characters", actually) for user-specified or application-specific character sets. What's wrong with using two of these escape sequences for Big5 blocks 1 and 2? Isn't that exactly what you did in Mule 1.0? If you do this, then you can support Big5 directly, within the ISO-2022 framework. There is no need to resort to the doubtful, indirect method of Big5<->CNS translation. Of course I have no objection to adding CNS support to Mule. But I think that Big5 should also be supported as a distinct character set. Big5 and CNS are not the same, no matter how much we might wish that they were. -- Steve From Ricky.Yeung@Eng.Sun.COM Wed Jun 8 17:03:00 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA23309; Wed, 8 Jun 94 17:03:00 CDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA10292; Wed, 8 Jun 94 15:00:16 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA29626; Wed, 8 Jun 94 15:02:38 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA03817; Wed, 8 Jun 1994 15:01:32 +0800 Date: Wed, 8 Jun 1994 15:01:32 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406082201.AA03817@hk.Eng.Sun.COM> To: soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Wed, 8 Jun 1994 13:33:03 -0400 <199406081733.NAA25473@boole.math.psu.edu> Subject: Re: comments on Ricky Yeung's MAT proposal Content-Length: 5717 Status: O [[ I'm replying several related messages altogether. ]] >There is also a somewhat different suggestion from Ricky Yeung, namely >to use something like ~{~y and ~{~z. I'm not sure whether this will >lead to a formal proposal. If it does, then I'm not sure how to >handle it. My proposal is to use ~{ to switch into non-ASCII (7-bit) mode. Inside the 7-bit mode, you can still switch between frequently-used and non-frequently-used codes. So you would do things like ~{~y......~z.....~y......~} Compare this to, say, HZ+L, ~y......~z.....~y......~} >Ricky Yeung writes: > > It's better than the other HZ proposals in terms of ASCII transparency, > > don't you agree? BTW, I didn't claim it's optimal but I think it's > > no worse than HZ and better than other HZ+ proposals. > >I agree. Your modification offers more ASCII transparency than the >existing HZ+ proposals. I'm not sure that your modification >is optimal in this respect, but it is an improvement. In fact, with my suggested (in my last message) changes to HZ, this is indeed optimal in ASCII transparency if we want to maintain HZ compatibility. > > Introducing new ~ escape sequence decreases ASCII transparency. HZ > > already introduced (basically) one non-transparency (lack of a better > > term) without much choice. I believe the best way would be not to > > further erode ASCII transparency but leveraging on the exisitng HZ > > sequence. > >Yes, Ricky's modification does have this technical advantage. The >only disadvantage that I see is that the total number of bytes used in >escape sequences is increased considerably. It's two bytes per SWITCH into 7-bit mode, and typically there is only one switch per line. I won't use the term "increased considerably," especially compare to other HZ+'s. > > I don't believe my proposal is more complicated and less clean; in fact, > > it would even make life simpler for future extensibility. The reason I > > posted my previous message is that I see a technical (not political) > > problem, namely the maximal ASCII transparency issue, with the current > > proposals. And I'm offering an enhencement solution that would maximize > > ASCII transparency within the HZ framework. > >From a technical point of view, I think Ricky may have a good point. >Politically, I don't want to hold up HZ+ any longer than necessary. > >How about the following way of dealing with this? We could go ahead >and vote on the mapping scheme issue, deciding between HZ+S and HZ-2. >Then we vote on the escape sequences, i.e. whether to use ~1 and ~2, >~y and ~z, etc. At that point we have a well-defined HZ+ standard. >Then we vote on whether to modify the standard in the way that Ricky >proposed. Would this procedure be agreeable to everybody? >Thinking more about Ricky Yeung's MAT proposal (MAT = maximal ascii >transparency), I can see some possible objections to it. > >Objection 1 > >Consider what would happen in a fairly typical case, when you convert >a Big5 file. Let's say your Big5 file looks like this: > > 8-bit Big5 data line one > 8-bit Big5 data line two > 8-bit Big5 data line three > >Under HZ+S this would convert to: > > ~>7-bit Big5 data line one~} > ~>7-bit Big5 data line two~} > ~>7-bit Big5 data line three~} > >Under HZ+S as modified by Ricky's proposal, this would become: > > ~{~>7-bit Big5 data line one~} > ~{~>7-bit Big5 data line two~} > ~{~>7-bit Big5 data line three~} > >I don't see that this is an improvement. It is longer, and the escape >sequences are just as likely to be misinterpreted, if not more so. It's two bytes longer per switch into 7-bit mode. Please see my example above. There is still a chance for ~{ to be misinterpreted (it's not 100% ASCII transparent), BUT we've been using it in HZ. Introducing new escape sequences decreases ASCII transparency further. My proposal closes the flood gate for further erosion of ASCII transparency. Imagine the effect on ASCII transparency, if it's decided to additionally support CNS or other 7-bit codes in the future. >Admittedly this is a rather small, unimportant matter. But if we are >going to have to spend time debating Ricky's proposal, we may as well >begin now. Escape sequence is not an unimportant matter. After all, the essential difference between HZ and ISO 2022(?) (the section about GB) is escape sequence. We just took the HZ escape sequence choice for granted. >Objection 2 > >Another objection to Ricky's proposal is that, politically, it isn't >fair to Big5. For GB users there is no objection, because ~{ is used >to switch into 7-bit GB mode, just as under the HZ standard. But for >Big5 users, it is unfair because whenever they want to switch into >Big5 mode, they are forced to switch to GB mode first. This is >politically objectionable. We want to treat GB and Big5 equally and >fairly. Big5 users who don't care about GB should be able to use HZ+ >for e-mail etc. without having to get involved with GB, just as GB >users don't have to be concerned with Big5. In order to switch among 7-bit codes inside the 7-bit mode, you need a prefix for GB anyway. I propose the use of ~z for 7-bit GB, and ~x and ~y for 7-bit Big5. Solely for maintaining HZ compatibility, the default would be 7-bit GB when you first switch into 7-bit mode. For example, ~{..GB..~x..big5-1..~y..big5-2..~z..GB..~} compare to HZ+S ~{..GB..~}~1..big5-1..~}~y..big5-2..~}~{..GB..~} or (is the following legel in HZ+S?) ~{..GB..~1..big5-1..~y..big5-2..~{..GB..~} -Ricky From lai@apple.com Wed Jun 8 19:26:08 1994 Return-Path: Received: from apple.com by ifcss.org (4.1/IFCSS-Mailer) id AA27848; Wed, 8 Jun 94 19:26:08 CDT Received: from [17.203.80.11] by apple.com with SMTP (5.61/8-Oct-1993-eef) id AA19411; Wed, 8 Jun 94 17:23:38 -0700 for soft-authors@ifcss.org Date: Wed, 8 Jun 94 17:23:38 -0700 Message-Id: <9406090023.AA19411@apple.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" To: ccnet%uga.BITNET@cunyvm.cuny.edu, soft-authors@ifcss.org, mule@etl.go.jp From: lai@apple.com (Edmund Lai) Subject: Re: Big5, CNS, Mule, and HZ+ Status: O > >In ETen Big5, the c6a1 - c8fe range contains some interesting things, >including Japanese kana and the Russian alphabet. > > > Non-chinese characters are not included in CNS-11643. > >If this is correct, then it is a significant obstacle to >Big5 <-> CNS conversion. ETen Big5 definitely does include >a lot of non-Chinese characters. Among these are Japanese kana, >the Russian alphabet, etc. > As pointed out by Christian Wittern, in a multi-lingual system it does not make sense to insist that kana and Cyrillic alphabet to be part of Big5. So it make sense to map it to the proper Japanese and Russian code range. In an extension of HZ, there is not yet and extension to Russian or Japanese, but as Christian pointed out kana and Cyrillic are already in GB, so it can be mapped to the range in GB. >Certainly these Big5 features are being used. I am referring to >Chinese characters above F9D6, As I pointed out before, it is wrong to post any user specific character on to a newsgroup which expect to be communicated with a common exchange code. > > > I think limiting Big5 as a subset of CNS and using other character > > sets (e.g. JISX0208 and ISO8859-5 (cyrillic)) mixtured with CNS is > > better. People using non-standard graphic characters are difficult > > to shift to standard world of ISO2022 or ISO10646 (including > > Unicode). > >Unfortunately, millions of Big5 users do not agree with you. I want >HZ+ to support Big5 as it is, not as we would like it to be. > Again we go through this so many times, millions of Big5 users want to use Big5 as the internal code in their Big5 document, they don't care what the exchange code is as long as it can be translated to and from Big5. Supporting CNS as an exchange code does not mean not supporting Big5 as the internal code. On alt.chinese.computing people from Taiwan has expressed again and again that they do not consider converting CNS and Big5 to be a problem. Steve is the one who keep ignoring what they said and keep using the myth that "Big5 user demands this" as a shield. It is a complete disrespect to keep twisting other people's view like this. It does not make sense to support both CNS and another form of 7 bit Big5 in the same exchange code. The two are the same set of Chinese characters (with the exception of two duplicate characters removed in CNS). To include both in the same exchange code violates the very meaning of exchange code. /* Disclaimer: All statments and opinions expressed are my own */ /* Edmund K. Lai */ /* Apple Computer, MS303-3A */ /* 20525 Mariani Ave, */ /* Cupertino, CA 95014 */ /* (408)974-6272 */ zW@h9cOi From simpson@math.psu.edu Wed Jun 8 19:48:25 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA28479; Wed, 8 Jun 94 19:48:25 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id UAA11649; Wed, 8 Jun 1994 20:46:20 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 8 Jun 1994 20:46:19 -0400 Date: Wed, 8 Jun 1994 20:46:19 -0400 Message-Id: <199406090046.UAA26253@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: Re: proposed changes in HZ+ In-Reply-To: <9406031747.AA29348@hk.Eng.Sun.COM> References: <199406021609.MAA08258@boole.math.psu.edu> <9406031747.AA29348@hk.Eng.Sun.COM> Reply-To: simpson@math.psu.edu Status: O Ricky Yeung writes: > The reason I posted my previous message is that I see a technical > (not political) problem, namely the maximal ASCII transparency > issue, with the current proposals. And I'm offering an enhencement > solution that would maximize ASCII transparency within the HZ > framework. I am unclear about exactly what you are offering. Are you planning to write your own HZ+ proposal? Or, alternatively, are you merely offering suggestions for Ya-Gui and me as to how we might improve our own HZ+ proposals? It is important for you to clarify this question, because otherwise I don't know how we can make a decision. Please remember that we are trying to reach a consensus within soft-authors on HZ+. This implies that, when proposing technical changes, you have to be very precise about the question of exactly what you are asking people to do. I admit that this is a political matter, but political matters are sometimes important too. -- Steve From simpson@math.psu.edu Wed Jun 8 20:17:57 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA29656; Wed, 8 Jun 94 20:17:57 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id VAA12118; Wed, 8 Jun 1994 21:15:50 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 8 Jun 1994 21:15:49 -0400 Date: Wed, 8 Jun 1994 21:15:49 -0400 Message-Id: <199406090115.VAA26299@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: Re: comments on Ricky Yeung's MAT proposal In-Reply-To: <9406082201.AA03817@hk.Eng.Sun.COM> References: <199406081733.NAA25473@boole.math.psu.edu> <9406082201.AA03817@hk.Eng.Sun.COM> Reply-To: simpson@math.psu.edu Status: O Ricky Yeung writes: > Escape sequence is not an unimportant matter. Uh oh. I get the feeling that we are in for a long debate on escape sequences. :-) > After all, the essential difference between HZ and ISO 2022(?) (the > section about GB) is escape sequence. I disagree. In my view, the essential differences between HZ and ISO-2022 are: (1) ISO-2022 is an official standard endorsed by national and international standards bodies. HZ is a popular standard used for practical purposes. (2) ISO-2022 uses a non-printable ASCII character, namely Esc or 0x21, to switch from one character set to another. HZ uses instead a printable ASCII character, ~. Compared to (1) and (2), the choice of whether to use ~{~y or ~> is a relatively unimportant issue, in my opinion. > In order to switch among 7-bit codes inside the 7-bit mode, you need a > prefix for GB anyway. No, you don't. You can just use ~{. If another escape sequence is used to switch into GB, that escape sequence is redundant. > I propose the use of ~z for 7-bit GB, and ~x and ~y for 7-bit Big5. Under this proposal, ~z is redundant, because you can always use ~{ instead of ~z. Does your ~x ~y ~z represent yet another proposal, or is it to be considered as an addition to your MAT proposal? (Sorry to bother you with these political questions.) > Solely for maintaining HZ compatibility, the default would be 7-bit > GB when you first switch into 7-bit mode. Once again, this would not be fair to Big5. Under HZ+S, the only default character set is ASCII, and neither GB nor Big5 is considered a default. Under your proposal, GB is positioned as the default 7-bit character set. Obviously this is not fair to Big5. Fairness and even-handedness as between GB and Big5 is an important political issue. One of the difficulties in gaining acceptance of HZ+ is that Big5 users will be hostile if they see it as an imposition of a GB-like standard. Some Big5 users are already suspicious of chinese.* for this reason. Your proposal tends to exacerbate this problem. (Again, sorry to bother you with political issues.) > or (is the following legel in HZ+S?) > > ~{..GB..~1..big5-1..~y..big5-2..~{..GB..~} Yes, of course, it is legal in HZ+S. But this assumes we are using ~1 for block 1 and ~y for block 2. I don't think anybody except you has proposed this particular combination of escape sequences. :-) All questions about what's legal in HZ+S are answered in my HZ+S document, available for anonymous FTP at ftp.math.psu.edu, /pub/simpson/chinese/hzp/hzp.doc. -- Steve From simpson@math.psu.edu Wed Jun 8 21:14:17 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA01694; Wed, 8 Jun 94 21:14:17 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id WAA12594; Wed, 8 Jun 1994 22:12:08 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 8 Jun 1994 22:12:06 -0400 Date: Wed, 8 Jun 1994 22:12:06 -0400 Message-Id: <199406090212.WAA26387@boole.math.psu.edu> To: ccnet%uga.BITNET@cunyvm.cuny.edu, soft-authors@ifcss.org, mule@etl.go.jp Subject: Re: Big5, CNS, Mule, and HZ+ In-Reply-To: <9406090023.AA19411@apple.com> References: <9406090023.AA19411@apple.com> Reply-To: simpson@math.psu.edu Status: O Edmund Lai writes: > As pointed out by Christian Wittern, in a multi-lingual system it > does not make sense to insist that kana and Cyrillic alphabet to > be part of Big5. So it make sense to map it to the proper Japanese > and Russian code range. In an extension of HZ, there is not yet > and extension to Russian or Japanese, but as Christian pointed > out kana and Cyrillic are already in GB, so it can be mapped to > the range in GB. Ed, are you talking about HZ+? If you are, I'm sorry to have to point out that this question has already been decided. Soft-authors voted to postpone consideration of CNS for the time being, so far as HZ+ is concerned. The only questions remaining are HZ+S versus HZ-2, and escape sequences. HZ+ is not intended to be a true multilingual system. Mule is another matter, of course. #ifdef FLAME Ed's idea of using GB codes to represent Big5 codes for Kana and Cyrillic strikes me as incredibly bizarre. Ed, please stop the insanity! #endif :-) > >Certainly these Big5 features are being used. I am referring to > >Chinese characters above F9D6, > > As I pointed out before, it is wrong to post any user specific > character on to a newsgroup which expect to be communicated with > a common exchange code. The Chinese characters above F9D6 that I was referring to are not user-specific. They may be ETen-specific, but I don't see anything wrong with posting them in newsgroups. After all, ETen Big5 is very popular. If users want to post those characters, they should be allowed to do so. It's not our job to prevent them. > On alt.chinese.computing people from Taiwan has expressed again and > again that they do not consider converting CNS and Big5 to be a > problem. I don't think people from Taiwan have said that. I know that Ed himself has said that, but Ed is not from Taiwan, right? Anyway, what exactly does Ed mean by "converting from CNS and Big5 is not a problem"? Obviously it's not a problem for you if it works for your particular document, or for the particular subsets of Big5 and CNS that you use. However, I want to accommodate the needs of *all* Big5 users, even those who use Big5 characters that Ed may not approve of. In this sense, CNS<->Big5 conversion is indeed a problem. If Ed really thinks it is not a problem, I challenge him to define precisely the conversion algorithm that he wants to use. > It does not make sense to support both CNS and another form of > 7 bit Big5 in the same exchange code. This statement from Ed is completely muddy. CNS is not "a form of 7-bit Big5". CNS and Big5 are two different coding systems and two different character sets. Probably the CNS design committee were hoping that CNS would replace Big5, but that is not how things have worked out. Big5 remains extremely popular, while Ed has not been able to find one person who uses CNS in his or her daily work. The biggest difference between CNS and Big5 is that CNS is official and unpopular, while Big5 is unofficial and popular. > The two are the same set of Chinese characters (with the exception > of two duplicate characters removed in CNS). This statement from Ed is factually incorrect. Big5 contains a lot of stuff in addition to the 13051 CNS standard Chinese characters. Big5 users want this additional stuff, otherwise it would not be there. CNS contains some of the additional stuff, but by no means all of it. Among the additional stuff, which Big5 users want but is missing from CNS, are Japanese kana and the Russian alphabet. Nelson pointed out that the Big5 Japanese kana are indeed used, for instance they are used heavily in Taiwan Big5 mirrors of Japanese-language newsgroups. Ed may be unhappy about this, but he ought to be willing to admit that it is true. > To include both in the same exchange code violates the very meaning > of exchange code. What is "the very meaning of exchange code"? I have repeatedly challenged Ed to define his concept of "exchange code," specifically as it applies to HZ and HZ+. Ed has repeatedly ignored this challenge. -- Steve From Ricky.Yeung@Eng.Sun.COM Wed Jun 8 22:27:46 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA04060; Wed, 8 Jun 94 22:27:46 CDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA26021; Wed, 8 Jun 94 20:25:42 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA28578; Wed, 8 Jun 94 20:28:09 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA08352; Wed, 8 Jun 1994 20:27:02 +0800 Date: Wed, 8 Jun 1994 20:27:02 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406090327.AA08352@hk.Eng.Sun.COM> To: soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Wed, 8 Jun 1994 21:15:49 -0400 <199406090115.VAA26299@boole.math.psu.edu> Subject: ASCII tranparency Content-Length: 4332 Status: O >Uh oh. I get the feeling that we are in for a long debate on escape >sequences. :-) Are we in sync. here that maximal ASCII tranparency is important? > > After all, the essential difference between HZ and ISO 2022(?) (the > > section about GB) is escape sequence. > > I disagree. In my view, the essential differences between HZ and > ISO-2022 are: (1) ISO-2022 is an official standard endorsed by > national and international standards bodies. HZ is a popular standard > used for practical purposes. (2) ISO-2022 uses a non-printable ASCII > character, namely Esc or 0x21, to switch from one character set to > another. HZ uses instead a printable ASCII character, ~. What I really meant is the "essential technical difference". So let's forget about your (1). So you do admit that it's important to choose printable ASCII characters as escape sequence, right? Do you imply that picking ~{ in HZ (not HZ+) is unimportant, i.e. one can use, say ~< and ~> instead of ~{ and ~}? > Compared to (1) and (2), the choice of whether to use ~{~y or ~> is a > relatively unimportant issue, in my opinion. My proposal is really using ~{ as the sole escape sequence into 7-bit mode. In my opinion, it's relatively UNIMPORTANT to choose the ASCII escape sequence once it's in 7-bit mode. So I don't want mine to be labeled as the ~{~y/~{~z proposal. > > In order to switch among 7-bit codes inside the 7-bit mode, you need a > > prefix for GB anyway. > > No, you don't. You can just use ~{. If another escape sequence is > used to switch into GB, that escape sequence is redundant. > > I propose the use of ~z for 7-bit GB, and ~x and ~y for 7-bit Big5. > > Under this proposal, ~z is redundant, because you can always use ~{ > instead of ~z. As I've been saying, in my proposal, the ~{ has the sole meaning of switching into 7-bit mode, NOT GB. Yes, you may want to overload it to mean the switching to GB, but I prefer not to. For example, it breaks the nice symmetric property of the ~{ ~} pair. > Does your ~x ~y ~z represent yet another proposal, or is it to be > considered as an addition to your MAT proposal? (Sorry to bother you > with these political questions.) To get my point across: the main idea of my proposal is to use the existing HZ escape sequence, namely ~{, as the sole escape sequence into 7-bit mode. Pick your choice inside 7-bit mode if you want, they are relatively unimportant. > > Solely for maintaining HZ compatibility, the default would be 7-bit > > GB when you first switch into 7-bit mode. > > Once again, this would not be fair to Big5. Under HZ+S, the only > default character set is ASCII, and neither GB nor Big5 is considered > a default. Under your proposal, GB is positioned as the default 7-bit > character set. Obviously this is not fair to Big5. If we want to maintain maximal ASCII tranparency and preserve HZ backward compatibility (which has been around for 5 years), I don't see other better alternatives, do you? > Fairness and even-handedness as between GB and Big5 is an important > political issue. One of the difficulties in gaining acceptance of HZ+ > is that Big5 users will be hostile if they see it as an imposition of > a GB-like standard. Some Big5 users are already suspicious of > chinese.* for this reason. Your proposal tends to exacerbate this > problem. (Again, sorry to bother you with political issues.) In terms of Taiwan and Mainland politics, there could be endless arguments like why HZ+ chooses ~1/~2 for Big5 and chooses ~{ for GB, which is literally (in terms of sort order) BEHIND the Taiwan Big5. One could also argue that some Big5 users would be hostile because the HZ+ bears the HZ label (which IS for GB), and is developed as an extension of HZ. They maybe already suspicious of chinese.* for this reason. :-) > > or (is the following legel in HZ+S?) > > > > ~{..GB..~1..big5-1..~y..big5-2..~{..GB..~} > > Yes, of course, it is legal in HZ+S. But this assumes we are using ~1 > for block 1 and ~y for block 2. I don't think anybody except you has > proposed this particular combination of escape sequences. :-) Oops, a typo. -Ricky From simpson@math.psu.edu Thu Jun 9 01:14:39 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA09460; Thu, 9 Jun 94 01:14:39 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id CAA13579; Thu, 9 Jun 1994 02:12:33 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 9 Jun 1994 02:12:31 -0400 Date: Thu, 9 Jun 1994 02:12:31 -0400 Message-Id: <199406090612.CAA26837@boole.math.psu.edu> To: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Cc: soft-authors@ifcss.org Subject: ASCII tranparency In-Reply-To: <9406090327.AA08352@hk.Eng.Sun.COM> References: <199406090115.VAA26299@boole.math.psu.edu> <9406090327.AA08352@hk.Eng.Sun.COM> Reply-To: simpson@math.psu.edu Status: O Ricky Yeung writes: > Are we in sync. here that maximal ASCII tranparency is important? I think it's somewhat desirable, but not essential. I don't think it's worth the effort that we have already spent on it. > > In my view, the essential differences between HZ and > > ISO-2022 are: (1) ISO-2022 is an official standard endorsed by > > national and international standards bodies. HZ is a popular standard > > used for practical purposes. (2) ISO-2022 uses a non-printable ASCII > > character, namely Esc or 0x21, to switch from one character set to > > another. HZ uses instead a printable ASCII character, ~. > > What I really meant is the "essential technical difference". So let's > forget about your (1). Sorry, I'm not willing to forget about (1). I think (1) is very important. Apparently you want to concentrate on technical issues and forget about political issues. My view is that, in reaching a consensus on HZ+, we have to consider both technical and political issues. > So you do admit that it's important to choose printable ASCII > characters as escape sequence, right? It's important for HZ and for HZ+. But I'm not against ISO-2022's approach, either. Both approaches may be useful for certain purposes. HZ and HZ+ are good for Chinese news and e-mail. ISO-2022 is good for multilingual applications like Mule. > Do you imply that picking ~{ in HZ (not HZ+) is unimportant, > i.e. one can use, say ~< and ~> instead of ~{ and ~}? I think ~{ is a good choice for HZ, perhaps the optimal choice. > My proposal is really using ~{ as the sole escape sequence into 7-bit > mode. I find your term "7-bit mode" confusing. ASCII is 7-bit too. In HZ and HZ+, all modes are 7-bit modes. Could you please think of a less confusing term? Perhaps "non-ASCII mode"? > So I don't want mine to be labeled as the ~{~y/~{~z proposal. How do you want it to be labeled? Apparently you are getting ready to make a formal proposal. I think you should give your proposal a name, so that we can discuss it and compare it to other proposals. I think we are in for at least three to four weeks of confusing and exhausting debate over your proposal, so could you please do us the courtesy of choosing a name for it? > As I've been saying, in my proposal, the ~{ has the sole meaning of > switching into 7-bit mode, NOT GB. Yes, you may want to overload it to > mean the switching to GB, but I prefer not to. I don't understand this remark. Your proposal overloads ~{ in exactly this way. Every time you type ~{ on a 2-byte boundary, you are in GB mode. If you don't prefer this, then why does your actual proposal demand this? > If we want to maintain maximal ASCII tranparency and preserve HZ > backward compatibility (which has been around for 5 years), I don't see > other better alternatives, do you? If the price of maximal ASCII transparency is unfairness to Big5, I don't want to pay that price. I don't want even the appearance of unfairness to Big5 or preferential treatment of GB. > One could also argue that some Big5 users would be hostile because > the HZ+ bears the HZ label (which IS for GB), and is developed as > an extension of HZ. They maybe already suspicious of chinese.* for > this reason. :-) This is no joke. I think that some Big5 users are hostile for exactly that reason. The hostility would be exacerbated if we treat Big5 unfairly, as you are proposing to do. I know you want to pretend that political issues are unimportant, but in fact they are very important. A communication standard that is not acceptable to large groups of people is useless. -- Steve From Ricky.Yeung@Eng.Sun.COM Thu Jun 9 02:33:47 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA11624; Thu, 9 Jun 94 02:33:47 CDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA05711; Thu, 9 Jun 94 00:31:41 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA03465; Thu, 9 Jun 94 00:34:08 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA08532; Thu, 9 Jun 1994 00:33:00 +0800 Date: Thu, 9 Jun 1994 00:33:00 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406090733.AA08532@hk.Eng.Sun.COM> To: soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Thu, 9 Jun 1994 02:12:31 -0400 <199406090612.CAA26837@boole.math.psu.edu> Subject: The hostile Big5 users? (was Re: ASCII tranparency) Content-Length: 895 Status: O > > One could also argue that some Big5 users would be hostile because > > the HZ+ bears the HZ label (which IS for GB), and is developed as > > an extension of HZ. They maybe already suspicious of chinese.* for > > this reason. :-) > > This is no joke. I think that some Big5 users are hostile for exactly > that reason. The hostility would be exacerbated if we treat Big5 > unfairly, as you are proposing to do. > > I know you want to pretend that political issues are unimportant, > but in fact they are very important. A communication standard that > is not acceptable to large groups of people is useless. I'm confident that the majority of Big5 users in Taiwan and all over the world won't be as narrow-minded as you think. They will recognize the relevant issues and eventually settle on the technically superior solution. -Ricky From Ricky.Yeung@Eng.Sun.COM Thu Jun 9 02:48:32 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA11965; Thu, 9 Jun 94 02:48:32 CDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA08016; Thu, 9 Jun 94 00:46:28 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA03833; Thu, 9 Jun 94 00:48:56 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA08539; Thu, 9 Jun 1994 00:47:48 +0800 Date: Thu, 9 Jun 1994 00:47:48 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406090747.AA08539@hk.Eng.Sun.COM> To: soft-authors@ifcss.org In-Reply-To: Ricky Yeung's message of Wed, 8 Jun 1994 15:01:32 +0800 <9406082201.AA03817@hk.Eng.Sun.COM> Subject: Re: comments on Ricky Yeung's MAT proposal Content-Length: 1314 Status: O > > ~>7-bit Big5 data line three~} > > > >Under HZ+S as modified by Ricky's proposal, this would become: > > > > ~{~>7-bit Big5 data line one~} > > ~{~>7-bit Big5 data line two~} > > ~{~>7-bit Big5 data line three~} > > > >I don't see that this is an improvement. It is longer, and the escape > >sequences are just as likely to be misinterpreted, if not more so. Oh, just realized that I may have misunderstood you. Did you mean either the ~{~> or the ~> inside that would be just as likely to be misinterpreted? The former would be no more likely to be misinterpreted compared to ~{ alone. The latter won't be misinterpreted because the decoder would interpret ~> as it is when it's in ASCII mode. -Ricky My original reply: > It's two bytes longer per switch into 7-bit mode. Please see my example > above. There is still a chance for ~{ to be misinterpreted (it's not > 100% ASCII transparent), BUT we've been using it in HZ. Introducing new > escape sequences decreases ASCII transparency further. My proposal > closes the flood gate for further erosion of ASCII transparency. > Imagine the effect on ASCII transparency, if it's decided to > additionally support CNS or other 7-bit codes in the future. From Ricky.Yeung@Eng.Sun.COM Thu Jun 9 03:47:07 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA13392; Thu, 9 Jun 94 03:47:07 CDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA16515; Thu, 9 Jun 94 01:45:03 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA04320; Thu, 9 Jun 94 01:47:33 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA08565; Thu, 9 Jun 1994 01:46:24 +0800 Date: Thu, 9 Jun 1994 01:46:24 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406090846.AA08565@hk.Eng.Sun.COM> To: soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Thu, 9 Jun 1994 02:12:31 -0400 <199406090612.CAA26837@boole.math.psu.edu> Subject: Re: ASCII tranparency Content-Length: 6068 Status: O I think you're somewhat contradictary. On the one hand, you've been telling me that the escape sequence is unimportant; on the other hand, you've been trying hard to tell me that my escape sequence is politically incorrect (in your opinion). > Ricky Yeung writes: > > Are we in sync. here that maximal ASCII tranparency is important? > > I think it's somewhat desirable, but not essential. I don't think > it's worth the effort that we have already spent on it. What effort? The e-mail discussions? Can you be specific? > > > In my view, the essential differences between HZ and > > > ISO-2022 are: (1) ISO-2022 is an official standard endorsed by > > > national and international standards bodies. HZ is a popular standard > > > used for practical purposes. (2) ISO-2022 uses a non-printable ASCII > > > character, namely Esc or 0x21, to switch from one character set to > > > another. HZ uses instead a printable ASCII character, ~. > > > > What I really meant is the "essential technical difference". So let's > > forget about your (1). > > Sorry, I'm not willing to forget about (1). I think (1) is very > important. > > Apparently you want to concentrate on technical issues and forget > about political issues. My view is that, in reaching a consensus on > HZ+, we have to consider both technical and political issues. Yes, I want to concentrate on technical issues for the time being. I didn't say forget the political issues. The "forget about your (1)" was in the context of "essential technical difference" (between HZ and ISO 2022). It's not part of the "technical difference," right? > > so you do admit that it's important to choose printable ASCII > > characters as escape sequence, right? > > It's important for HZ and for HZ+. But I'm not against ISO-2022's > approach, either. Both approaches may be useful for certain purposes. > HZ and HZ+ are good for Chinese news and e-mail. ISO-2022 is good for > multilingual applications like Mule. You missed the point. I was saying do you now admit the escape sequence is important. > > Do you imply that picking ~{ in HZ (not HZ+) is unimportant, > > i.e. one can use, say ~< and ~> instead of ~{ and ~}? > > I think ~{ is a good choice for HZ, perhaps the optimal choice. > > My proposal is really using ~{ as the sole escape sequence into 7-bit > > mode. > > I find your term "7-bit mode" confusing. ASCII is 7-bit too. In HZ > and HZ+, all modes are 7-bit modes. Could you please think of a less > confusing term? Perhaps "non-ASCII mode"? Thanks for pointing out. When I started using this term, I did use "non-ASCII (7-bit) mode" in my message. Yes, I should use non-ASCII instead of 7-bit as short hand. > > So I don't want mine to be labeled as the ~{~y/~{~z proposal. > > How do you want it to be labeled? Apparently you are getting ready to > make a formal proposal. I think you should give your proposal a name, > so that we can discuss it and compare it to other proposals. I think > we are in for at least three to four weeks of confusing and exhausting > debate over your proposal, so could you please do us the courtesy of > choosing a name for it? My original intention was to point out the issue of maximal ASCCI tranparency. I offered an enhencement that would work well with any of the proposed ones (even 7-bit CNS). I feel being forced to make a proposal like the others so that it would be treated as yet another proposal. The real issue here is about ASCII tranparency. > > As I've been saying, in my proposal, the ~{ has the sole meaning of > > switching into 7-bit mode, NOT GB. Yes, you may want to overload it to > > mean the switching to GB, but I prefer not to. > > I don't understand this remark. Your proposal overloads ~{ in exactly > this way. Every time you type ~{ on a 2-byte boundary, you are in GB > mode. If you don't prefer this, then why does your actual proposal > demand this? No, you're wrong. Every time you hit ~{ you switch to non-ASCII mode, not GB mode, only when the following character is not ~, then you're in GB mode. If your parser switches to GB whenever you see ~{, then it's semantically wrong (according to my proposal). > > If we want to maintain maximal ASCII tranparency and preserve HZ > > backward compatibility (which has been around for 5 years), I don't see > > other better alternatives, do you? > > If the price of maximal ASCII transparency is unfairness to Big5, I > don't want to pay that price. I don't want even the appearance I am, and I think many others will be, willing to pay that small known cost to close the flood gate for the further unlimited erosion of ASCII transparency in the unknown future. > of unfairness to Big5 or preferential treatment of GB. I don't think Big5 is being treated unfairly by my enhencement. One may argue that Big5 was treated unfairly when HZ was born, namely, it's not supported. To be really fair, should we throw away HZ and start afresh "fair" 7-bit exchange code scheme? The rest was answered in the message "The hostile Big5 users?" that I sent out earlier. > > One could also argue that some Big5 users would be hostile because > > the HZ+ bears the HZ label (which IS for GB), and is developed as > > an extension of HZ. They maybe already suspicious of chinese.* for > > this reason. :-) > > This is no joke. I think that some Big5 users are hostile for exactly > that reason. The hostility would be exacerbated if we treat Big5 > unfairly, as you are proposing to do. > > I know you want to pretend that political issues are unimportant, > but in fact they are very important. A communication standard that > is not acceptable to large groups of people is useless. > > -- Steve -Ricky From rap@doc.ic.ac.uk Thu Jun 9 07:05:56 1994 Return-Path: Received: from santos.doc.ic.ac.uk by ifcss.org (4.1/IFCSS-Mailer) id AA18223; Thu, 9 Jun 94 07:05:56 CDT Received: by santos.doc.ic.ac.uk (Smail3.1.28.1 #7) id m0qBipl-00070kC; Thu, 9 Jun 94 13:03 BST Message-Id: Date: Thu, 9 Jun 94 13:03 BST From: Ross Paterson To: soft-authors@ifcss.org Subject: Re: ASCII tranparency Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Status: O Ricky Yeung writes: > [...] So you do admit that it's important to choose printable ASCII > characters as escape sequence, right? Why is this important? Contrariwise, if you use ESC you get complete transparency of printable ASCII. From shin@hermes.ee.nthu.edu.tw Thu Jun 9 07:19:48 1994 Return-Path: Received: from nthu.edu.tw by ifcss.org (4.1/IFCSS-Mailer) id AA18695; Thu, 9 Jun 94 07:19:48 CDT Received: from hermes.ee.nthu.edu.tw ([140.114.28.27]) by nthu.edu.tw with SMTP id <181991(2)>; Thu, 9 Jun 1994 20:23:28 +0800 Received: by hermes.ee.nthu.edu.tw (4.1/SMI-4.0/shin-1.0) id AA10411; Thu, 9 Jun 94 20:15:40 CST Date: Fri, 10 Jun 1994 10:15:40 +0800 From: shin@hermes.ee.nthu.edu.tw (Jing-Shin Chang) Message-Id: <9406091215.AA10411@hermes.ee.nthu.edu.tw> To: ccnet%uga.BITNET@cunyvm.cuny.edu, lai@apple.com, mule@etl.go.jp, soft-authors@ifcss.org Subject: Re: Big5, CNS, Mule, and HZ+ Status: O |> |> > |> >In ETen Big5, the c6a1 - c8fe range contains some interesting things, |> >including Japanese kana and the Russian alphabet. |> > |> > > Non-chinese characters are not included in CNS-11643. |> > |> >If this is correct, then it is a significant obstacle to |> >Big5 <-> CNS conversion. ETen Big5 definitely does include |> >a lot of non-Chinese characters. Among these are Japanese kana, |> >the Russian alphabet, etc. |> > |> These two days, I inspected ETen's implementation on CNS. I found that ETen had extended the CNS to include such Japanese characters. In the Big5 extension, such characters are allocated in a continuous code space. In its CNS counterparts, they are separated in 2 or 3 continuous code space. So there shouldn't be any problem to convert to/from Big5 and CNS. I guess the other code sets are also extended in the same way because the chars (bitmaps) are already there. I noticed this problem because Nelson was asking me to update the ccf (taiwanses code conversion package) to include such kanas. And I had an A-Plan to go, so ... please expect my A-Plan. -^^- Jing-Shin -^^- From shin@hermes.ee.nthu.edu.tw Thu Jun 9 07:51:27 1994 Return-Path: Received: from nthu.edu.tw by ifcss.org (4.1/IFCSS-Mailer) id AA19521; Thu, 9 Jun 94 07:51:27 CDT Received: from hermes.ee.nthu.edu.tw ([140.114.28.27]) by nthu.edu.tw with SMTP id <182036(1)>; Thu, 9 Jun 1994 20:55:10 +0800 Received: by hermes.ee.nthu.edu.tw (4.1/SMI-4.0/shin-1.0) id AA10481; Thu, 9 Jun 94 20:47:40 CST Date: Fri, 10 Jun 1994 10:47:40 +0800 From: shin@hermes.ee.nthu.edu.tw (Jing-Shin Chang) Message-Id: <9406091247.AA10481@hermes.ee.nthu.edu.tw> To: Ricky.Yeung@Eng.Sun.COM, soft-authors@ifcss.org Subject: Re: The hostile Big5 users? (was Re: ASCII tranparency) Status: O |> |> I'm confident that the majority of Big5 users in Taiwan and all over the |> world won't be as narrow-minded as you think. They will recognize the |> relevant issues and eventually settle on the technically superior |> solution. |> |> -Ricky |> YES. In fact, HanZi is more of a neutral expression than Chinese or Taiwanese in political sense. Furthermore, I think people in this list are more of technical background than political ground. We only show political attitude when unreasonable political attitude are imposed on us. Thanks Ricky. Shin -^^- From simpson@math.psu.edu Thu Jun 9 09:23:24 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA22576; Thu, 9 Jun 94 09:23:24 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id KAA19286; Thu, 9 Jun 1994 10:21:18 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 9 Jun 1994 10:21:17 -0400 Date: Thu, 9 Jun 1994 10:21:17 -0400 Message-Id: <199406091421.KAA01486@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: Re: The hostile Big5 users? In-Reply-To: <9406091247.AA10481@hermes.ee.nthu.edu.tw> References: <9406091247.AA10481@hermes.ee.nthu.edu.tw> Reply-To: simpson@math.psu.edu Status: O Ricky Yeung wrote: > |> I'm confident that the majority of Big5 users in Taiwan and all over the > |> world won't be as narrow-minded as you think. They will recognize the > |> relevant issues and eventually settle on the technically superior > |> solution. I'd like to believe that. But I have found that people sometimes lose perspective and get upset for irrational reasons, and this sometimes affects the climate. This is not only Big5 users, this is everybody. As an example of a hostile Big5 user, there is a guy named Jeffrey Chen who has been flaming in alt.chinese.computing about soft-gods, self-appointed US experts, HZ imperialism, etc. I try to handle him politely, but wow! Jing-Shin writes: > YES. In fact, HanZi is more of a neutral expression than Chinese or > Taiwanese in political sense. Yes, I agree. The name HZ+ is not biased in favor of GB or mainland users, so Big5 users cannot find anything upsetting there. On the other hand, they may still feel suspicious because HZ+ is derived from HZ, and HZ was a GB artifact. Jeffrey Chen is an example of this attitude. Jeffrey is apparently a non-programmer, but for better or worse, non-programmers will have to play a large role in gaining widespread acceptance of HZ+. This is a political fact of life. > Furthermore, I think people in this list are more of technical > background than political ground. We only show political attitude > when unreasonable political attitude are imposed on us. Thanks Yes, people in soft-authors should be able and willing to decide things on technical merit. But we must also anticipate that political considerations will come in later, when we present HZ+ to the outside world. Ricky Yeung again: > I don't think Big5 is being treated unfairly by my enhencement. In a purely technical sense, you have to admit that Big5 is being treated a little bit unfairly, because you are designating GB as the default non-ASCII mode. > One may argue that Big5 was treated unfairly when HZ was born, > namely, it's not supported. This would be comparing apples and oranges. In the case of HZ, there was no need and no attempt to attract Big5 users, because Big5 was not supported. For HZ+, participation of Big5 users is absolutely essential, so we have got to be scrupulously fair to them. > To be really fair, should we throw away HZ and start afresh "fair" > 7-bit exchange code scheme? Perhaps you meant this ironically, but there may be some validity to it. Official international standards like ISO-2022 may have this kind of advantage -- they have the appearance of neutrality and are equally inconvenient for everybody. My hope with HZ+ was to build on the success of HZ, but that may not be possible, because of the appearance of unfairness to Big5 users. -- Steve From simpson@math.psu.edu Thu Jun 9 09:43:37 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA23489; Thu, 9 Jun 94 09:43:37 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id KAA19418; Thu, 9 Jun 1994 10:40:44 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 9 Jun 1994 10:40:42 -0400 Date: Thu, 9 Jun 1994 10:40:42 -0400 Message-Id: <199406091440.KAA01517@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org, mule@etl.go.jp Subject: Re: Big5, CNS, Mule, and HZ+ In-Reply-To: <199406091302.JAA18865@leibniz.math.psu.edu> References: <199406091302.JAA18865@leibniz.math.psu.edu> Reply-To: simpson@math.psu.edu Status: O Jing-Shin Chang writes: > I found that ETen had extended the CNS to include such Japanese > characters. In the Big5 extension, such characters are allocated > in a continuous code space. In its CNS counterparts, they are > separated in 2 or 3 continuous code space. What do you mean by "CNS counterparts"? Does this mean that Japanese kana are in CNS planes 1 and 2 after all? Somebody has said that they aren't. If they are, that's good to know. > So there shouldn't be any problem to convert to/from Big5 and CNS. It seems to me that there may still be a problem concerning Japanese kana. The reason is that the Japanese kana in ETen Big5 are apparently ETen-specific. Your ETen-specific Big5<->CNS conversion won't make any sense when applied to non-ETen versions of Big5. This would be a serious problem. We can expect a whole series of similar problems, due to the fact of vendor-specific Big5 versions. PLEASE NOTE: All such problems are neatly avoided by my HZ+S mapping of Big5 into the graphical ASCII code space. This is because my mapping is simple and 100 percent invertible and is defined for the entire Big5 code range, independently of what vendors may choose to put in this range. Under my mapping, Big5 -> HZ+S -> Big5 conversion will always work 100 percent correctly. (Actually, I shouldn't call it *my* mapping. I got the idea from Mule 1.0.) -- Steve From Ricky.Yeung@Eng.Sun.COM Thu Jun 9 12:28:11 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA29115; Thu, 9 Jun 94 12:28:11 CDT Received: from Eng.Sun.COM (engmail2.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA04825; Thu, 9 Jun 94 10:26:05 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (8.6.9/SMI-4.1) id KAA00577; Thu, 9 Jun 1994 10:26:28 -0700 Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA09715; Thu, 9 Jun 1994 10:27:12 +0800 Date: Thu, 9 Jun 1994 10:27:12 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406091727.AA09715@hk.Eng.Sun.COM> To: soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Thu, 9 Jun 1994 10:21:17 -0400 <199406091421.KAA01486@boole.math.psu.edu> Subject: HZ+ & "unfairness" to Big5 users. Re: The hostile Big5 users? Status: O > From: Stephen G Simpson > > My hope with HZ+ was to build on the success of HZ, but that may not > be possible, because of the appearance of unfairness to Big5 users. So what's your point? Are you suggesting dropping HZ+ because of its "appearance of unfairness" (in your opinion) to Big5 users? -Ricky From cyl Thu Jun 9 12:52:23 1994 Return-Path: Received: by ifcss.org (4.1/IFCSS-Mailer) id AA29911; Thu, 9 Jun 94 12:52:23 CDT Date: Thu, 9 Jun 94 12:52:23 CDT From: cyl (Yilong Chen) Message-Id: <9406091752.AA29911@ifcss.org> To: mule@etl.go.jp, soft-authors Status: O Subject: Re: Big5, CNS, Mule, and HZ+ Newsgroups: alt.chinese.computing References: <199406091302.JAA18865@leibniz.math.psu.edu> <199406091442.JAA20786@news.cs.utexas.edu> Organization: IFCSS Chinese Community Info. Center Distribution: Stephen G Simpson (simpson@math.psu.edu) ´£¨ì: : It seems to me that there may still be a problem concerning Japanese : kana. The reason is that the Japanese kana in ETen Big5 are : apparently ETen-specific. Your ETen-specific Big5<->CNS conversion : won't make any sense when applied to non-ETen versions of Big5. This : would be a serious problem. : We can expect a whole series of similar problems, due to the fact of : vendor-specific Big5 versions. But you have said ETen Big5 is by far (90%?) the most popular version, and thus it's a minor problem. By just supporting ETen BIG5, you will motivate the other 10% to only post in ETen BIG5. This way there is uniformity in the postings and won't create a headache as to what vendor BIG5 you used. IF the 10% of the users find that their software doesn't have an ETen BIG5 mode, then they will ask the authors to support it in the next version, because ETen BIG5 is "very, very popular". In the 8-bit world, this has already been implemented, see the csie.bbs.fj.* BIG5 mirrors of the japanese fj.* heirarchy. They only use ETen BIG5, so that it keeps the code in the vendor specific region uniform, and thus there is no problem as to what the exchange code is. : PLEASE NOTE: All such problems are neatly avoided by my HZ+S mapping : of Big5 into the graphical ASCII code space. This is because my : mapping is simple and 100 percent invertible and is defined for the : entire Big5 code range, independently of what vendors may choose to : put in this range. Under my mapping, Big5 -> HZ+S -> Big5 conversion : will always work 100 percent correctly. : (Actually, I shouldn't call it *my* mapping. I got the idea from Mule : 1.0.) if you insist that your HZ+S avoids certain problems, I find that it creates bigger problems. 1) No standardization in the Vendor specific area of BIG5 code. 2) Loss of communication in the Vendor specific area of BIG5 code, esp if not ETen specific, and if the user didn't specify what vendor BIG5 the posting is in, resulting in loss of communication with 90% of users. 3) HZ+S doesn't promote standardization to the most popular form of BIG5 ie, ETen BIG5 4) Thus mapping to other character sets (unless you use some user defined area) is impossible, in the vender specific area, due to lack of standardization. 5) If you insist on emphasizing the importance of the ETen specific area, then HZ+S only creates more problems, because of its loopholes. -- ³¯ Nelson Chin, Chen Yilong ====102 Brook Street Quincy MA 02170 1508 024 USA ¶h CIIP: nchin@hnt.com USENET: bu.mule CICC: cyl@ifcss.org PH:+1 617 472-2851 Às PRODIGY: GWBV10E@prodigy.com BUTTA: ttennis@bu.edu GIS: butta1@crsa.bu.edu From simpson@math.psu.edu Thu Jun 9 13:57:10 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA02858; Thu, 9 Jun 94 13:57:10 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id OAA01538; Thu, 9 Jun 1994 14:54:58 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 9 Jun 1994 14:54:56 -0400 Date: Thu, 9 Jun 1994 14:54:56 -0400 Message-Id: <199406091854.OAA01944@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: HZ+ & "unfairness" to Big5 users. Re: The hostile Big5 users? In-Reply-To: <9406091727.AA09715@hk.Eng.Sun.COM> References: <199406091421.KAA01486@boole.math.psu.edu> <9406091727.AA09715@hk.Eng.Sun.COM> Reply-To: simpson@math.psu.edu Status: O I wrote: > My hope with HZ+ was to build on the success of HZ, but that may not > be possible, because of the appearance of unfairness to Big5 users. Ricky replied: > So what's your point? Are you suggesting dropping HZ+ because of its > "appearance of unfairness" (in your opinion) to Big5 users? Certainly not. I want to keep pushing HZ+ and trying to convince Big5 users that HZ+ is fair. -- Steve From simpson@math.psu.edu Thu Jun 9 14:01:36 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA03240; Thu, 9 Jun 94 14:01:36 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id OAA01596; Thu, 9 Jun 1994 14:59:31 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 9 Jun 1994 14:59:30 -0400 Date: Thu, 9 Jun 1994 14:59:30 -0400 Message-Id: <199406091859.OAA01954@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: Re: ASCII tranparency In-Reply-To: <9406090846.AA08565@hk.Eng.Sun.COM> References: <199406090612.CAA26837@boole.math.psu.edu> <9406090846.AA08565@hk.Eng.Sun.COM> Reply-To: simpson@math.psu.edu Status: O Ricky Yeung writes: > I think you're somewhat contradictary. On the one hand, you've been > telling me that the escape sequence is unimportant; on the other hand, > you've been trying hard to tell me that my escape sequence is > politically incorrect (in your opinion). There is no contradiction here. I didn't say that your escape sequences are politically incorrect. What I said is that your idea of making GB mode the default non-ASCII mode is unfair to Big5. > You missed the point. I was saying do you now admit the escape sequence > is important. Escape sequences are a many-sided issue. Some aspects of escape sequences are more important, some are less important, some are not important at all. > My original intention was to point out the issue of maximal ASCCI > tranparency. I offered an enhencement that would work well with > any of the proposed ones (even 7-bit CNS). I feel being forced to > make a proposal like the others so that it would be treated as yet > another proposal. I don't want to force you to make your own proposal. I think that we have already had enough proposals and counterproposals. Instead of your making your own formal proposal, I would prefer that you simply point out the advantages or benefits of your suggestions regarding escape sequences, in terms of maximal ASCII transparency or whatever benefits you see, and then leave it up to the authors of formal proposals whether or how to make use of your suggestions. At the present time there are only two formal proposals on the table, my HZ+S and Ya-Gui's HZ-2. I have no idea how Ya-Gui feels about your suggestions. My view is that your suggestions make a lot of technical sense, but politically I'm wondering whether they are a good idea. Politically, I think it is a little late in the game to be making changes to HZ+S. Also, I am afraid that your suggestions may give the appearance of unfairness to Big5. > The real issue here is about ASCII tranparency. That is one issue, but not the only issue. I said: > If the price of maximal ASCII transparency is unfairness to Big5, I > don't want to pay that price. Ricky responded: > I am, and I think many others will be, willing to pay that small known > cost to close the flood gate for the further unlimited erosion of ASCII > transparency in the unknown future. Ricky has a good point. This is a matter of judgement. Both technical and political considerations must be weighed. -- Steve From simpson@math.psu.edu Thu Jun 9 14:03:35 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA03361; Thu, 9 Jun 94 14:03:35 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id PAA01614; Thu, 9 Jun 1994 15:01:25 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 9 Jun 1994 15:01:23 -0400 Date: Thu, 9 Jun 1994 15:01:23 -0400 Message-Id: <199406091901.PAA01963@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: Re: ASCII tranparency In-Reply-To: References: Reply-To: simpson@math.psu.edu Status: O Ross Paterson writes: > Ricky Yeung writes: > > [...] So you do admit that it's important to choose printable ASCII > > characters as escape sequence, right? > > Why is this important? Contrariwise, if you use ESC you get complete > transparency of printable ASCII. Many people regard HZ's use of the printable ~ (rather than ESC or another non-printable) as very beneficial. ESC and other non-printables confuse some mail and news programs and terminals. -- Steve From simpson@math.psu.edu Thu Jun 9 14:17:52 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA04163; Thu, 9 Jun 94 14:17:52 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id PAA01665; Thu, 9 Jun 1994 15:13:22 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 9 Jun 1994 15:13:21 -0400 Date: Thu, 9 Jun 1994 15:13:21 -0400 Message-Id: <199406091913.PAA01984@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org, mule@etl.go.jp Subject: Re: Big5, CNS, Mule, and HZ+ In-Reply-To: <9406091537.AA10829@hermes.ee.nthu.edu.tw> References: <9406091537.AA10829@hermes.ee.nthu.edu.tw> Reply-To: simpson@math.psu.edu Status: O Jing-Shin Chang writes: > YES, ETen places these kana in planes 1 and 2. OK, let me make sure I understand. Standard CNS does not contain Japanese Kana. ETen's extension of CNS does contain them. Is that right? > |> It seems to me that there may still be a problem concerning Japanese > |> kana. The reason is that the Japanese kana in ETen Big5 are > |> apparently ETen-specific. Your ETen-specific Big5<->CNS conversion > |> won't make any sense when applied to non-ETen versions of Big5. This > |> would be a serious problem. > > >From the begining, we are talking about ETen Big5. And all Big5/8<-> > Big5/7 wouln't have any conversion problem for any specific vender. However, Big5 <-> CNS *would* have such problems, because some ETen-specific Big5 or other vendor-specific Big5 characters may not be translatable to standard CNS. > The problem only arise when, say, we are using ETen Big5 to communicate > KC Big5 (assuming that they have different vender-specific part), then > the communication of course will occurs. In this case, any ETbig5 <->X > <->KCbig5 won't remady this problem. Here, X could be any HZ+ (e.g., HZ-S) > or even ETbig5 (or KCBig5, in their original form.) So this is not the > problem of that "X" but the problem between the two different charsets: > ETbig5 and KCbig5. (Don't blame "X" -^^-.) This is correct, provided X is something like HZ+S. However, if X is standard CNS, then additional problems arise, because standard CNS may not contain the vendor-specific extensions of Big5. > So long as we insist on one character set (and extend it > consistently), there should be no problem. Unfortunately, this is not a practical solution. Vendor-specific Big5 extensions are a fact of life. If we want excellent Big5 support, we cannot insist on only one character set. We have to support ETen Big5, KC Big5, Chinese Windows Big5, etc. etc. > So when you are saying "independently of what vendors may > choose..." you are actually referring to "the same vendor (i.e., > x==y)." In this case, other HZ+ do it as well. No. Let's be very precise. What I am saying is this. If x is any Big5 vendor, then Big5(x) -> HZ+S -> Big5(x) will always work perfectly. This will be true no matter which vendor x is, be it ETen or KC or Chinese Windows or Chinese OS/2 or Chinese Mac or whatever. If we try to replace HZ+S by CNS, then this feature will fall apart. CNS may not contain the vendor-specific Big5 codes. Also, the translations of the vendor-specific stuff to CNS may be different. -- Steve From Ricky.Yeung@Eng.Sun.COM Thu Jun 9 19:23:56 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA17857; Thu, 9 Jun 94 19:23:56 CDT Received: from Eng.Sun.COM (engmail2.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA26156; Thu, 9 Jun 94 17:21:37 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (8.6.9/SMI-4.1) id RAA02427; Thu, 9 Jun 1994 17:20:53 -0700 Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA11799; Thu, 9 Jun 1994 17:22:39 +0800 Date: Thu, 9 Jun 1994 17:22:39 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406100022.AA11799@hk.Eng.Sun.COM> To: soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Thu, 9 Jun 1994 14:54:56 -0400 <199406091854.OAA01944@boole.math.psu.edu> Subject: Re: HZ+ & "unfairness" to Big5 users. (Was Re: The hostile Big5 users?) Status: O > > My hope with HZ+ was to build on the success of HZ, but that may not > > be possible, because of the appearance of unfairness to Big5 users. > > Ricky replied: > > So what's your point? Are you suggesting dropping HZ+ because of its > > "appearance of unfairness" (in your opinion) to Big5 users? > > Certainly not. I want to keep pushing HZ+ and trying to convince Big5 > users that HZ+ is fair. IF the majority of Big5 users are as hostile as you think, and with your public acknowledgement that HZ+ has the "appearance of unfairness" to Big5 users, I think you'll have a hard time convincing them, with or without my enhencement. -Ricky From lai@apple.com Thu Jun 9 19:52:34 1994 Return-Path: Received: from apple.com by ifcss.org (4.1/IFCSS-Mailer) id AA18953; Thu, 9 Jun 94 19:52:34 CDT Received: from [17.203.80.11] by apple.com with SMTP (5.61/8-Oct-1993-eef) id AA20725; Thu, 9 Jun 94 17:49:58 -0700 for soft-authors@ifcss.org Date: Thu, 9 Jun 94 17:49:58 -0700 Message-Id: <9406100049.AA20725@apple.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" To: ccnet%uga.BITNET@cunyvm.cuny.edu, soft-authors@ifcss.org, mule@etl.go.jp From: lai@apple.com (Edmund Lai) Subject: Re: Big5, CNS, Mule, and HZ+ Status: O >#ifdef FLAME >Ed's idea of using GB codes to represent Big5 codes for Kana and >Cyrillic strikes me as incredibly bizarre. Ed, please stop the insanity! >#endif :-) > HZ alrady has code for Kana and Cyrillic. Kana is Kana, Cyrillic is Cyrillic. It is not GB Kana or ETen Big5 Kana, so if a document has Kana and you want to translate your document to HZ, then it should go to the code space in HZ for Kana. So obviously it is Steve that is insane. > > >Certainly these Big5 features are being used. I am referring to > > >Chinese characters above F9D6, > > > > As I pointed out before, it is wrong to post any user specific > > character on to a newsgroup which expect to be communicated with > > a common exchange code. > >The Chinese characters above F9D6 that I was referring to are not >user-specific. They may be ETen-specific, but I don't see anything >wrong with posting them in newsgroups. After all, ETen Big5 is very >popular. If users want to post those characters, they should be >allowed to do so. It's not our job to prevent them. > > > On alt.chinese.computing people from Taiwan has expressed again and > > again that they do not consider converting CNS and Big5 to be a > > problem. > >I don't think people from Taiwan have said that. I know that Ed >himself has said that, but Ed is not from Taiwan, right? > Let us quote from Shann Wei-Chang "The conversion program between Big-5 and first two plains of CNS is written, tested and used for some months. I distrusted CNS code before. After a few days of study and talking with people, I have changed my mind. Now I would support CNS as an exchange code, if not an internal code yet. Jing-Shin said he can modify Simpson's conversion program a little bit to let it convert Big-5 to CNS. Even if he does not have time to do it, there are already conversion programs. For instance chcode-1.4." So at least two people from Taiwan said so. And there were a few previous post on alt.chinese.computing before that says the same thing. Of course Steve is famous for only seeing thing he want to see. >Anyway, what exactly does Ed mean by "converting from CNS and Big5 is >not a problem"? Obviously it's not a problem for you if it works for >your particular document, or for the particular subsets of Big5 and >CNS that you use. However, I want to accommodate the needs of *all* >Big5 users, even those who use Big5 characters that Ed may not approve >of. In this sense, CNS<->Big5 conversion is indeed a problem. If >Ed really thinks it is not a problem, I challenge him to define >precisely the conversion algorithm that he wants to use. > > > It does not make sense to support both CNS and another form of > > 7 bit Big5 in the same exchange code. > >This statement from Ed is completely muddy. CNS is not "a form of >7-bit Big5". CNS and Big5 are two different coding systems and two >different character sets. > >Probably the CNS design committee were hoping that CNS would replace >Big5, but that is not how things have worked out. Big5 remains >extremely popular, while Ed has not been able to find one person who >uses CNS in his or her daily work. > I thought Steve had apologized before for claiming not a single person uses CNS. Now he is back to his old self again (of course he is trying to be tricky with the escape clause Ed cannot find that person himself). As it is pointed out so many times, the issue is not how popular Big5 is. We are not trying to replace Big5 internal code by CNS, we are talking about using CNS as exchange code for Big5. Of course Steve never listen to anything he does not want to listen. > > > To include both in the same exchange code violates the very meaning > > of exchange code. > >What is "the very meaning of exchange code"? I have repeatedly >challenged Ed to define his concept of "exchange code," specifically >as it applies to HZ and HZ+. Ed has repeatedly ignored this >challenge. > Steve don't even understand what exhange code is. No wondor he makes so many mistakes in his arguments. When you post something to the newsgroup, the news will be read by user who uses very diverse machine, so it need to have a common format so that the document can be read by everyone. The exchange code can either be a union of everything or a intersection of everything but it need to be a common format. As pointed out by Jing-Shin Chang: "As I said in the above, Big5(x) -> HZ+ -> Big5(x) is always 100% convertable with any HZ+ we knew of. But Big5(x) -> HZ+ -> Big5(y) is not so fortune. That's not the fault of any HZ+, but the fault between x and y. So when you are saying "independently of what vendors may choose..." you are actually referring to "the same vendor (i.e., x==y)." So blindly sending out vendor specific code violates a common exchange code. If it is really felt that certain ETen extension is absolutelty necessary, then at least it should be declared to be part of the exchange code so that if one uses other Big5 machine and other Chinese system can have a chance to translate to and from the common code. Of course if someone is just using this as an excuse to declare that one can never translate between CNS and Big5, then he would not want to understand antyghing ohter than his own scheme. /* Disclaimer: All statments and opinions expressed are my own */ /* Edmund K. Lai */ /* Apple Computer, MS303-3A */ /* 20525 Mariani Ave, */ /* Cupertino, CA 95014 */ /* (408)974-6272 */ zW@h9cOi From Ricky.Yeung@Eng.Sun.COM Thu Jun 9 19:54:51 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA19035; Thu, 9 Jun 94 19:54:51 CDT Received: from Eng.Sun.COM (engmail2.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AB02264; Thu, 9 Jun 94 17:52:45 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (8.6.9/SMI-4.1) id RAA06303; Thu, 9 Jun 1994 17:52:11 -0700 Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA11814; Thu, 9 Jun 1994 17:54:01 +0800 Date: Thu, 9 Jun 1994 17:54:01 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406100054.AA11814@hk.Eng.Sun.COM> To: soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Thu, 9 Jun 1994 10:21:17 -0400 <199406091421.KAA01486@boole.math.psu.edu> Subject: Politics (Was Re: The hostile Big5 users?) Status: O > suggestions. My view is that your suggestions make a lot of technical > sense, but politically I'm wondering whether they are a good idea. > Politically, I think it is a little late in the game to be making Ok, now we're settled on the technical front. Let's talk about politics. You seemed to be pessimistic about Big5 users. I do think otherwise. Let's look at history, the majority of GB users picked the technically superior solution, namely HZ, over the other one. Are you telling us that the Big5 users are not capable of making the judgement call like the GB users? Yes, there are always radical hostile users like the one you referred to. But would one be naive enough to believe that this kind of individuals represents the majority? I don't. > Ricky Yeung wrote: > > |> I'm confident that the majority of Big5 users in Taiwan and all over the > > |> world won't be as narrow-minded as you think. They will recognize the > > |> relevant issues and eventually settle on the technically superior > > |> solution. > > I'd like to believe that. But I have found that people sometimes lose > perspective and get upset for irrational reasons, and this sometimes > affects the climate. This is not only Big5 users, this is everybody. > > As an example of a hostile Big5 user, there is a guy named Jeffrey > Chen who has been flaming in alt.chinese.computing about soft-gods, > self-appointed US experts, HZ imperialism, etc. I try to handle him > politely, but wow! -Ricky From simpson@math.psu.edu Thu Jun 9 20:09:12 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA19571; Thu, 9 Jun 94 20:09:12 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id VAA07320; Thu, 9 Jun 1994 21:06:51 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 9 Jun 1994 21:06:50 -0400 Date: Thu, 9 Jun 1994 21:06:50 -0400 Message-Id: <199406100106.VAA02606@boole.math.psu.edu> To: mule@etl.go.jp, ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: Re: Big5, CNS, Mule, and HZ+ In-Reply-To: <9406091752.AA29911@ifcss.org> References: <9406091752.AA29911@ifcss.org> Reply-To: simpson@math.psu.edu Status: O I wrote: > : ETen-specific Big5<->CNS conversion won't make any sense when > : applied to non-ETen versions of Big5. This would be a serious > : problem. We can expect a whole series of similar problems, due > : to the fact of vendor-specific Big5 versions. Yilong Chen replied: > But you have said ETen Big5 is by far (90%?) the most popular version, > and thus it's a minor problem. Well, it's a problem for everybody who uses an implementation of Big5 other than ETen's. I wouldn't call this a minor problem. For the people affected, it's serious. If Yilong and Jing-Shin want to write programs to convert ETen Big5 into CNS, or maybe into ETen CNS, there is certainly nothing wrong with that. (In fact, I think somebody was mentioning that the ETen software package includes a utility which does exactly this.) However, you have to expect that these ETen-specific programs won't work well with other vendor-specific versions of Big5. Users of different versions of Big5 will almost certainly need different Big5<->CNS conversion programs. > By just supporting ETen BIG5, you will motivate the other 10% to > only post in ETen BIG5. This way there is uniformity in the > postings and won't create a headache as to what vendor BIG5 you > used. This stikes me as a bad idea. Motivating people to use only ETen Big5 is not my job, nor is it HZ+'s job. If people want to post in Chinese Windows Big5, HZ+ should not stand in their way. Standardizing Big5 is an interesting and difficult problem, but it is not HZ+'s problem. > if you insist that your HZ+S avoids certain problems, > I find that it creates bigger problems. > 1) No standardization in the Vendor specific area of BIG5 code. > 2) Loss of communication in the Vendor specific area of BIG5 code, esp > if not ETen specific, and if the user didn't specify what vendor BIG5 > the posting is in, resulting in loss of communication with 90% of users. > 3) HZ+S doesn't promote standardization to the most popular form of BIG5 > ie, ETen BIG5 > 4) Thus mapping to other character sets (unless you use some user > defined area) is impossible, in the vender specific area, due to lack > of standardization. I disagree with Yilong's reasoning here. He seems to be saying that HZ+ creates problems by *not* forcing Big5 users to standardize on ETen Big5. I disagree. First, the existence of vendor-specific versions of Big5 other than ETen Big5 is not necessarily a problem. Second, even if Yilong thinks it is a problem, HZ+ did not create that problem. Why blame HZ+ for a problem that HZ+ did not create? The 7-bit representation of Big5 provided by HZ+ is vendor-neutral and 100 percent accurate and invertible for all Big5 vendors. HZ+ works with anybody's Big5, not only ETen's Big5. I regard this as an advantage and a positive feature of HZ+. Yilong regards it as a disadvantage and a negative feature of HZ+. Who is right? I think I'm right, because I want HZ+ to do one job and do it well. (The job I want HZ+ to do is to provide a convenient 7-bit representation of mixed Big5/GB/ASCII for e-mail and news.) Yilong thinks he's right, because he wants HZ+ to take on an extra job, namely to force standardization of Big5. > 5) If you insist on emphasizing the importance of the ETen specific area, > then HZ+S only creates more problems, because of its loopholes. OK, let me state it in a vendor-neutral way. The ETen-specific part of Big5 is important to ETen users. The CWin-specific part of Big5 is important to CWin users. The Chinese-OS/2-specific part of Big5 is important to users of Chinese-OS/2. Etc. Etc. One of the virtues of HZ+S is that it easily accommodates all of these users. -- Steve From yawei@cs.indiana.edu Thu Jun 9 22:13:44 1994 Return-Path: Received: from moose.cs.indiana.edu by ifcss.org (4.1/IFCSS-Mailer) id AA23746; Thu, 9 Jun 94 22:13:44 CDT Message-Id: <9406100313.AA23746@ifcss.org> Received: by moose.cs.indiana.edu (5.65c/9.4jsm) id AA14286; Thu, 9 Jun 1994 22:11:33 -0500 Date: Thu, 9 Jun 1994 22:11:33 -0500 From: "Ya-Gui Wei" To: chinesenet@uclink.berkeley.edu, soft-authors@ifcss.org Subject: Re: response to a good-natured flame Status: O >Jing-Shin has withdrawn B5E1 and B5E2 and B5E3, but if you are still >interested, his information may still be available at his FTP site. >To the best of my knowledge, nobody has written any code for HZ-2, >which exists only as a proposal document. The proposal document itself contains all the C code one needs to convert between HZ-2 and Big5. If anyone needs equivalent code in another language let me know. Ya-Gui From simpson@math.psu.edu Fri Jun 10 00:03:15 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA27899; Fri, 10 Jun 94 00:03:15 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id BAA08304; Fri, 10 Jun 1994 01:00:55 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 10 Jun 1994 01:00:54 -0400 Date: Fri, 10 Jun 1994 01:00:54 -0400 Message-Id: <199406100500.BAA03002@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org, mule@etl.go.jp Subject: Re: Big5, CNS, Mule, and HZ+ In-Reply-To: <9406100049.AA20725@apple.com> References: <9406100049.AA20725@apple.com> Reply-To: simpson@math.psu.edu Status: O Edmund Lai quotes Wei-Chang Shann as saying: > The conversion program between Big-5 and first two plains of CNS is > written, tested and used for some months. > I distrusted CNS code before. After a few days of study and talking > with people, I have changed my mind. The experience of one or two users doesn't prove that there are no problems with Big5<->CNS conversion. The actual situation is that some users will experience no problems, while other users will experience severe problems. One obvious source of trouble is vendor-specific Big5 characters. I am willing to stipulate that chcode-1.4 may work pretty well (99.9 percent or more) so long as your files contain only the standard 13053 Big5 Chinese characters and the 406 standard Big5 symbols. However, chcode-1.4 may break down completely for other Big5 stuff, for example ETen's Japanese kana. Furthermore, chcode-1.4 may not give exactly the same results as other Big5<->CNS conversion programs. The only way to really answer this would be to perform some exhaustive, careful tests. I don't know that anyone has done this. Ed says: > So at least two people from Taiwan said so. Who is the other one, besides Wei-Chang? Ed says: > I thought Steve had apologized before for claiming not a single person > uses CNS. Now he is back to his old self again I apologized when a certain Taiwanese user said that CNS is standard on DEC workstations in Taiwan. However, after my apology and in response to my questions, that same Taiwanese user reported that nobody does any actual word processing on these workstations! Everybody does their word processing on a PC in Big5. The workstations are used only as servers, to store Big5 files. In light of this information, I retracted my apology. I still don't know of one single person who actually uses CNS. > When you post something to the newsgroup, the news will be read by > user who uses very diverse machine, so it need to have a common > format so that the document can be read by everyone. The exchange > code can either be a union of everything or a intersection of > everything but it need to be a common format. OK Ed, thanks for finally defining your concept of "exchange code". > So blindly sending out vendor specific code violates a common exchange > code. So, what you are really saying is that it is morally wrong to post vendor-specific codes in a newsgroup, because then the posting cannot be read by everybody, only by those who have that particular vendor's software. Your statement makes a certain amount of sense, but I don't see it as an inviolable moral rule. For instance, if an ETen-specific message is clearly labeled as such, do you still think that posting it is morally wrong? What about e-mail? Do you think it is morally wrong to send vendor-specific codes in e-mail? What if the sender and the receiver know that both are using the same vendor-specific version of Big5? If vendor specific codes are OK in e-mail but not in news, this would seem to mean that we need two different exchange codes, one for news and one for e-mail. Don't you think that would be a little ridiculous? Ed disapproves of vendor-specific codes. I am more forgiving. My opinion is that HZ+ should be as transparent as possible. Big5 users should be able to post in HZ+ in the same way they would post in 8-bit Big5. If they want to post vendor-specific Big5 codes, HZ+ should not stand in their way. > Of course if someone is just using this as an excuse to declare that one > can never translate between CNS and Big5, then he would not want to > understand antyghing ohter than his own scheme. I never declared that one can never translate between CNS and Big5. What I am saying is that the general concept of Big5<->CNS translation is subject to pitfalls and ambiguities. These pitfalls and ambiguities are mostly due to the fact that, although CNS is an official standard, Big5 is not completely standardized. By contrast, Big5<->HZ+S translation is not subject to any pitfalls or ambiguities. Big5<->HZ+S translation is always 100 percent well-defined and 100 percent accurate and 100 percent invertible. Not only that, but it is very, very simple. -- Steve From simpson@math.psu.edu Fri Jun 10 00:09:26 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA28494; Fri, 10 Jun 94 00:09:26 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id BAA08343; Fri, 10 Jun 1994 01:07:21 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 10 Jun 1994 01:07:20 -0400 Date: Fri, 10 Jun 1994 01:07:20 -0400 Message-Id: <199406100507.BAA03019@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: Politics (Was Re: The hostile Big5 users?) In-Reply-To: <9406100054.AA11814@hk.Eng.Sun.COM> References: <199406091421.KAA01486@boole.math.psu.edu> <9406100054.AA11814@hk.Eng.Sun.COM> Reply-To: simpson@math.psu.edu Status: O Ricky Yeung writes: > Let's look at history, the majority of GB users picked the technically > superior solution, namely HZ, over the other one. What other one? There was really no alternative. E-mail and news in the USA has to be 7-bit clean. HZ was a practical proposal for 7-bit GB. After 6/4/89, HZ or something very much like it was more or less inevitable. By contrast, Taiwan has a completely 8-bit clean environment. Users in Taiwan have an obvious alternative to 7-bit Big5, namely 8-bit Big5. So far as I can tell, the only reason they might want 7-bit Big5 would be for *international* news and e-mail. This limited need is the one that HZ+ is trying to fill. > Are you telling us that the Big5 users are not capable of making > the judgement call like the GB users? No, that's not what I'm saying. I'm sure Big5 users are just as intelligent as GB users. However, there are other factors at work, for instance 8-bit cleanness within Taiwan. If it were not for these other factors, I'm sure the Taiwanese computer industry would have developed 7-bit Big5 a long time ago. My feeling is that HZ+ is rather simple from a technical standpoint, but tricky and delicate from a political standpoint. Consider all of the silly wrangling over CNS and mapping schemes. Obviously there are political issues here, which overshadow the technical issues. And this is within soft-authors, supposedly a group of technically oriented people. What will happen in the wider world? -- Steve From Ricky.Yeung@Eng.Sun.COM Fri Jun 10 00:28:41 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA29163; Fri, 10 Jun 94 00:28:41 CDT Received: from Eng.Sun.COM (engmail2.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA25325; Thu, 9 Jun 94 22:26:37 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (8.6.9/SMI-4.1) id WAA29517; Thu, 9 Jun 1994 22:26:04 -0700 Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA00685; Thu, 9 Jun 1994 22:27:57 +0800 Date: Thu, 9 Jun 1994 22:27:57 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406100527.AA00685@hk.Eng.Sun.COM> To: soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Fri, 10 Jun 1994 01:07:20 -0400 <199406100507.BAA03019@boole.math.psu.edu> Subject: Re: Politics (Was Re: The hostile Big5 users?) Status: O > From: Stephen G Simpson > Date: Fri, 10 Jun 1994 01:07:20 -0400 > > Ricky Yeung writes: > > Let's look at history, the majority of GB users picked the technically > > superior solution, namely HZ, over the other one. > > What other one? There was really no alternative. E-mail and news in > the USA has to be 7-bit clean. HZ was a practical proposal for 7-bit > GB. After 6/4/89, HZ or something very much like it was more or less > inevitable. Don't you know why zWDOS is named? Have you read the FAQ of a.c.t.? -Ricky From Ricky.Yeung@Eng.Sun.COM Fri Jun 10 00:37:30 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA29466; Fri, 10 Jun 94 00:37:30 CDT Received: from Eng.Sun.COM (engmail2.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA25856; Thu, 9 Jun 94 22:35:25 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (8.6.9/SMI-4.1) id WAA00431; Thu, 9 Jun 1994 22:34:53 -0700 Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA00681; Thu, 9 Jun 1994 22:13:33 +0800 Date: Thu, 9 Jun 1994 22:13:33 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406100513.AA00681@hk.Eng.Sun.COM> To: soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Thu, 9 Jun 1994 10:21:17 -0400 <199406091421.KAA01486@boole.math.psu.edu> Subject: The issue of fairness Status: O > From: Stephen G Simpson > Date: Thu, 9 Jun 1994 10:21:17 -0400 > In a purely technical sense, you have to admit that Big5 is being > treated a little bit unfairly, because you are designating GB as the > default non-ASCII mode. No. I think Big5 is being treated unfairly by your HZ+S proposal. In your proposal, with two ~ escape sequences, obviously the Big5 users lose more ASCII transparency. Is it fair that the GB users can enjoy more ASCII transparency? On the other hand, my proposal is fair because both GB and Big5 enjoy the same level of ASCII transparency. -Ricky From haa@unda.fi Fri Jun 10 01:58:48 1994 Return-Path: Received: from goliath.unda.fi by ifcss.org (4.1/IFCSS-Mailer) id AA02247; Fri, 10 Jun 94 01:58:48 CDT Received: from unda.fi (titan.unda.fi) by goliath.unda.fi (5.65c/93.6.3-MAI) id AA15014; Fri, 10 Jun 1994 09:55:36 +0300 Received: by unda.fi (4.1/MAI-94.2.12) id AA28986; Fri, 10 Jun 94 09:55:53 +0300 Date: Fri, 10 Jun 94 09:55:53 +0300 Message-Id: <9406100655.AA28986@unda.fi> From: haa@cs.hut.fi (Hannu Aronsson) Sender: haa@unda.fi To: simpson@math.psu.edu, mule@etl.go.jp, soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu Subject: Re: Big5, CNS, Mule, and HZ+ References: <9406100049.AA20725@apple.com> <199406100500.BAA03002@boole.math.psu.edu> Status: O Sorry to intervene in this "fight", but to me it seems it is mostly over terminology and other not so important things, and it isn't going anywhere. We should be working together, not fighting each other, as i18n is hard enough anyway ;-( [Stephen G Simpson writes:] > For instance, if an ETen-specific message > is clearly labeled as such, do you still think that posting it is > morally wrong? The *big* item here is labeling. You really need to label what kind of Big5 you are encoding, so that users will be able to do the needed conversions from Big5(whatever sender used) to Big5(whatever they happen to have) > By contrast, Big5<->HZ+S translation is not subject to any pitfalls or > ambiguities. Yes, unfortunately it is, because you're grouping several different versions of Big5 under one name in your encoding. I think you should not be talking about "The one and only HZ+S", but instead HZ+S_Eten, HZ+S_CWin, HZ+S_whatever, ... because they are not 100% compatible with each other (for the vendor defined space). Without specific tagging (preferably *in* the coding, not relying on some headers or such easily removed external things), many users will see junk if their Big5 doesn't happen to match the sender's. This is definitely *bad*. The whole idea of using CNS (or something well defined) as a interchange code would be that everybody is required to convert to/from a well defined *single* standard character set. In this case, everybody would only need *one* CNS<->Their system conversion, in your scheme, everybody will need *many* conversion programs (from different Big5's). Maybe they can't express every character in their system in that standard set, but they can be 100% sure their message will get to the receiver reliably. Many think reliability is more important than support for any exotic non-standard feature or characters. > Big5<->HZ+S translation is always 100 percent > well-defined and 100 percent accurate and 100 percent invertible. The important thing here is that without some internal or external tagging scheme, plain HS+Z is *not* invertible or accurate or even well-defined, if the senders and receivers use different versions of Big5. We don't just need Big5(x)<-->Big5(x), but we really need and want Big5(any)<-->Big5(any), so that people can convey their message reliably. For an interchange code, it is most important that it can convey information without ambiguities. If you're now designing a new encoding, *please* make it completely unambiguous. It will save a lot of grief in the long run. I think nobody would complain, if you encode into the message well and reliably the version of Big5 used to send the message. Then everybody is able to convert to anything without ambiguity. But if there is no reliable way to determine what version of Big5 was used, then the code will not work as a reliable interchange code, and it will only add to the confusion already out there. Disclaimer: I don't know so much about Big5 or CNS. Personally I would like to see a move to something (currently ISO10646 would seem to be the best choice) to end the character set wars. Yours, Hannu -- Hannu.Aronsson@cs.hut.fi Helsinki University of Technology, Finland Hannu Aronsson /pub/culture/japan archives at ftp.funet.fi From simpson@math.psu.edu Fri Jun 10 07:23:44 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA10961; Fri, 10 Jun 94 07:23:44 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id IAA09959; Fri, 10 Jun 1994 08:21:35 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 10 Jun 1994 08:21:34 -0400 Date: Fri, 10 Jun 1994 08:21:34 -0400 Message-Id: <199406101221.IAA03738@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: Re: Politics (Was Re: The hostile Big5 users?) In-Reply-To: <9406100527.AA00685@hk.Eng.Sun.COM> References: <199406100507.BAA03019@boole.math.psu.edu> <9406100527.AA00685@hk.Eng.Sun.COM> Reply-To: simpson@math.psu.edu Status: O I wrote: > > What other one? There was really no alternative. E-mail and news in > > the USA has to be 7-bit clean. HZ was a practical proposal for 7-bit > > GB. After 6/4/89, HZ or something very much like it was more or less > > inevitable. Ricky replied: > Don't you know why zWDOS is named? Have you read the FAQ of a.c.t.? I guess you are referring to ZW, an alternative to HZ that was considered for a while before HZ took over. I regard ZW as very similar to HZ. Both of them use the same 7-bit representation of GB (by stripping high bits) and both of them use ASCII escape sequences. When I said that something like HZ was inevitable, I was including ZW in this statement. And yes, I have read the a.c.t FAQ. Also, I am very familiar with the history of ZW and HZ, having followed them closely from the beginning. -- Steve From simpson@math.psu.edu Fri Jun 10 07:53:11 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA11818; Fri, 10 Jun 94 07:53:11 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id IAA10105; Fri, 10 Jun 1994 08:50:36 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 10 Jun 1994 08:50:35 -0400 Date: Fri, 10 Jun 1994 08:50:35 -0400 Message-Id: <199406101250.IAA03784@boole.math.psu.edu> To: mule@etl.go.jp, soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu Newsgroups: alt.chinese.text.big5 Subject: a question for users of 8-bit Big5 newsgroups In-Reply-To: <9406100655.AA28986@unda.fi> References: <9406100049.AA20725@apple.com> <199406100500.BAA03002@boole.math.psu.edu> <9406100655.AA28986@unda.fi> Reply-To: simpson@math.psu.edu Status: O Hannu Aronsson writes: > I think you should > not be talking about "The one and only HZ+S", but instead > HZ+S_Eten, > HZ+S_CWin, > HZ+S_whatever, > ... > because they are not 100% compatible with each other (for the vendor > defined space). I reject this suggestion. There is only one HZ+S. There is no need for monsters such as HZ+S_ETen, HZ+S_CWin, etc. I think some people are severely confused over the purpose of HZ+. The purpose is *not* to create a new coding standard or to standardize Big5. The only purpose of HZ+ is to transparently represent mixed ASCII/GB/Big5 in 7-bit environments. To the extent that Big5 is messy because of vendor-specific codes, HZ+ will transparently reflect that messiness. But HZ+ itself is 100 percent accurate and will not add any messiness of its own. Regarding vendor-specific Big5 codes, let me pose some questions for users of 8-bit Big5 newsgroups. In these newsgroups, is there any policy or controversy about vendor-specific Big5 codes? Is there any attempt to censor or get rid of such codes? Is there any attempt to standardize on one vendor-specific set of Big5 extensions? -- Steve Stephen G. Simpson Department of Mathematics, Pennsylvania State University 333 McAllister Building, University Park, State College, PA 16802 Office phone +1 814 863-0775 Home phone +1 814 238-2274 E-mail simpson@math.psu.edu FAX +1 814 865-3735 World Wide Web http://www.math.psu.edu/simpson/ From butta1@crsa.bu.edu Fri Jun 10 08:28:29 1994 Return-Path: Received: from crsa.bu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA13118; Fri, 10 Jun 94 08:28:29 CDT Received: from ithake.bu.edu by crsa.bu.edu (8.6.4/Spike-2.1) id JAA14169; Fri, 10 Jun 1994 09:26:06 -0400 Received: from localhost by ithake.bu.edu (8.6.4/Spike-2.1) id JAA24432; Fri, 10 Jun 1994 09:25:21 -0400 Date: Fri, 10 Jun 1994 09:25:19 -0400 (EDT) From: C I I Project Sender: C I I Project Reply-To: C I I Project Subject: Re: a question for users of 8-bit Big5 newsgroups To: Stephen G Simpson Cc: mule@etl.go.jp, soft-authors@ifcss.org, alt.chinese.computing@usenet.ucs.indiana.edu In-Reply-To: <199406101250.IAA03784@boole.math.psu.edu> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Status: O On Fri, 10 Jun 1994, Stephen G Simpson wrote: > I reject this suggestion. There is only one HZ+S. There is no > need for monsters such as HZ+S_ETen, HZ+S_CWin, etc. So are you telling vendors(or anyone who wants to transmit vendor-specific code) to provide conversion utilities for every conceivable vendor-big5, after it's converted back to 8-bit BIG5 from HZ+S? Are you saying the HZ+ standard is just the chinese way of say UUDECODE, UUENCODE (of course w/o the defects of UUEN/DECODE). > > Regarding vendor-specific Big5 codes, let me pose some questions for > users of 8-bit Big5 newsgroups. In these newsgroups, is there any > policy or controversy about vendor-specific Big5 codes? Is there any > attempt to censor or get rid of such codes? Is there any attempt to > standardize on one vendor-specific set of Big5 extensions? HOW NEWSGROUPS HANDLE VENDOR SPECIFIC BIG5 ========================================== alt.chinese.text.big5 and alt.chinese.text conversion gateway assumes ETen specific code. software used: ifcss.org:/software/unix/sym-supp.tab, hc-supp.tab, in-corr.tab, hc-30.tar.gz csie.bbs.fj.* big5 mirrors of fj.* japanese regional hierachy assumes ETen specific code. software used: sinocode, you can do an archie on it tw.bbs.lang.japanese, direct BIG5 input, no conversions. From the postings everyone conforms to ETen BIG5 code. CONCLUSION ========== It is safe to to choose a vendor like ETen to be the standard of communications, and it has been done so already. In an international setting, we must standardize, otherwise it'll be another case of a chicken talking to a duck. In a local homogenous setting or people in the same vendor group, HZ+S will not be a problem, but we are dealing with International communication. Simpson expressed concerns of making it messier with converting it to HZ+ if we try to manipulate the code other than just transforming it to 7-bit, and that BIG5 is messy already, but as we can see from the newsgroups, by standardizing we are actually making BIG5 unmessier. Therefore i propose that HZ+ also deal with this issue of standardizing to a specific vendor, namely ETen BIG5 for all code range. This will release the burden to require so many different conversion utilities of the vendor-specific area, if this vendor-specific code is necessary for international communication. Nelson Chin From haa@unda.fi Fri Jun 10 09:40:10 1994 Return-Path: Received: from goliath.unda.fi by ifcss.org (4.1/IFCSS-Mailer) id AA16340; Fri, 10 Jun 94 09:40:10 CDT Received: from unda.fi (titan.unda.fi) by goliath.unda.fi (5.65c/93.6.3-MAI) id AA28971; Fri, 10 Jun 1994 17:37:12 +0300 Received: by unda.fi (4.1/MAI-94.2.12) id AA21457; Fri, 10 Jun 94 17:37:30 +0300 Date: Fri, 10 Jun 94 17:37:30 +0300 Message-Id: <9406101437.AA21457@unda.fi> From: haa@cs.hut.fi (Hannu Aronsson) Sender: haa@unda.fi To: simpson@math.psu.edu, soft-authors@ifcss.org, mule@etl.go.jp Subject: Re: a question for users of 8-bit Big5 newsgroups Newsgroups: alt.chinese.text.big5 References: <9406100049.AA20725@apple.com> <199406100500.BAA03002@boole.math.psu.edu> <9406100655.AA28986@unda.fi> <199406101250.IAA03784@boole.math.psu.edu> Status: O Stephen G Simpson writes: > To the extent that Big5 is messy because of vendor-specific codes, HZ+ > will transparently reflect that messiness. ... But why don't you try to solve this mess? You're designing a new 7-bit encoding that probably will be quite widely used in many, many places. Just *now* you have the chance of a lifetime to make your encoding somehow reflect the version of Big5 used. Then those who don't care about Big5 versions, they could just ignore the different "version identifiers" in incoming data. Those, who do care, it would allow correctly identifying incoming data. Making data coming out of the systems have the correct identification would be easy to implement (#ifdef big5_eten_system #define big5_id "some_eten_magic_id" #else ...) A few hour's work *now* could save hundreds of hours of work and gigabytes of slightly corrupt data in the future. Why not try to make the world a little better place, if you could do it? > ... 8-bit Big5 newsgroups ... is there any policy or controversy > about vendor-specific Big5 codes? I would guess they gave up on making it work right, as it is almost impossible to change any existing jungle of systems :-( But, when you're designing a *new* encoding, you could build it to be better, incorporating what we have learned about the flaws of old encodings. > I think some people are severely confused over the purpose of HZ+. I may well be. But then, there seems to be so much *political* confusion in the i18n arena, that often technical issues are over-run by political nonsense. Maybe I'm just too idealistic, in which case I can only offer my naive apologies. Hoping for a better world, Hannu -- Hannu.Aronsson@cs.hut.fi Helsinki University of Technology, Finland ``If you only had one tooth, would you brush it longer?'' From czha@CATI.CSUFresno.EDU Fri Jun 10 11:40:23 1994 Return-Path: Received: from CATI.CSUFresno.EDU (caticsuf.cati.csufresno.edu) by ifcss.org (4.1/IFCSS-Mailer) id AA22103; Fri, 10 Jun 94 11:40:23 CDT Received: (czha@localhost) by CATI.CSUFresno.EDU (8.6.8.1/8.6.5) id JAA26042 for soft-authors@ifcss.org; Fri, 10 Jun 1994 09:37:51 -0700 Date: Fri, 10 Jun 1994 09:37:51 -0700 From: Charlie Zha Message-Id: <199406101637.JAA26042@CATI.CSUFresno.EDU> X-Mailer: idsmail 4.0 To: soft-authors@ifcss.org Subject: All the fights over Big5 Status: O Everyday, my mailbox is full of e-mails fighting over which Big5 is better. The sad thing is that it does not seem to go anywhere. Everyday I have to spend time just to dump them off my mailbox for more meaningful e-mails. What I am more interested at this point is to define an application architecture that is flexible enough to support different coding schemes. I am not a pro in this area and do not intend to pretend to be one. I will start by reading cxterm source code. Hopefully, I can contribute something more interesting than my opinions over which Big5 (or CNS) is better. Have a nice weekend and nice fights over everything. Charlie Zha From czha@CATI.CSUFresno.EDU Fri Jun 10 11:41:38 1994 Return-Path: Received: from CATI.CSUFresno.EDU (caticsuf.cati.csufresno.edu) by ifcss.org (4.1/IFCSS-Mailer) id AA22138; Fri, 10 Jun 94 11:41:38 CDT Received: (czha@localhost) by CATI.CSUFresno.EDU (8.6.8.1/8.6.5) id JAA26131 for soft-authors@ifcss.org; Fri, 10 Jun 1994 09:39:07 -0700 Date: Fri, 10 Jun 1994 09:39:07 -0700 From: Charlie Zha Message-Id: <199406101639.JAA26131@CATI.CSUFresno.EDU> X-Mailer: idsmail 4.0 To: soft-authors@ifcss.org Subject: UNSUBSCRIBE Status: O UNSUBSCRIBE From Ricky.Yeung@Eng.Sun.COM Fri Jun 10 13:40:31 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA27293; Fri, 10 Jun 94 13:40:31 CDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA13824; Fri, 10 Jun 94 11:38:22 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA18301; Fri, 10 Jun 94 11:38:25 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA02252; Fri, 10 Jun 1994 11:39:36 +0800 Date: Fri, 10 Jun 1994 11:39:36 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406101839.AA02252@hk.Eng.Sun.COM> To: soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Fri, 10 Jun 1994 08:21:34 -0400 <199406101221.IAA03738@boole.math.psu.edu> Subject: HZ vs. zW (Re: Politics (Was Re: The hostile Big5 users?)) Content-Length: 1518 Status: O > From: Stephen G Simpson > Date: Fri, 10 Jun 1994 08:21:34 -0400 > > I guess you are referring to ZW, an alternative to HZ that was > considered for a while before HZ took over. I regard ZW as very > similar to HZ. Both of them use the same 7-bit representation of GB > (by stripping high bits) and both of them use ASCII escape sequences. > When I said that something like HZ was inevitable, I was including ZW > in this statement. It's really interesting to see someone who could say the zW escape sequence, with restrictions like it has to switch at the beginning of the line, and can't do in-line switch from/to ASCII, etc, as VERY similar to HZ. Maybe you could say there is no difference between HZ+S and my proposal. :-) It's interesting to note that in terms of byte count, zW vs. HZ is 2 vs. 4. A 1:2 ratio. BTW, the byte count ratio between HZ+ and my proposal is 2n + 4 : 2n + 6 where n is the number of times you switch to other non-ASCII code schemes while you're in non-ASCII mode. (If it happens to switch into GB first, the ratio is 1:1). > And yes, I have read the a.c.t FAQ. Also, I am very familiar with the > history of ZW and HZ, having followed them closely from the beginning. If you're really familiar with the history of zW and HZ, you should have noticed that there was no consensus reached. The GB users picked HZ not because some software authors voted to use it exclusively; there was no vote. -Ricky From lai@apple.com Fri Jun 10 16:54:30 1994 Return-Path: Received: from apple.com by ifcss.org (4.1/IFCSS-Mailer) id AA06573; Fri, 10 Jun 94 16:54:30 CDT Received: from [17.203.80.11] by apple.com with SMTP (5.61/8-Oct-1993-eef) id AA25033; Fri, 10 Jun 94 14:51:55 -0700 for soft-authors@ifcss.org Date: Fri, 10 Jun 94 14:51:55 -0700 Message-Id: <9406102151.AA25033@apple.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" To: mule@etl.go.jp, soft-authors@ifcss.org, alt.chinese.computing@usenet.ucs.indiana.edu From: lai@apple.com (Edmund Lai) Subject: Vendor Specifc code Status: O Hannu Aronsson wrote >Just *now* you have the chance of a lifetime to make your encoding >somehow reflect the version of Big5 used. > >Then those who don't care about Big5 versions, they could just ignore >the different "version identifiers" in incoming data. Those, who do >care, it would allow correctly identifying incoming data. > >Making data coming out of the systems have the correct identification >would be easy to implement (#ifdef big5_eten_system #define big5_id >"some_eten_magic_id" #else ...) > >A few hour's work *now* could save hundreds of hours of work and >gigabytes of slightly corrupt data in the future. > Nelson Chin wrotes >It is safe to to choose a vendor like ETen to be the standard of >communications, and it has been done so already. In an international >setting, we must standardize, otherwise it'll be another case of >a chicken talking to a duck. In a local homogenous setting or people in >the same vendor group, HZ+S will not be a problem, but we are dealing >with International communication. Simpson expressed concerns of making it >messier with converting it to HZ+ if we try to manipulate the code other >than just transforming it to 7-bit, and that BIG5 is messy already, >but as we can see from the newsgroups, by standardizing we are actually >making BIG5 unmessier. Therefore i propose that HZ+ also deal with this >issue of standardizing to a specific vendor, namely ETen BIG5 for all >code range. This will release the burden to require so many different >conversion utilities of the vendor-specific area, if this vendor-specific >code is necessary for international communication. > >Nelson Chin Stephen G Simpson wrote: >So, what you are really saying is that it is morally wrong to post >vendor-specific codes in a newsgroup, because then the posting cannot >be read by everybody, only by those who have that particular vendor's >software. > >Your statement makes a certain amount of sense, but I don't see it as >an inviolable moral rule. For instance, if an ETen-specific message >is clearly labeled as such, do you still think that posting it is >morally wrong? > Well, it look like most people want the exchange code to be unambiguous. This is not exactly what Steve said, but he seem to leave open the possiblity of labeling, which then is no different from what Hannu Aronsson wanted. I have no problem if any vendor specific code or user specific code is posted in a manner specified by the exchange code so that it would not conflict with the exchange code, and that there is a machine readable label so that the vendor specific code can be handled by software. Note that the software can choose to handle them in several way. It can be translated into the vendor specific internal code of my own machine if the label matches my own machine and the label identifies a vendor that I know how to translate. The software can choose to ignore vendor specific code that it cannot recognize. The software can choose to use the missing character symbol commonly used in the Mac to denote character it cannot display. It is not perfect because if the software is a terminal program then the label can be missed, and there is no clear demaraction from one news to another. But for NewsReader type of program it is not a problem. So this is something worth looking into. Actaully once we try to extend HZ to Big5 we enter the multi-lingual arena. On the Mac GB and Big5 are consider two different languages. If you have a Big5 machine then half of materials you read may be from the GB half. When we dicuss extension to HZ, we keep talking like it is a new exchange code that involves Big5 only. That is a mistake. For example, there are GB characters, in particular the radicals, that cannot be translated into Big5. Perhaps they can be handled as "vendor specific" code. In my opinon there is really very little difference between using CNS or other form of 7 bit Big5. A few people have expressed the same view. My take is then if there is little difference why not use CNS. Other people take the opposite view that since there is little difference, then why use CNS. I can understand that. However I cannot accept some of arguments against CNS because they are just not true. Some one may ask why do care at all if the final result does not make a lot of difference. The problem is that in the attempt to exclude CNS, HZ+S has to take some very wrong decision, such as the code should not be unambiguous, and that would result in a poor standard. /* Disclaimer: All statments and opinions expressed are my own */ /* Edmund K. Lai */ /* Apple Computer, MS303-3A */ /* 20525 Mariani Ave, */ /* Cupertino, CA 95014 */ /* (408)974-6272 */ zW@h9cOi From simpson@math.psu.edu Fri Jun 10 17:02:35 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA07045; Fri, 10 Jun 94 17:02:35 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id SAA13874; Fri, 10 Jun 1994 18:00:26 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 10 Jun 1994 18:00:24 -0400 Date: Fri, 10 Jun 1994 18:00:24 -0400 Message-Id: <199406102200.SAA04653@boole.math.psu.edu> To: soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu, mule@etl.go.jp Subject: cleaning up Big5 isn't HZ+'s job Newsgroups: alt.chinese.text.big5 In-Reply-To: <9406101437.AA21457@unda.fi> References: <9406100049.AA20725@apple.com> <199406100500.BAA03002@boole.math.psu.edu> <9406100655.AA28986@unda.fi> <199406101250.IAA03784@boole.math.psu.edu> <9406101437.AA21457@unda.fi> Reply-To: simpson@math.psu.edu Status: O I wrote: > To the extent that Big5 is messy because of vendor-specific codes, HZ+ > will transparently reflect that messiness. ... Hannu Aronsson replied: > But why don't you try to solve this mess? You're designing a new 7-bit > encoding that probably will be quite widely used in many, many places. I hope you are right in saying that HZ+ will be widely used. But this depends on a lot of factors. In order to make HZ+ acceptable to large groups of people, I want to keep HZ+ extremely clean and simple. I want HZ+ to do only one job, and to do that job very, very well. That job is to provide a simple, convenient representation of mixed Big5/GB/ASCII for 7-bit environments. I don't want to burden HZ+ with extra baggage such as attempting to clean up Big5. Government and industry groups in Taiwan have tried very hard to clean up Big5, and they have failed miserably. (I think CNS was the result of one of these failed attempts.) If they couldn't do it, why do you think I can do it? > Just *now* you have the chance of a lifetime to make your encoding > somehow reflect the version of Big5 used. Then those who don't > care about Big5 versions, they could just ignore the different > "version identifiers" in incoming data. Those, who do care, it > would allow correctly identifying incoming data. My guess is that many users don't care. For users who don't care, forcing them to designate a Big5 version is imposing an extra burden. I don't want to make HZ+ any more burdensome than necessary. Users who care about vendor-specific Big5 codes are always free to designate this in an informal way, or to agree among themselves. I don't see any reason why HZ+ should take on the task of standardizing how this is to be done. > Making data coming out of the systems have the correct identification > would be easy to implement (#ifdef big5_eten_system #define big5_id > "some_eten_magic_id" #else ...) This has nothing to do with HZ+. The Big5<->HZ+ translation algorithm doesn't depend on which Big5 version you are using. There is no reason for HZ+ to force the user to specify the Big5 version. > Why not try to make the world a little better place, if you could > do it? I think HZ+ will make the world a better place, provided HZ+ is successful. But HZ+ will not be successful if we burden it with a lot of irrelevant baggage. > when you're designing a *new* encoding, you could build it to be > better, incorporating what we have learned about the flaws of old > encodings. I'm not designing a new encoding. HZ+ is not really a new encoding. HZ+ is nothing but a simple, 7-bit representation of mixed ASCII/GB/Big5. It inherits all the flaws and all the virtues of the old encodings: ASCII, GB, Big5. It adds nothing, and it takes nothing away. Translation to and from HZ+ is simple and 100 percent invertible. This is the way I want HZ+ to be. I hope you can sympathize and understand what I am trying to accomplish. -- Steve From simpson@math.psu.edu Fri Jun 10 17:16:03 1994 Return-Path: Received: from cnd.org (cnd.cnd.org) by ifcss.org (4.1/IFCSS-Mailer) id AA07664; Fri, 10 Jun 94 17:16:03 CDT Received: from leibniz.math.psu.edu by cnd.org (4.1/4.7) id AA24816; Fri, 10 Jun 94 15:14:35 PDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id SAA13935; Fri, 10 Jun 1994 18:11:57 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 10 Jun 1994 18:11:55 -0400 Date: Fri, 10 Jun 1994 18:11:55 -0400 Message-Id: <199406102211.SAA04673@boole.math.psu.edu> To: mule@etl.go.jp, ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: HZ+ etc. In-Reply-To: <9406101815.AA01971@> References: <9406101815.AA01971@> Reply-To: simpson@math.psu.edu Status: O stampe@uhunix.uhcc.Hawaii.Edu writes: > Would you mind taking this discussion off mule@etl.go.jp? It is not > going anywhere, and in any event it is completely unrelated to mule. OK, sure. Guys, let's stop sending stuff about HZ+ to the Mule list. > If you really think that Handa is interested, Cc to him privately. > But please not to the whole list. I have no way of knowing how interested Mr Handa is. I know that he briefly mentioned plans to change Mule's method of Big5 support. The plans were vague but had something to do with CNS. HZ+S is based on an idea taken from Mule 1.0's Big5 support. > Most mule users don't use Chinese! I wasn't aware of that. Does anyone have any statistics or guesses about the number of Mule users, or the number of Mule users using specific languages such as Japanese, Korean, Vietnamese, Russian, etc.? -- Steve (a Mule contributor) Stephen G. Simpson Department of Mathematics, Pennsylvania State University 333 McAllister Building, University Park, State College, PA 16802 Office phone +1 814 863-0775 Home phone +1 814 238-2274 E-mail simpson@math.psu.edu FAX +1 814 865-3735 World Wide Web http://www.math.psu.edu/simpson/ From simpson@math.psu.edu Fri Jun 10 17:22:54 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA08126; Fri, 10 Jun 94 17:22:54 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id SAA14066; Fri, 10 Jun 1994 18:20:49 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 10 Jun 1994 18:20:47 -0400 Date: Fri, 10 Jun 1994 18:20:47 -0400 Message-Id: <199406102220.SAA04708@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: HZ vs. zW (Re: Politics (Was Re: The hostile Big5 users?)) In-Reply-To: <9406101839.AA02252@hk.Eng.Sun.COM> References: <199406101221.IAA03738@boole.math.psu.edu> <9406101839.AA02252@hk.Eng.Sun.COM> Reply-To: simpson@math.psu.edu Status: O Ricky Yeung writes: > Maybe you could say there is no difference between HZ+S > and my proposal. :-) You haven't made a proposal. It would be possible to modify HZ+S in accordance with your suggestions, and the result would be very similar to HZ+S in all essential respects. > If you're really familiar with the history of zW and HZ, you should have > noticed that there was no consensus reached. The GB users picked HZ not > because some software authors voted to use it exclusively; there was no > vote. I'd put it this way: A consensus was reached, but it wasn't reached by voting. Yes, I'm aware of this history. What's your point, if any? -- Steve From simpson@math.psu.edu Fri Jun 10 18:03:51 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA10405; Fri, 10 Jun 94 18:03:51 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id TAA14895; Fri, 10 Jun 1994 19:01:44 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 10 Jun 1994 19:01:42 -0400 Date: Fri, 10 Jun 1994 19:01:42 -0400 Message-Id: <199406102301.TAA04771@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: Vendor Specifc code In-Reply-To: <9406102151.AA25033@apple.com> References: <9406102151.AA25033@apple.com> Reply-To: simpson@math.psu.edu Status: O Edmund Lai writes: > This is not exactly what Steve said, but he seem to leave open the > possiblity of labeling, which then is no different from what > Hannu Aronsson wanted. By "labeling" Ed is referring to the idea that we should label our Big5 news postings to indicate which vendor-specific Big5 we are using. I would make a distinction between (1) labeling enforced by HZ+, and (2) informal labeling. Hannu was calling for (1), where HZ+ would force you to decide which vendor-specific Big5 you are using and to label every document, the label to have a specified format. I reject (1), because I think labeling is an extra burden that doesn't really have much to do with HZ+. On the other hand, as Ed says, I did leave open the possibility of informal labeling. What I mean by informal labeling is that people who are using some vendor-specific Big5 codes could indicate this informally, for instance a little warning note at the beginning of a document, something like "The following message contains CWin-specific Big5 codes" or whatever, but this warning is not required and its format is not standardized. Such warnings could conceivably be useful not only in chinese.* (using HZ+) but also in 8-bit Big5 newsgroups. This would be a courteous and straightforward way of coping with vendor-specific Big5. Another possibility is to develop a labeling standard, where Big5 users would be encouraged to label all of their news postings or documents, according to a specified label format. Developing such a standard strikes me as somewhat Quixotic, but if somebody wants to work on this, please go ahead. I left open the possibility of labeling, but I really don't care about it very much. Above all, I don't want to get into a silly debate about enforced labeling versus informal labeling versus no labeling. I don't see that labeling has much to do with HZ+. Labeling is really a Big5 issue, not an HZ+ issue. All that HZ+ does is to represent the Big5 codes transparently, in a 100 percent invertible way. Ed says: > So this is something worth looking into. Sure, labeling is worth looking into. By all means, let's look into it. But let's separate it from HZ+. We need to move ahead and reach a consensus on HZ+. -- Steve From simpson@math.psu.edu Fri Jun 10 18:07:03 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA10523; Fri, 10 Jun 94 18:07:03 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id TAA14909; Fri, 10 Jun 1994 19:04:57 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 10 Jun 1994 19:04:56 -0400 Date: Fri, 10 Jun 1994 19:04:56 -0400 Message-Id: <199406102304.TAA04776@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: Can we finally put CNS aside? Reply-To: simpson@math.psu.edu Status: O Ed Lai writes: > In my opinon there is really very little difference between using CNS > or other form of 7 bit Big5. A few people have expressed the same view. > My take is then if there is little difference why not use CNS. Other > people take the opposite view that since there is little difference, > then why use CNS. I can understand that. OK, good. Since Ed is saying that there is very little difference, let's settle it quickly by a vote. Oops, sorry, I forgot, we already voted. CNS lost. OK, fine, now can we be done with this? Can we now please move on and settle the remaining issues and reach a consensus on HZ+? -- Steve From Ricky.Yeung@Eng.Sun.COM Fri Jun 10 18:41:59 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA11595; Fri, 10 Jun 94 18:41:59 CDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA12960; Fri, 10 Jun 94 16:39:54 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA18264; Fri, 10 Jun 94 16:40:05 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA03862; Fri, 10 Jun 1994 16:41:14 +0800 Date: Fri, 10 Jun 1994 16:41:14 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406102341.AA03862@hk.Eng.Sun.COM> To: soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Fri, 10 Jun 1994 18:20:47 -0400 <199406102220.SAA04708@boole.math.psu.edu> Subject: Re: HZ vs. zW (Re: Politics (Was Re: The hostile Big5 users?)) Content-Length: 639 Status: O > From: Stephen G Simpson > Date: Fri, 10 Jun 1994 18:20:47 -0400 > > > If you're really familiar with the history of zW and HZ, you should have > > noticed that there was no consensus reached. The GB users picked HZ not > > because some software authors voted to use it exclusively; there was no > > vote. > > I'd put it this way: A consensus was reached, but it wasn't reached by There was no consensus reached among the competing authors. > voting. Yes, I'm aware of this history. What's your point, if any? Isn't it obvious? I think any fool can get it. -Ricky From simpson@math.psu.edu Fri Jun 10 18:44:16 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA11663; Fri, 10 Jun 94 18:44:16 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id TAA15733; Fri, 10 Jun 1994 19:42:09 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 10 Jun 1994 19:42:07 -0400 Date: Fri, 10 Jun 1994 19:42:07 -0400 Message-Id: <199406102342.TAA04858@boole.math.psu.edu> To: soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu Subject: HZ+ versus UUencode In-Reply-To: References: <199406101250.IAA03784@boole.math.psu.edu> Reply-To: simpson@math.psu.edu Status: O I wrote: > > There is only one HZ+S. There is no > > need for monsters such as HZ+S_ETen, HZ+S_CWin, etc. Nelson replied: > So are you telling vendors(or anyone who wants to transmit > vendor-specific code) to provide conversion utilities for every > conceivable vendor-big5, after it's converted back to 8-bit BIG5 from > HZ+S? No, I never told them to do this. Vendors can support their vendor-specific characters any way they want to. They can try to provide conversion utilities, or they can just assume a homogenous environment, which will often be the case anyway. The question of how Big5 vendors should support their own vendor-specific characters is a question for the vendors, not a question for HZ+. HZ+ provides a 7-bit representation of the codes, that's all. > Are you saying the HZ+ standard is just the chinese way of say > UUDECODE, UUENCODE (of course w/o the defects of UUEN/DECODE). No, I never said this. I don't know how Nelson can read so much into what I said. :-) UUencode provides a convenient 7-bit representation of binary files. HZ+S provides something that UUencode can't provide, namely a convenient 7-bit representation of mixed ASCII/GB/Big5 text. One of the many ways in which HZ+S differs from UUencode is that HZ+ fully respects the structure of the GB and Big5 code sets. -- Steve From simpson@math.psu.edu Fri Jun 10 19:09:35 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA12434; Fri, 10 Jun 94 19:09:35 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id UAA16406; Fri, 10 Jun 1994 20:07:29 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 10 Jun 1994 20:07:27 -0400 Date: Fri, 10 Jun 1994 20:07:27 -0400 Message-Id: <199406110007.UAA04903@boole.math.psu.edu> To: soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu Subject: Re: a question for users of 8-bit Big5 newsgroups In-Reply-To: References: <199406101250.IAA03784@boole.math.psu.edu> Reply-To: simpson@math.psu.edu Status: O C I I Project writes: > HOW NEWSGROUPS HANDLE VENDOR SPECIFIC BIG5 > ========================================== > alt.chinese.text.big5 and alt.chinese.text conversion gateway assumes > ETen specific code. > software used: > ifcss.org:/software/unix/sym-supp.tab, hc-supp.tab, in-corr.tab, > hc-30.tar.gz > csie.bbs.fj.* big5 mirrors of fj.* japanese regional hierachy assumes > ETen specific code. > software used: sinocode, you can do an archie on it > tw.bbs.lang.japanese, direct BIG5 input, no conversions. From the > postings everyone conforms to ETen BIG5 code. Interesting. So ETen Big5 seems to be at least a de facto standard in these newsgroups. How much (if at all) is this standard enforced? Are postings rejected if they contain vendor-specific Big5 other than ETen Big5? > CONCLUSION > ========== > It is safe to to choose a vendor like ETen to be the standard of > communications, and it has been done so already. In an international > setting, we must standardize, otherwise it'll be another case of > a chicken talking to a duck. >From what you are saying, ETen Big5 appears to already be a de facto standard. Good, I'm all in favor of this. As you know, I emphasized ETen Big5 in my HZ+ document. And long ago I wrote a careful description of ETen Big5, for use in the soft-authors HBF discussion, precisely because I wanted to promote ETen Big5. > In a local homogenous setting or people in > the same vendor group, HZ+S will not be a problem, but we are dealing > with International communication. Even in international communication, I don't see that HZ+S as such will create many problems. If there are problems due to vendor-specific Big5, HZ+S will not solve those problems, but it will also not make them any worse. > Simpson expressed concerns of making it messier with converting it > to HZ+ if we try to manipulate the code other than just > transforming it to 7-bit, and that BIG5 is messy already, but as we > can see from the newsgroups, by standardizing we are actually > making BIG5 unmessier. Therefore i propose that HZ+ also deal with > this issue of standardizing to a specific vendor, namely ETen BIG5 > for all code range. That's a good suggestion. Thanks! I don't want to change the HZ+S mapping scheme. But I'm thinking I could implement your suggestion by adding language to the HZ+S document, strongly encouraging standardization on ETen Big5. The document already emphasizes ETen Big5, but I could make the language even stronger. What do you say? > This will release the burden to require so many different > conversion utilities of the vendor-specific area, if this > vendor-specific code is necessary for international communication. Yes, I agree. ETen Big5 isn't perfect, but because of its popularity it is the closest thing we have to a standardized Big5 with wide acceptance. -- Steve From Ricky.Yeung@Eng.Sun.COM Fri Jun 10 19:12:39 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA12565; Fri, 10 Jun 94 19:12:39 CDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA17235; Fri, 10 Jun 94 17:10:34 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA20949; Fri, 10 Jun 94 17:10:45 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA03902; Fri, 10 Jun 1994 17:11:54 +0800 Date: Fri, 10 Jun 1994 17:11:54 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406110011.AA03902@hk.Eng.Sun.COM> To: soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Fri, 10 Jun 1994 18:20:47 -0400 <199406102220.SAA04708@boole.math.psu.edu> Subject: Proposal Content-Length: 474 Status: O > From: Stephen G Simpson > Date: Fri, 10 Jun 1994 18:20:47 -0400 > > You haven't made a proposal. It would be possible to modify HZ+S > in accordance with your suggestions, and the result would be very > similar to HZ+S in all essential respects. You are pretty clear on my PROPOSED changes to HZ through the e-mail discussions. Is a "proposal" like your HZ+S document format a prerequisite to talk about HZ issues? -Ricky From simpson@math.psu.edu Fri Jun 10 19:32:53 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA13395; Fri, 10 Jun 94 19:32:53 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id UAA16939; Fri, 10 Jun 1994 20:30:48 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 10 Jun 1994 20:30:46 -0400 Date: Fri, 10 Jun 1994 20:30:46 -0400 Message-Id: <199406110030.UAA04941@boole.math.psu.edu> To: Bob Sandstrom Cc: soft-authors@ifcss.org Subject: User specific Big5 codes In-Reply-To: References: Reply-To: simpson@math.psu.edu Status: O Bob Sandstrom writes: > please note that 0xa140-0xfefe is not the entire Big5 code space. > ... > Eten, for example, also uses 0x8e40-0xa0fe (for the > second section of USRFONT) and 0x8140-0x8dfe (for the third section > of USRFONT). My Eten information is from an April 1992 Eten > manual, ISBN 957-504-044-9, page 5-2 in Appendix 5. This is interesting. It is also a little bit dismaying. I didn't know about these code ranges reserved for user-defined characters. Tell me, what are the allowable first bytes in these ranges? All along I have been assuming that the entire Big5 code space is included in 0xa140-0xfefe, but maybe we need to expand this. I have an idea for how to deal with this. I can extend HZ+S by packing the user-specific character range into the graphical ASCII range, 0x2121-0xfdfe, and designating an additional escape sequence for this. The escape sequence could be ~0 if we use ~1 and ~2 for blocks 1 and 2. Thus the user-defined range is treated as block 0. Note that this extension of HZ+S would be downward compatible with the current HZ+S. It could be designated as an optional extension, needed only in special situations involving user-defined characters. -- Steve From lai@apple.com Fri Jun 10 19:33:32 1994 Return-Path: Received: from apple.com by ifcss.org (4.1/IFCSS-Mailer) id AA13414; Fri, 10 Jun 94 19:33:32 CDT Received: from [17.203.80.11] by apple.com with SMTP (5.61/8-Oct-1993-eef) id AA15320; Fri, 10 Jun 94 17:31:23 -0700 for soft-authors@ifcss.org Date: Fri, 10 Jun 94 17:31:23 -0700 Message-Id: <9406110031.AA15320@apple.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org From: lai@apple.com (Edmund Lai) Subject: Re: Vendor Specifc code Status: O >Edmund Lai writes: > > This is not exactly what Steve said, but he seem to leave open the > > possiblity of labeling, which then is no different from what > > Hannu Aronsson wanted. > >By "labeling" Ed is referring to the idea that we should label our >Big5 news postings to indicate which vendor-specific Big5 we are >using. I would make a distinction between (1) labeling enforced by >HZ+, and (2) informal labeling. Hannu was calling for (1), where HZ+ >would force you to decide which vendor-specific Big5 you are using and >to label every document, the label to have a specified format. > >I reject (1), because I think labeling is an extra burden that doesn't >really have much to do with HZ+. On the other hand, as Ed says, I did >leave open the possibility of informal labeling. > >What I mean by informal labeling is that people who are using some >vendor-specific Big5 codes could indicate this informally, for >instance a little warning note at the beginning of a document, >something like "The following message contains CWin-specific Big5 >codes" or whatever, but this warning is not required and its format is >not standardized. Such warnings could conceivably be useful not only >in chinese.* (using HZ+) but also in 8-bit Big5 newsgroups. This >would be a courteous and straightforward way of coping with >vendor-specific Big5. > Informal labeling is next to useless. I stated very clearly that the label need to be able to processed by the software automatically, so that the software can do any translation that it can do. If there is no vendor specific code, then no labeling is needed. In reality that means that most document will have no label. /* Disclaimer: All statments and opinions expressed are my own */ /* Edmund K. Lai */ /* Apple Computer, MS303-3A */ /* 20525 Mariani Ave, */ /* Cupertino, CA 95014 */ /* (408)974-6272 */ zW@h9cOi From lai@apple.com Fri Jun 10 19:38:27 1994 Return-Path: Received: from apple.com by ifcss.org (4.1/IFCSS-Mailer) id AA13542; Fri, 10 Jun 94 19:38:27 CDT Received: from [17.203.80.11] by apple.com with SMTP (5.61/8-Oct-1993-eef) id AA15691; Fri, 10 Jun 94 17:36:15 -0700 for soft-authors@ifcss.org Date: Fri, 10 Jun 94 17:36:15 -0700 Message-Id: <9406110036.AA15691@apple.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" To: soft-authors@ifcss.org From: lai@apple.com (Edmund Lai) Subject: Re: Can we finally put CNS aside? Status: O >Ed Lai writes: > > In my opinon there is really very little difference between using CNS > > or other form of 7 bit Big5. A few people have expressed the same view. > > My take is then if there is little difference why not use CNS. Other > > people take the opposite view that since there is little difference, > > then why use CNS. I can understand that. > >OK, good. Since Ed is saying that there is very little difference, >let's settle it quickly by a vote. Oops, sorry, I forgot, we already >voted. CNS lost. OK, fine, now can we be done with this? Can we now >please move on and settle the remaining issues and reach a consensus >on HZ+? > If indeed it was a fair vote, then it would have been settled. But the voting was under the misrepsentation that CNS cannot be a 7 bit representation of Big5, a vote for CNS is a vote for Big5. /* Disclaimer: All statments and opinions expressed are my own */ /* Edmund K. Lai */ /* Apple Computer, MS303-3A */ /* 20525 Mariani Ave, */ /* Cupertino, CA 95014 */ /* (408)974-6272 */ zW@h9cOi From handa@etlken.etl.go.jp Fri Jun 10 20:22:04 1994 Received: from etlpost (etlpost.etl.go.jp) by ifcss.org (4.1/IFCSS-Mailer) id AA14816; Fri, 10 Jun 94 20:22:04 CDT Received: from etlpom.etl.go.jp by etlpost (5.67+1.6W/2.7W) id AA05539; Sat, 11 Jun 94 10:06:15 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA06813; Sat, 11 Jun 94 10:06:08 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA23461; Sat, 11 Jun 94 10:04:56 JST Date: Sat, 11 Jun 94 10:04:56 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9406110104.AA23461@etlken.etl.go.jp> To: mule@etl.go.jp Cc: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Fri, 10 Jun 1994 18:11:55 -0400 <199406102211.SAA04673@boole.math.psu.edu> Subject: Re: HZ+ etc. Status: O On Fri, 10 Jun 1994 18:11:55 -0400, Stephen G Simpson said: > stampe@uhunix.uhcc.Hawaii.Edu writes: > > Would you mind taking this discussion off mule@etl.go.jp? It is not > > going anywhere, and in any event it is completely unrelated to mule. > OK, sure. Guys, let's stop sending stuff about HZ+ to the Mule list. I also think it's better. I hope this is the last mail to . But please keep including me in this dicussion. I would like to join or . Could someone please send me the quideline of these mailing list? OR, how about moving the discussion to alt.chinese.software? I just now found some mails on this thread are also posted to the newsgroup. I may have missed some articles related on this thread in the newsgroup. > I have no way of knowing how interested Mr Handa is. I know that he I'm VERY interested in this discussion and reading all mails on this thread. But, since I'm now very busy, my response will be lazy. > briefly mentioned plans to change Mule's method of Big5 support. The > plans were vague but had something to do with CNS. HZ+S is based on > an idea taken from Mule 1.0's Big5 support. I have not yet decided the detail. I'm waiting some agreement on this matter. > > Most mule users don't use Chinese! > I wasn't aware of that. Does anyone have any statistics or guesses > about the number of Mule users, or the number of Mule users using > specific languages such as Japanese, Korean, Vietnamese, Russian, > etc.? I think Japanese users are the largest, next Chinese, but don't have any statistics. --- Ken'ichi HANDA handa@etl.go.jp From handa@etlken.etl.go.jp Fri Jun 10 20:34:18 1994 Received: from etlpost (etlpost.etl.go.jp) by ifcss.org (4.1/IFCSS-Mailer) id AA15132; Fri, 10 Jun 94 20:34:18 CDT Received: from etlpom.etl.go.jp by etlpost (5.67+1.6W/2.7W) id AA05599; Sat, 11 Jun 94 10:19:45 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA06842; Sat, 11 Jun 94 10:19:37 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA23481; Sat, 11 Jun 94 10:19:45 JST Date: Sat, 11 Jun 94 10:19:45 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9406110119.AA23481@etlken.etl.go.jp> To: mule@etl.go.jp Cc: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Fri, 10 Jun 1994 18:11:55 -0400 <199406102211.SAA04673@boole.math.psu.edu> Subject: Re: HZ+ etc. Status: O On Fri, 10 Jun 1994 18:11:55 -0400, Stephen G Simpson said: > stampe@uhunix.uhcc.Hawaii.Edu writes: > > Would you mind taking this discussion off mule@etl.go.jp? It is not > > going anywhere, and in any event it is completely unrelated to mule. > OK, sure. Guys, let's stop sending stuff about HZ+ to the Mule list. I also think it's better. I hope this is the last mail to . But please keep including me in this dicussion. I would like to join or . Could someone please send me the quideline of these mailing list? OR, how about moving the discussion to alt.chinese.software? I just now found some mails on this thread are also posted to the newsgroup. I may have missed some articles related on this thread in the newsgroup. > I have no way of knowing how interested Mr Handa is. I know that he I'm VERY interested in this discussion and reading all mails on this thread. But, since I'm now very busy, my response will be lazy. > briefly mentioned plans to change Mule's method of Big5 support. The > plans were vague but had something to do with CNS. HZ+S is based on > an idea taken from Mule 1.0's Big5 support. I have not yet decided the detail. I'm waiting some agreement on this matter. > > Most mule users don't use Chinese! > I wasn't aware of that. Does anyone have any statistics or guesses > about the number of Mule users, or the number of Mule users using > specific languages such as Japanese, Korean, Vietnamese, Russian, > etc.? I think Japanese users are the largest (many of them shifted from Nemacs to Mule), next Chinese, but don't have any statistics. --- Ken'ichi HANDA handa@etl.go.jp From simpson@math.psu.edu Fri Jun 10 23:36:13 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA20824; Fri, 10 Jun 94 23:36:13 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id AAA17991; Sat, 11 Jun 1994 00:34:08 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Sat, 11 Jun 1994 00:34:06 -0400 Date: Sat, 11 Jun 1994 00:34:06 -0400 Message-Id: <199406110434.AAA05351@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: call for a vote on HZ+ mapping schemes Reply-To: soft-authors@ifcss.org Status: O Recently there has been nothing new on the 7-bit Big5 mapping scheme issue. I would therefore like to call for a formal vote within the soft-authors@ifcss.org mail alias. HZ+ is defined as HZ plus some form of 7-bit Big5. One of the issues that we have discussed extensively is 7-bit Big5 mapping schemes, i.e. how shall HZ+ map Big5 into the graphical ASCII code space. The purpose of the vote is to decide this issue. There are two proposals under consideration: A. my proposal, HZ+S. B. Ya-Gui Wei's proposal, HZ-2. Please note that this vote concerns only 7-bit Big5 mapping schemes. Another outstanding issue is which tilde escape sequences to use, but this will be decided separately. If you receive this mail, you are on the soft-authors alias. Could everybody who receives this mail please vote? The way to vote is to reply to soft-authors@ifcss.org, declaring your preference for option A or option B. I urge everybody to vote. By voting, you are helping us to reach a consensus on HZ+. Such a consensus is urgently needed. The deadline for votes is June 15. -- Steve From cyl Sat Jun 11 00:58:29 1994 Return-Path: Received: by ifcss.org (4.1/IFCSS-Mailer) id AA24463; Sat, 11 Jun 94 00:58:29 CDT Date: Sat, 11 Jun 1994 00:58:26 -0500 (CDT) From: Nelson CHIN Sender: Nelson CHIN Reply-To: Nelson CHIN Subject: new solution: approve both HZ+ and ISO2022 To: soft-authors@ifcss.org, handa@etl.go.jp Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=ISO-8859-1 Status: O I've a suggestion for 7-bit transmission. How about soft-authors give their stamp of approval to an HZ+ standard to fulfill the low road of popular defacto standard like BIG5 and also the existing HZ standard, and also give their stamp of approval to ISO2022 standard of 7-bit transmission, and we can use ifcss.org:/software/unix/convert/b5cns.tar.gz as an example, to mobilize users into International arena of the high road. This way, the passage of time and as users see post in ISO format they will ask what's that junk and will get the required conversion software or will ask their vendor to support it in their software, and vendors will more likely willing to support ISO standards as it'll be upcompatible with other iso approved character codes. It's important that soft-authors give stamp of approval to both short-term and long term goals, and allow users to post/email freely in both codes, and the passage of time will bring about the transition. We should start the transition stage now as we introduce 7-bit exchange code for mixed ascii/big5/gb. We must not just look at the here and the now, we must provide a way to transition into the future and be an encouragement to users to take that path, so we therefore should not pick one or the other, but give users the choice to transition, with the backing of soft-authors. The goal of transmitting in 7-bit code is still accomplished by stamping approval for both HZ+ and ISO-2022 (eg, b5cns, and maybe someone can come up with a 7-bit GB convertor that's iso compliant), this will make it alot easier for internationized software to support chinese in the future. Nelson Chin From shin@hermes.ee.nthu.edu.tw Sat Jun 11 06:08:41 1994 Return-Path: Received: from nthu.edu.tw by ifcss.org (4.1/IFCSS-Mailer) id AA01498; Sat, 11 Jun 94 06:08:41 CDT Received: from hermes.ee.nthu.edu.tw ([140.114.28.27]) by nthu.edu.tw with SMTP id <181972(1)>; Sat, 11 Jun 1994 19:12:25 +0800 Received: by hermes.ee.nthu.edu.tw (4.1/SMI-4.0/shin-1.0) id AA13196; Sat, 11 Jun 94 19:04:39 CST Date: Sun, 12 Jun 1994 09:04:39 +0800 From: shin@hermes.ee.nthu.edu.tw (Jing-Shin Chang) Message-Id: <9406111104.AA13196@hermes.ee.nthu.edu.tw> To: ccnet-l@uga.uga.edu, soft-authors@ifcss.org Subject: My A-Plan Status: RO [My A-Plan and My Positions] For the last few days, esp. after Steve's reply to my many questions was posted (-^^-), I received at least half a dozen of emails and a phone call, mostly from the Big5 user group. Some of them push me hardly to speak louder and more vocally for my B5E1/2/3. And I saw posting in B5E3 and CNS/7 in chinese.test from some of these heartly and kind persons. I thanks all of them. Sincerely. Obviously, my plan to be a "happy end user" is terminated by these kind people. So I decide to defer for one month+ before I really become a happy end user. Many of the people are considering that I was not advocate vocally and repeatedly on my 7-bit encoding schemes. I think that was not the major points. And I believe the software contributors' own judgement; whether they care about simplicity, file sizes, XXX-compatibility, sorting sequence, etc. are their own judgement. I don't need to tell them what they should choose. When people are still talking about such encoding schemes, I had noticed the User Interface aspects (esp. for the Big5 users) we can offer to the end users (it is even much earlier than the "ONE problem with HZ+" issue was raised.) As a response to these kind persons who support me, I will post my proposals again to some of the chinese.*. And I may put an (admitly 5% buggy) Betty to my ftp site so that people could use it to read news posting encoded in HZ-?. (I need to clean some of the codes, and add HZ-S and HZ-2 supports. So you may need to wait a little later.) Since I had withdraw my B5E1/2/3 from the "official" voting list, I will not push them back to the "official" list of voting. (It sounds strange to me.) For people who are interested in doing something with such encoding schemes, you could use them for your personal communication, say, between you and your dear girl friend or boy friend. It's copy free. In fact, my position is toward going back to the CNS proposals; this is not an abrupt decision but a long period of consideration. I had considered this problem for a long time even during the fighting period of those HZ+ debates. I get information from Ross, Erik, Mule, and many others; I also see similar implementations in the workstation environments; I got informations from Lai, Yuan, Nelson, John; plus my original idea of not inventing new wheels. I guess we could open our eyes toward a more international high road. (People who noticed that I was posting the first article to ask people not to invent new wheels should not be surprised at this; it is much earlier even than Lai, Yuan and Steve's debate on the CNS stuffs.) Admittedly, Big5/8 to Big5/7 mapping "looks" simpler (esp. with HZ-2, B5E1, B5E3 and an unannounced method used in Sun), so, as a tradeoff, the Big5/7 option will still be in my implementation as a compilation option. One of my A-Plan is to demonstrate that CNS<->Big5 is not more difficult than Big5/8<->Big5/7 with essentially the same packing algorithms as HZ-S, even with those ETen vender-specific extension. (Of course, I think it is possible to do it better than using table lookup -^^-. Carlos, sorry I didn't mention this idea when I told you that HZ-S could be optimized by using table lookup -^^-. At that time, I didn't think that the searching time of the table lookup process may take some times for a large table.) I think the previous problem I raised (and other unraised problems) could be eliminated if well designed HZ+-aware (and esp. Big5-User-aware) software is available. I hope I could do it directly when debating doesn't seem to be helpful. Obviously, only a few sample off-line filter programs are there at the present time. When we have enought HZ-+aware software, I have no objection that anybody call the problem I raised as a *FAKE* problem. (In fact, there are other incompatibility problems of the HZ and HZ+ standard with the current day computer systems, where different ESC sequences, special characters from your applications, from the shells may interrupt the HZ(+) character streams such that the HZ(+) decoders are confused with such sequences. By doing it, other than the filter implementation, we will be better aware of such problems, and would not call such problems *FAKE* problems.) So, instead of debating non-constructively on problems in the list, I would invite all people interested in making the chinese.* the first potential multi-lingual newsgroups to face them by doing real works. Here is my A-Plan (** Not B5E4 ** -^^-): 1. I hope that anyone who had proposed any 7-bit representation for Big5 code could spend one month or so to write or modify AT LEAST ONE (and better AS MANY AS POSSIBLE) news reader or mailer that is most widely used in your campus or organization so that at least some HZ+-aware softwares are available when we feel that an official announcement of a HZ+ standard is appropriate. For people who did not present any formal proposal, like John and Ross, maybe you could push your friends who are doing programming to try to support a HZ+-aware news reader or mailer. 2. I guess we could give Ya-Gui two months or more to modify his new version of ZWDOS, because there is the "rumor" from XiaoFu that Ya-Gui had got a beautiful daughter recently. (How long have we made the debates, probably more than a month+, such that we ignore such a big news in ACT(.B).-^^-) Probably I am not qualified to suggest Ya-Gui for any improvement on his ZWDOS, but I think, in addition to taking over the control of the DOS display, he could also take over the file open/save/close INT's such that a HZ(+) file could be presented (pre-filtered) to the users' favorite form at a much cleaner appearance without the esc. characters (and even those unnoticeable symbols for substituting those esc. sequences.); just like we handle a Words document and so on, without presenting those non-text elements (underlying attributes, typeface info., etc.) to the users. (I am here referring to ZWDOS2.2 and older. If such problems no longer exist in newer versions, please forgive my ignorance.) 3. For myself, I plan to implement all the Big5/7 proposals, probably including a CNS version, for a news reader. (My first choice is the "rn" news reader for Unix, which is widely used in my campus and several other universities here, and is not as big as Mule to me). I also plan to clean my online-converter (Betty) for reading news without changing the news reader. With the materials at my hand, I think it is possible to use Betty with any popular taiwanese terminal, be it Big5(PC/HP/OS2), EUC (Sun/AIX), CNS (PC/Sun/AIX), IBM5550(OS/2) or TCA(PC) terminals, to any normalized 7-bit representation. As for the vender-specific chars, such as Japanese kana, the ETen version had allocate some unused code space in its corresponding CNS implementation. So I am not esp. worry about it. And may use it to standardize such extension. (The bad thing is, ETen did not seems to allocate a better code space in the unused CNS code space; it seems that ETen could allocate a larger, unused, continuous code space in CNS, *not* allocating several smaller pieces like its current implementation. I need to re-consider this better. As for other Big5 vender-specific codes, such as KC Big5, if any, I would like to normalize them to the extended CNS version if the corresponding characters are already there. If there are no corresponding characters, maybe a new allocation is necessary. Anyway, it's the other vender's (or their supporters') responsibility to map such vender-specific code to the extended CNS. NOT ME.) (I need some times to correct the ccf/chcode package for this.) And finally, I may come up with a mailer so that even with an old UNIX OS, 8-bit mails does not suffer from any practical problems. Of course, my major problem is the time for implementation. With so many commitments, it is very likely that many works will be done partially. Normally, I worked 10-14hr a day for my professional research. So the free time in a month means only about 3 or 4 Sundays to me. (Plus I have to prepare for a conference in New Mexico State Univ. at the end of this June.) It is really a big burden to me. If I could not offers such solutions or bug-free solutions on time, please just forgive me. Anyway, I am more of an empiricist rather than a rationalist; we deal with tens of thousands of sentences of language materials every day in our research works, so I want to do it after thorough considerations were made, rather than debating problems that is already clear enough or that is subject to debate without justification. By executing such an A-Plan, I think we at least have the following advantages: 0. In view of the current status of the ACT(B) users and their postings, I don't see any urgency to get a quick agreement (within one month+) on the HZ+ proposals. (For the present time, we have actually only 1.x official proposals. -^^-) In fact, what I am concerned is: if we are proposing lousy solutions to the end users based on a new "standard", I would, instead, suggest regional mirroring, in which cases, nobody needs to change their software environment. 1. We don't need to debate any issues on a very loose ground. When we are tired of implementing our solutions, we could take our times to see what users are talking about in the ACT(B) or look at something interesting like LiangSheng's TTF research. 2. We don't need to prepare, at any time, for fighting to somebody's debates/rebuttals. That's time consuming, and sometimes just makes a very simple fact messed or introduces incorrect information. 3. After the months of practical implementation, we can better discuss user's needs on a more solid basis, without limiting ourselves to thoses minor encoding techniques. (Believe it or not, in Sun's 8-bit mails, they even did not differentiate any kind of Plane 1/2 (or Block 1/2), and people are using it successfully and unconsciously.) 4. It's good to our health (esp. to me, who suffered from a little irregular heart beating these days.) If anyone is interested in this A-Plan and would like to help, please give me a message. Don't be afraid that you are not familiar with anything about this. Nothing is really too difficult. (People from around the Hsinchu area are especially convenient for direct communication. People could also reach me electronically if she/he is not in the Hsinchu area.) [Some Experiences and Technical Stuffs] ... a few ways to implement HZ+-aware stuffs from within system kernel, from within the applications, and from outside the applications ... ... DELETED ... (It's too long to post, and it is not completed ready yet. So I decide to post it next time. -^^-. At least after this weekend. -^^-) From shin@hermes.ee.nthu.edu.tw Sat Jun 11 07:18:30 1994 Return-Path: Received: from nthu.edu.tw by ifcss.org (4.1/IFCSS-Mailer) id AA02812; Sat, 11 Jun 94 07:18:30 CDT Received: from hermes.ee.nthu.edu.tw ([140.114.28.27]) by nthu.edu.tw with SMTP id <181970(2)>; Sat, 11 Jun 1994 20:22:13 +0800 Received: by hermes.ee.nthu.edu.tw (4.1/SMI-4.0/shin-1.0) id AA13278; Sat, 11 Jun 94 20:14:28 CST Date: Sun, 12 Jun 1994 10:14:28 +0800 From: shin@hermes.ee.nthu.edu.tw (Jing-Shin Chang) Message-Id: <9406111214.AA13278@hermes.ee.nthu.edu.tw> To: ccnet-l@uga.uga.edu, soft-authors@ifcss.org Subject: [A-Plan: Appendix] (repost) 3 7-bit Big5 encoding methods Status: RO This is a reposting which acts as the appendix of my Plan-A. Please refer to the previous posting for details. Thanks for your attention. If you want to find the HZ-S encoding method, you could ftp to math.psu.edu:/pub/simpson/chinese/hzps. You can also find HZ-2 spec and the original HZ spec, other ISO2022 doc, CNS materials in my ftp site: hermes.ee.nthu.edu.tw:/shin/* Have a nice weekend. Shin -^^- P.S. If you have little time to read it at once, I suggest that your reading order be B5E1, B5E3 and B5E2. This order is ranked in terms of simplicity. P.S. For CNS/8 and CNS/7 mapping, it is just to clear the high bits of the two bytes of a chinese char, which is the same as B5E1 does (The high bits of the 2nd bytes of the plane 2 chars need not be cleared since they are already '0'.) Furthermore, CNS does not have the problem of switching frequently between block 1 and block2 since frequently used chars are already allocated in plane1 (i.e., block 1) and less frequently used chars are allocated in plane 2. =================== [HZ+B5E1/HZ+B5E2/HZ+B5E3] @(#)$Id: hzb5e.spec,v 1.3 1994/05/21 21:19:21 shin Exp $ Three Proposals for Mixed ASCII/GB/Big5 Representation Jing-Shin Chang (shin@hermes.ee.nthu.edu.tw) --- ** all documents mentioned here are available from: hermes.ee.nthu.edu.tw(140.114.28.27):/shin/hzb5encode/* --- 0. Acknowledgements The original work of the HZ specification for representing mixed ASCII/GB character sets are due to Fung-Fung Lee (lee@umunhum.stanford.edu,09/04/1989) and his fellows. (See "HZ.spec" for a detailed specification of the HZ standard.) The escape sequences for switching into GB mode and switching to the default ASCII mode in the proposals are borrowed from the original HZ specification. (*) The idea to extend the current HZ specification are initiated by Professor Stephen G. Simpson (simpson@math.psu.edu). The escape sequences for entering Big5 block 1 and block 2 are borrowed from Professor Simpson's current HZ+S specification (v. 0.6), which is another proposal for mixing ASCII/GB/Big5 character sets. (See math.psu.edu:/pub/simpson/chinese/hzps/hzps.txt for details on HZ+S specifications.) (*) There are other kind persons who directly or indirectly contribute their ideas on discussing or commenting on the advantages or disadvantages of the current proposals. In particular, I had learned much from Ya-Gui Wei's HZ-2, which is yet another very valuable mixed ASCII/GB/Big5 character sets scheme. The B5E3 encoding scheme has borrowed the idea of cutting the high-byte range of 0xa1-0xfe at the middle from HZ-2. All credits concerning the related materials and efforts should be given to the respective authors or contributors. Thanks are due to all these kind persons. I love all of them. (**) --- P.S. (*) If they don't borrow me such escape sequences, I will have big trouble. I will then need to invent my own wheels. -^^- (**) What would happen if I don't love them ?? -^^- --- 1. Motivation Due to the very complex communication environment in the world-wide network systems, there is the demand for communicating with chinese safely. One possible way for safe communication with Chinese characters through the network is to use 7-bit encoding of Chinese characters. There are two major chinese user groups: one of them uses the GB code for their daily life and the other group uses the Big5 code for computer applications. To allow communication between these two groups, we need a way to integrate these two systems into a unified one. We thus need a 7-bit standard for such communication purposes. Currently, there is already a HZ standard for mixing ASCII and GB with 7-bit printable ASCII characters (0x21-0x7e). In this document, I will propose two solutions for representing the Big5 code with 7-bit printable ASCII characters. These two encoding schemes will be called B5encode-1/ B5decode-1 and B5encode-2/B5decode-2, or B5E1 and B5E2, respectively. Such encoding schemes for converting 8-bit Big5 to/from 7-bit Big5, when combined with appropriate escape sequences, will enable us to mix ASCII/ GB/Big5 with 7-bit printable ASCII characters. The mixed character set encoding schemes will be referred to as HZ+B5E1 and HZ+B5E2, respectly. (When there is no confusion, you could still call them B5E1 and B5E2.) 2. The Big5 Map The big5 code set is a two-byte code set with the following code space: /* * @(#) Jing-Shin Chang, EE, NTHU, Hsinchu, Taiwan, ROC. * ASCII: 0x00 - 0x1f - control characters 0x20 - 0x2f - space and symbols 0x30 - 0x39 - digits (0 - 9) 0x3a - 0x40 - symbols (0x20- 0x40 - symbols and digits) 0x41 - 0x5a - upper case 0x5b - 0x60 - symbols 0x61 - 0x7a - lower case 0x7b - 0x7e - symbols 0x7f (127) - DEL Big5: High Bytes: 0x81-0x8d,0x8e-0xa0,0xa1-0xfe (0x01-0x0d,0x0e-0x20,0x21-0x7e, when high bit stripped) Low Bytes: 0x40-0x7e,0xa1-0xfe a0: ' ' | 0x80, a1: '!' | 0x80, fe: '~' | 0x80 High Bytes 00 7e 80 81 8d 8e a0 a1 a3 a4 c6 c9 f9 fa fe ff 00 +----------------------------------------------------------------+ | | | | 3f +----------------------------------------------------------------+ 40 +--------------------*----*-.-------.-*---*-#----#-!----!-x---x--+ : | 40-7e: 63 chars * U * . U . * S * # S # ! A ! x U x | : | * 3 * . 2 . * P * # T # ! L ! x 1 x | : | * * . . * C * # D # ! T ! x x | 7e +--------------------*----*-.-------.-*---*-#----#-!----!-x---x--+ 80 (NULL|0x80) 81 +----------------------------------------------------------------+ a1 +--------------------*----*-.-------.-*---*-#---#--!----!-x---x--+ : | a1-fe: 94 chars * U * . U . * * # S # ! A ! x U x | : | * * . . * bf* # # ! ! x x | : | * 3 * . 2 . * c0$ # T # ! L ! x 1 x | : | * * . . * $ # # ! ! x x | : | * * . . * e0$ # D # ! Td5! x x | : | * * . . * # # ! x x | fe +--------------------*----*-.-------.-*-----#---#--!------x---x--+ ff (DEL) SPC: special symbols STD: standard (commonly used) characters ALT: alternative set of (less frequently used) characters U1: user-defined code area #1 U2: user-defined code area #2 U3: user-defined code area #3 */ There are different vender-specific extension to the original Big5 code. Therefore, I will handle the following 4 blocks of codes to cover all such possible extension: Freq-2 (F2): (0xa1-0xc8,0x40-0x7e) Freq-1 (F1): (0xa1-0xc8,0xa1-0xfe) Rare-2 : (0xc9-0xfe,0x40-0x7e) Rare-1 (R1): (0xc9-0xfe,0xa1-0xfe) The 2nd and 3rd user-defined code space (U2 and U3) will not be handled because they have very little chance to be used. In fact, we probably should not even touch such non-uniquely defined code area. Note that: Blocks Freq-1 and Freq-2 contain frequently used characters and symbols. Blocks Rare-1 and Rare-2 contain less frequently or rarely used characters. (We use the suffix '1' to designate the code space with more codes and '2' for the code space that has a smaller area.) 3. Escape Sequences for Switching between Different Character Sets The following escape sequences are used for switching from one character set to another and locked in that character set. The default character set is the ASCII character set. HZ Escape Sequences: (the same as in the original HZ standard) ~{ -- enters GB mode, each double-byte is regarded as a GB character ~} -- enters ASCII mode, each byte is interpreted as an ASCII character with the following two exceptions: ~~ -- represents the '~' character (0x7e) ~\n -- line continuation mark, the characters in the next line should be concatenated with the current line HZ+S Escape Sequences: (the same as in HZ+S, v. 0.6, proposal. However, they could be changed if desirable.) ~1 -- switches into Big5 block 1, each double-byte is regarded as a Big5 character in Block 1 ~2 -- switches into Big5 block 2, each double-byte is regarded as a Big5 character in Block 2 The division of Big5 codes into two blocks is essential since there are only 94x94=8836 double-byte printable ASCII characters (0x21-0x7e) available while there are more than 130K Big5 codes. We will sometimes refer to the Big5 block 1 simply as ~1 and block 2 as ~2 in the following sections. Different division of the original Big5 code space into two blocks and the mapping method from the codes in one block into the (0x21-0x7e,0x21-0x7e) code space determine the characteristics of the transformed 7-bit Big5 codes (hereinafter, Big5/7, in contrast to their 8-bit counterpart, Big5/8). Two proposals are given here based on two different criteria. 4. Mapping Method I: The B5E1 Mapping Method --- (* Based on B5E1/B5encode-1: 1994/05/02, B5encode/B5decode, Rev. 1.2) --- The first way to divide the original code space is shown as follows: original code space ==> transformed code space High Bytes a1 fe 21 7e L 40 ************************ 40 ************************ o * * * * w * Block 2 * ==> * Block 2 * * * * * B 7e ************************ 7e ************************ y a1 ++++++++++++++++++++++++ 21 ++++++++++++++++++++++++ t + + + + e + + + + s + Block 1 + ==> + Block 1 + + + + + + + + + fe ++++++++++++++++++++++++ 7e ++++++++++++++++++++++++ The codes in Block 1 have the 8-th bit set in both their high bytes and low bytes. On the other hand, in Block 2, the 8-th bit of the high bytes are set BUT the 8-th bit of the low bytes are cleared. For such partition, we can map the big5 codes into the printable area simply as follows: 1. clear all the 8-th bits of the high bytes when we encounter a big5 code. 2. clear the 8-th bits of the low bytes only for codes in Block 1 but keep them untouched if the codes belong to Block2 3. use the appropriate escape sequence to distinguish codes in block 1 and block 2 The encoding and decoding for each character requires at most two LOGICAL operations (ANDs or ORs) for clearing or setting the 8-th bits, instead of using multiplications and divisions for address calculation, or using table lookup for adding offsets in the mapping process. Therefore, the encoding and decoding is extremely simple. To implement the encoding and decoding of HZ+B5E1 scheme, we only need two simple finite state machines with a very small number of states. The main operations will be logical ANDs and ORs without using multiplications and divisions or table lookup. The computation load can thus be ignored. And switching between the original and the transformed code space can be done easily with two C statements. Two programs, called "b5encode.c" (rev. 1.2) and "b5decode.c" (rev. 1.2) are available in hermes.ee.nthu.edu.tw:/shin/hzb5encode/b5encode-1 to show the simple implementation. The sample decoder could parse a mixed ASCII/GB/Big5 text in the current decoder implementation, with characters in the GB modes pass-through to the output. At least 1M bytes of big5 files had been tested. Such a 8-bit to 7-bit mapping method is also used in some CNS or EUC-based chinese email implementation. For instance, the SunMicro's implementation of the Chinese/Japanese/Korean email transmission in its Asia Language Environment also used the same mapping process to change the 8-bit codes (Big5 or EUC) to 7-bit streams before the email is deliveried. (The EUC for chinese character set used in Taiwan has a very trivial mapping with CNS-11643/86). The internal codes (8 bit) and the exchange codes (7 bit) for CNS/11643-86 also has the same simple mapping. Therefore, we can easily modify the same b5encode and b5decode procedures to support existing implementations in news and email postings. (* The HZ/GB mapping is essentially using bit set/clear too.) The major criticism to such a division is: the switching between Big5 block 1 and block 2 will be frequent, and thus introduces more ~2 escape sequences, and a larger file size. However, for its extremely simple mapping and negligible computation load, people (software contributors, users) may find it a good choice as file size is no more a big deal with extremely cheap storage devices, high communication speeds and data compression protocols of the present day computing environments. Note that: the EUC/8 <-> EUC/7 and CNS/8 <-> CNS/7 does not have such a switching problem since all frequently used characters in EUC or CNS are already arranged in their 1st plane, and less frequently used characters are arranged in the 2nd plane. EUC/8 or CNS/8 here refers to their 8-bit version (e.g., internal code representation), and EUC/7 or CNS/7 refers to their 7-bit version. 5. Mapping Method II: The B5E2 Mapping Method --- (* Based on B5E2/B5encode-2: 1994/05/07, B5encode/B5decode, Rev. 1.4) --- Another way to map the Big5/8 code to Big5/7 code is to divide the code space according to the frequency of use into two blocks, then make the 8-bit bytes into 7-bit bytes. In the second encoding scheme, we therefore designate codes in Freq-1 and Freq-2 area as Block 1 codes, and codes in Rare-1 and Rare-2 areas as Block 2 codes. Under such circumstances, the switching between Block 1 and Block 2 will be reduced to the same amount as any other proposals which divide the Block boundaries in the same way. 5.1. Mapping of Frequently Used Characters It is easy to map codes in Freq-1 (F1) by bit-clear, and map codes in Freq-2 (F2) by adding a constant X-offset to the X-coordinate (high byte) without changing the Y-coordinate (low byte) of the code. The mapping table is shown as follows: ** Mapping for Frequently Used Chars: (note: all geometries are retained) ** F1 = Freq-1, F2 = Freq-2 High Bytes a1 c8 L 40 +---------+ o | F2 | w | | 7e +---------+ B a1 +---------+ +---------+ 21 y | F1 | => | F1 | t | | | | 40 +---------+ e | | | | | F2 | s | | | | | | fe +---------+ +---------+ 7e +---------+ 21 48 49 70 ---------------------------------------------------------- Hi Lo Hop Lop hi lo ---------------------------------------------------------- F1 a1-c8 a1-fe CLR CLR 21-48 21-7e F2 a1-c8 40-7e -58 NOP 49-70 40-7e ---------------------------------------------------------- ================================================================ Notations: Hi: Hi byte ranges (in Big5 code) Lo: Lo byte ranges (in Big5 code) hi: Hi byte ranges (in B5E2 representation) lo: Lo byte ranges (in B5E2 representation) Hop: operations on Hi bytes (Big5->B5E2) Lop: operations on Li bytes (Big5->B5E2) NOP: no operation CLR: clear the 8th bit -YY: add a negative offset 0xYY * all numbers are HEX numbers * ================================================================ 5.2. Mapping of Less Frequently or Rarely Used Characters The mapping of codes in the rarely used area is also easy. The largest code space Rare-1 (R1) can still be changed to 7-bit simply by clearing the 8th bits of the high bytes and low bytes. Also, most of the codes in Rare-2 region (i.e., the 40-column R2 region as shown in the next figure) can be changed to 7-bit by adding a constant X-offset to the X-coordinates without changing the Y-coordinates. The mapping table is shown as follows: ** Mapping for Rarely Used Chars : ** c9 f0 f1 fe 40 +------------------+ +------+ 40 | | |R3.1 | | | | | | | +------+ 5e | R2 | +------+ 5f | | |R3.2 | | | | | | | +------+ 7d 7e +------------------+ +----+++ <- R3.3 + R3.4 ( 0xfd7e, 0xfe7e ) f1 fc fd fe a1 +---------------------------+ | | | | | | | | | | | R1 | | | | | | | | | | | fe +---------------------------+ c9 fe | | V 21 2e 2f 3c 3d 48 21 +------++------++--+ 21 +---------------------------+ 21 (+--+ : R3.3) |R3.1 ||R3.2 |++ 22 | | 22 (++ : R3.4) | || | 3e | | 3f +------++------+ 3f | | 40 +------------------+ | | | | | | | | | R1 | | | | | | R2 | | | | | | | | | | | | | | | 7e +------------------+ +---------------------------+ 21 48 49 7e ---------------------------------------------------------- Hi Lo Hop Lop hi lo ---------------------------------------------------------- R1 c9-fe a1-fe CLR CLR 49-7e 21-7e R2 c9-f0 40-7e -a8 NOP 21-48 40-7e R3.1 f1-fe 40-5e -d0 -1f 21-2e 21-3f R3.2 f1-fe 5f-7d -c2 -3e 2f-3c 21-3f R3.3 f1-fc 7e -b4 -5d 3d-48 21 R3.4 fd-fe 7e -c0 -5c 3d-3e 22 ---------------------------------------------------------- Note that: the transformed X-coordinates of R1+R2 occupy all the high byte ranges of 0x21-0x7e. Therefore, we are left with 14 columns of codes in the rarely used region Rare-2 (i.e., 882 codes, BUT, including 5 columns of user defined codes). All these columns can be further divided into two small rectangular areas and one small row of codes so that they are moved to the unused part of the (0x21-0x7e,0x21-0x7e) area simply by adding a constant (X,Y) translation vector to each rectangular region. --- ** We can discard the R3.4 region, which contains only two user-defined codes in practical implementation. --- If we regard bit-clear (&0x7f or -0x80) and NOP (no operation) as two special cases for adding constant offsets, then the above mapping can be implemented with 8 (or 7) offset vectors. We can therefore map the 8-bit big5 codes into 7-bit stream without using any multiplication or division operations or resorting to any table lookup. (Note that at most 6 byte-comparisons are needed to decide which region the current code belongs to. Mostly, only about 3 comparisons are needed.) Since the major operation for the largest blocks in F1, F2, R1 and R2 is the bit-clear operation, as in the B5E1 proposal, the computation load is thus comparable with the B5E1 approach while reducing the block switching frequency to a very negligible extent. The b5encode.c and b5decode.c for this proposal is accessible from the ftp site: hermes.ee.nthu.edu.tw:/shin/hzb5encode/b5encode-2 to show the simple implementation. When choosing the 'appropriate blocks', the philosophy is to retain the geometry of the blocks of continuous codes and their sizes as closely to their BIG5 origins as possible, and the largest blocks are fit to the printable area first with the simplest possible operations. With this strategy, only those less frequently used chars, whose low bytes lie in the 0x40-0x7e range are actually divided somewhat differently from their Big5 origins. 6. Mapping Method III: The B5E3 Mapping Method (YAB5E) Another method, which retains the simplicity of encoding/decoding by bit-clear/set and removes the undesirable block switching effects, is described here as B5E3 (or YAB5E, namely, Yet Another Big5 Encoding scheme. Since the triggering idea is borrowed from Ya-Gui Wei's HZ-2, it could also be called YA-gui's value-added Big5 Encoding scheme. -^^-) The major idea is to divide the code space into two blocks so that half of the high-byte ranges (0xa1-0xcf) are designated as Block 1 and the remaining half (0xd0-0x7e) are designated as Block 2 of the Big5 code. With such an arrangement, the encoding and decoding are both easy and symmetric. The mapping tables are shown as follows. High Bytes a1 cf L 40 +---------+ o | F2+ | w | | 7e +---------+ B a1 +---------+ +---------+ 21 y | F1+ | => | F1+ | t | | | | 40 +---------+ e | | | | | F2+ | s | | | | | | fe +---------+ +---------+ 7e +---------+ 21 4f 50 7e ---------------------------------------------------------- Hi Lo Hop Lop hi lo ---------------------------------------------------------- F1+ a1-cf a1-fe CLR CLR 21-4f 21-7e F2+ a1-cf 40-7e -51 NOP 50-7e 40-7e ---------------------------------------------------------- High Bytes d0 fe L 40 +---------+ o | R2+ | w | | 7e +---------+ B a1 +---------+ +---------+ 21 y | R1+ | => | R1+ | t | | +---------+ 40 | | e | | | R2+ | | | s | | | | | | fe +---------+ +---------+ 7e +---------+ 7e 21 4f 50 7e ---------------------------------------------------------- Hi Lo Hop Lop hi lo ---------------------------------------------------------- R1+ d0-fe a1-fe CLR CLR 50-7e 21-7e R2+ d0-fe 40-7e -af NOP 21-4f 40-7e ---------------------------------------------------------- For codes whose low bytes lie in the range of 0xa1-0xfe (F1+ and R1+), the 8-th bit of their high bytes and low bytes are cleared during the encoding process. For codes with low bytes in the range of 0x40-0x7e (F2+ and R2+), their high bytes are shifted by adding a constant offset so that they fall into the remaining half of the printable ASCII area. The offsets for Block 1 and Block 2 codes are -0x51 and -0xaf respectively. The low bytes need no change since they are already within 0x21-0x7e. The decoding is also easy and symmetric. For the left half of the Block 1 codes in the printable ASCII area (0x21-0x4f) and Block 2 codes in the right half (0x50-0x7e), the 8-th bit of their high bytes and low bytes are set to '1'. For Block 1 codes in the right half (0x50-0x7e) and Block 2 codes in the left half (0x21-0x4f), the high bytes are shifted by adding a constant offset (+0x51 and +0xaf, respectively) without changing the low bytes. The sample implementation of the b5encode.c and b5decode.c for this proposal is accessible from the ftp site: hermes.ee.nthu.edu.tw:/shin/hzb5encode/b5encode-3 to show the simple implementation. The encoding/decoding process is almost as easy as for B5E1, except for different numbers of range checking. (In B5E1, we check whether a code belongs to the upper half or lower half; in B5E3, we also check whether a code belongs to the right half or left half to decide the mapping operation: bit-clear or byte offset.) In the current arrangement of block boundary, all frequently used characters plus several columns of less frequently used characters (with high byte in the range of 0xc9-0xcf) are designated as Block 1. The remaining less frequently used characters are designated as Block 2. As a result, as indicated by Ya-Gui, the switching operations is even less than the case where the block boundary is defined at the start of the less frequently used characters. (This fact is also observed in a test over about 1M big5 files.) 7. Pros and Cons There are major criticisms on not preserving the sorting sequence of the original codes EXPLICITLY in B5E2. Since the current design is aimed at acting as a code exchange media, not for information processing directly over the 7-bit exchange codes (especially for the computation environments of the Big5 users), the chances to use such sorting sequence is close to zero. (In fact, it is almost equally true in the original Big5 code space for using such sorting sequence.) Even in the very rare cases, the switching between the two code spaces could be done within bit or byte changes. Therefore, it means little for designing a mapping scheme. There are also concerns for future ISO-2022 compatibility in terms of the division of the Big5 code. However, the designation of which characters as block 1 chars and which characters as block 2 chars are not yet decided for Big5 except for some private designations. So it will probably depend on what division is widely accepted after the Big5/7 standard is settled by the chinese computing societies. The major discussions on the advantages or disadvantages of the above proposals are accessible at: hermes.ee.nthu.edu.tw:/shin/hzb5encode/doc/disc.b5e Since B5E3 is new to the discussion list, further comments from all parties will be added to the file. Other discussions on 7-bit representations of Big5 code are also available in the same directory. You can read them as ordinary emails, say, by using 'mail -f disc.b5e' in the UNIX environment for all such files. ----------------------------------------------------------------------- Natural Language Processing Lab Department of Electrical Engineering Jing-Shin Chang National Tsing-Hua University shin@hermes.ee.nthu.edu.tw Hsinchu, Taiwan 30043, R.O.C. ----------------------------------------------------------------------- From shin@hermes.ee.nthu.edu.tw Sat Jun 11 07:28:27 1994 Return-Path: Received: from nthu.edu.tw by ifcss.org (4.1/IFCSS-Mailer) id AA02981; Sat, 11 Jun 94 07:28:27 CDT Received: from hermes.ee.nthu.edu.tw ([140.114.28.27]) by nthu.edu.tw with SMTP id <181965(2)>; Sat, 11 Jun 1994 20:32:04 +0800 Received: by hermes.ee.nthu.edu.tw (4.1/SMI-4.0/shin-1.0) id AA13294; Sat, 11 Jun 94 20:24:22 CST Date: Sun, 12 Jun 1994 10:24:22 +0800 From: shin@hermes.ee.nthu.edu.tw (Jing-Shin Chang) Message-Id: <9406111224.AA13294@hermes.ee.nthu.edu.tw> To: cyl@ifcss.org, handa@etl.go.jp, soft-authors@ifcss.org Subject: Re: new solution: approve both HZ+ and ISO2022 Status: RO Nelson, I am especially admire of your international view. I think Ross had also pointed this out. And the Chinese people should not only limit our views in the Chinese language only. I think many of the soft-authors are qualified in writing international grade software to support what we need, no matter how slow this step may go. If we do not go a step further, we may always follow MS, IBM, Apple's policies. Shin -^^- |> I've a suggestion for 7-bit transmission. How about soft-authors give |> their stamp of approval to an HZ+ standard to fulfill the low road of |> popular defacto standard like BIG5 and also the existing HZ standard, and |> also give their stamp of approval to ISO2022 standard of 7-bit |> transmission, and we can use ifcss.org:/software/unix/convert/b5cns.tar.gz |> as an example, to mobilize users into International arena of the high road. |> |> This way, the passage of time and as users see post in ISO format they |> will ask what's that junk and will get the required conversion software |> or will ask their vendor to support it in their software, and vendors |> will more likely willing to support ISO standards as it'll be upcompatible |> with other iso approved character codes. |> |> |> It's important that soft-authors give stamp of approval to both |> short-term and long term goals, and allow users to post/email freely in |> both codes, and the passage of time will bring about the transition. |> We should start the transition stage now as we introduce 7-bit exchange |> code for mixed ascii/big5/gb. We must not just look at the here and the |> now, we must provide a way to transition into the future and be an |> encouragement to users to take that path, so we therefore should not pick |> one or the other, but give users the choice to transition, with the |> backing of soft-authors. |> |> The goal of transmitting in 7-bit code is still accomplished by stamping |> approval for both HZ+ and ISO-2022 (eg, b5cns, and maybe someone can |> come up with a 7-bit GB convertor that's iso compliant), this will make |> it alot easier for internationized software to support chinese in the |> future. |> |> Nelson Chin |> From haa@unda.fi Sat Jun 11 07:42:12 1994 Return-Path: Received: from goliath.unda.fi by ifcss.org (4.1/IFCSS-Mailer) id AA03246; Sat, 11 Jun 94 07:42:12 CDT Received: from unda.fi (titan.unda.fi) by goliath.unda.fi (5.65c/93.6.3-MAI) id AA05892; Sat, 11 Jun 1994 15:39:45 +0300 Received: by unda.fi (4.1/MAI-94.2.12) id AA28941; Sat, 11 Jun 94 15:40:03 +0300 Date: Sat, 11 Jun 94 15:40:03 +0300 Message-Id: <9406111240.AA28941@unda.fi> From: haa@cs.hut.fi (Hannu Aronsson) Sender: haa@unda.fi To: simpson@math.psu.edu Cc: soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu Subject: Re: cleaning up Big5 isn't HZ+'s job Status: O Well, as I'm only on the Mule list, so I won't see what will happen in the future about this. I hope something good would come out, anyway. > My guess is that many users don't care. For users who don't care, > forcing them to designate a Big5 version is imposing an extra burden. The *users* don't even have to know that there is some kind of Big5 version information in the coding. Only the programmer porting the conversion to system X needs to set one setting according to what Big5 version the system X software or platform is using. You probably have some "preamble" and "postamble" codes to designate Big5 data beginning and ending. One very easy solution would just be to have several preamble codes, each also designating the version if Big5 used in the following data (and one for "unspecified Big5 data"). Simple readers might then just handle all these few preambles as one, but the better software would be able distinguish between them and do any conversions needed for the vendor specific part. Without version tagging, it will be *impossible* to make any program even try to do the right thing. > Users who care about vendor-specific Big5 codes are always free to > designate this in an informal way, or to agree among themselves. But this creates the problem that most users data won't have any clue to whatever version of Big5 it is in, because most users won't use any external tagging mechanism anyway. This is mostly because I would guess not many Big5 users even know or care about this problem, until it hurts them (some wrong characters appear in an important document). I would still argue that there is no additional burden to the user, and only very very small additional work to any programmer implementing HZ+S. > This has nothing to do with HZ+. The Big5<->HZ+ translation algorithm > doesn't depend on which Big5 version you are using. Probably you won't need anything in the translation algorithm itself, just in the preamble or something like that... Almost no complexity added. > There is no > reason for HZ+ to force the user to specify the Big5 version. Again, the user doesn't have to care, the programmer providing the conversion software for system X knows what Big5 version is being used on that platform, and make the conversion output the necessary information. > I think HZ+ will make the world a better place, provided HZ+ is > successful. But HZ+ will not be successful if we burden it with a lot > of irrelevant baggage. But HZ+ will not be successful, if it continues to provide people with partly corrupt data without any clue to what might be the right version of Big5 to use with any piece of HZ+ data. > I hope you can sympathize and understand what I am trying to > accomplish. I can see that you want to avoid eny extra complexity. But worrying about the cost of almost no additional complexity, you're also passing the chance to make most 7-bit Big-5 communication better than what we have today. Also, if you want to push HZ+S instead of the current mess, you should have some positive arguments why yours is better. Providing means to help people avoid vendor-space corruption would be a BIG drive for your encoding, I think. Files which look mostly OK (common part of any standard) with only some small percentage of corrupted data (vendor/system specific part) are the worst kind of data corruption, as they look OK to the casual reader. You probably have never been hurt by that kind of data corruption in any important way. Tagging all data with the exact code set used if the only way to avoid this problem. I guess you have already decided against making any improvements in this with your encoding, so we've lost this chance too to make any progress. If everybody had this kind of attitude, we would never get any progress. I know it's not your fault the Big5 arena is such a mess, but it strikes odd to me to also see that there is no desire for improvement in the Big5 world. But then, you're in the USA, the leading provider of anti-internationalized software in the world. Maybe the stereotype of US programmers having no clue about the real problems of the "outside world" is right, after all... I guess we should start developing standards and conventions which are not based on US-ASCII, to force everybody to learn how to do things right ;-) If/when the HZ+S specification if finished some day, I'd like to see it, anyway. Maybe someone could then develop a HZ+S+ based on it, with version tagging, and start pushing that instead ;-) I apologize for the slightly over-flamey style of this e-mail. It just seems that, once again, a new encoding is being created, but no real progress is going to be made. Yours, Hannu -- Hannu.Aronsson@cs.hut.fi Helsinki University of Technology, Finland From simpson@math.psu.edu Sat Jun 11 08:40:18 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA04615; Sat, 11 Jun 94 08:40:18 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id JAA19768; Sat, 11 Jun 1994 09:38:02 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Sat, 11 Jun 1994 09:38:01 -0400 Date: Sat, 11 Jun 1994 09:38:01 -0400 Message-Id: <199406111338.JAA06261@boole.math.psu.edu> To: soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu, handa@etl.go.jp Subject: new solution: approve both HZ+ and ISO2022 In-Reply-To: References: Reply-To: simpson@math.psu.edu Status: RO Nelson CHIN writes: > I've a suggestion for 7-bit transmission. How about soft-authors give > their stamp of approval to an HZ+ standard to fulfill the low road of > popular defacto standard like BIG5 and also the existing HZ standard, and > also give their stamp of approval to ISO2022 standard of 7-bit > transmission, and we can use ifcss.org:/software/unix/convert/b5cns.tar.gz > as an example, to mobilize users into International arena of the high road. I applaud and support this suggestion. This was exactly the point of my posting in which I introduced the terms "low road" and "hig road". The low road means immediate, practical solutions based on popular standards such as HZ and Big5. The high road means long term, multilingual solutions based on official standards such as ISO 2022 and CNS. Chinese computing needs both kinds of solutions. I have called for a vote on HZ+ mapping schemes. I hope everybody on soft-authors will vote. I think we can reach a consensus on HZ+ very soon now. I hope that this can be our low road solution for the immediate future. At the same time, I am very eager to promote and support any reasonable high road consensus for Chinese computing. Ross's idea looks very good to me. Mule looks very good to me. I hope that Ross or somebody else will take the lead. I will support this effort in any way I can. -- Steve Stephen G. Simpson Department of Mathematics, Pennsylvania State University 333 McAllister Building, University Park, State College, PA 16802 Office phone +1 814 863-0775 Home phone +1 814 238-2274 E-mail simpson@math.psu.edu FAX +1 814 865-3735 World Wide Web http://www.math.psu.edu/simpson/ From lee@simd.stanford.edu Sat Jun 11 13:38:32 1994 Return-Path: Received: from simd.stanford.edu by ifcss.org (4.1/IFCSS-Mailer) id AA13202; Sat, 11 Jun 94 13:38:32 CDT Received: by simd.stanford.edu (5.65/inc-1.0) id AA29339; Sat, 11 Jun 1994 11:36:13 -0700 Date: Sat, 11 Jun 1994 11:36:13 -0700 From: lee@simd.stanford.edu (Fung Fung Lee) Message-Id: <9406111836.AA29339@simd.stanford.edu> To: soft-authors@ifcss.org, handa@etlken.etl.go.jp In-Reply-To: Kenichi Handa's message of Sat, 11 Jun 94 10:19:45 JST <9406110119.AA23481@etlken.etl.go.jp> Subject: Let's discuss HZ+ on alt.chinese.computing (Was: HZ+ etc.) Status: O >Date: Sat, 11 Jun 94 10:19:45 JST >From: handa@etlken.etl.go.jp (Kenichi Handa) > ... >OR, how about moving the discussion to alt.chinese.software? ^^^^^^^^^^^^^^^^^^^^ alt.chinese.computing? >I just now found some mails on this thread are also posted >to the newsgroup. I may have missed some articles related >on this thread in the newsgroup. With the recent creation of the newsgroup alt.chinese.computing, why don't we use it as the ONLY public discussion platform for issues such as HZ+? In fact, I've seen that some recent postings to soft-authors were already cross-posted to alt.chinese.computing. The practical matter is that my mailbox (and perhaps mailboxes of many others) has been flooded with HZ+ related messages. Just for the first 10 days in June, there are about 70 to 80 (?) such messages, about 40 from Stephen G Simpson alone. I remember seeing one or a few persons started "unsubscribing" from soft-authors recently. I strongly suspect there is a cause-and-effect relation there. The mailing list soft-authors@ifcss.org was created originally for the discussion of HBF (Hanzi Bitmap Font File Format) in early 1993. It started as a small group, and since then has grown to 59 members as of this morning. I think, soft-authors at this size is no longer effective as a small group discussion forum. From now on, I think it would be nice to send mail to soft-authors about announcement, such as "invitation to join HZ+ discussions on alt.chinese.computing", but NOT the actual discussions. Since there are many threads of discussions going on alt.chinese.computing, as a convention we may use "HZ+: " as a prefix of the subjects of the articles to be posted that are related to HZ+. In case some people on the current soft-authors list cannot read alt.chinese.computing but wish to continue read and/or participate in the discussions, one possible solution is to create a separate mailing list for them. (Perhaps sending such requests to xiaofei@ifcss.org?) Of course, it may not be necessary at all. Best regards, Fung Fung Lee From lee@simd.stanford.edu Sat Jun 11 16:23:40 1994 Return-Path: Received: from simd.stanford.edu by ifcss.org (4.1/IFCSS-Mailer) id AA18338; Sat, 11 Jun 94 16:23:40 CDT Received: by simd.stanford.edu (5.65/inc-1.0) id AA00184; Sat, 11 Jun 1994 14:21:30 -0700 Date: Sat, 11 Jun 1994 14:21:30 -0700 From: lee@simd.stanford.edu (Fung Fung Lee) Message-Id: <9406112121.AA00184@simd.stanford.edu> To: soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Sat, 11 Jun 1994 00:34:06 -0400 <199406110434.AAA05351@boole.math.psu.edu> Subject: call for a vote on HZ+ mapping schemes Status: RO >From: Stephen G Simpson >Date: Sat, 11 Jun 1994 00:34:06 -0400 >Reply-To: soft-authors@ifcss.org > >Recently there has been nothing new on the 7-bit Big5 mapping scheme ?????? ??? >issue. I would therefore like to call for a formal vote within the >soft-authors@ifcss.org mail alias. > >HZ+ is defined as HZ plus some form of 7-bit Big5. One of the issues >that we have discussed extensively is 7-bit Big5 mapping schemes, >i.e. how shall HZ+ map Big5 into the graphical ASCII code space. The >purpose of the vote is to decide this issue. > >There are two proposals under consideration: > >A. my proposal, HZ+S. > >B. Ya-Gui Wei's proposal, HZ-2. > >Please note that this vote concerns only 7-bit Big5 mapping schemes. >Another outstanding issue is which tilde escape sequences to use, but >this will be decided separately. I think we can't separate the two issues, namely, choice of escape sequences and code point remapping. In my previous message that attempted to "rate" various proposals, the "score" is slightly in favor of HZ+S over HZ-2, because HZ+S allows a tiny advantage in the choice of escape sequences, that is, ~x, ~y (vs. Simpson's ~1 and ~2 or even ~< and ~>). Given the possibility of accomodation of CNS, the plane 1 characters of which occupy 0x2121 to 0x7D4B, it is no longer possible to use ~x, ~y for CNS anyway. Since it is still possible to search for beginning of escape sequence with regular expressions, now I think that the previously tiny advantage becomes even more tiny. In short, it's a tie between HZ+S and HZ-2 using my previous criteria (plus this slight update). In contrast, it now appears that the "maximal ASCII transparency principle" as explained by Ricky Yeung is much more important. Yet none of the above two proposals embody the proposal. I can't vote or endorse a part of one proposal with the uncertainty that I may not get the other parts I like. Anyway, one of the purpose of this message is that you CAN'T judge (and then even vote) on a proposal by just looking at its individual pieces instead of the whole. >If you receive this mail, you are on the soft-authors alias. Could >everybody who receives this mail please vote? The way to vote is to >reply to soft-authors@ifcss.org, declaring your preference for option A >or option B. > >I urge everybody to vote. By voting, you are helping us to reach >a consensus on HZ+. Such a consensus is urgently needed. > >The deadline for votes is June 15. > >-- Steve Apparently no consensus has been reached and will be reached in the near term, even among the active members of soft-authors, let alone among the much larger Chinese computing communities. At the last call for vote, myself and 7 (?) others voted to put aside CNS for the time being. Then I began to see statements such as "the soft-authors has voted to put aside CNS ..." This is unfair. Only 8 out of 59 soft-authors members is by no means a majority. Strictly speaking, even if 58 out of 59 members voted yes to some proposal, it would still be unfair to the one who voted no or absent, if statements such as "the soft-authors ..." were used against him/her. This is because of the informal non-bidding ad-hoc nature of soft-authors list. It's just a mailing list. It's not Congress. As this message regards a call for vote specific to soft-authors, I have to reply to soft-authors rather than to alt.chinese.computing. >From now on, regarding issues related to HZ+, please don't expect my response to soft-authors@ifcss.org. For those who want to continue, let's move on to alt.chinese.computing. Fung Fung Lee From simpson@math.psu.edu Sat Jun 11 17:39:05 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA20240; Sat, 11 Jun 94 17:39:05 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id SAA21480; Sat, 11 Jun 1994 18:36:53 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Sat, 11 Jun 1994 18:36:52 -0400 Date: Sat, 11 Jun 1994 18:36:52 -0400 Message-Id: <199406112236.SAA07027@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: call for vote is withdrawn In-Reply-To: <9406112121.AA00184@simd.stanford.edu> References: <199406110434.AAA05351@boole.math.psu.edu> <9406112121.AA00184@simd.stanford.edu> Reply-To: simpson@math.psu.edu Status: RO Fung posted in soft-authors raising various technical and political objections to my call for a vote on HZ+. I don't completely understand Fung's reasoning, but it's enough to convince me that we had better postpone a vote for at least a couple more days. I therefore now withdraw my call for a vote. Fung Fung Lee writes: > I think we can't separate the two issues, namely, choice of escape > sequences and code point remapping. ... Some time ago I proposed separating these two issues, and nobody objected. However, Fung has now come up with some plausible technical reasons for considering them together, and Ricky Yeung's ideas on maximal ASCII transparency raise some interesting possibilities. I still want to have a vote on HZ+, but maybe we need to rethink the options. Fung, from your perspective, what would you suggest as a list of options to vote on? > Apparently no consensus has been reached and will be reached in the > near term, even among the active members of soft-authors, let alone > among the much larger Chinese computing communities. I am still holding out hope that we can reach some sort of HZ+ consensus in soft-authors, or maybe in ccnet-l. Fung, are you opposed to this? Or are you perhaps saying it can't be done? Why? > At the last call for vote, myself and 7 (?) others voted to put aside > CNS for the time being. The actual count was 11 for putting aside CNS for the time being, 1 for supporting CNS and dropping Big5 (that was Ed Lai's vote), total 12 votes. The list of voters included most of the people who have spoken up recently on soft-authors. > Then I began to see statements such as "the soft-authors has voted > to put aside CNS ..." This is unfair. Only 8 out of 59 > soft-authors members is by no means a majority. Everybody on soft-authors received the call for votes. 11 out of 12 is a majority of those who voted. I think it's fair to say that this decision was the outcome of the vote. I posted this outcome at the time, and nobody objected. Why are you objecting now? If somebody has better ideas about how to reach a consensus in soft-authors, please suggest it. I'd be happy if someone else wants to take the lead here in calling for votes or whatever. I don't have any wonderful ideas about parliamentary procedure. My only desire is to reach some sort of consensus on HZ+. > Strictly speaking, even if 58 out of 59 members voted yes to some > proposal, it would still be unfair to the one who voted no or > absent, if statements such as "the soft-authors ..." were used > against him/her. This is because of the informal non-bidding > ad-hoc nature of soft-authors list. It's just a mailing list. It's > not Congress. I'm not sure I understand. If a majority of those voting vote for a certain outcome, don't you regard this as significant? Those who don't express an opinion presumably don't have one, so why is it unfair to them? Furthermore, the vote on CNS has not been used against any person, merely against the idea that HZ+ should support CNS. This was anyway the outcome of the vote, so why do you think it's objectionable? With your kind of criterion, it would seem impossible to reach any kind of consensus by voting, so long as even one person remains silent. Is this the conclusion you want? > From now on, regarding issues related to HZ+, please don't expect > my response to soft-authors@ifcss.org. I'm somewhat mystified. I guess Fung is saying that he plans to boycott all HZ+ votes and discussions in soft-authors. If everyone does this, then obviously we can't reach any sort of HZ+ consensus in soft-authors. Is this what Fung wants? -- Steve From jiang Mon Jun 13 09:30:17 1994 Return-Path: Received: by ifcss.org (4.1/IFCSS-Mailer) id AA25782; Mon, 13 Jun 94 09:30:17 CDT Date: Mon, 13 Jun 94 09:30:17 CDT From: jiang (Yuan Jiang) Message-Id: <9406131430.AA25782@ifcss.org> To: soft-authors Subject: Re: call for vote is withdrawn > > Apparently no consensus has been reached and will be reached in the > > near term, even among the active members of soft-authors, let alone > > among the much larger Chinese computing communities. > >I am still holding out hope that we can reach some sort of HZ+ >consensus in soft-authors, or maybe in ccnet-l. Fung, are you opposed If you think a consensus could be reached, what's the need for voting? >The actual count was 11 for putting aside CNS for the time being, 1 >for supporting CNS and dropping Big5 (that was Ed Lai's vote), total >12 votes. The list of voters included most of the people who have This is not what Ed voted for. He voted for something out of his own list. To say the least, it's not a valid vote for your list. >certain outcome, don't you regard this as significant? Those who >don't express an opinion presumably don't have one, so why is it Maybe they boycotted the voting. I did. It would be funny to say that Yagui and Jing-Shin don't have opinions. I did not call for others to boycot, but I will when you resume your call for a vote a couple of days later. Soft-authors list was not created to vote on something. There is no rule saying whether a voting with 1/5 participation is valid. Our common sense tells us that its result is not significant. >unfair to them? Furthermore, the vote on CNS has not been used >against any person, merely against the idea that HZ+ should support >CNS. This was anyway the outcome of the vote, so why do you think >it's objectionable? You played a trick in calling for the vote and in interpreting the meaning of the voting. I pointed out that there are two issues 1) which internal code to be represented (supported), 2) which exchange code to be used to represent the internal code. You admitted that theses were separate issues and would be voted on separately. Your previous call for vote could only be on 1) since "supporting Big5" is mentioned. Yet, you interpret the result of the voting to cover issue 2) -- dropping the consideration of CNS as an exchange code. As a mathematician, such a twist of logic can only be an intentional trick. Yuan >I called for a vote in soft-authors@ifcss.org on whether HZ+ will >support CNS. The options were: > >A. Add CNS support, in addition to the existing Big5 support. > >B. Drop Big5 support. Support CNS instead. > >C. Postpone a decision on CNS for the time being. > >-- Steve > > From jiang Mon Jun 13 09:59:30 1994 Return-Path: Received: by ifcss.org (4.1/IFCSS-Mailer) id AA26853; Mon, 13 Jun 94 09:59:30 CDT Date: Mon, 13 Jun 94 09:59:30 CDT From: jiang (Yuan Jiang) Message-Id: <9406131459.AA26853@ifcss.org> To: ccnet-l@uga.uga.edu, shin@hermes.ee.nthu.edu.tw, soft-authors Subject: Re: My A-Plan My opinion is in line with Jing-Shin's -- spend the time on applications instead of debates. Let you software to say what you mean. Jing-Shin raised a valid issue -- the interference of the escape code -- which is outside of the debate of HZ+. It is actually an issue concerning the implementation of HZ to start with. In principle, HZ as an exchange code is only to go thru the network system and should not be dealt with by end users. But I don't think it's possible to avoid this. ZWDOS and HZTTY all allow users to deal with HZ directly. It's convenient to be able to use your unix vi to edit HZ file under HZTTY. It's simple enough for GB users to determine where to insert ~{ and ~}. However, the plain arrangement of Big5 may discourage Big5 users to work on HZ+ directly. It's not simple to determine where to use ~1 and ~2 (or x and y or G and H, to the effect). But I don't know whether this is enough to discourage zwdos and hztty to allow end users to deal with the exchange code. One middle road is to hide the escape code when the text is viewed but not edited. This avoids messing up the number of charaters in a line. However, when the text is edited, the user has to know where the escape codes are. Yuan From simpson@math.psu.edu Mon Jun 13 10:51:51 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA29052; Mon, 13 Jun 94 10:51:51 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id LAA03530; Mon, 13 Jun 1994 11:49:43 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Mon, 13 Jun 1994 11:49:42 -0400 Date: Mon, 13 Jun 1994 11:49:42 -0400 Message-Id: <199406131549.LAA14643@boole.math.psu.edu> To: soft-authors@ifcss.org Subject: Re: call for vote is withdrawn In-Reply-To: <9406131430.AA25782@ifcss.org> References: <9406131430.AA25782@ifcss.org> Reply-To: simpson@math.psu.edu I wrote: > >I am still holding out hope that we can reach some sort of HZ+ > >consensus in soft-authors, or maybe in ccnet-l. Fung, are you opposed Yuan Jiang writes: > If you think a consensus could be reached, what's the need for voting? My hope is that voting will help us to reach a consensus, by showing which version of HZ+ is favored by the people who are interested. I wrote: > >The actual count was 11 for putting aside CNS for the time being, 1 > >for supporting CNS and dropping Big5 (that was Ed Lai's vote), total > >12 votes. Yuan Jiang replied: > This is not what Ed voted for. > He voted for something out of his own list. > To say the least, it's not a valid vote for your list. Sorry, excuse my ignorance, I'm not aware of Ed's list. Did he post it on soft-authors? On a.c.c? In any case, Yuan is in error. Ed did indeed vote for option B, to drop Big5 support and support CNS instead. From Ed's e-mail: > Indeed, I would vote for option B. If Mr. HANDA, who first came up > with Mule Big5, now realize CNS is a better idea and plan to replace > Mule Big5 with CNS. I should do the same. Yuan writes: > You played a trick in calling for the vote and in interpreting the meaning > of the voting. I pointed out that there are two issues 1) which internal > code to be represented (supported), 2) which exchange code to be used to > represent the internal code. You admitted that theses were separate issues > and would be voted on separately. Your previous call for vote could > only be on 1) since "supporting Big5" is mentioned. Yet, you interpret the > result of the voting to cover issue 2) -- dropping the consideration of > CNS as an exchange code. As a mathematician, such a twist of logic > can only be an intentional trick. This argment of Yuan's is too vague for me to untangle. I'll only say that I did not intend any trick. I explained the options and issues as clearly as I was able. If anyone had any objections to my explanation or my list of options, I allowed plenty of time for objections to be raised. Nobody raised any objections until after the outcome of the vote was clear. It seems to me that the CNS fans are unhappy with the outcome of the vote, so they are trying to argue that the vote was somehow invalid. -- Steve From simpson@math.psu.edu Mon Jun 13 11:26:27 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA00514; Mon, 13 Jun 94 11:26:27 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id MAA03774; Mon, 13 Jun 1994 12:24:18 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Mon, 13 Jun 1994 12:24:16 -0400 Date: Mon, 13 Jun 1994 12:24:16 -0400 Message-Id: <199406131624.MAA14703@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: tradeoffs in HZ+ implementation In-Reply-To: <9406131459.AA26853@ifcss.org> References: <9406131459.AA26853@ifcss.org> Reply-To: simpson@math.psu.edu Yuan Jiang writes: > My opinion is in line with Jing-Shin's -- spend the time on applications > instead of debates. Let you software to say what you mean. I also think that the debates have been largely a waste of time (despite the fact that some interesting facts have emerged from them). I have been wanting to spend a little time working up Emacs Lisp news and mail interfaces for HZ+, but I have been too busy answering flames from Ed, Jing-Shin, and you. One of the things that spoiled the CNS debate was that people like Ed, who want HZ+ to "use CNS as the exchange code for Big5", never bothered to put forward a specific proposal or to write any demonstration software. I wrote and announced some software to show how CNS support could be added to HZ+. Unfortunately, Ed and company never bothered to look at this software. > Jing-Shin raised a valid issue -- the interference of the escape code -- > which is outside of the debate of HZ+. It is actually an issue concerning > the implementation of HZ to start with. This issue is indeed valid. As I also pointed out, there are various tradeoffs involved. The trouble started because Jing-Shin raised this issue in a phony way, in his "ONE problem with HZ+" posting, by declaring that HZ+ would insert ~2 in front of common Chinese characters in the middle of most lines of Big5 text. This statement of Jing-Shin's is absolutely false. This alleged problem is a problem only for Jing-Shin's B5E1. It is not a problem for HZ+. It's interesting to note that Jing-Shin has not yet bothered to retract his false statement, which unfairly slanders HZ+. > In principle, HZ as an exchange code is only to go thru the network system > and should not be dealt with by end users. But I don't think it's possible > to avoid this. ZWDOS and HZTTY all allow users to deal with HZ directly. > It's convenient to be able to use your unix vi to edit HZ file under HZTTY. > It's simple enough for GB users to determine where to insert ~{ and ~}. > > However, the plain arrangement of Big5 may discourage Big5 users to work > on HZ+ directly. It's not simple to determine where to use ~1 and ~2 > (or x and y or G and H, to the effect). Usually you will use ~1. This is because all of the more frequently used characters, as well as the standard Big5 symbols and alphabets, are in block 1. > But I don't know whether this is enough to discourage zwdos and hztty to > allow end users to deal with the exchange code. One middle road is > to hide the escape code when the text is viewed but not edited. This > avoids messing up the number of charaters in a line. However, when > the text is edited, the user has to know where the escape codes are. This is one of the tradeoffs I was referring to. The simplest interim solution is to use a plain ASCII text editor to edit HZ+ files directly, if this is desired. If you want something a little more convenient, you could for instance set up Emacs to switch back and forth at will between two modes, one in which the escape sequences would be hidden and one in which they would be visible. Thank you for now doing what Jing-Shin did not do, namely to attempt to discuss this issue in a sensible, balanced way. -- Steve From cyl Mon Jun 13 12:32:27 1994 Return-Path: Received: by ifcss.org (4.1/IFCSS-Mailer) id AA02929; Mon, 13 Jun 94 12:32:27 CDT Date: Mon, 13 Jun 1994 12:32:26 -0500 (CDT) From: Nelson CHIN Subject: use acc for HZ+ discussions; my call for CONSENSUS To: soft-authors@ifcss.org Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII i also support using alt.chinese.computing for HZ+ discussions, and other discussions esp if there's going to be alot of posting. a newsgroup is ideal for this and no one's going to complain if there's lots of discussion in a newsgroup. If you site doesn't carry acc, ask you news admin, and you can set your NNTPSERVER to aidebbs.edu.tw for now. it allows both reading and posting, and you can also check out the rest of the tw.bbs.* heirarchy if you find any interesting bbs's. on another note, i would like to know if there's a consensus for ISO-2022-JP (which is an official 7-bit transmission of multilingual text and is also MIME-registered for emailing) as the high road? I would also like to know if there's a consensus to introduce ISO-2022-JP now to start the transistion onto the high road. practically: 1) let chinese.* know that we have consensus to encourage ISO-2022-JP postings in chinese.* starting now to start transitioning into the high road. HZ+ will be introduced along side with ISO-2022-JP, onto chinese.* once a consensus of some sort is reached in soft-authors. soft-authors has reached consensus that chinese users should post test articles in their favorite 7-bit encoding in chinese.test. This is invaluable input for soft-authors as to what chinese users wants for 7-bit encoding. They may also post in 8-bit encoding in chinese.test, if they prefer to have 8-bit mirrors for chinese.* newgroups. 2) let chinese and international software authors know that soft-authors has consensus to now begin the high road to iso-2022-jp for 7-bit transmission of chinese text which is compatible with any other language text. 3) soft-authors has reached consensus that HZ+ is only a temporary solution for 7-bit international communication, to accomodate the existing popular 7-bit GB standard (HZ), and to accomodate BIG5 coexisting with HZ. note: i know some of you want to directly jumped onto the highroad, without accomodating BIG5 with the existing HZ standard. But may i suggest that you instead just give the approval for an HZ+ standard and let those who's interested in creating this HZ+ to create it, and you put your effort into advocating ISO-2022-JP to chinese users to move onto the high road. As long as everyone has consensus that we want to reach the high road, then each person can play a role along that road to move users to the right destination. If you feel you're only interested in ISO-2022-JP, then stand on the road and cheer the users to continue walking. If you feel like providing HZ+, then provide it to the users at the start line, and make sure you tell them that this is just something to help you start on your journey, and we at soft-authors will like to make this journey as most comfortable a journey as possible. If anybody on soft-authors has anything to disagree upon, let me know and i'll remove anything i proposed as our consensus. Even if this boils down to one line of consensus, at least we found a common ground from which to based our discussions on and stop wasting our time in debating out of whatever that's fueling us to debate, and which comes down to nothing. Nelson Chin From butta1@cs.bu.edu Mon Jun 13 13:06:01 1994 Return-Path: Received: from cs.bu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA04247; Mon, 13 Jun 94 13:06:01 CDT Received: from csa.bu.edu by cs.bu.edu (8.6.4/Spike-2.1) id OAA28830; Mon, 13 Jun 1994 14:03:09 -0400 Received: from localhost by csa.bu.edu (8.6.4/Spike-2.1) id OAA14984; Mon, 13 Jun 1994 14:03:41 -0400 Date: Mon, 13 Jun 1994 14:03:41 -0400 (EDT) From: C I I Project Sender: C I I Project Reply-To: C I I Project Subject: demo at UN To: soft-authors@ifcss.org, handa@etl.go.jp Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII I'll be at United Nations,NYC, leaving tomorrow nite, to help demo a networking proposal for the Global Environmental Facility (GEF) geared for China, (gopher ftp.worldbank.org for more info on GEF) and depending on what computer system they have for me to demo on, i will need to use some programs to emulate chinese. I think for X-windows, cxterm will be ideal, and PC's i think ZWdos will be good. I'll be using HZ code, but i am really interested telling the china specialists at UN about what chinese networking community has plans for, esp what soft-author is up to. PLease do tell me if my previous emailing about reaching a consensus about how everything comes together for the highroad sounds ASAP, of which i'll bring up during the demo. And i can share their feedback to you guys also once i come back on thursday nite. Nelson Chin From Ricky.Yeung@Eng.Sun.COM Mon Jun 13 13:10:17 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA04393; Mon, 13 Jun 94 13:10:17 CDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA09056; Mon, 13 Jun 94 11:07:14 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA09625; Mon, 13 Jun 94 11:07:52 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA10972; Mon, 13 Jun 1994 11:08:31 +0800 Date: Mon, 13 Jun 1994 11:08:31 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406131808.AA10972@hk.Eng.Sun.COM> To: soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu, handa@etl.go.jp Subject: A framework for extending HZ (Draft) Content-Length: 6300 [ Here is a draft of my proposal to extend HZ to handle non-ASCII character code sets. I agree that further discussion related to HZ/HZ+ should be moved to alt.chinese.computing. -Ricky ] An Extension to HZ - A Framework for 7-bit Encoding Scheme for Arbitrarily Mixed ASCII and Non-ASCII characters (Preliminary Draft) Ricky Yeung* June 12, 1994 1. Introduction HZ has been the popular 7-bit data encoding scheme for GuoBiao (GB) for five years. There has been recent interest on extending HZ to handle popular Chinese character codes like BIG-5. Although this need is transient in nature, as systems are getting more 8-bit clean and universal 16-bit character coding schemes like Unicode are emerging, it is anyway desirable to have a 7-bit data exchange format because it seems that this transient period would not end soon in the foreseeable future. HZ was originally designed as an exchange code for files. However, because of its choice of graphical escape sequence, it makes it possible to write real-time encode/decode programs, such as hztty, that would allow existing applications to run on top without the awareness of HZ. This extension is an attempt to try to leverage on the success of HZ and extending it to handle other non-ASCII coding schemes. The purpose of this specification is to provide a framework and it does not target any SPECIFIC coding scheme. 2. Specification of the Extensions To support a non-ASCII code scheme, the extension requires the following. There exists a mapping to map the non-ASCII character codes to a fixed number of ASCII byte codes, such that each byte is within the range of $21-$7E (hexadecimals), except the first byte, which can only be in the range of $21-$7D. A graphical ASCII character is a byte in the range $21-$7E. A non-graphical ASCII character is a byte in the range $0-$20 or of the value $7F. The code $7E is ~. There are two modes, namely ASCII mode and non-ASCII mode. Inside non-ASCII mode, there could be sub-modes, such as GB and BIG-5 sub-modes. By convention, a non-graphical ASCII character should only appear in ASCII mode. The default mode is ASCII mode. In ASCII mode, a byte is interpreted as an ASCII character, unless a '~' is encountered. The character '~' is an escape character. Depends on the character immediately followed, it has the following special meaning. o The escape sequence "~~" is interpreted as a "~". o The escape-to-non-ASCII sequence "~{" switches the mode from ASCII to non-ASCII. o The escape sequence "~\n" is a line-continuation marker to be consumed with no output produced. o Any character c, other than the three characters mentioned above, that follows "~" is interpreted as ~c (i.e., "~" followed by itself). In non-ASCII mode, characters are interpreted as non-ASCII codes until the escape-from-non-ASCII code "~}" is read. This code switches the mode from non-ASCII back to ASCII. (Note that the escape-from-non-ASCII code "~}" ($7E7D) is outside the defined non-ASCII range.) Inside non-ASCII mode, a tilde escape sequence is used to switch into specific non-ASCII code scheme. To maintain backward compatibility with HZ, if the first byte that follows "~{" is not "~", by default it switches to GB sub-mode (as defined in the HZ specification). When picking code scheme specific escape sequences in the non-ASCII mode, it is recommended that graphical ASCII sequences similar to the ISO standards are used. For examples, use ~A for GB, and ~G and ~H for CNS. (Note that there is no such obvious escape sequences for BIG-5, as it is not a registered ISO Standard. Perhaps ~g and ~h could be chosen.) Also it is recommended that specific ISO code mappings from 16-bit non-ASCII codes to 7-bit ASCII to be used if possible. 3. Rationales a. The ISO 7-bit exchange standards. One of the biggest advantages of HZ is that it is possible to use many standard ASCII applications such as editors, mailers, and news readers for GB without any modification to the applications. For example, running a real-time HZ encoder/decoder program such as hztty on top of cxterm, which by itself has no knowledge of HZ, is sufficient to allow mixed ASCII and GB code processing. It would be extremely difficult, if not impossible, to re-use ASCII applications in this "HZ-unaware" manner, had ISO-type of escape sequences been used. It is because ISO chooses the non-graphical ASCII character ESC as the beginning of the escape sequences. The problem with ESC is that it does not have a unique graphical representation. Notice that the HZ Extension defines the ASCII and non-ASCII modes while the ISO has no explicit non-ASCII mode. b. The issue of maximal ASCII transparency. Informally, ASCII transparency is defined as how much ASCII information is lost when a pure ASCII file goes through a decoder for exchange format like HZ. Obviously, the more tilde escape sequences are defined, the more ASCII transparency will be lost. Therefore the approach of defining individual tilde escape sequences for various coding schemes was not taken. Instead, ASCII and non-ASCII modes are explicitly distinguished so that only one tilde escape sequence is needed to switch from ASCII to non-ASCII mode. Non-ASCII code schemes are then localized in the non-ASCII mode. Also to prevent ASCII transparency from further erosion, the HZ Extension defines the other two-byte tilde escape sequences that are originally undefined in HZ. Please refer to the HZ specification for the other rationales and recommendations regarding HZ. They are generally applicable to this HZ Extension. Acknowledgement The idea of creating a general framework for non-ASCII codes and the issue of ASCII transparency were inspired by the discussion on extending HZ to handle BIG-5 among the author, Fung Fung Lee, and Yagui Wei about a year ago, and the recent attempt on the net to come up with a unified 7-bit exchange format for Big-5. Fung Fung Lee has been very helpful in shaping the ideas in this article. I also thank him for his careful review of this draft and numerous interesting suggestions. *e-mail address: Ricky.Yeung@Sun.Com Disclaimer: This article has no connection with my employer. From simpson@math.psu.edu Mon Jun 13 14:56:24 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA08528; Mon, 13 Jun 94 14:56:24 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id PAA05505; Mon, 13 Jun 1994 15:54:08 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Mon, 13 Jun 1994 15:54:07 -0400 Date: Mon, 13 Jun 1994 15:54:07 -0400 Message-Id: <199406131954.PAA15059@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: cyl@ifcss.ifcss.org's message of 13 Jun 1994 16:16:11 GMT Subject: Re: Mule support for CNS and Big5 Reply-To: simpson@math.psu.edu From: cyl@ifcss.ifcss.org (Yilong Chen) Newsgroups: alt.chinese.computing Date: 13 Jun 1994 16:16:11 GMT References: <9406121317.AA23903@etlken.etl.go.jp> <199406131535.KAA24600@news.cs.utexas.edu> I wrote: : My suggestion for Kenichi is to make no changes, i.e. to continue to : support Big5 and CNS as two different character sets, as in Mule 1.0 : and 1.1. I think that merging Big5 and CNS into one character set : will present technical difficulties, due mostly to the fact that Big5 : is a somewhat chaotic popular standard while CNS is a rigid official : standard. Yilong writes: BIG5 is chaotic, but ETen BIG5 is stable, and maintained by ETen, and also a very popular form of BIG5 at least from the postings i see in the tw.bbs.lang.japanese and csie.bbs.fj.* hierarchy. Yes, ETen Big5 is stable and very popular. So maybe Mule should support ETen Big5 specifically. The current Big5 support in Mule 1.0 and 1.1 works well with ETen Big5, but if Kenichi wants to go further and specify ETen Big5 as standard (as I do in HZ+), that would make sense. Note however that ETen Big5 and CNS are still two different character sets, so it still seems important for Mule to support them separately, as is currently the case in Mule 1.0 and 1.1. -- Steve From simpson@math.psu.edu Mon Jun 13 15:45:46 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA10431; Mon, 13 Jun 94 15:45:46 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id QAA05868; Mon, 13 Jun 1994 16:43:33 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Mon, 13 Jun 1994 16:43:31 -0400 Date: Mon, 13 Jun 1994 16:43:31 -0400 Message-Id: <199406132043.QAA15136@boole.math.psu.edu> To: Nelson CHIN Cc: soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu Subject: Nelson's call for consensus In-Reply-To: References: Reply-To: simpson@math.psu.edu Nelson CHIN writes: > I would also like to know if there's a consensus to introduce > ISO-2022-JP now to start the transistion onto the high road. Could you please tell us a little more about ISO-2022-JP? Does it support CNS? I know that ISO-2022-JP-2 does, but that's different, right? Also, does ISO-2022-JP specify 7-bit only? Be careful, ISO-2022 includes some 8-bit as well as 7-bit schemes. Also, does ISO-2022-JP include Ross's proposal? Before I fully endorse your call for consensus, I'd like to hear what Ross says. On the whole, I am very much in favor of your high road idea based on something like 7-bit ISO-2022-JP. At the same time, and also in line with your posting, I would like to see some movement on a low road consensus for something like HZ+. Currently my efforts in this directon are running into some problems. Some people (notably Fung, Ricky, Yuan Jiang, and Ed Lai) seem to disapprove of my methods for attempting to reach a consensus. Or maybe these people are against HZ+, I'm not sure which, but in any case they are throwing a lot of roadblocks in my way, for whatever reasons. > note: i know some of you want to directly jumped onto the highroad, without > accomodating BIG5 with the existing HZ standard. But may i suggest > that you instead just give the approval for an HZ+ standard and let > those who's interested in creating this HZ+ to create it, and you put > your effort into advocating ISO-2022-JP to chinese users to move > onto the high road. As long as everyone has consensus that we want > to reach the high road, then each person can play a role along that > road to move users to the right destination. If you feel you're only > interested in ISO-2022-JP, then stand on the road and cheer the > users to continue walking. If you feel like providing HZ+, then > provide it to the users at the start line, and make sure you tell > them that this is just something to help you start on your journey, > and we at soft-authors will like to make this journey as most > comfortable a journey as possible. These comments by Nelson make a lot of sense to me. HZ+ can meet an immediate practical need, but it can also play a useful role in the transition from HZ to high-road multilingual solutions. > If anybody on soft-authors has anything to disagree upon, let me know and > i'll remove anything i proposed as our consensus. Nelson, I would urge you to be very cautious about using the term "consensus" with respect to soft-authors. People seem to be very touchy. A vote or a lack of response may not be enough. You may find yourself in a situation like I am, where people are crying foul even after you have taken every precaution to insure fairness and openness. I don't have any specific advice to offer except, be careful. -- Steve From simpson@math.psu.edu Mon Jun 13 16:14:23 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA11723; Mon, 13 Jun 94 16:14:23 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id RAA06138; Mon, 13 Jun 1994 17:12:01 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Mon, 13 Jun 1994 17:11:59 -0400 Date: Mon, 13 Jun 1994 17:11:59 -0400 Message-Id: <199406132111.RAA15186@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: Re: new solution: approve both HZ+ and ISO2022 In-Reply-To: <199406111858.OAA20808@leibniz.math.psu.edu> References: <199406111858.OAA20808@leibniz.math.psu.edu> Reply-To: simpson@math.psu.edu I wrote: > >The low road means immediate, practical solutions based on popular > >standards such as HZ and Big5. The high road means long term, > >multilingual solutions based on official standards such as ISO 2022 > >and CNS. Chinese computing needs both kinds of solutions. John Delacour replied: > I don't see the difference. Why is there any need to take the low road at > all? HZ has served its purpose well for a long while but now there is a HZ is still of great interest and usefulness to a lot of people. There are various reasons for this. One reason is that the Escape character 0x1E causes problems for some terminals and news and mail software. Several Chinese newsgroups currently use HZ and the chinese.* organizers are calling for some form of HZ+ which would incorporate Big5. > chance to introduce a single conversion procedure that will process both GB > and Big-5 and also comply with international standards, I don't see any hope that Big5 will ever be brought into compliance. Big5 is very popular, but it is not and never will be an official standard. > GB has never been > a problem but now we have a way (or rather, several ways) to convert either > GB or Big-5 to CNS and back again. The matter is not so simple. This conversion software is subject to various ambiguities and pitfalls. In the case of Big5 -> GB and CNS->GB, there is the notorious n-to-1 problem. In the case of Big5 -> CNS, there are problems caused by vendor-specific codes and eliminated duplicates. > Surely no software author is going to complain if he has to throw out the > HZ<->GB code from his programme in favour of the new code, because the > whole thing is so simple. I don't think its so simple, and I think there will be lots of complaints from both software authors and users. Our problem is to choose good solutions with reasonable hope for success, but we shouldn't underestimate the difficulties. From simpson@math.psu.edu Mon Jun 13 16:48:35 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA13030; Mon, 13 Jun 94 16:48:35 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id RAA07046; Mon, 13 Jun 1994 17:46:17 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Mon, 13 Jun 1994 17:46:16 -0400 Date: Mon, 13 Jun 1994 17:46:16 -0400 Message-Id: <199406132146.RAA15256@boole.math.psu.edu> To: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Cc: soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu, handa@etl.go.jp Subject: Ricky's framework; future of HZ+ In-Reply-To: <9406131808.AA10972@hk.Eng.Sun.COM> References: <9406131808.AA10972@hk.Eng.Sun.COM> Reply-To: simpson@math.psu.edu Ricky Yeung writes: > An Extension to HZ - A Framework for > 7-bit Encoding Scheme for Arbitrarily Mixed > ASCII and Non-ASCII characters This framework looks good. I am thinking that maybe I will change HZ+S to conform to Ricky's framework. I'm thinking of using ~0, ~1, and ~2 as the escape sequences for Big5. (~1 and ~2 for ETen Big5 blocks 1 and 2, ~0 for the ETen user-defined area as pointed out by Sandstrom. Note that this choice of escapes is in a sense ISO 2022 compliant, since 0, 1, 2, ... are reserved as ISO 2022 final characters for application-specific character sets.) Are there any comments? Is there anybody who is for or against my changing HZ+S in this way? But first I need to somehow get disentangled from the mess in soft-authors. I don't see a really clean way to resolve the uncertainty and confusion about CNS, HZ-2, B5E1/2/3, etc. Fung, Ricky, and Yuan are putting roadblocks in the way of a vote. I'm thinking maybe I should stop trying to seek a soft-authors consensus, and instead just take control of HZ+ myself and do it the way I want to do it, and if anybody likes what I am doing they can speak up in favor. Comments? -- Steve From simpson@math.psu.edu Tue Jun 14 10:26:24 1994 Return-Path: Received: from cnd.org (cnd.cnd.org) by ifcss.org (4.1/IFCSS-Mailer) id AA14783; Tue, 14 Jun 94 10:26:24 CDT Received: from leibniz.math.psu.edu by cnd.org (4.1/4.7) id AA12043; Tue, 14 Jun 94 08:24:59 PDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id LAA14170; Tue, 14 Jun 1994 11:22:54 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Tue, 14 Jun 1994 11:22:52 -0400 Date: Tue, 14 Jun 1994 11:22:52 -0400 Message-Id: <199406141522.LAA16974@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: cyl@ifcss.ifcss.org's message of 13 Jun 1994 21:44:32 GMT Subject: Re: Mule support for CNS and Big5 Reply-To: simpson@math.psu.edu I wrote: : Note however that ETen Big5 and CNS are still two different character : sets, so it still seems important for Mule to support them separately, : as is currently the case in Mule 1.0 and 1.1. Nelson replied: support is done by providing a BIG5 bdf font and the correct mapping from whatever mule uses for its internal coding. You can input BIG5 and CNS as two different characters, and it'll come out as two different character sets, provided you have the fonts to view them. Why distinguish it when it's mappable to any combination of official character sets within ISO2022. No one but the program has to deal with the mappings and stuff. You input what you want to input, and you tell mule what you want it to output. It is unclear to me how this would work in practice. The current setup is clear enough -- Mule supports CNS and Big5 as two different character sets with different final characters (G and H for CNS, 0 and 1 for Big5), and different fonts. I'm not sure whether Nelson is proposing to keep this arrangement or not. I'm not sure what Nelson means by "inputting Big5 and CNS as two different characters" and "mapping to combinations of official character sets within ISO 2022". For example, does the latter mean that ETen Big5's Japanese kana get mapped to JIS? I think that questions of this sort can only be answered by Kenichi, since Kenichi is making the design decisions for Mule. -- Steve From simpson@math.psu.edu Tue Jun 14 15:24:25 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA27301; Tue, 14 Jun 94 15:24:25 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id QAA16958; Tue, 14 Jun 1994 16:22:15 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Tue, 14 Jun 1994 16:22:13 -0400 Date: Tue, 14 Jun 1994 16:22:13 -0400 Message-Id: <199406142022.QAA17463@boole.math.psu.edu> To: soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu, chinesenet@uclink.berkeley.edu Subject: Re: future of HZ+ In-Reply-To: <2tim61$dnh@ifcss.org> References: <9406131808.AA10972@hk.Eng.Sun.COM> <199406132146.QAA18108@news.cs.utexas.edu> <2tim61$dnh@ifcss.org> Reply-To: simpson@math.psu.edu I wrote: > : thinking maybe I should stop trying to seek a soft-authors consensus, > : and instead just take control of HZ+ myself and do it the way I want > : to do it, and if anybody likes what I am doing they can speak up in > : favor. Comments? Yilong Chen replied: > I would say continue on your work on HZ+S, and give a periodical update of > changes/revisions etc, and ask for people to join your development team, > and get some beta testers.. etc, and if you find other proposals seem > to go in line of what you want, merge with others and build a bigger > team. Yeah, maybe that is the best course of action. I never did have much patience with committees, and soft-authors seems to be an unusually difficult committee. Let me think this over a little more, but I may well end up following your advice. Does anyone else have anything to say? > I think because of the confusion and mess floating in soft-authors, some > feel that a consensus should not be made at this time in soft-authors, and Yes. There seems to be some kind of anti-consensus dynamic at work. The only thing everyone seems to agree on is that consensus is undesirable. I can't understand this attitude. I find it weird. Don't these people realize that the world needs HZ+ now? > it seems that people are now just working on what they think is the ^^^^^^^^^^^^^^^^^^^^^^ > best proposal(s) and let the users/software implementers decide. ^^^^^^^^^^^^^^^^ No. If you think about it, that is not actually the case. That's another frustrating thing about this whole discussion. None of the vocal participants (except me) seem to be interested in the question of which proposal is best. For example, the author of B5E1/2/3 *hasn't* said that he thinks his many proposals are better than others. Rather, his position seems to be that there are various criteria and he himself can't decide which criterion is important. He seems to think that we should wait a few months for him to make up his mind. Similarly, Ed Lai and company have *not* claimed that Ed's vague idea of "using CNS codes as the exchange codes for Big5 codes" is better than HZ+. Furthermore, they never offered any specification or implementation. Rather, their main concerns were procedural -- complaints about my alleged unfairness in calling for votes, etc. As for Ya-Gui, his participation in this whole discussion has been minimal to nonexistent, and he also has not offered any implementation. It's interesting to note that Ya-Gui sat on his ideas for more than a year (4/93-4/94), because of mysterious doubts related to future extensions of GB. He released his ideas only after I released HZ+S. My approach has been quite different. I have backed up my HZ+S proposal with specification documents and sample implementations and careful explanations of why I think it is much preferable to the other proposals (simplicity, full Big5 compatibility, etc.). Furthermore, I have worked hard trying to promote open discussion and to reach a consensus in soft-authors. I don't understand why some people are so negative about this. -- Steve From simpson@math.psu.edu Wed Jun 15 06:24:46 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA25080; Wed, 15 Jun 94 06:24:46 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id HAA21686; Wed, 15 Jun 1994 07:22:36 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 15 Jun 1994 07:22:34 -0400 Date: Wed, 15 Jun 1994 07:22:34 -0400 Message-Id: <199406151122.HAA18991@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: clarification of the term HZ+ In-Reply-To: <199406140641.CAA11930@leibniz.math.psu.edu> References: <199406140641.CAA11930@leibniz.math.psu.edu> Reply-To: simpson@math.psu.edu John Delacour writes: > "a fully reversible conversion routine to translate either 8-bit GB > or 8+7-bit Big-5 double-byte characters into US-ASCII for the > reliable transport of Chinese over the Internet etc." I am not > aware that HZ+ has been more narrowly defined or even that the name > of the standard that will eventually be adopted has been > determined. It seems that I need to once again clarify the term HZ+, which I introduced in my original HZ+S proposal. I defined HZ+ to mean "HZ plus some form of 7-bit Big5". If a proposal for 7-bit Chinese does not include HZ, it cannot be called HZ+ (assuming we want to communicate clearly). The context in which I introduced the term HZ+ was that there were several competing HZ+ proposals and it was thought that soft-authors could vote to reach a consensus on one of these, which would thereafter take the name HZ+. Since consensus has proved elusive and various people are throwing roadblocks in the way of a vote, I am now thinking that I should reclaim the term HZ+ and use it more narrowly to designate only my own proposal, HZ+S. > rather than convert Big-5 into an arbitrary 7-bit form it seems HZ+S does not "convert Big-5 into an arbitrary 7-bit form". HZ+S respects the natural division of the Big5 code space into blocks 1 and 2, and it packs these blocks into the available 7-bit code space, in order, with no gaps. This particular mapping is therefore by no means arbitrary. Some of the other proposed mapping schemes can be accused of arbitrariness, but not HZ+S. > sensible to use an existing standard representation of Big-5, > i.e. CNS, for the conversion. CNS is not a representation of Big5. CNS and Big5 are universally recognized as different coding systems and different character sets. Nowhere in the CNS documentation does it say that CNS is a representation of Big5. Mule treats Big5 and CNS as two different character sets, each designated by its own ISO-2022 final characters. ETen treats Big5 and CNS as two different coding systems, and nowhere in ETen's documentation does it say that they are really the same or that one is a representation of the other. There are in fact numerous differences between them. For instance, ETen Big5 contains some Chinese characters, Japanese kana, etc. that CNS does not contain. On a sociological level, CNS and Big5 are also very different, because CNS is an unpopular official standard, while Big5 is a popular unofficial standard. -- Steve From simpson@math.psu.edu Wed Jun 15 06:47:37 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA25544; Wed, 15 Jun 94 06:47:37 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id HAA21768; Wed, 15 Jun 1994 07:45:28 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 15 Jun 1994 07:45:26 -0400 Date: Wed, 15 Jun 1994 07:45:26 -0400 Message-Id: <199406151145.HAA19027@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: HZ and the Escape character In-Reply-To: <199406142015.QAA16708@leibniz.math.psu.edu> References: <199406142015.QAA16708@leibniz.math.psu.edu> Reply-To: simpson@math.psu.edu John Delacour writes: > The character (see 'em?) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > Here I have to ask some questions... > 1. It has been said by Steve Simpson and Ricky Yeung that this presents a > problem. What proportion of users suffer from this problem and why can't > it be resolved? This is really a question about the design decisions that led up to the creation of HZ. I'd like to hear from Fung-Fung Lee, Ed Lai, Ricky Yeung, and the other people who were actively involved, as to why they decided to use ~ rather than Escape or some other non-printable. My understanding is that they felt Escape would cause problems because (1) it is non-printable, (2) it confuses some terminals and software. I wrote: > HZ is still of great interest and usefulness to a lot of people. > There are various reasons for this. One reason is that the Escape > character 0x1E causes problems for some terminals. John replied: > That "escape character" would cause my terminal the most > terrible problems, but I wouldn't blame my terminal, even if I had > one, because 0x1E is not the escape character in question. It's a > ghost and it belongs to the Apple Mafia :-) Sorry, I meant 0x1B, ASCII 27, the Escape character. It is a fact that DEC terminals (e.g. the venerable VT100) and most other character terminals and terminal emulation programs use escape sequences starting with Escape, to designate things like cursor motion, underline mode, etc. etc. I'm not sure how much of a problem this is for ISO 2022. Maybe there are technical difficulties in running ISO 2022 applications on a character terminal, maybe not. I'd like to hear from someone who has implemented one of them. In any case, aside from purely technical questions about Escape versus tilde, the biggest reason to maintain compatibility with HZ is that HZ is popular. > BIG-5 = CNS False. > GB | B5 can = cns I can't parse this. If it means what I think it means, it's false. -- Steve From simpson@math.psu.edu Wed Jun 15 13:51:44 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA10843; Wed, 15 Jun 94 13:51:44 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id OAA24829; Wed, 15 Jun 1994 14:49:35 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 15 Jun 1994 14:49:33 -0400 Date: Wed, 15 Jun 1994 14:49:33 -0400 Message-Id: <199406151849.OAA19729@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: problems with the Escape character? In-Reply-To: <14JUN94.19121266@neurophys.wisc.edu> References: <199406142202.RAA01576@news.cs.utexas.edu> <14JUN94.19121266@neurophys.wisc.edu> Reply-To: simpson@math.psu.edu Speaking of the Escape character, John Delacour wrote: > >2. Why is it that it's a problem for Chinese users and apparently no > problem at all for Japanese users. Every news posting and mail item I get cai@neurophys.wisc.edu replied: > ---------- This is NOT true. Otherwise we don't need to have the "famous" > jis repair program by Ken Lunde. Pardon my ignorance. What is the famous JIS repair program? What kind of damage does it repair? Kenichi has already pointed out that there is a difference between 7-bit representations of JIS (used in e-mail, etc.) and the 8-bit shift-JIS representation (used in PCs), so of course there must be some translation software. This is not really a problem, just a necessary annoyance. But perhaps there is some other damage? -- Steve From sandstro@cac.washington.edu Wed Jun 15 14:09:51 1994 Return-Path: Received: from shiva1.cac.washington.edu by ifcss.org (4.1/IFCSS-Mailer) id AA11560; Wed, 15 Jun 94 14:09:51 CDT Received: by shiva1.cac.washington.edu (5.65+UW94.4/UW-NDC Revision: 2.30 ) id AA08642; Wed, 15 Jun 94 12:07:40 -0700 Date: Wed, 15 Jun 1994 12:07:39 -0700 (PDT) From: Bob Sandstrom Subject: Re: User specific Big5 codes To: Stephen G Simpson Cc: soft-authors@ifcss.org In-Reply-To: <199406110030.UAA04941@boole.math.psu.edu> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII On Fri, 10 Jun 1994, Stephen G Simpson wrote: > Bob Sandstrom writes: > > please note that 0xa140-0xfefe is not the entire Big5 code space. > > ... > > Eten, for example, also uses 0x8e40-0xa0fe (for the > > second section of USRFONT) and 0x8140-0x8dfe (for the third section > > of USRFONT). My Eten information is from an April 1992 Eten > > manual, ISBN 957-504-044-9, page 5-2 in Appendix 5. > > This is interesting. It is also a little bit dismaying. I didn't > know about these code ranges reserved for user-defined characters. > > Tell me, what are the allowable first bytes in these ranges? > ... Here is a uuencoded Big5 quote from Appendix 5, page 5-1, of my Eten manual, ISBN 957-504-044-9. begin 644 etb5range.et M0DE'+34@O5BH=++.K+"HXJ;LI+BRUX<3$Y+#,X1$@*(" @ M(" @(""G0Z;LI+BRU2 M+2 T,$BAXS=%2" H*C$U-RD*(" @(" @(" @(" @ 2(" @(" @("!!,4BAXT9%2 H* end To summarize in English, the Eten company feels that Big5 is as follows: high byte -- 0xa1 to 0xfe, 0x8e to 0xa0, 0x81 to 0x8d low byte -- 0x40 to 0x7e, 0xa1 to 0xfe 19782 codes total Bob Sandstrom U of Washington From CAI%MVF.DecNet@wigate.nic.wisc.edu Wed Jun 15 14:25:01 1994 Return-Path: Received: from vms.macc.wisc.edu by ifcss.org (4.1/IFCSS-Mailer) id AA12105; Wed, 15 Jun 94 14:25:01 CDT Received: from VMSmail by wircs1.DecNet; Wed, 15 Jun 94 14:21 CDT Message-Id: <24061514215856@wircs1.DecNet> Date: Wed, 15 Jun 94 14:21 CDT From: CAI%MVF.DecNet@wigate.nic.wisc.edu Subject: Escape in Japanese To: soft-authors@ifcss.org, simpson@math.psu.edu X-Vms-To: WIRCS1::IN%"soft-authors@ifcss.org", WIRCS1::IN%"simpson@math.psu.edu" >From: IN%"simpson@math.psu.EDU" 15-JUN-1994 13:55:50.25 Subj: problems with the Escape character? ....... cai@neurophys.wisc.edu replied: > ---------- This is NOT true. Otherwise we don't need to have the "famous" > jis repair program by Ken Lunde. Pardon my ignorance. What is the famous JIS repair program? What kind of damage does it repair? Kenichi has already pointed out that there is a difference between .... -- Steve I think Kenichi was talking about the situation within Japan. I believe most of the damage occure outside Japan. One of the damage is that the ESC characters are lost in the network. For details, please study the program itself: jconv.c at ftp.ora.com:/pub/examples/nutshell/ujip/src. With ISO-2022 (?), you need to introduce another layer, namely EUC, as pointed out by Kenichi. In my opinion, it is not even a "high road" at all for Chinese-only applications (I understand it is desirable for those situations that you have to deal with several languages). I know with Pentium now and P6 next year, 486 is going to be out of date in 2 or 3 years, but I still brought a 486 2 weeks ago. Yidao CAI, MD, not yet a Ph.D From handa@etlken.etl.go.jp Wed Jun 15 18:57:04 1994 Received: from etlpost (etlpost.etl.go.jp) by ifcss.org (4.1/IFCSS-Mailer) id AA23767; Wed, 15 Jun 94 18:57:04 CDT Received: from etlpom.etl.go.jp by etlpost (5.67+1.6W/2.7W) id AA15684; Thu, 16 Jun 94 08:54:49 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA06242; Thu, 16 Jun 94 08:54:48 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA26155; Thu, 16 Jun 94 08:54:56 JST Date: Thu, 16 Jun 94 08:54:56 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9406152354.AA26155@etlken.etl.go.jp> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Wed, 15 Jun 1994 14:49:33 -0400 <199406151849.OAA19729@boole.math.psu.edu> Subject: Re: problems with the Escape character? On Wed, 15 Jun 1994 14:49:33 -0400, Stephen G Simpson said: > cai@neurophys.wisc.edu replied: > > ---------- This is NOT true. Otherwise we don't need to have the "famous" > > jis repair program by Ken Lunde. > Pardon my ignorance. What is the famous JIS repair program? What > kind of damage does it repair? Perhaps "jis repair program" means "a program which recovers character of an news-article which is sent or received by -eating news system". At least in Japan, we have already modified news system (i.e, bnews or inews) not to eat . Eating (or dropping) is just a bug concerning RFC-xxxx (I forgot the number "xxxx"). I heard that the patch is already reflected in the original `inews' but it's true that there still exist many sites using old version of inews or bnews. It was the decision of Japanse internet community to insist on compatibility with ISO2022 even though we have to correct the original (US-made) `bnews/inews'. --- Ken'ichi HANDA handa@etl.go.jp From A7621GAC@AWIUNI11.EDVZ.UNIVIE.AC.AT Thu Jun 16 02:01:02 1994 Return-Path: Received: from AWIUNI11.EDVZ.UniVie.AC.AT (helios.edvz.univie.ac.at) by ifcss.org (4.1/IFCSS-Mailer) id AA07154; Thu, 16 Jun 94 02:01:02 CDT Message-Id: <9406160701.AA07154@ifcss.org> Received: from AWIUNI11.EDVZ.UNIVIE.AC.AT by AWIUNI11.EDVZ.UniVie.AC.AT (IBM VM SMTP V2R2) with BSMTP id 7999; Thu, 16 Jun 94 08:58:31 MEZ Received: from AWIUNI11.EDVZ.UNIVIE.AC.AT (NJE origin A7621GAC@AWIUNI11) by AWIUNI11.EDVZ.UNIVIE.AC.AT (LMail V1.2a/1.8a) with BSMTP id 6227; Thu, 16 Jun 1994 08:58:29 +0100 Date: Thu, 16 Jun 94 08:55:35 MEZ From: Werner Lemberg To: soft-authors@ifcss.org Subject: Re: HBF standard reexamined In-Reply-To: Your message of Mon, 6 Jun 94 11:11 BST Because I don't know whether my previous mail reached this list (I usually get my own mail resent), I will send it again. Sorry for answering so late. Ross Paterson said: >> o a simple function HBF_CheckCode() to check if a specified code >> is valid or not (with similar error codes to HBF_GetBitmap() ) > >How do you intend to use this? I've written a CJK-package for LaTeX2e (not yet finished) which will use compressed fonts (i.e. 256 characters per font instead of one plane per font), and I need to convert bitmap fonts into the needed format. This job is easy for GB fonts, but a bit more difficult for Big 5 fonts. My package will not recognize the gaps in the code range, therefore I have to insert null bytes if an invalid character code is encountered. hbfGetEach will jump over the gaps, which is not useful for my program. > >> o It is not explained exactly how to handle multiple entries of the >> same keyword like HBF_CODE_RANGE if using the function >> HBF_GetProperty(). I suggest to change this API-function: > >> int HBF_GetProperty( >> OUT char * property >> IN HBF_Handle handle, >> IN char * propertyName ); > >How about just saying that the following properties are inaccessible: > > HBF_START_BYTE_2_RANGES > HBF_BYTE_2_RANGE > HBF_START_CODE_RANGES > HBF_CODE_RANGE After trying to implement my suggestion, I support your view :-) I now think it is better to hide these entries, but my program NEEDS the information about byte- and code ranges. Please add functions like HBF_GetCodeRange(...) and HBF_GetByte2Range(...) to access these values. Werner From A7621GAC@AWIUNI11.EDVZ.UNIVIE.AC.AT Thu Jun 16 02:11:11 1994 Return-Path: Received: from AWIUNI11.EDVZ.UniVie.AC.AT (helios.edvz.univie.ac.at) by ifcss.org (4.1/IFCSS-Mailer) id AA07369; Thu, 16 Jun 94 02:11:11 CDT Message-Id: <9406160711.AA07369@ifcss.org> Received: from AWIUNI11.EDVZ.UNIVIE.AC.AT by AWIUNI11.EDVZ.UniVie.AC.AT (IBM VM SMTP V2R2) with BSMTP id 8089; Thu, 16 Jun 94 09:07:45 MEZ Received: from AWIUNI11.EDVZ.UNIVIE.AC.AT (NJE origin A7621GAC@AWIUNI11) by AWIUNI11.EDVZ.UNIVIE.AC.AT (LMail V1.2a/1.8a) with BSMTP id 6426; Thu, 16 Jun 1994 09:07:42 +0100 Date: Thu, 16 Jun 94 09:00:05 MEZ From: Werner Lemberg Subject: Error in HBF standard To: soft-authors@ifcss.org I found an error in the Backus-Naur description of HBFByte2Range and HBFCodeRange: in the beginning of Appendix 1 it is said that terminal tokens "can be separated by one or more blank or tab characters" which I interpret that at least one blank or tab character must be there. But we have: HBFCodeRange ::= 'HBF_CODE_RANGE' firstCode '-' lastCode ... ^ ^ (the same holds for HBFByte2Range) thus writing e.g. 0xA140-0xA3BF would be wrong, which is of course not intended Werner From simpson@math.psu.edu Fri Jun 17 09:06:40 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA10119; Fri, 17 Jun 94 09:06:40 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id KAA19489; Fri, 17 Jun 1994 10:04:27 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 17 Jun 1994 10:04:25 -0400 Date: Fri, 17 Jun 1994 10:04:25 -0400 Message-Id: <199406171404.KAA27505@boole.math.psu.edu> To: chungdan@Xenon.Stanford.EDU (ZHONG Qiyao) Cc: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: chungdan@Xenon.Stanford.EDU's message of 17 Jun 1994 10:58:59 GMT Subject: (7-BITS) HZ extention to Big 5 etc. Reply-To: simpson@math.psu.edu chungdan@Xenon.Stanford.EDU (ZHONG Qiyao) writes: > It seems that one way to "extend" HZ to HZ+ is the following: > > - ~{~} > - ~{~0~} > - ~{~1~} > - ~{~2~} > > Because ~0, ~1 and ~2 currently do not have any meaning within ~{ ~}. > So we can define them for mode-switching. I am also thinking along these lines. Has Ricky Yeung posted his "extended HZ" proposal to alt.chinese.computing? If I modify my HZ+ (see ftp.math.psu.edu, /pub/simpson/chinese/hzp) in line with Ricky's proposal, I get something like this: ~{...ordinary HZ codes, i.e. 7-bit GB...~} ~{~A...ordinary HZ codes, i.e. 7-bit GB...~} ~{~1...7-bit no-gap representation of Big5 block 1...~} ~{~2...7-bit no-gap representation of Big5 block 2...~} (But this description is over-simplified -- see Ricky's document for precise details about the structure of the escape sequences.) Here block 1 is 0xA140-0xC8FE, the ETen Big5 area containing the more frequently used Chinese characters plus various alphabets, numerals, symbols, Japanese kana, etc., and block 2 is 0xC940-0xFEFE, the ETen Big5 area containing the less frequently used Chinese characters plus a few special characters and some graphical symbols. In addition, I am thinking of adding ~{~0...7-bit no-gap representation of ETen Big5 user area...~} Here "block 0" is 0x8140-0xA040, the ETen Big5 area reserved for user-defined characters and symbols, as pointed out by Bob Sandstrom. Ricky's reason for using ~A is that A is the ISO 2022 registered final character for GB. Big5 is not registered with ISO 2022, and it is therefore not so clear what escapes to use for Big5. Ricky very tentatively suggests ~g and ~h. I think ~0, ~1, and ~2 as above make more sense, because according to Ross, ISO 2022 reserves 0, 1, 2, ... as final characters for user-specific or application-specific character sets. In summary, I am thinking of modifying my HZ+ in accordance with Ricky's ideas and using ~0, ~1, ~2 for ETen Big5 as above. Ricky and Fung have been silent for a few days, but I hope that they will comment on this when they get time. -- Steve From simpson@math.psu.edu Fri Jun 17 09:20:57 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA10819; Fri, 17 Jun 94 09:20:57 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id KAA19560; Fri, 17 Jun 1994 10:18:49 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 17 Jun 1994 10:18:48 -0400 Date: Fri, 17 Jun 1994 10:18:48 -0400 Message-Id: <199406171418.KAA27520@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: (7-BITS) HZ extention to Big 5 etc. In-Reply-To: <199406171404.KAA27505@boole.math.psu.edu> References: <199406171404.KAA27505@boole.math.psu.edu> Reply-To: simpson@math.psu.edu Stephen G Simpson writes: > Here "block 0" is 0x8140-0xA040, the ETen Big5 area reserved for > user-defined characters and symbols, as pointed out by Bob Sandstrom. Sorry, I meant 0x8140-0xA0FE. A couple of people in the Chinese computing newsgroups have complained that I make too many mistakes. That's probably true, but at least when I learn of my mistakes I publish corrections of them. -- Steve From simpson@math.psu.edu Fri Jun 17 10:28:46 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA13447; Fri, 17 Jun 94 10:28:46 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id LAA20035; Fri, 17 Jun 1994 11:26:35 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 17 Jun 1994 11:26:33 -0400 Date: Fri, 17 Jun 1994 11:26:33 -0400 Message-Id: <199406171526.LAA27636@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org, chinesenet@uclink.berkeley.edu Subject: from now on, HZ+ equals HZ+S Reply-To: simpson@math.psu.edu There has been much confusion about the term HZ+, leading to numerous misunderstandings. Some people have somehow gotten the idea that HZ+ means any form of 7-bit Chinese whatsoever. We need to straighten out this mess. A couple of days ago I wrote: > The context in which I introduced the term HZ+ was that there were > several competing HZ+ proposals and it was thought that soft-authors > could vote to reach a consensus on one of these, which would > thereafter take the name HZ+. Since consensus has proved elusive and > various people are throwing roadblocks in the way of a vote, I am now > thinking that I should reclaim the term HZ+ and use it more narrowly > to designate only my own proposal, HZ+S. I have now decided that this is the right policy to follow. From now on, to avoid confusion, I would appreciate it if everybody would use the term HZ+ to designate only HZ+S, i.e. my specific proposal, the key features of which are HZ compatibility and complete faithfulness to the structure of Big5, achieved by means of no-gap mappings. Thus, from now on, HZ+ equals HZ+S. The only official source of information on HZ+ will be ftp.math.psu.edu, /pub/simpson/chinese/hzp. If there are any changes to be made in HZ+, I will make them and announce them. To avoid further confusion, proposals other than HZ+ (= HZ+S) can and should be designated by other names. The 7-bit Chinese proposals that I have heard of, in order of when they were publicly proposed: 1. HZ+ or HZ+S, Steve Simpson's proposal 2. HZ-2, Ya-Gui Wei's proposal 3. B5E1, a 7-bit Big5 mapping scheme from Jing-Shin Chang 4. B5E2, another 7-bit Big5 mapping scheme from Jing-Shin Chang 5. (no name), Ross Paterson's proposal using CNS and Escape $ G, etc. 6. B5E3, another 7-bit Big5 mapping scheme from Jing-Shin Chang 7. (no name), Ed Lai's proposal to "use CNS as the exchange codes for Big5" (no details available) 8. (no name), Jing-Shin's proposal to use CNS (no details available) 9. (no name) Ricky Yeung's extended HZ framework proposal To avoid further confusion, the authors of the no-name proposals should designate names and, if they have not already done so, provide specification details. -- Steve From simpson@math.psu.edu Fri Jun 17 12:09:41 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA17806; Fri, 17 Jun 94 12:09:41 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id NAA20577; Fri, 17 Jun 1994 13:07:29 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 17 Jun 1994 13:07:28 -0400 Date: Fri, 17 Jun 1994 13:07:28 -0400 Message-Id: <199406171707.NAA27823@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: HZ+ filters are available for MS-DOS Reply-To: simpson@math.psu.edu The C source code for my HZ+ filters is available at ftp.math.psu.edu, /pub/simpson/chinese/hzp. The makefile is for Unix, but the same source code also works under Turbo C for MS-DOS. The resulting MS-DOS executables are in /pub/simpson/chinese/hzp/msdos. The method of using them is the same as for Unix, as described in howto.doc (except use "type" instead of "cat", etc.) -- Steve From simpson@math.psu.edu Fri Jun 17 12:24:29 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA18734; Fri, 17 Jun 94 12:24:29 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id NAA20655; Fri, 17 Jun 1994 13:22:21 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 17 Jun 1994 13:22:20 -0400 Date: Fri, 17 Jun 1994 13:22:20 -0400 Message-Id: <199406171722.NAA27849@boole.math.psu.edu> To: soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu Subject: thoughts on Emacs interfaces for HZ and HZ+ Reply-To: simpson@math.psu.edu Does anyone have any thoughts or ideas to share with me about Emacs news and mail interfaces for HZ and HZ+? Under Unix, we can make good use of Emacs's asynchronous process capabilities. For example, if you are using VM or GNUS to read mail or news, and if you notice that a particular message is in HZ, you can type "|hz2gb" to see the message instantly translated into GB Chinese (assuming you have are in a GB environment and have Fung's hz2gb filter). There are various hooks to automate and customize this process. Similar tricks can be done with HZ+ using my HZ+ filters in a GB or Big5 environment. If you are running Mule instead of plain Emacs, there are even more possibilities. -- Steve From simpson@math.psu.edu Sat Jun 18 09:58:09 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA27466; Sat, 18 Jun 94 09:58:09 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id KAA25885; Sat, 18 Jun 1994 10:54:24 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Sat, 18 Jun 1994 10:55:57 -0400 Date: Sat, 18 Jun 1994 10:55:57 -0400 Message-Id: <199406181455.KAA29973@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: JD@EREMITA.DEMON.CO.UK's message of 17 Jun 1994 15:05:56 -0500 Subject: HZ+ and ISO 2022 Reply-To: simpson@math.psu.edu I wrote: >Here "block 0" is 0x8140-0xA040, the ETen Big5 area reserved for >user-defined characters and symbols, as pointed out by Bob Sandstrom. Sorry, I meant 0x8140-0xA0FE. This is one of the ETen Big5 user areas, but as Sandstrom pointed out, 0xFA40-0xFEFE (part of Big5 block 2) is also designated by ETen as a user area. John Delacour wrote: > Well, at least it makes sense to have a special escape sequence for > the ETEN user area, but I see it uses control characters from the > extension. Intuitively I feel this is a problem and would exclude > HZ+ from any integration with a multilingual environmment. This is > just a feeling I have, so please, ye gods, comment. I don't see any problem here. 0x81-0xA0 are not really control characters. We already know that 8-bit Big5 is not compatible with ISO 2022, so 0x8140-0xA0FE does not make Big5 any worse in this respect. As for 7-bit Big5 in HZ+, adding "block 0" would preserve ISO 2022 compatibility. We would still be able to embed HZ+ into a Ross-like brand of 7-bit ISO 2022, by translating ~{ to Escape $ A ~} to Escape ( B ~0 to Escape $ ( 0 ~1 to Escape $ ( 1 ~2 to Escape $ ( 2 John wrote: > If there is a problem with 0x1B There are indeed problems with 0x1B, i.e. Escape. Postings by Kenichi Handa and Yidao Cai indicate that Escape-eating news feeds have been exterminated in Japan, but not in the rest of the world. Jianqing Hu's posting points to problems involving Escape and Unix tty drivers. I feel that this information further confirms the need for something like HZ+ for international Chinese news and e-mail. Of course HZ+ is intended only as a temporary, low-road solution, to tide us over until the arrival of high-road solutions, like ISO-2022 and Unicode. But I do feel that something like HZ+ can be useful now. -- Steve From simpson@math.psu.edu Sun Jun 19 10:37:22 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA06630; Sun, 19 Jun 94 10:37:22 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id LAA00440; Sun, 19 Jun 1994 11:35:01 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Sun, 19 Jun 1994 11:34:59 -0400 Date: Sun, 19 Jun 1994 11:34:59 -0400 Message-Id: <199406191534.LAA02379@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: Re: reply to John's points about Big5 and CNS In-Reply-To: <9406191408.aa12679@hinge.demon.co.uk> References: <9406191408.aa12679@hinge.demon.co.uk> Reply-To: simpson@math.psu.edu I wrote: > >No, here I must emphatically disagree. The division into more > >frequently used and less frequently used characters is not irrelevant. > >It is an essential structural feature of Big5. .... John replied: > I see no point of disagreement in any of this. You seem to be looking for > roadblocks where there are none. Well, maybe I'm a little touchy or over-sensitive on this issue. If I'm touchy, it's a result of my having had to combat absurd statements such as Jing-Shin's: "The designation of which characters as block 1 chars and which characters as block 2 chars are not yet decided for Big5 except for some private designations." (This is a quote from one of Jing-Shin's mapping scheme documents.) Here Jing-Shin is trying to deny the existence of a significant structural feature of Big5, namely the natural division into two blocks containing the lists of more and less frequently used characters, respectively, ordered by stroke count and radical. This feature of Big5 is obvious to anyone who has looked at a Big5 table, but Jing-Shin denies it. The reason Jing-Shin denies it is that he wants to push his own mapping schemes, which don't respect it. This is the kind of deliberate disinformation that I have had to deal with. > There is nothing wrong with your code at all -- very nice and tidy > and perfectly practical and efficient, but what of the future? Thanks for the complement. As for the future, I imagine that the world will eventually migrate to ISO 2022 and/or Unicode. But this may be a long way off, and in the meantime, popular standards such as HZ and Big5 will play a significant role. That is the entire rationale for HZ+. > Where are planes 3, 4 and 5 of Big-5? They don't exist, of course. My understanding is that CNS planes 3, 4, 5, ... contain obscure characters that are sometimes needed in Taiwan, e.g. for police logs where people's names have to be recorded accurately. This is certainly of some value. If it bothers you that Big5 doesn't contain anything similar, just use the CNS stuff. But please, don't mess with Big5. > How in future is a Swede or a Russian going to mix Swedish or > Russian text with Chinese text? Unicode and ISO-2022 provide ways to do this. Big5 can also fit in, because you can translate Big5 -> Unicode as shown in the Unicode tables, and Big5 -> HZ+ -> ISO-2022 as mentioned in my earlier posting. There is no real need for CNS here. I grant that CNS has an inside track in the race with Big5, because it is an official ROC government standard. But Big5 is winning the race anyway. The reason is that Big5 is popular, while CNS is not. > Please let me have some examples in hexadecimal notation of Big-5 > characters that cannot be converted to CNS and back again. I find > it hard to believe but I await your proof. OK. How about the 365 characters at C6A1-C8D3 (including some numerals, special symbols, Japanese kana, and the Russian alphabet), and the 41 characters at F9D6-F9FE (including some ETen-specific Chinese characters and some graphical characters). Of course I don't claim that these characters *cannot* be translated. Anybody could easily find some unused code space in CNS and define a mapping of the ranges that I mentioned into that code space. My only point is that there is no *standard* translation into CNS. In other words, Big5 -> CNS translation is subject to ambiguities and pitfalls. HZ+ neatly avoids all these problems. I wrote: > >Anybody who says there are no problems with Big5<->CNS translation is > >whistling in the dark. John replied: > wheeee whooo wee wee hooo! :-) Whistle all you want. However, a more practical course of action is to come up with a way to neatly avoid these problems. HZ+ is the way. -- Steve From simpson@math.psu.edu Sun Jun 19 15:51:23 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA16002; Sun, 19 Jun 94 15:51:23 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id QAA01222; Sun, 19 Jun 1994 16:49:13 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Sun, 19 Jun 1994 16:49:12 -0400 Date: Sun, 19 Jun 1994 16:49:12 -0400 Message-Id: <199406192049.QAA02765@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: Re: reply to John's points about Big5 and CNS Reply-To: simpson@math.psu.edu I wrote: > >OK. How about the 365 characters at C6A1-C8D3 (including some > >numerals, special symbols, Japanese kana, and the Russian alphabet), > >and the 41 characters at F9D6-F9FE (including some ETen-specific > >Chinese characters and some graphical characters). John Delacour replied: > What about them? I asked for Big-5 characters. Are any of these included > in the 13,053 characters of Big-5? or does your "Big-5" contain 14,758 > characters? The 365 + 41 characters that I mentioned are not included in the 5401 + 7652 = 13053 that constitute the Big5 lists of more and less frequently used Chinese characters ordered by stroke count and radical, in blocks 1 and 2. However, Big5 does not consist solely of these lists, and my 365 + 41 characters are indeed part of Big5. I'm defining Big5 the way ETen defines Big5. References for my definition are the ETen manual and my document ftp.math.psu.edu, /pub/simpson/chinese/hzp/etfonts.doc. My definition makes a lot of sense, because ETen is by far the most popular Big5 environment. ETen Big5 is a de facto standard not only for ETen users but also more widely, e.g. on tw.* newsgroups and Internet BBS systems (according to information from Nelson). You haven't mentioned how you're defining Big5. > In that case we're not talking about the same thing. Maybe not. In any case, my point still stands. My point, you may recall, was that Big5 to CNS conversion is subject to ambiguities and pitfalls. Some pitfalls may arise from the possibility that, for instance, your Big5 may be different from my Big5, and therefore your idea of Big5 to CNS conversion may be different from my idea of Big5 to CNS conversion. Do you now concede my point? Another point I'd like to emphasize is that Big5 to HZ+ conversion is *not* subject to any of these ambiguities and pitfalls. Big5 to HZ+ conversion is always the same, no matter whose Big5 we are talking about. This implies that Big5 to HZ+ to Big5 will always work 100 percent correctly, something that cannot be said for Big5 to CNS to Big5. This is one aspect of how HZ+ supports Big5 much better than any imaginable scheme based on CNS conversion. -- Steve From simpson@math.psu.edu Mon Jun 20 10:50:06 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA17873; Mon, 20 Jun 94 10:50:06 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id LAA10043; Mon, 20 Jun 1994 11:47:38 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Mon, 20 Jun 1994 11:47:36 -0400 Date: Mon, 20 Jun 1994 11:47:36 -0400 Message-Id: <199406201547.LAA08289@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: Big5 to CNS conversion algorithms In-Reply-To: <199406192349.TAA01730@leibniz.math.psu.edu> References: <199406192349.TAA01730@leibniz.math.psu.edu> Reply-To: simpson@math.psu.edu I wrote: > >I'm defining Big5 the way ETen defines Big5....... John Delacour writes: > Yes, exactly! And only yesterday you were telling us all not to mess about > with Big-5 and here you go messing about with it yourself and redefining it > according to ETEN, whoever he is :-) I'm not messing with Big5, and I don't know why you say that I am doing so. Nelson and I gave good reasons why, if we have to choose one vendor's Big5 as a standard, ETen's would be a good choice. If you want to claim that I am messing with Big5 and you are not, then why have you never gotten around to defining exactly what you mean by Big5? See here John, let's cut through the rhetoric and talk about the facts. It is a fact of life that Big5 is a popular standard, not an official standard. This implies that Big5 comes in many different variants. I'm not saying that you have to accept ETen Big5 as the one and only Big5, but you do have to accept the existence of vendor-specific and other variants of Big5. If you can't accept this fact, then you will never be able to come to terms with Big5 (in a psychological sense). It's no good your trying to tell millions of ETen users that their Big5 is not really Big5. If you try to ignore reality in this way, the product that you come up with will be unacceptable. > >.................. This is one aspect of how HZ+ supports Big5 > >much better than any imaginable scheme based on CNS conversion. > > So you say, but it's not so. If it's not so, could you please point to the facts which disprove it? > I have no argument with HZ+S -- it works fine, > but Ross's function using CNS does just as well. I question whether Ross's function works just as well. Could you please run Ross's code on the 365 + 41 Big5 characters that I mentioned earlier (in response to your challenge), and tell me what you get? I suspect that you will get garbage. > HZ+S translates 0xF9DA to 0x724B > Ross's Big5>CNS code translates it to 0x7249 > Very close, by chance, but both are invertible. In examining the merits of Big5 to CNS conversion algorithms, invertibility is not the only relevant point. Another relevant point is whether, to use your example, 0x7249 is really a CNS code for the same Chinese character as ETen Big5's 0xF9DA. I suspect that it isn't. If it isn't, then we can surmise that it is somehow peculiar to Ross's particular Big5 to CNS conversion algorithm, and that other Big5 to CNS conversion algorithms (for example III's chcode) will give different results. This is an example of the pitfalls and ambiguities in Big5 to CNS conversion that I am talking about. -- Steve From shin@hermes.ee.nthu.edu.tw Mon Jun 20 14:02:22 1994 Return-Path: Received: from nthu.edu.tw by ifcss.org (4.1/IFCSS-Mailer) id AA25881; Mon, 20 Jun 94 14:02:22 CDT Received: from hermes.ee.nthu.edu.tw ([140.114.28.27]) by nthu.edu.tw with SMTP id <181968(2)>; Tue, 21 Jun 1994 03:06:01 +0800 Received: by hermes.ee.nthu.edu.tw (4.1/SMI-4.0/shin-1.0) id AA22027; Tue, 21 Jun 94 02:58:06 CST Date: Tue, 21 Jun 1994 16:58:06 +0800 From: shin@hermes.ee.nthu.edu.tw (Jing-Shin Chang) Message-Id: <9406201858.AA22027@hermes.ee.nthu.edu.tw> To: ccnet-l@uga.cc.uga.edu, simpson@math.psu.edu, soft-authors@ifcss.org Subject: Re: Big5 to CNS conversion algorithms Ah-, I wish I had more time to clarify something (and probably dis-flame, which is not good to health.-^^-). It's strange that the author of the B5E1/2/3 stuffs are called by name with a very high frequency during the last week when he is silent ... and why people are keeping silent -^^- The following are not answers, but some questions (for everybody). 1. If you preferred ETen/Big5, how about ETen/CNS? I mean serious. There are good stuffs like Japanese Kana, circle numbers, etc. which are not either in the official Big5 code space nor in the official CNS spec. (The circled numbers are in official CNS though.) 2. Do you think ETen/Big5 <-> ETen/CNS conversion has any ambiguity? 100% convertible? 3. I think, if we have to choose one vendor's CNS as a standard, ETen's would be a good choice. How do you like it? 4. Should we sumarise that some conversion algorithms are peculiar because the ETen/Big5 <-> CNS produce something? (* P.S. for people who cannot carefully distinguish the user areas from the Eten extensions and stuffs like that, you may want to consult my B5MAP or b5.map in hermes.ee.nthu.edu.tw:/shin/hzb5encode/doc. The ETen extensions are following immediately after the official code space, and no codes are in the U1-U3 areas in ETen's implementation.) Jing-Shin -^^- |> From simpson@math.psu.edu Tue Jun 21 00:07:52 1994 |> |> I wrote: |> > >I'm defining Big5 the way ETen defines Big5....... |> |> John Delacour writes: |> > Yes, exactly! And only yesterday you were telling us all not to mess about |> > with Big-5 and here you go messing about with it yourself and redefining it |> > according to ETEN, whoever he is :-) |> |> I'm not messing with Big5, and I don't know why you say that I am |> doing so. Nelson and I gave good reasons why, if we have to choose |> one vendor's Big5 as a standard, ETen's would be a good choice. If |> you want to claim that I am messing with Big5 and you are not, then |> why have you never gotten around to defining exactly what you mean by |> Big5? |> |> See here John, let's cut through the rhetoric and talk about the |> facts. It is a fact of life that Big5 is a popular standard, not an |> official standard. This implies that Big5 comes in many different |> variants. I'm not saying that you have to accept ETen Big5 as the one |> and only Big5, but you do have to accept the existence of |> vendor-specific and other variants of Big5. If you can't accept this |> fact, then you will never be able to come to terms with Big5 (in a |> psychological sense). |> |> It's no good your trying to tell millions of ETen users that their |> Big5 is not really Big5. If you try to ignore reality in this way, |> the product that you come up with will be unacceptable. |> |> > >.................. This is one aspect of how HZ+ supports Big5 |> > >much better than any imaginable scheme based on CNS conversion. |> > |> > So you say, but it's not so. |> |> If it's not so, could you please point to the facts which disprove it? |> |> > I have no argument with HZ+S -- it works fine, |> > but Ross's function using CNS does just as well. |> |> I question whether Ross's function works just as well. Could you |> please run Ross's code on the 365 + 41 Big5 characters that I |> mentioned earlier (in response to your challenge), and tell me what |> you get? I suspect that you will get garbage. If you asked me about the III's code, I would say: I got a few blanks. But please don't blame the author (not me -^^-), they didn't know that the CCNET-L the tw.* newsgroups are asking for those Kana when the codes are written a few years ago. (Don't blame Ross too. -^^-) Someday, we may also be blamed for using something. So just don't blame anybody. So probably we just need to make up that part. |> |> > HZ+S translates 0xF9DA to 0x724B |> > Ross's Big5>CNS code translates it to 0x7249 |> > Very close, by chance, but both are invertible. |> |> In examining the merits of Big5 to CNS conversion algorithms, |> invertibility is not the only relevant point. Another relevant point |> is whether, to use your example, 0x7249 is really a CNS code for the |> same Chinese character as ETen Big5's 0xF9DA. I suspect that it |> isn't. If it isn't, then we can surmise that it is somehow peculiar |> to Ross's particular Big5 to CNS conversion algorithm, and that other |> Big5 to CNS conversion algorithms (for example III's chcode) will give |> different results. |> |> This is an example of the pitfalls and ambiguities in Big5 to CNS |> conversion that I am talking about. |> |> -- Steve |> From handa@etlken.etl.go.jp Mon Jun 20 21:03:06 1994 Received: from etlpost (etlpost.etl.go.jp) by ifcss.org (4.1/IFCSS-Mailer) id AA10562; Mon, 20 Jun 94 21:03:06 CDT Received: from etlpom.etl.go.jp by etlpost (5.67+1.6W/2.7W) id AA22945; Tue, 21 Jun 94 11:00:39 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA25707; Tue, 21 Jun 94 11:00:38 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA28304; Tue, 21 Jun 94 11:00:45 JST Date: Tue, 21 Jun 94 11:00:45 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9406210200.AA28304@etlken.etl.go.jp> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: HZ-EA [Re: Big5 to CNS conversion algorithms] I've just got font of Big5-ETen Ver.3.11 and found many characters in the range C6A1-C8D3. My feeling is WHAT A MESS!! There seems no policy on selected characters and the order of them. Apparently they borrowed characters from CNS11643, GB2312, JISX0208, and somewhere I don't know. Big5-ETen's free space is narrowing. When all code space in the range C6A1-C8FE is filled, they will add more characters over F9FE. And when space upto FEFE are filled, what comes? They may notice that they can even use FFxx, ... I don't know. Anyway, I feel very anxious about putting basis of communication in Internet on such an uncertainty. Now, my point of view on Big5-ETen is that it is not a simple character set but just an encoding mechanism of multiple character sets (CNS11643, GB2312, JISX0208, and several private characters) in 2-byte. So, I propose to design HZ-EA (East Asia) which is an upward compatible with HZ and can encode not only GB2312, but also CNS11643 and JISX0208 (and perhaps KSC5601). It also supplies a way to encode private characters. Then, what Big5-ETen (or other Big5) supporters should do is to find a map from their Big5 to GB2312, CNS11643, or JISX0208. Only in the case that they can't find a map, they may use private area. These works can be done separately from the design of HZ-EA. And, here's my proposal to HZ-EA. (although it may be too late and too confusing to propose a new one... :-( ) ------------------------------------------------------------ HZ-EA := [ ASCII* | '~{' GBSEQ* EACODE* '~}' ]* EACODE := '~A' GBSEQ | '~B' JISSEQ | '~G' CNS1SEQ | '~H' CNS2SEQ | '~0' PRVSEQ ASCII := sequence of ASCII code (TAB, LF, CR, FF, 0x20-0x7E) GBSEQ := 7-bit sequence of GB2312 code (0x2121-0x7E7E) JISSEQ := 7-bit sequence of JISX0208 code (0x2121-0x7E7E) CNS1SEQ := 7-bit sequence of CNS11643 Plane1 code (0x2121-0x7E7E) CNS2SEQ := 7-bit sequence of CNS11643 Plane2 code (0x2121-0x7E7E) PRVSEQ := 7-bit sequence of private character code (0x2121-0x7E7E) ------------------------------------------------------------ The usage of private characters should be discussed within Big5-ETen community or Big5-other communities. Mapping of Kana or Cyrillic to GB or JIS is also a matter of discussion within each communities. --- Ken'ichi HANDA handa@etl.go.jp PS. The next step of HZ-EA is perhaps HZ-INT (International). From lee@simd.stanford.edu Tue Jun 21 01:01:23 1994 Return-Path: Received: from simd.stanford.edu by ifcss.org (4.1/IFCSS-Mailer) id AA18356; Tue, 21 Jun 94 01:01:23 CDT Received: by simd.stanford.edu (5.65/inc-1.0) id AA13472; Mon, 20 Jun 1994 22:59:10 -0700 Date: Mon, 20 Jun 1994 22:59:10 -0700 From: lee@simd.stanford.edu (Fung Fung Lee) Message-Id: <9406210559.AA13472@simd.stanford.edu> To: soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu Cc: handa@etlken.etl.go.jp Subject: EHZ (Extended HZ for GB, CNS and BIG5) EHZ: Extended HZ for GB, CNS and BIG5 (Preliminary Draft) Fung Fung Lee * June 18, 1994 1. Introduction Following the HZ extension framework recently proposed by Ricky Yeung, I hereby propose two specific extensions to HZ for 7-bit encoding of CNS11643-1986 and BIG5. First, let me explain the terminology. The original HZ encoding scheme I proposed in 1989 can now be referred as HZ-1989, or just HZ for short. The current extended HZ will be referred as EHZ-1994, or just EHZ for short. Possible future extensions will be designated with the "EHZ" prefix. EHZ is a superset of HZ (the set of strings that are "accepted" by the EHZ parser is a superset of that acceptable by the HZ parser). Any future EHZ should always be a superset of previous EHZ. This is possible due to the highly extensible extension framework proposed by Yeung (see his proposal for details). The subset of EHZ that deals only with ASCII and a non-ASCII code X can be referred as EHZ-X. EHZ-CNS and EHZ-BIG5 are the topics of this paper. By following the methodolgy implied in this paper, it should be relatively straightforward to define other extensions, such as EHZ-JIS, in the future, if such goal is desirable. 2. Extension Scheme To allow for direct switching from one non-ASCII mode to another non-ASCII mode, say, BIG5 Plane 1 to BIG5 Plane 2, it is most natural to choose a two-byte escape sequence with '~' as the leading byte. Motivated by the escape sequences of ISO-2022 chosen for GB and CNS, we choose the following escape sequences: ~A GB ~G CNS Plane 1 ~H CNS Plane 2 ~g BIG5 Plane 1 ~h BIG5 Plane 2 ~f BIG5 Plane 0 As BIG5 is not an ISO registered standard, no escape sequence has been pre-defined for BIG5. The sequences ~g and ~h are chosen to reflect the approximate equivalence of the CNS and BIG5 character sets. The sequence ~f is for potential user-defined BIG5 codes (as explained later). The reader is reminded that ISO-2022 is used as a reference to limit the arbitrariness of the choice of the second bytes of the two-byte escape sequences. EHZ is NOT ISO-2022. As proposed by Yeung, the default non-ASCII mode is GB, mandated by backward compatibility, i.e., HZ should be a subset of EHZ. For example, the following examples are syntactically correct EHZ strings: This is ASCII. This is GB: ~{cccccc~}. This is also GB: ~{~Acccccc~}. This is CNS: ~{~Gcccccc~}. This is also CNS: ~{~Gcccc~Hcccc~Gcccccc~}. This is BIG5: ~{~gcccccc~}. This is also BIG5: ~{~hcc~gcccccccccc~}. This is mixed: ~{gcccc~Gcccccc~gcccccc~Acccc~Hcc~hcc~Acccc~}. For CNS11643-1986 (a subset of CNS11643-1992), or CNS for short, the choice of mapping is trivial, because CNS was defined as a 7-bit code based on graphical ASCII character in the range 0x21-0x7E. There exists a CNS internal code that has a straightforward one-to-one correspodence to the CNS exchange code: setting the high bit of the first byte, and setting or not setting the high bit of the second byte depending on whether the code is in Plane 1 or Plane 2. As for BIG5, the situation is somewhat tricky, because there is no pre-defined 7-bit mapping as in the case of CNS and GB. However, a natural starting point is to find "anchor code points" using the approximate correspondence between CNS and BIG5. Three such anchor points are the first member of the common symbols (the space symbol), the first member of the frequently-used hanzi, and the first member of the less-frequently-used hanzi. This "anchoring" principle says that each of these three members should share the same 7-bit code point in both EHZ-CNS and EHZ-BIG5. Successive members of the three above ranges in BIG5 are then assigned successive code points in EHZ-BIG5, as shown in Table 1. These three regions cover all the vendor-independent common parts of BIG5. For the two vendor-defined ranges (0xA3C0-0xA3FE, 0xC6A1-0xC8FE), the BIG5 codes could not be mapped to successive EHZ-BIG5 codes, because images of codes in the former range would overlap some existing CNS symbol code range, which is undesirable, and images of codes in the later range would extend beyond the end of Plane 1 (217D7E, 0x7D7E of Plane 1). Hence, a mapping is defined so that they do not overlap any defined CNS code ranges, are packed in order and assigned integer codes smaller than those of existing CNS symbols and larger than those of hanzi. In addition, it seems nice to map 0xA3C0 to 212940, and map 0xC6A1 to 212A21, because the respective second bytes differ by 0x80. There is indeed some arbitrariness in this process. Note that this is NOT an attempt to unify CNS and BIG5. For example, the 30 CNS numerals (212621-21263E) are present in one or the other of the above two vendor-defined ranges, depending on which vendors, such as KuoChiao and ETen. Such a unification would imply some kind of vendor affinity. This kind of bias is not desirable because it is not necessary at all in the first place. As for the remaining code range (0xF9D6-0xFEFE) within 0xA140-0xFEFE, the BIG5 codes can take on successive EHZ-BIG5 codes following the mapping of 0xF9D5. The BIG5 user-defined code range 0x8140-0xA0FE is too large to fit into the remaining unused code space in Planes 1 and 2 of EHZ-BIG5, even if the range is fragmented. A new code plane, Plane 0, is needed for the completeness of the BIG5 to EHZ-BIG5 transformation, although the use of any user-defined code on Internet for open communication is discouraged in general. Table 1: EHZ mapping tables --------------------------------------------------------------------------- EHZ-GB (HZ) GB2312-1980 internal code ("high-bit set") 2121-7D7E <-> A1A1-FDFE (2nd byte range: A1-FE) in more details: 2121-297E <-> A1A1-A9FE 846 symbols (including "blank" ones) 3021-5779 <-> B0A1-D7F9 3755 frequently-used hanzi 5821-777E <-> D8A1-F7FE 3008 less-frequently-used hanzi other code points no offical glyphs assigned ----------------------------------------------------------------------------- EHZ-CNS CNS11643-1986 internal code (ETen) (CNS11643-1992 Planes 1 and 2) 212121-217D7E <-> A1A1-FDFE (2nd byte range: A1-FE) 222121-227D7E <-> A121-FD7E (2nd byte range: 21-7E) in more details: a. *212121-212570 <-> A1A1-A5F0 408 common symbols + 48 "blanks" b. 212621-21263E <-> A6A1-A6BE 30 numerals c. 212721-212939 <-> A7A1-A9B9 213 radicals f. 214221-214241 <-> C2A1-C2C1 33 other symbols g. *214421-217D4B <-> C4A1-FDCB 5401 frequently-used hanzi h. *222121-227244 <-> A121-F244 7650 less-frequently-used hanzi *227245-227246 <-> F245-F246 no official glyphs assigned other code points no official glyphs assigned ----------------------------------------------------------------------------- EHZ-BIG5 BIG5 internal code (ETen, Chinese Windows 3.1, MacOS) (2nd byte ranges: 40-7E, A1-FE) u. 202121-20564A <-> 8140-A0FE 5024 user defined symbols a. *212121-212540 <-> A140-A3BF 408 common symbols d. 212940-21297E <-> A3C0-A3FE 63 vendor defined hanzi/symbols e. 212A21-212E40 <-> C6A1-C8FE 408 vendor defined hanzi/symbols g. *214421-217D4B <-> A440-C67E 5401 frequently-used hanzi h. *222121-227246 <-> C940-F9D5 7652 less-frequently-used hanzi i. 227247-227B32 <-> F9D6-FEFE 826 vendor defined hanzi/symbols other code points unused ----------------------------------------------------------------------------- Note that the common symbols (a), the frequently-used hanzi (g), and the less-frequently-used hanzi (h) of CNS and BIG5 both map to the same code ranges by conscious design (with the minor exception of BIG5 having 2 more hanzi in region h). However, two identical code points, one from EHZ-CNS, another from EHZ-BIG5, do not necessarily correspond to the same glyph. This is because CNS was supposed to be a cleanup of BIG5, involving re-ordering of glyphs according to proper stroke counts, and removal of redundant and/or incorrect glyphs. Even ignoring the difference in the non-hanzi part and the vendor-specific part of the character sets, conversion between CNS and BIG5 is far from straightforward. ----------------------------------------------------------------------------- * Author's email address: lee@csl.stanford.edu Disclaimer: This work has no relation to my employer. From simpson@math.psu.edu Tue Jun 21 11:31:39 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA08454; Tue, 21 Jun 94 11:31:39 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id MAA05744; Tue, 21 Jun 1994 12:29:23 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Tue, 21 Jun 1994 12:29:22 -0400 Date: Tue, 21 Jun 1994 12:29:22 -0400 Message-Id: <199406211629.MAA10550@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: Re: Big5 to CNS conversion algorithms In-Reply-To: <9406201858.AA22027@hermes.ee.nthu.edu.tw> References: <9406201858.AA22027@hermes.ee.nthu.edu.tw> Reply-To: simpson@math.psu.edu Jing-Shin Chang writes: > I wish I had more time to clarify something (and probably > dis-flame, which is not good to health.-^^-). It's strange that the > author of the B5E1/2/3 stuffs are called by name with a very high > frequency during the last week when he is silent ... and why people > are keeping silent -^^- All of this is too subtle for me to understand. If you want to say something, please say it. I know that your time is precious, but I still feel that you should try to express yourself clearly, so that people can understand you. > 1. If you preferred ETen/Big5, how about ETen/CNS? I mean serious. There > are good stuffs like Japanese Kana, circle numbers, etc. which > are not either in the official Big5 code space nor in the official > CNS spec. (The circled numbers are in official CNS though.) I'm not enthusiastic about any version of CNS. The reason I'm not enthusiastic is that CNS is not popular. I prefer Big5 over CNS, because Big5 is about 1000 times as popular as CNS. However, if somebody required a vendor-specific CNS for some specialized application, I might recommend ETen CNS over other vendor-specific versions of CNS (depending on the application), simply because millions of people use ETen. I regard all of this as irrelevant to our main problem, namely to decide what kind of 7-bit Chinese we want to use on the Internet. In my opinion, our low-road solution should focus on the two popular Chinese coding standards, GB/HZ and Big5. CNS is irrelevant. > 2. Do you think ETen/Big5 <-> ETen/CNS conversion has any ambiguity? > 100% convertible? This kind of conversion is probably 99.9 percent invertible. One small problem that I know of arose because CNS eliminated two duplicate characters in Big5. However, even though there may be only a little ambiguity in ETen Big5 <-> ETen CNS conversion, there is still a lot of ambiguity and pitfalls in the *general* concept of Big5 <-> CNS conversion. This is one reason why it makes no sense to speak of "using CNS codes as the exchange codes for Big5 codes". > 4. Should we sumarise that some conversion algorithms are peculiar because > the ETen/Big5 <-> CNS produce something? I can't understand this question. -- Steve From simpson@math.psu.edu Tue Jun 21 11:47:05 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA09186; Tue, 21 Jun 94 11:47:05 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id MAA05865; Tue, 21 Jun 1994 12:44:54 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Tue, 21 Jun 1994 12:44:52 -0400 Date: Tue, 21 Jun 1994 12:44:52 -0400 Message-Id: <199406211644.MAA10574@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: Kenichi Handa's HZ-EA proposal In-Reply-To: <199406210203.WAA01473@leibniz.math.psu.edu> References: <199406210203.WAA01473@leibniz.math.psu.edu> Reply-To: simpson@math.psu.edu Kenichi Handa writes: > So, I propose to design HZ-EA (East Asia) which is an upward > compatible with HZ and can encode not only GB2312, but also > CNS11643 and JISX0208 (and perhaps KSC5601). It also > supplies a way to encode private characters. > > Then, what Big5-ETen (or other Big5) supporters should do is > to find a map from their Big5 to GB2312, CNS11643, or > JISX0208. Only in the case that they can't find a map, they > may use private area. These works can be done separately > from the design of HZ-EA. Kenichi correctly pointed out that Big5 is messy. However, this fact does not allow us to ignore the needs of millions of Big5 users. In my opinion, HZ-EA as a popular standard will never be successful, because it does not support Big5. -- Steve From simpson@math.psu.edu Tue Jun 21 16:01:24 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA21106; Tue, 21 Jun 94 16:01:24 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id QAA08385; Tue, 21 Jun 1994 16:59:12 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Tue, 21 Jun 1994 16:59:11 -0400 Date: Tue, 21 Jun 1994 16:59:11 -0400 Message-Id: <199406212059.QAA10988@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: Re: Big5 to CNS conversion algorithms In-Reply-To: <9406212028.aa05861@hinge.demon.co.uk> References: <9406212028.aa05861@hinge.demon.co.uk> Reply-To: simpson@math.psu.edu John Delacour writes: > Every specification of Big-5 I have ever seen mentioned in print or > on the 'net comprises 13,053 points in a 157*94 grid Really? Could you point me to one such? All specifications of Big5 that I have seen (in print or elsewhere) include in addition the 408 "common symbols" (Roman and Greek alphabets, punctuation, Arabic numerals, etc.) at A140-A3BF. These are over and above the list of 5401 frequently used Chinese characters at A440-C67E and the list of 7652 less frequently used Chinese characters at C940-F9D5, comprising the 13053 that you mention. So far as I know, the 408 common symbols are not vendor-specific, i.e. they are common to all vendors' Big5 implementations. But in addition to these 408 common symbols, there is vendor-specific Big5 stuff such as ETen's Japanese kana and additional Chinese characters above F9D5. > ETEN has nothing to do with the discussion. I disagree. ETen is the most popular Big5 software, so obviously it's relevant. I wrote: > >I question whether Ross's function works just as well. Could you > >please run Ross's code on the 365 + 41 Big5 characters that I > >mentioned earlier (in response to your challenge), and tell me what > >you get? I suspect that you will get garbage. You retorted: > Well... NO! Obviously if I have no Chinese fonts installed, I'll get > garbage, and if I have JIS installed instead of Big-5 I'll get garbage. I suspect you will get garbage even if you have Big5 and/or CNS fonts and software. -- Steve From A7621GAC@AWIUNI11.EDVZ.UNIVIE.AC.AT Tue Jun 21 16:21:20 1994 Return-Path: Received: from AWIUNI11.EDVZ.UniVie.AC.AT (helios.edvz.univie.ac.at) by ifcss.org (4.1/IFCSS-Mailer) id AA21847; Tue, 21 Jun 94 16:21:20 CDT Message-Id: <9406212121.AA21847@ifcss.org> Received: from AWIUNI11.EDVZ.UNIVIE.AC.AT by AWIUNI11.EDVZ.UniVie.AC.AT (IBM VM SMTP V2R2) with BSMTP id 9104; Tue, 21 Jun 94 23:18:50 MEZ Received: from AWIUNI11.EDVZ.UNIVIE.AC.AT (NJE origin A7621GAC@AWIUNI11) by AWIUNI11.EDVZ.UNIVIE.AC.AT (LMail V1.2a/1.8a) with BSMTP id 8216; Tue, 21 Jun 1994 23:18:49 +0100 Date: Tue, 21 Jun 94 23:15:07 MEZ From: Werner Lemberg Subject: ANNOUNCING CJK.sty To: soft-authors@ifcss.org I've written a LaTeX2e package for Chinese/Japanese/Korean. You can find it soon at all CTAN archives (pip.shsu.edu, ftp.dante.de and one English host I always forget :-) in tex-archive/language/chinese/CJK. Here is the readme file. Werner This is the LaTeX2e style package CJK.zip Version 1.0 (1-June-1994) =================================================================== It contains the following files: CJK.txt This file CJK.sty A LaTeX2e style file to enable CJK (Chinese/Japanese/Korean) logographs (i.e. Hanzi/Kanji/Hanja) with LaTeX2e CJK.enc Encoding Schemes (at the moment only for Chinese) UBg5.fd UGBs.fd UGBt.fd Font definition files (examples only!) UpmC-Bg5.fd UpmC-GBs.fd UpmC-GBt.fd Font definition files for (old) pmC-fonts This is freely distributable under the GNU Public License. Use --- Use CJK.sty as a package, e.g. \documentclass{article} \usepackage{CJK} . A new environment \begin{CJK}{encoding}{shape} ... \end{CJK} is defined; encoding at the moment only Bg5 (Big5), GBs (GuoBiao with simplified characters), and GBt (GuoBiao with traditional characters) are defined in CJK.enc, but it's very easy to add different encoding schemes. For compatibility with the pmC package three additional encodings are defined: pmC-Bg5, pmC-GBs, and pmC-GBt. It's not encouraged to use these encodings because of wasting fonts. If possible, convert your original CJK-bitmaps with bmf2gf (see below) to CJK-encodings. shape It is impossible to know what fonts are available at your site; look at the example .fd-files how to create appropriate .fd-files suiting your needs. This is a very realistic example: \begin{CJK}{GBs}{kai} ... \end{CJK} How it works ------------ Asiatic logographs can't be represented with one byte per character. (At least) two bytes are needed, and the most common encoding schemes (GB, Big5, JIS, KS etc.) have a certain range for the first byte (usually 0xA1-OxFE or a part of it) which signales that this and the next byte represents an Asiatic logograph. This means that plain ASCII-text (i.e. characters between 0x00 and 0x7F) will be left undisturbed, and most characters of the extended ASCII character set (0x80-0xFF) will be assigned to a CJK encoding. CJK.sty will make the characters 0xA1-OxFE active (and in the file CJK.enc you should inactivate those characters not needed to be active in a special encoding scheme) inside of an environment and assigns the macros \CJK@char and \CJK@charx to the active characters which select the proper font. The real mechanism is a bit more complicated to assure robustness (it was borrowed and modified from german.sty). The encodings ------------- CJK.sty defines \CJK@standardEncoding, \CJK@BigfiveEncoding and, for compatibility with pmC, \CJK@pmCsmallEncoding and \CJK@pmCbigEncoding. \CJK@standardEncoding should be used for encodings with the second byte in the range 0xA0-0xFE (GB, KS, JIS etc). \CJK@BigfiveEncoding should be used for Big5 encodings (e.g. ETen fonts) with the second byte in the range 0x40-0xFE. \CJK@pmCsmallEncoding and \CJK@pmCbigEncoding can be activated with \pmCsmall (this is the default) and \pmCbig inside the CJK environment. Note that the original pmC fonts have two character sizes per font (the bigger ones with an offset of -128); pmC-Big5 encoded fonts cannot contain big characters. The names of the fonts in the UpmC-xxx.fd files reflect the modifications added by Marc Leisher (mleisher@nmsu.edu) to the original poor man's Chinese (pmC) package written by Thomas Ridgeway (ridgeway@blackbox.hacc.washington.edu). The fonts --------- CJK.sty uses NFSS (New Font Selection Scheme) which has some advantages over the font selection offered with pmC: o TeX fonts are loaded only on demand. This is especially useful with Asiatic logographs. If you have e.g. three Chinese characters in your text, pmC must load the whole Chinese font (about 85 TeX fonts), whereas LaTeX2e loads at least three fonts. o As long as the limit of 256 TeX fonts will not be exceeded, you can use as many CJK fonts as you like (e.g. simplified and traditional Chinese characters together with Japanese fonts in different sizes). pmC is limited to two sizes and can only have two CJK fonts at the same time. o You need not to care about the right size of the CJK fonts in footnotes etc. It will obey the NFSS (although changing any other attribute like the family or the shape will not affect the CJK font). Of course you must have proper sizes of your CJK fonts --- if you have, say, 48x48 and 64x64 bitmap fonts (or more), I recommend bmf2gf to convert them directly to .pk-fonts. If you have only 24x24 size (as most of the public domain fonts are), you can magnify them using bmf2gf but only with integer magnifications. If you chose one font per active character as with the pmC macros, you would waste character space (256 characters per font are possible with TeX 3). Therefore CJK.sty expects the whole Chinese font splitted in TeX fonts with 256 characters each. An example: GuoBiao-encoded simplified characters in song style 12pt: ^ ^ ^^ ^^ first byte second byte TeX font offset ---------------------------------------------- 0xA1 0xA1-OxFE gsso0112 0 0xA2 0xA1-0xFE gsso0112 94 0xA3 0xA1-0xE4 gsso0112 188 0xA3 0xE5-0xFE gsso0212 0 0xA4 0xA1-0xFE gsso0212 26 0xA5 0xA1-0xFE gsso0212 120 . . . 0xFE 0xA1-OxFE gsso3512 38 To create such splitted fonts you should use the package bmf2gf.zip (written by me) to be found at ifcss.org for example. Big5 encoded fonts are a bit more complicated to convert because of a gap in the range of the second byte: 0x7F-0xA0 is not used. Additionally some of the possible code points are unused --- I'm writing a program hbf2bmf which will enable automatic conversion using HBF (Hanzi Bitmap Font) headers into a form bmf2gf can understand. You should get the appropriate HBF header file from ifcss.org (or create if you can't find the right one); almost all Chinese bitmap fonts in the public domain together with their HBF headers are collected there. Caveats ------- o You can of course use CJK-environments inside of a CJK-environment, but it is possible that you must increase the so called save size (with emTeX you can adjust this with -ms=...). The CJK package has an optional argument which controls the scope of CJK environments: global \lccode, \uccode, \catcode and the activation of the characters 0xA1-0xFE will be globally modified (\lccode and \uccode reset to 0). This is the most economical mode concerning save size, but you can't have CJK environments inside of CJK environments or other environments which manipulate the character range 0xA1-0xFE. Packages which change some of the above values only once (e.g. in the preamble) will also not work after the first use of a CJK environment. local Only \lccode and \uccode will be modified globally. This is the default. You can stack environments. encapsulated If you want to use DC fonts outside of the CJK environment with \uppercase and \lowercase working correctly, you must use this option. All values mentioned above will be local, so you can stack environments. This option probably causes an overflow of the save size. Say \usepackage[option]{CJK} to activate 'option'. o The characters \, {, and } are used as second bytes in the Big5 encoding. If you write Big5 text, you should use the Bg5text environment which changes the category codes of these characters. The command prefix is now the forward slash /, and the grouping characters are ( and ) respectively. An example: \begin{CJK}{Bg5}{song} \begin{Bg5text} .... /begin(center) .... /end(center) .... /end(Bg5text) \end{CJK} o Be careful not to use any commands inside the Bg5text environment which write something in an external file (commands like \chapter etc.). To get the /, (, and ) characters, write //, /(, and /) inside the Bg5text environment. o If it's not possible to avoid Big5 character codes with \, {, or } outside of the Bg5text environment (e.g. having Big5-text in a \chapter or \section command), you can replace them with the \CJKchar macro manually: \section{This is a problematic Big5 character: \CJKchar{169}{92}} The parameters are the first and second byte of the Big5 character code. You can also use hexadecimal or octal notation. o You can't use Big5 characters in any command which uses \uppercase or \lowercase, i.e. \chapter, \article etc. --- this affects all standard document classes, which convert page headers to uppercase. It's possible to overcome this restriction by substituting all affected Big5 characters with the appropriate \CJKchar, i.e. for \uppercase characters with the second byte in the range to a-z (and A-Z for \lowercase respectively). A quick and dirty solution to disable \uppercase is to say \let\uppercase\relax before \begin{document}, but be aware that some commands like \Roman will behave unexpectedly. The clean way is either to use a document class which does not convert headers to uppercase or to write a small package modifying the sectioning commands to remove the \uppercase commands. o The same holds for ALL CJK encoding schemes if you use DC fonts because these 8-bit fonts have most \lccode and \uccode set in the range 0x80-0xFF. Possible errors --------------- o If you get the error message: "\CJK@min (or \CJK@max) undefined", you should insert a \newpage before saying \end{CJK}. This can happen if LaTeX writes the headers (or footers) of a page containing Chinese characters after closing the CJK environment. o If you get overfull hboxes caused by CJK characters, try to increase \CJKglue. It defines the glue between CJK characters; default value is \hskip 0pt plus 0.8pt . Author ------ Werner Lemberg (A7621GAC@univie.ac.at) Please report any errors or suggestions to this email-address. From A7621GAC@AWIUNI11.EDVZ.UNIVIE.AC.AT Tue Jun 21 16:26:55 1994 Return-Path: Received: from AWIUNI11.EDVZ.UniVie.AC.AT (helios.edvz.univie.ac.at) by ifcss.org (4.1/IFCSS-Mailer) id AA22026; Tue, 21 Jun 94 16:26:55 CDT Message-Id: <9406212126.AA22026@ifcss.org> Received: from AWIUNI11.EDVZ.UNIVIE.AC.AT by AWIUNI11.EDVZ.UniVie.AC.AT (IBM VM SMTP V2R2) with BSMTP id 9123; Tue, 21 Jun 94 23:24:27 MEZ Received: from AWIUNI11.EDVZ.UNIVIE.AC.AT (NJE origin A7621GAC@AWIUNI11) by AWIUNI11.EDVZ.UNIVIE.AC.AT (LMail V1.2a/1.8a) with BSMTP id 8264; Tue, 21 Jun 1994 23:24:24 +0100 Date: Tue, 21 Jun 94 23:19:12 MEZ From: Werner Lemberg Subject: ANNOUNCING bmf2gf 1.0 To: soft-authors@ifcss.org This is a major update to bmf2gf 0.1 . I've completely redesigned the program; it uses a configuration file now. You will need this program to convert your CJK bitmap fonts into appropriate TeX fonts for the CJK package and poor man's Chinese. It will be soon at all CTAN archives in tex-archive/language/chinese/bmf2gf. Here is the readme file. Werner This is the bmf2gf package Version 1.0 (1-June-1994) ======================================================== Copyright (C) 1994 Werner Lemberg bmf2gf will transform bitmap fonts (usually Chinese/Japanese/Korean fonts containing several thousands of characters) into TeX .gf--fonts (which will then be converted into .pk--fonts) according to the poor man's Chinese (pmC) macros for TeX and the CJK package for LaTeX2e. The programs PLtoTF and GFtoPK (should be part of every TeX package) must be in the path. bmf2gf.zip contains the following files: bmf2gf.txt this file history.txt the history file Makefile a makefile to compile bmf2gf for UNIX, OS/2, and DOS bmf2gf.def definition file (needed for compiling under OS/2 only) COPYING the GNU Copying License bmf2gf.w a CWEB file of the program bmf2gf.exe an executable for DOS and OS/2 bmf2gf.cfg sample configuration file for bmf2gf bmf2gf.dvi The cweb file in the final .dvi--form (see below) et2bmf.w Eten fonts to bitmap fonts (CWEB file) (see below) et2bmf.dvi The cweb file in the final .dvi--form (see below) emx.exe the emx DOS--extender (ver. 0.8h) (for VCPI) rsx.exe the rsx DOS--extender (rev. 3a) (for DPMI) emx.dll the emx runtime library for OS/2 (ver 0.8h) If you alter the CWEB files and you want to create a .dvi file again, you should (but not must) use the CWEAVE of the c2cweb--package (found at the CTAN archives, e.g. pip.shsu.edu) and then say make documentation This (fully compatible) CWEAVE can transform CWEB--files with alternative output rules the author (it's me too :-) prefer. Otherwise this file will be formatted traditionally. Anyway, you shouldn't alter the CWEB file! Instead, write a change file for bmf2gf.w . Copying ------- This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License version 2 as published by the Free Software Foundation. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. Remarks for UNIX systems user ----------------------------- Because of the many different UNIX implementations, no executables are included. bmf2gf should compile without great problems; you should say make unix to build the executable. You will need CTANGLE of the CWEB package to convert the .w files into .c files. How to use ---------- Chinese bitmap fonts can't be handled by TeX directly because TeX fonts are limited to 256 characters per font. bmf2gf will now break a CJK bitmap font (CJK stands for Chinese/Japanese/Korean) into .gf--fonts using the key values found in a configuration file. Additionally a batch file is created to convert the .gf--files (and one .pl--file) into .pk (and .tfm)--files. Here is a sample configuration file myconfig.cfg to convert the 24x24 font fansong.24 into 48x48 TeX fonts for use with CJK.sty of the CJK package: small_file fansong.24 small_size 24 small_mag 2 small_x_offset 3 small_y_offset -8 text_small fanti songti 24x24 pixel font magnified and adapted for 12 pt nmb_planes 1000 nmb_char_in_plane 256 first_plane 1 first_char 0 output_name gsso%02i12 dpi_x 300 dpi_y 300 checksum 123456789 coding codingscheme GuoBiao encoded TeX text job_extension .bat rm_command del cp_command copy pk_directory c:\emtex\texfonts\pixel.lj\300dpi\ tfm_directory c:\emtex\texfonts\tfm\ Some explanations: small_... pmC macros expect two sizes per font; there are pendants big_... for the second size small_mag only integer magnifications are allowed nmb_planes bmf2gf recognizes the end of a file, thus setting nmb_planes to an unrealistic high value you get all files. If e.g. nmb_files is set to 10, you get the first 10 TeX fonts. nmb_char_in_plane pmC correlates each TeX font with the first byte of the 2 byte CJK encoding (usually called a `plane'). See the .dvi file for a pmC example and a discussion of different encodings. Usually nmb_char_in_plane is the number of valid second byte values. CJK.sty always needs 256. first_plane CJK.sty normally expects 1 (as defined in the .fd files). first_char CJK.sty always needs 0. output_name This is a printf formatted string with one integer parameter which will be incremented by one after writing one TeX font. Keywords must start a line; the appropriate values must be on the same line separated with one or more blanks or tabs. Please read the bmf2gf.dvi file for a complete discussion of all keywords. The call bmf2gf myconfig.cfg creates the files gsso0112.gf, gsso0212.gf, ... , gsso3212.gf, gsso0112.pl, and gsso0012.bat . After calling gsso0012.bat you will find the .pk--files in the c:\emtex\texfonts\pixel.lj\300dpi directory and the .tfm--files in the c:\emtex\texfonts\tfm directory; all .gf--files and gsso0112.pl will be deleted. Big5 fonts ---------- This version of bmf2gf can only handle fonts with no gaps in the complete code range, which is not valid for Big5 fonts. If you want to convert such a font into .pk--files, you first must transform the Big5 file(s) into one large font file. Coming with this package you will find et2bmf.w which converts ETen fonts into the needed input file. This quick and dirty written program must be adapted to suit your needs. The next version of bmf2gf will support HBF--headers to get rid of this restriction. Author ------ Werner Lemberg (a7621gac@univie.ac.at) Please report any errors, comments or suggestions to this email--address. N.B.: If you like this program, send me a postcard ! Werner Lemberg Goldschlagstr. 52/14 A-1150 Vienna/Austria From handa@etlken.etl.go.jp Tue Jun 21 19:45:27 1994 Received: from etlpost (etlpost.etl.go.jp) by ifcss.org (4.1/IFCSS-Mailer) id AA29187; Tue, 21 Jun 94 19:45:27 CDT Received: from etlpom.etl.go.jp by etlpost (5.67+1.6W/2.7W) id AA04456; Wed, 22 Jun 94 09:43:12 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA10166; Wed, 22 Jun 94 09:43:11 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA28889; Wed, 22 Jun 94 09:43:18 JST Date: Wed, 22 Jun 94 09:43:18 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9406220043.AA28889@etlken.etl.go.jp> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Tue, 21 Jun 1994 12:44:52 -0400 <199406211644.MAA10574@boole.math.psu.edu> Subject: Re: Kenichi Handa's HZ-EA proposal On Tue, 21 Jun 1994 12:44:52 -0400, Stephen G Simpson said: > Kenichi correctly pointed out that Big5 is messy. However, this fact > does not allow us to ignore the needs of millions of Big5 users. In > my opinion, HZ-EA as a popular standard will never be successful, > because it does not support Big5. Millions of Big5 users won't care about it. They only care that their text can be exchanged correctly, which can be done if Big5 supporters decide the mapping from/to their Big5 variations to/from GB/CNS/JIS/private. Even in HZ+S, Big5 is not directly supported but encoder/decoder is required. The encoder/decoder of HZ-EA is longer than that of HZ+S's, the task is not that hard. --- Ken'ichi HANDA handa@etl.go.jp From yawei@cs.indiana.edu Tue Jun 21 20:56:15 1994 Return-Path: Received: from moose.cs.indiana.edu by ifcss.org (4.1/IFCSS-Mailer) id AA01417; Tue, 21 Jun 94 20:56:15 CDT Message-Id: <9406220156.AA01417@ifcss.org> Received: by moose.cs.indiana.edu (5.65c/9.4jsm) id AA15400; Tue, 21 Jun 1994 20:53:32 -0500 Date: Tue, 21 Jun 1994 20:53:32 -0500 From: "Ya-Gui Wei" To: alt.chinese.computing@usenet, ccnet-l@uga.cc.uga.edu, handa@etlken.etl.go.jp, soft-authors@ifcss.org Subject: Re: HZ-EA [Re: Big5 to CNS conversion algorithms] I am for Mr. Handa and Mr. Lee's proposals mostly. The following is some old mail which discussed some aspects that may have been missed in their proposals. (Original was posted to soft-authors some 14 months ago. Some errors have been corrected in this reposting.) My only concern lies in selection of the escape sequences in their proposals. I suggest that we eschew "similar-looking-ness" to ISO-2022 -- it makes no point in my opinion. I suggest that in assigning the escape sequences, we try to use the sequences that are not likely to occur in commonly used applications first -- such as Berkeley mail and the C language. (This may carry a bit of the programmer's bias, but I do write C code with embedded HZ text strings. It gets messy if ~A could mean "complement of A" as well as "start of GB".) If Big5 is to be included, I don't have serious objection to the current Mule's mapping of Big5 to ASCII symbols. But I do think CNS is a serious candidate and should be considered instead. Ya-Gui 6/21/94 -- A Proposal To Extended the Current "HZ" Specification To Accommodate Other East Asian Coding Standards (Draft 1.0a) Ya-Gui Wei April 07, 1993 The "HZ" representation (Fung Fung Lee, 1989) of GB2312 Chinese text has been implemented in various software packages under different platforms, and tested in the usenet newsgroup "alt.chinese.text" and various other forums with considerable success. This proposal seeks to expand the specification to _simultaneously_ accommodate other East Asian coding standards under an umbrella scheme that is analogous to the original "HZ" scheme. This proposal is first posted to soft-authors@ifcss.org to solicit comments, and may be posted to other forums at a later time. * ESSENSE of "HZ" The "HZ" standard defines a way of integrating ASCII characters with GB2312 Chinese characters (represented as two printable ASCII characters). The following ASCII sequences are defined as special: ~{ -- escape from ASCII to GB2312 ~} -- escape from GB2312 to ASCII ~~ -- the ASCII character ~ ~\n -- line continuation "HZ" further stipulates that each ASCII line must start in the ASCII context. Furthermore, the ~\n line continuation sequence is permitted in the ASCII context only. * CONSOLIDATION OF CODES I hereby propose to extend the "HZ" specification to create a Consolidated Collection of East Asian Coding Standards (CCEACODE, hereafter), to allow for the simultaneous transport of text of the different languages or coding standards. Briefly, all the remaining tilde- sequences of "HZ" will be defined as special under CCEACODE, which will be used as escape signals for transition among East Asian Language (EAL, hereafter) code blocks, as well as between EAL code blocks and ASCII. CODE BLOCKS: Code Blocks of 93 rows-by-94 columns will be defined by CCEACODE, with each character within an code block represented by two printable ASCII characters. If two consecutive characters are from the same code block, no special attention is needed. If two adjacent characters are from different code blocks, a special tilde- sequence is inserted in between to signal the transition between code blocks. For example, if ~{ is used for transition into GB (Chinese) text, and ~[ is used for transition into Japanese JIS text, and ~} is used for transition into ASCII text, then examples of mixed ASCII, GB and JIS text may look like: aaaa~{gbgbgbgb~[jijijiji~}aaaa aaaa~[jijijiji~{gbgbgbgb~}aaaa * PROPERTIES - Large Code Space. CCEACODE can accommodate 92 code blocks, with each block having 93 rows, and each row having 94 characters. This results in a code space of up to 804,000 characters, or 19.6 bit equivalence, while each character takes up only about 14 bits in the data stream. (Contrast this to 16-bit schemes, which would have a mere 65,000 character code space, with each character taking up 16 bits in the data stream.) - CCEACODE will be a superset of ASCII. Under CCEACODE, ASCII text will remain as 7-bit (single-byte) characters, with no need to widen them into multiple byte sequences. The only exception being that an ASCII "~" character will be represented by the two byte "~~" sequence. (Contrast this to 16-bit uniform width schemes, which would widen each ASCII character to 16 bits -- requiring all existing software to be re-written.) - CCEACODE will be a superset or partial superset of all the East Asian coding standards it seeks to accommodate. - CCEACODE will be a superset of the original "HZ" Specification. - CCEACODE encoded multilingual text will be indistinguishable from ASCII text at the byte level, thus ensuring compatibility with existing software and hardware. * SUPPORTED CODING STANDARDS CCEACODE does not attempt to define any of the characters within its code space. Instead, its code space will be allocated to the various national or industrial coding standards in blocks, and individual characters will be defined by these national or industrial coding standards. CCEACODE will also standardize the tilde- escape sequences and the exact positioning of national/industrial standard character sets within the CCEACODE code space. Only standards of wide-spread recognition and/or prominence will be accommodated in CCEACODE. Code blocks, each 93x94 in size, are allocated to accommodated standards by assigning a tilde- special sequence to the standard, indicating that text following the sequence will be interpreted using the said standard. Proposed CCEACODE tilde- special sequence assignments (undefined tilde- sequences are reserved for future expansion): ~} -- escape to ASCII ~{ -- escape to Chinese (GB2312) ~( -- escape to Chinese Big-5 primary (*) ~+ -- escape to Chinese Big-5 secondary ~[ -- escape to Japanese (JIS) ~] -- escape to Korean (KSC-5601) ~~ -- the ASCII character ~ ~\n -- line continuation (in ASCII context only) (\n = new line sequence, may be platform-specific.) (*) In lieu of Big-5, CNS may be a better candidate, mainly to avoid the problem of vender specific extensions. Vender specific extensions to member character sets, in principle, should not be supported. * NAME OF THE GAME This proposed specification shall be referred to as "Consolidated Collection of East Asian Coding Standards", with acronym CCEACODE (pronounced "see code"). I have intentionally avoided the term "Unified" in the name so as to avoid discussions of "Han Unification". I am not against "Han Unification" per se, but I believe it is incompatible with the spirit of accommodating existing standards. [End of Document] From Ricky.Yeung@Eng.Sun.COM Wed Jun 22 01:05:57 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA09219; Wed, 22 Jun 94 01:05:57 CDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA06511; Tue, 21 Jun 94 23:03:36 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA20779; Tue, 21 Jun 94 23:05:45 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA25514; Tue, 21 Jun 1994 23:04:51 +0800 Date: Tue, 21 Jun 1994 23:04:51 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406220604.AA25514@hk.Eng.Sun.COM> To: yawei@cs.indiana.edu Cc: alt.chinese.computing@usenet.ifcss.org, ccnet-l@uga.cc.uga.edu, handa@etlken.etl.go.jp, soft-authors@ifcss.org In-Reply-To: "Ya-Gui Wei"'s message of Tue, 21 Jun 1994 20:53:32 -0500 <9406220156.AA01417@ifcss.org> Subject: Escape sequence (Was Re: HZ-EA) Content-Length: 1217 > My only concern lies in selection of the escape > sequences in their proposals. I suggest that > we eschew "similar-looking-ness" to ISO-2022 -- > it makes no point in my opinion. I suggest that in > assigning the escape sequences, we try to use the > sequences that are not likely to occur in commonly > used applications first -- such as Berkeley mail > and the C language. (This may carry a bit of the > programmer's bias, but I do write C code with embedded > HZ text strings. It gets messy if ~A could mean > "complement of A" as well as "start of GB".) This ASCII transparency problem is the exactly the reason why I proposed the Extended HZ framework. In EHZ, choosing escape sequences like ~A for GB, is NOT a problem at all, because "start of GB" (or whatever non-ASCII code) only occurs in non-ASCII mode inside the ~{ ~} pair. A (correct) EHZ decoder can't mess up the "complement of A" in ASCII mode. In Ya-Gui's proposal, > aaaa~{gbgbgbgb~[jijijiji~}aaaa > aaaa~[jijijiji~{gbgbgbgb~}aaaa Note in my framework the second line would be aaaa~{~Bjijijiji~Agbgbgbgb~}aaaa if we follow ISO and use A for GB, and B for JIS. -Ricky From jiang Wed Jun 22 11:21:23 1994 Return-Path: Received: by ifcss.org (4.1/IFCSS-Mailer) id AA26073; Wed, 22 Jun 94 11:21:23 CDT Date: Wed, 22 Jun 94 11:21:23 CDT From: jiang (Yuan Jiang) Message-Id: <9406221621.AA26073@ifcss.org> To: ccnet-l@uga.bitnet, soft-authors Subject: Re: HZ-EA [Re: Big5 to CNS conversion algorithms] I agree with Yagui and like Fung Fung Lee's proposal the most. The escape code is a secondary issue. I'm more concerned about software implementation. To what extend ZWDOS and HZTTY will let end users to deal with EHZ or HZE directly. It's easy for a user to know whether to insert ~{ or ~}, but how can he know whether a character is in the 1st bloc or 2nd bloc in order to insert ~{g or ~{h? From simpson@math.psu.edu Wed Jun 22 15:14:08 1994 Return-Path: Received: from cnd.org (cnd.cnd.org) by ifcss.org (4.1/IFCSS-Mailer) id AA06165; Wed, 22 Jun 94 15:14:08 CDT Received: from leibniz.math.psu.edu by cnd.org (4.1/4.7) id AA02930; Wed, 22 Jun 94 13:12:06 PDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id QAA17658; Wed, 22 Jun 1994 16:10:31 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 22 Jun 1994 16:10:29 -0400 Date: Wed, 22 Jun 1994 16:10:29 -0400 Message-Id: <199406222010.QAA13237@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: Re: HZ-EA [Re: Big5 to CNS conversion algorithms] In-Reply-To: <9406220156.AA01417@ifcss.org> References: <9406220156.AA01417@ifcss.org> Reply-To: simpson@math.psu.edu Ya-Gui Wei writes: > If Big5 is to be included, I don't have serious > objection to the current Mule's mapping of Big5 to > ASCII symbols. But I do think CNS is a serious > candidate and should be considered instead. This statement is ambiguous. Do you mean that we should consider both Big5 and CNS, with CNS as a possible alternative to Big5? Or do you mean that we should drop consideration of Big5 and only consider CNS? My own view is that we should support Big5 now, but keep CNS in mind for possible support later. This viewpoint has also been expressed as the outcome of a vote taken in soft-authors. > (*) In lieu of Big-5, CNS may be a better candidate, mainly > to avoid the problem of vender specific extensions. Vender > specific extensions to member character sets, in principle, > should not be supported. I disagree, on several levels. First, what principle are you talking about? None of my moral principles forbid support of vendor-specific extensions. Second, I feel that on principle we *should* support vendor-specific extensions of Big5, to the extent that we can conveniently do so. The principle involved here is that we should try to make our standard as useful as possible. One aspect of usefulness is compatibility with existing software and popular usage. Third, even if we accept your "principle" opposed to vendor-specific extensions, this would not be sufficient reason to abandon the popular Big5 standard in favor of the unpopular CNS. If you don't want to support vendor-specific extensions of Big5, you can still find a way to support the common part of Big5 which is not vendor-specific. Fourth, even if you insist on abandoning Big5 in favor of CNS (an irrational course of action, in my opinion), you will not thereby completely avoid "the problem of vendor-specific extensions". The reason is that there is also such a thing as a vendor-specific extension of CNS. ETen's CNS includes vendor-specific extensions. It remains a mystery to me why otherwise sensible people are so mesmerized by CNS. I find that the arguments that have been given in favor of CNS are incredibly weak and illogical. -- Steve From simpson@math.psu.edu Wed Jun 22 16:25:37 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA09235; Wed, 22 Jun 94 16:25:37 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id RAA18483; Wed, 22 Jun 1994 17:23:15 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Wed, 22 Jun 1994 17:23:13 -0400 Date: Wed, 22 Jun 1994 17:23:13 -0400 Message-Id: <199406222123.RAA13374@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org, chinesenet@uclink.berkeley.edu Subject: HZ+ for the rest of us In-Reply-To: <199406220424.AAA12286@leibniz.math.psu.edu> References: <199406220424.AAA12286@leibniz.math.psu.edu> Reply-To: simpson@math.psu.edu Avron A. Boretz writes: > position. But as Elder Brother Handa suggests, as an end user I don't > care what goes on in between there and here, as long as I can send and > receive Big5 text without having to do _anything_ by hand to convert from > one codeset to another. Your target end user has no time for or interest > in playing with command lines with lots of parameters. Even a Windows > encoder-decoder program is too much to ask the average user to bother > with. Conversion has to happen automatically, on the fly. Automatic conversion is certainly a valid point which we should keep in mind at all times. However, in order to bring about this degree of user friendliness, the programmers who write software for news, mail, etc will have to implement the conversion method. And they can't do this until some significant group of people have first agreed on what conversion standard to follow. The basic problem is that the Internet is not 8-bit clean, which means that Big5 text will often be irreparably destroyed when you try to e-mail it or post it to newsgroups. Therefore, we need to define a method to convert Big5 text to a form which the Internet is capable of transmitting accurately. In an ideal world, the end user will never have to concern himself with this conversion method, because the programmers will have implemented the conversion method for him in such a way that he will never have to look at it. However, the programmers and other affected parties must first agree on which conversion method to use. People like Ed Lai have argued that *it* *doesn't* *matter* what conversion method we use, because the end user doesn't care. I find this argument silly. I have consistently opposed this silly argument. To me, this silly argument seems like the last desperate resort of those who favor or oppose some particular conversion method but have no other argument to offer. The reason I think the choice of conversion method *does* matter is *not* that I don't care about end users. Far from it. I am very concerned to make sure that we come up with a standard that will serve end users well. My point is that, even though the conversion method will be invisible to end users, it can affect end users in important ways. I want us to pick a conversion method that will serve both end users and programmers well. The following technical considerations are involved: 1. We need a conversion method that is in fact capable of representing Big5 with guaranteed, complete, 100 percent accuracy (including vendor-specific extensions of Big5). Some of the conversion methods that have been proposed (e.g. Ed Lai's vague proposal to "use CNS codes as the exchange codes for Big5 codes") do not meet even this basic, minimal criterion. 2. We need a conversion method that produces text that will not be destroyed by Internet news and mail software. Since there is still some escape-eating news software lurking in the Internet, this means we need to avoid the Escape character. So ISO-2022 is out, at least for the short term. Considerations such as Ricky Yeung's "maximal ASCII transparency" are also relevant. 3. It is desirable to maintain compatibility with existing popular standards, such as HZ and Big5. This is why I have consistently favored some form of HZ extended with new escape sequences to handle 7-bit Big5. I have also sought to define a conversion method which respects and preserves the structure of Big5 as much as possible. These considerations are not so important as 1 and 2, but they are still important. 4. Finally, it is desirable that our conversion method be as simple and straightforward as possible, for ease of programming and to maximize the speed of on-the-fly conversion. This issue is not as important as 1, 2, and 3, but it is still desirable. I think that my HZ+ proposal (also known as HZ+S) is pretty good with respect to all four of the above criteria. > where the great HZ+ debate is going, please set me straight and > explain to me (and all those poor, lost souls who think C++ is just > below a B-) what all the fuss is about. Yes, the great HZ+ debate must look awfully strange to end users. What is all the fuss about? Let me try to explain it by saying that everybody is at least professing to want to do the best possible job for end users, but other factors sometimes come into play. Among the other factors are: narrow-mindedness (GB programmers do not always understand the need for Big5), egomania (some people want to leave their mark on the future of Chinese computing at all costs), and politics (PRC versus ROC versus computer users and programmers). -- Steve From handa@etlken.etl.go.jp Wed Jun 22 20:37:04 1994 Received: from etlpost (etlpost.etl.go.jp) by ifcss.org (4.1/IFCSS-Mailer) id AA16865; Wed, 22 Jun 94 20:37:04 CDT Received: from etlpom.etl.go.jp by etlpost (5.67+1.6W/2.7W) id AA17517; Thu, 23 Jun 94 10:32:23 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA25777; Thu, 23 Jun 94 10:32:22 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA29765; Thu, 23 Jun 94 10:32:29 JST Date: Thu, 23 Jun 94 10:32:29 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9406230132.AA29765@etlken.etl.go.jp> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org, chinesenet@uclink.berkeley.edu In-Reply-To: Stephen G Simpson's message of Wed, 22 Jun 1994 17:23:13 -0400 <199406222123.RAA13374@boole.math.psu.edu> Subject: Re: HZ+ for the rest of us On Wed, 22 Jun 1994 17:23:13 -0400, Stephen G Simpson said: > People like Ed Lai have argued that *it* *doesn't* *matter* what > conversion method we use, because the end user doesn't care. I find > this argument silly. I have consistently opposed this silly argument. Could you please explain why it's silly. Since I joined this argument recently, I may have missed your statement. > The following technical considerations are involved: > 1. We need a conversion method that is in fact capable of > representing Big5 with guaranteed, complete, 100 percent accuracy > (including vendor-specific extensions of Big5). Some of the > conversion methods that have been proposed (e.g. Ed Lai's vague > proposal to "use CNS codes as the exchange codes for Big5 codes") do > not meet even this basic, minimal criterion. HZ+S doesn't provide 100 percent accuracy between users of different variants of Big5. > 4. Finally, it is desirable that our conversion method be as simple > and straightforward as possible, for ease of programming and to > maximize the speed of on-the-fly conversion. This issue is not as > important as 1, 2, and 3, but it is still desirable. This is desirable but not as important as clearness of what we are exchanging. In addition, the conversion of Big5-ETen <-> GB&CNS&JIS&private is not that hard task. I'll add "APPENDIX" part to my proposal how to handle Big5-ETen with HZ-EA. Please give me a time. --- Ken'ichi HANDA handa@etl.go.jp From handa@etlken.etl.go.jp Wed Jun 22 20:46:39 1994 Received: from etlpost (etlpost.etl.go.jp) by ifcss.org (4.1/IFCSS-Mailer) id AA17267; Wed, 22 Jun 94 20:46:39 CDT Received: from etlpom.etl.go.jp by etlpost (5.67+1.6W/2.7W) id AA17109; Thu, 23 Jun 94 10:08:43 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA25398; Thu, 23 Jun 94 10:08:42 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA29756; Thu, 23 Jun 94 10:08:49 JST Date: Thu, 23 Jun 94 10:08:49 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9406230108.AA29756@etlken.etl.go.jp> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Wed, 22 Jun 1994 16:10:29 -0400 <199406222010.QAA13237@boole.math.psu.edu> Subject: Re: HZ-EA [Re: Big5 to CNS conversion algorithms] On Wed, 22 Jun 1994 16:10:29 -0400, Stephen G Simpson said: > It remains a mystery to me why otherwise sensible people are so > mesmerized by CNS. I find that the arguments that have been given in > favor of CNS are incredibly weak and illogical. Why I prefer CNS to Big5 on internet communication. 0. First of all, on internet communication, the important thing is clearness of what we are exchanging. When one receives a code, he should be able to know what character is corresponding to it. 1. CNS11643 Plane1&2 (here after just CNS) is registered in ISO, which means everyone can get information about it through official route. 2. CNS has no variation and the set of characters are fixed. 3. As for a character set, CNS is more well-defined than Big5 (i.e. no dumplicated-encoding, meaningfull ordering). --- Ken'ichi HANDA handa@etl.go.jp From yawei@cs.indiana.edu Wed Jun 22 22:23:13 1994 Return-Path: Received: from moose.cs.indiana.edu by ifcss.org (4.1/IFCSS-Mailer) id AA20734; Wed, 22 Jun 94 22:23:13 CDT Message-Id: <9406230323.AA20734@ifcss.org> Received: by moose.cs.indiana.edu (5.65c/9.4jsm) id AA25108; Wed, 22 Jun 1994 22:20:29 -0500 Date: Wed, 22 Jun 1994 22:20:29 -0500 From: "Ya-Gui Wei" To: Ricky.Yeung@Eng.Sun.COM, yawei@cs.indiana.edu Subject: Re: Escape sequence (Was Re: HZ-EA) Cc: alt.chinese.computing@usenet.ifcss.org, alt.chinese.computing@usenet.ucs.indiana.edu, ccnet-l@uga.cc.uga.edu, handa@etlken.etl.go.jp, soft-authors@ifcss.org I agree with the need for ASCII transparency, but I feel that 4 byte escape sequence is a bit too much. Wouldn't it be nice if we can have two byte escape sequences and still achieve a similar level of result? -- What we call ASCII transparency here really should be called "accidental ASCII transparency", since HZ itself is 100% ASCII transparent (100% reversible encode/decoding.) What we are trying to deal with is when someone accidentally decodes a non-HZ file with an HZ decoder -- a bit of a secondary issue if you ask me. As to vender specific extensions, there are maybe 2 dozen vender specific extensions to GB, yet there has never been a discussion about HZ supporting them. I think that may be a desirable situation for an HZ extension. But I have not made up my mind between Big5/CNS since I have not had the time to look at the CNS conversion algorithm yet. Ya-Gui To Yuan - I don't view the 4 proposals as competing proposals. In many ways they are really complementary aspects of the same issue. Therefore the word "favorite" has no meaning here. I also hope we don't need to "vote" among them. :-) From handa@etlken.etl.go.jp Wed Jun 22 22:46:39 1994 Received: from etlpost (etlpost.etl.go.jp) by ifcss.org (4.1/IFCSS-Mailer) id AA21498; Wed, 22 Jun 94 22:46:39 CDT Received: from etlpom.etl.go.jp by etlpost (5.67+1.6W/2.7W) id AA19090; Thu, 23 Jun 94 12:43:46 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA27549; Thu, 23 Jun 94 12:43:45 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA29909; Thu, 23 Jun 94 12:43:52 JST Date: Thu, 23 Jun 94 12:43:52 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9406230343.AA29909@etlken.etl.go.jp> To: yawei@cs.indiana.edu Cc: Ricky.Yeung@Eng.Sun.COM, yawei@cs.indiana.edu, alt.chinese.computing@usenet.ifcss.org, alt.chinese.computing@usenet.ucs.indiana.edu, ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: "Ya-Gui Wei"'s message of Wed, 22 Jun 1994 22:20:29 -0500 <9406230320.AA18887@etlpost> Subject: Re: Escape sequence (Was Re: HZ-EA) On Wed, 22 Jun 1994 22:20:29 -0500, "Ya-Gui Wei" said: > As to vender specific extensions, there are maybe 2 dozen vender > specific extensions to GB, yet there has never been a discussion > about HZ supporting them. I think that may be a desirable > situation for an HZ extension. But I have not made up my > mind between Big5/CNS since I have not had the time to look > at the CNS conversion algorithm yet. I'm now writing a conversion routine between Big5-ETen and CNS/GB/JIS along the following note. --- Ken'ichi HANDA handa@etl.go.jp ------------------------------------------------------------ ################################################# # Proposal of ETen-Big5 <-> GB/CNS/JIS # # Ver.0.0 1994.6.23 # # by K.Handa # ################################################# >From my point of view, ETen-Big5 is not a single character set but an encoding mechanism of multiple character sets, CNS11642 Plane 1, CNS Plane 2, GB2312, and JISX0208. The code range of ETen-Big5 (Ver.3.11?) is divided as follows: (1) Graphic symbols 0xA140..0xA3BF ==> 0x2121..0x2570 of CNS11642 Plane 1 (2) Chinese standard characters (frequently used) 0xA440..0xC67E ==> 0x4421..0x7D4B of CNS11642 Plane 1 (code order is different) (3) Various numbers (circled, parenthesized, lower-roman) 0xC6A1..0xC6BE ==> 0x2621..0x263E of GB2312 (4) Component of Chinese characters 0xC6BF ==> 0x2723 of CNS11642 Plane 1 0xC6C0 ==> 0x2724 of CNS11642 Plane 1 0xC6C1 ==> 0x2726 of CNS11642 Plane 1 0xC6C2 ==> 0x2728 of CNS11642 Plane 1 0xC6C3 ==> 0x272D of CNS11642 Plane 1 0xC6C4 ==> 0x272E of CNS11642 Plane 1 0xC6C5 ==> 0x272F of CNS11642 Plane 1 0xC6C6 ==> 0x2734 of CNS11642 Plane 1 0xC6C7 ==> 0x2737 of CNS11642 Plane 1 0xC6C8 ==> 0x273A of CNS11642 Plane 1 0xC6C9 ==> 0x273C of CNS11642 Plane 1 0xC6CA ==> 0x2742 of CNS11642 Plane 1 0xC6CB ==> 0x2747 of CNS11642 Plane 1 0xC6CC ==> 0x274E of CNS11642 Plane 1 0xC6CD ==> 0x2753 of CNS11642 Plane 1 0xC6CE ==> 0x2754 of CNS11642 Plane 1 0xC6CF ==> 0x2755 of CNS11642 Plane 1 0xC6D0 ==> 0x2759 of CNS11642 Plane 1 0xC6D1 ==> 0x275A of CNS11642 Plane 1 0xC6D2 ==> 0x2761 of CNS11642 Plane 1 0xC6D3 ==> 0x2766 of CNS11642 Plane 1 0xC6D4 ==> 0x2829 of CNS11642 Plane 1 0xC6D5 ==> 0x282A of CNS11642 Plane 1 0xC6D6 ==> 0x2863 of CNS11642 Plane 1 0xC6D7 ==> 0x286C of CNS11642 Plane 1 (5) Japanese Symbols 0xC6D8 ==> 0x212F of JISX0208 0xC6D9 ==> 0x2130 of JISX0208 0xC6DA ==> 0x2133 of JISX0208 0xC6DB ==> 0x2134 of JISX0208 0xC6DC ==> 0x2135 of JISX0208 0xC6DD ==> 0x2136 of JISX0208 0xC6DE ==> 0x2137 of JISX0208 0xC6DF ==> 0x2138 of JISX0208 0xC6E0 ==> 0x2139 of JISX0208 0xC6E1 ==> 0x213A of JISX0208 0xC6E2 ==> 0x213B of JISX0208 0xC6E3 ==> 0x213C of JISX0208 0xC6E4 ==> 0x214E of JISX0208 0xC6E5 ==> 0x214F of JISX0208 0xC6E6 ==> 0x2176 of JISX0208 (6) Japanese Hiragana and Katakana 0xC6E7..0xC7F2 ==> 0x2421..0x2576 of JISX0208 or GB2312 (7) Cyrillic alphabet 0xC7F3..0xC875 ==> 0x2721..0x2771 of JISX0208 or GB2312 (8) Miscellaneous private symbols (PART1) most messy part! 0xC876..0xC8D3 ==> ???? (9) Chinese standard characters (less-frequently used) 0xC940..0xC949 ==> 0x2121..0x212A of CNS11642 Plane 2 0xC94A ==> 0x4442 of CNS11642 Plane 1 (duplicate 0xA461 of Big5) 0xC94B..0xDDFB ==> 0x212B..0x4423 of CNS11642 Plane 2 (code order is different) 0xDDFC ==> 0x4176 of CNS11642 Plane 1 (duplicate 0xDCD1 of Big5) 0xDDFD..0xF9D5 ==> 0x4424..0x7244 of CNS11642 Plane 2 (code order is different) (10) Chinese standard characters (extra) 0xF9D6 ==> 0x4665 of GB2312 or 0x386B of JISX0208 0xF9D7 ==> 0x5062 of GB2312 or 0x6E78 of JISX0208 0xF9D8 ==> 0x406F of GB2312 or 0x4E22 of JISX0208 0xF9D9 ==> 0x5899 of GB2312 or 0x5456 of JISX0208 0xF9DA ==> 0x3A63 of GB2312 or 0x3931 of JISX0208 0xF9DB ==> 0x5731 of GB2312 or 0x3E51 of JISX0208 0xF9DC ==> 0x6635 of GB2312 or 0x5546 of JISX0208 (these can be mapped to higher planes of CNS11643) (11) Miscellaneous private symbols (PART2) (ruled double lines and black box) 0xF9DD..0xF9FE ==> ???? (The following mapping is the alternative consideration of this area. The reason why CNS11643 is chosen is that GB and JIS don't have rounded corners.) 0xF9DD ==> 0x233C of CNS11642 Plane 1 0xF9DE ==> 0x2335 of CNS11642 Plane 1 0xF9DF ==> 0x233D of CNS11642 Plane 1 0xF9E0 ==> 0x2337 of CNS11642 Plane 1 0xF9E1 ==> 0x2333 of CNS11642 Plane 1 0xF9E2 ==> 0x2336 of CNS11642 Plane 1 0xF9E3 ==> 0x233E of CNS11642 Plane 1 0xF9E4 ==> 0x2334 of CNS11642 Plane 1 0xF9E5 ==> 0x233F of CNS11642 Plane 1 0xF9E6 ==> 0x233C of CNS11642 Plane 1 0xF9E7 ==> 0x2335 of CNS11642 Plane 1 0xF9E8 ==> 0x233D of CNS11642 Plane 1 0xF9E9 ==> 0x2345 of CNS11642 Plane 1 0xF9EA ==> 0x2346 of CNS11642 Plane 1 0xF9EB ==> 0x2347 of CNS11642 Plane 1 0xF9EC ==> 0x233E of CNS11642 Plane 1 0xF9ED ==> 0x2334 of CNS11642 Plane 1 0xF9EE ==> 0x233F of CNS11642 Plane 1 0xF9EF ==> 0x233C of CNS11642 Plane 1 0xF9F0 ==> 0x2335 of CNS11642 Plane 1 0xF9F1 ==> 0x233D of CNS11642 Plane 1 0xF9F2 ==> 0x2337 of CNS11642 Plane 1 0xF9F3 ==> 0x2333 of CNS11642 Plane 1 0xF9F4 ==> 0x2336 of CNS11642 Plane 1 0xF9F5 ==> 0x233E of CNS11642 Plane 1 0xF9F6 ==> 0x2334 of CNS11642 Plane 1 0xF9F7 ==> 0x233F of CNS11642 Plane 1 0xF9F8 ==> 0x233A of CNS11642 Plane 1 0xF9F9 ==> 0x2339 of CNS11642 Plane 1 0xF9FA ==> 0x2340 of CNS11642 Plane 1 0xF9FB ==> 0x2341 of CNS11642 Plane 1 0xF9FC ==> 0x2342 of CNS11642 Plane 1 0xF9FD ==> 0x2343 of CNS11642 Plane 1 0xF9FE ==> 0x232B of CNS11642 Plane 1 In the table above, (3-8),(10-11) are Big5-ETen specific area among variants of Big5. We should also consider just ignoring (8) and (11) to avoid using private area in HZ-EA. From handa@etlken.etl.go.jp Wed Jun 22 23:09:50 1994 Received: from etlpost (etlpost.etl.go.jp) by ifcss.org (4.1/IFCSS-Mailer) id AA22345; Wed, 22 Jun 94 23:09:50 CDT Received: from etlpom.etl.go.jp by etlpost (5.67+1.6W/2.7W) id AA19351; Thu, 23 Jun 94 13:07:20 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA27826; Thu, 23 Jun 94 13:07:19 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA29974; Thu, 23 Jun 94 13:07:26 JST Date: Thu, 23 Jun 94 13:07:26 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9406230407.AA29974@etlken.etl.go.jp> To: handa@etlken.etl.go.jp Cc: yawei@cs.indiana.edu, Ricky.Yeung@Eng.Sun.COM, yawei@cs.indiana.edu, alt.chinese.computing@usenet.ifcss.org, alt.chinese.computing@usenet.ucs.indiana.edu, ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: Kenichi Handa's message of Thu, 23 Jun 94 12:43:52 JST <9406230343.AA29909@etlken.etl.go.jp> Subject: Big5-ETen <-> GB/CNS/JIS [Re: Escape sequence (Was Re: HZ-EA)] On Thu, 23 Jun 94 12:43:52 JST, handa@etlken.etl.go.jp (Kenichi Handa) said: > I'm now writing a conversion routine between Big5-ETen and > CNS/GB/JIS along the following note. I found a mistake in the node. 1) "CNS11642" should be "CNS11643". ^.^;;; > (3) Various numbers (circled, parenthesized, lower-roman) > 0xC6A1..0xC6BE ==> 0x2621..0x263E of GB2312 ^^^^^^-> CNS11643 --- Ken'ichi HANDA handa@etl.go.jp From Ricky.Yeung@Eng.Sun.COM Thu Jun 23 02:31:54 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA28126; Thu, 23 Jun 94 02:31:54 CDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA23802; Thu, 23 Jun 94 00:28:03 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA24844; Thu, 23 Jun 94 00:30:23 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA28207; Thu, 23 Jun 1994 00:29:18 +0800 Date: Thu, 23 Jun 1994 00:29:18 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406230729.AA28207@hk.Eng.Sun.COM> To: alt.chinese.computing@usenet.ifcss.org, ccnet-l@uga.cc.uga.edu, handa@etlken.etl.go.jp, soft-authors@ifcss.org Subject: EHZ == HZ-EA ? Content-Length: 737 I really want to know how the HZ-EA proposal (posted by Mr. Handa on 6/21/94) differs from my Extension to HZ (EHZ) proposal (posted on 6/13/94, and archived in ifcss.org:/software/soft-author-discussion/ehz.spec). -Ricky Excerpt from ehz.spec: "When picking code scheme specific escape sequences in the non-ASCII mode, it is recommended that graphical ASCII sequences similar to the ISO standards are used. For examples, use ~A for GB, and ~G and ~H for CNS. (Note that there is no such obvious escape sequences for BIG-5, as it is not a registered ISO Standard. Perhaps ~g and ~h could be chosen.) Also it is recommended that specific ISO code mappings from 16-bit non-ASCII codes to 7-bit ASCII to be used if possible." From handa@etlken.etl.go.jp Thu Jun 23 02:49:48 1994 Received: from etlpost (etlpost.etl.go.jp) by ifcss.org (4.1/IFCSS-Mailer) id AA28475; Thu, 23 Jun 94 02:49:48 CDT Received: from etlpom.etl.go.jp by etlpost (5.67+1.6W/2.7W) id AA22302; Thu, 23 Jun 94 16:46:48 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA02322; Thu, 23 Jun 94 16:46:47 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA00365; Thu, 23 Jun 94 16:46:54 JST Date: Thu, 23 Jun 94 16:46:54 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9406230746.AA00365@etlken.etl.go.jp> To: Ricky.Yeung@Eng.Sun.COM Cc: alt.chinese.computing@usenet.ifcss.org, ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: Ricky Yeung's message of Thu, 23 Jun 1994 00:29:18 +0800 <9406230729.AA28207@hk.Eng.Sun.COM> Subject: Re: EHZ == HZ-EA ? On Thu, 23 Jun 1994 00:29:18 +0800, Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) said: > I really want to know how the HZ-EA proposal (posted by Mr. Handa on > 6/21/94) differs from my Extension to HZ (EHZ) proposal > (posted on 6/13/94, and archived in > ifcss.org:/software/soft-author-discussion/ehz.spec). "EHZ != HZ-EA" because... 1) HZ-EA includes JISX0208 from the first but EHZ makes it a future extension. In HZ-EA, it is necessary to include JISX0208 because some characters in Big5-ETen are not in GB nor CNS. 2) HZ-EA doesn't include Big5. In HZ-EA, Big5 should be mapped to CNS, GB, or JIS. Although the mapping should be decided by each Big5 variant community, it would be better to attach a description about the recommended mapping to the proposal of HZ-EA. As for tilde-sequences (e.g. ~{, ~A), I'm not insisting on the current proposal of HZ-EA. --- Ken'ichi HANDA handa@etl.go.jp From Ricky.Yeung@Eng.Sun.COM Thu Jun 23 02:59:02 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA28734; Thu, 23 Jun 94 02:59:02 CDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA28887; Thu, 23 Jun 94 00:56:49 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA25141; Thu, 23 Jun 94 00:59:09 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA28239; Thu, 23 Jun 1994 00:58:04 +0800 Date: Thu, 23 Jun 1994 00:58:04 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406230758.AA28239@hk.Eng.Sun.COM> To: yawei@cs.indiana.edu Cc: alt.chinese.computing@usenet.ifcss.org, ccnet-l@uga.cc.uga.edu, handa@etlken.etl.go.jp, soft-authors@ifcss.org In-Reply-To: "Ya-Gui Wei"'s message of Wed, 22 Jun 1994 22:20:29 -0500 <9406230320.AA21616@Sun.COM> Subject: ASCII transparency (Was Re: Escape sequence) Content-Length: 1079 > a similar level of result? -- What we call ASCII > transparency here really should be called "accidental ASCII > transparency", since HZ itself is 100% ASCII transparent (100% > reversible encode/decoding.) What we are trying to deal with > is when someone accidentally decodes a non-HZ file with an > HZ decoder -- a bit of a secondary issue if you ask me. I think you completely missed the point of my proposal. Obviously HZ is NOT 100% ASCII transparent. (BTW, you always "accidentally" decode non-HZ data with an HZ decoder if you view them through programs like zWDOS or hztty. Moreover, you're probably experiencing the problem of (lack of 100%) ASCII transparency, if you're using zWDOS or hztty to view this message :-) > To Yuan - I don't view the 4 proposals as competing proposals. > In many ways they are really complementary aspects of the > same issue. Therefore the word "favorite" has no meaning here. > I also hope we don't need to "vote" among them. :-) Which 4 proposals are you referring to? -Ricky From lee@simd.stanford.edu Thu Jun 23 12:14:18 1994 Return-Path: Received: from simd.stanford.edu by ifcss.org (4.1/IFCSS-Mailer) id AA16310; Thu, 23 Jun 94 12:14:18 CDT Received: by simd.stanford.edu (5.65/inc-1.0) id AA29217; Thu, 23 Jun 1994 10:11:24 -0700 Date: Thu, 23 Jun 1994 10:11:24 -0700 From: lee@simd.stanford.edu (Fung Fung Lee) Message-Id: <9406231711.AA29217@simd.stanford.edu> To: handa@etlken.etl.go.jp Cc: Ricky.Yeung@Eng.Sun.COM, alt.chinese.computing@usenet.ifcss.org, ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: Kenichi Handa's message of Thu, 23 Jun 94 16:46:54 JST <9406230746.AA00365@etlken.etl.go.jp> Subject: EHZ == HZ-EA ? I am very disturbed by Mr. Handa's response to Dr. Ricky Yeung's message. Dr. Yeung was the one who first invented and published the concept of his EHZ (Extension to HZ) framework. As described in his draft proposal, he has outlined the structure of how EHZ can extend HZ to handle CNS and similarly other ISO-registered character sets. As clearly inidicated in the title, Dr. Yeung was proposing a "framework", the major essence of which is the special role played by "~{" to distinguish ASCII and non-ASCII modes. It is significantly different from every other escape sequence proposals that I have seen, including ISO-2022. Obviously, Mr. Handa has borrowed every major piece of Dr. Yeung's EHZ proposal without ever properly acknowledging or even referring to it. As for my own proposal EHZ (Extended HZ), I took the liberty to change Ricky's "Extension to" to "Extended". (If Ricky has any objection, I'll glad to discuss openly with him on this naming issue.) In my proposal, I explicitly said that EHZ followed exactly his framework. What my propsoal did was to fill out the simple details concerning CNS, and the more complicated details concerning BIG5. EHZ-BIG5's 7-bit mapping scheme is clearly different from every other BIG5 7-bit mapping proposals previously posted. Strictly speaking, what cannot be straightforwardly deduced from Ricky's EHZ proposal is the EHZ-BIG5 part. In this spirit of respecting the original inventor of EHZ, I think the proper way to label the JIS part would be to call it EHZ-JIS, which can be straightforwardly deduced from Ricky's EHZ proposal anyway. >Date: Thu, 23 Jun 94 16:46:54 JST >From: handa@etlken.etl.go.jp (Kenichi Handa) > >On Thu, 23 Jun 1994 00:29:18 +0800, >Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) said: >> I really want to know how the HZ-EA proposal (posted by Mr. Handa on >> 6/21/94) differs from my Extension to HZ (EHZ) proposal >> (posted on 6/13/94, and archived in >> ifcss.org:/software/soft-author-discussion/ehz.spec).> > >"EHZ != HZ-EA" because... > >1) HZ-EA includes JISX0208 from the first but EHZ makes it a >future extension. In HZ-EA, it is necessary to include >JISX0208 because some characters in Big5-ETen are not in GB >nor CNS. Didn't you notice that Dr. Yeung was proposing a framework, and the document is a "preliminary draft"? You have followed his framework, and given it more details, yet chose not to refer to it and relabel the concept in your own term. This is serious. I am willing now to give you the benefit of doubt that you were just careless. :-| >2) HZ-EA doesn't include Big5. In HZ-EA, Big5 should be >mapped to CNS, GB, or JIS. Although the mapping should be >decided by each Big5 variant community, it would be better >to attach a description about the recommended mapping to the >proposal of HZ-EA. Isn't it true that the mapping between BIG5 and CNS has already been defined in Taiwan back in the 80's ? There is nothing left for something like HZ-EA to define. If ETen comes up with a new version with additional characters, is this HZ-EA's job to define the mapping? What you were really saying is that you don't want to see anything like EHZ-BIG5. Well, say, something like EHZ-BIG5 is missing, then the BIG5 user community will be "forced" to use a combination of CNS, GB and even JIS (7-bit forms) to post their 7-bit messages, originally in one code, namely BIG5. I can't imagine they will like or accept this kind of arrangement. Thus, the BIG5 issue will still remain unresolved, waiting for newer and better proposals. Why not give BIG5 and CNS an equal chance as in EHZ, and let the users decide what is (are) best for them? For those who are concerned about vendor independent BIG5, I urge them to read my EHZ proposal carefully. By adopting something like EHZ-BIG5 that simply tries to do one-to-one code space mapping, we can actually achieve vendor-independence of BIG5 in a way similar to what HZ has achieved on GB. As pointed out by Yagui Wei, yes, there are many vendor-dependent additions to 8-bit GB implemenations. >As for tilde-sequences (e.g. ~{, ~A), I'm not insisting on >the current proposal of HZ-EA. > >--- >Ken'ichi HANDA >handa@etl.go.jp Fung Fung Lee From simpson@math.psu.edu Thu Jun 23 15:45:45 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA24520; Thu, 23 Jun 94 15:45:45 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id QAA02005; Thu, 23 Jun 1994 16:43:32 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 23 Jun 1994 16:43:30 -0400 Date: Thu, 23 Jun 1994 16:43:30 -0400 Message-Id: <199406232043.QAA19317@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: Re: HZ-EA [Re: Big5 to CNS conversion algorithms] In-Reply-To: <199406230127.VAA22019@leibniz.math.psu.edu> References: <199406222010.QAA13237@boole.math.psu.edu> <199406230127.VAA22019@leibniz.math.psu.edu> Reply-To: simpson@math.psu.edu I wrote: > I find that the arguments that have been given in > favor of CNS are incredibly weak and illogical. Kenichi Handa replied: > Why I prefer CNS to Big5 on internet communication. > > 0. First of all, on internet communication, the important > thing is clearness of what we are exchanging. When one > receives a code, he should be able to know what character is > corresponding to it. I agree that clarity is important. However, this is not a reason to prefer CNS to Big5. Big5 is not an obstacle to clarity of communication. Millions of people use Big5 to communicate every day. The "common" (i.e. non-vendor-specific) part of Big5 is perfectly well defined and readily available from many sources. Even if we want to use vendor-specific extensions of Big5, we are free to label them as such or to agree among ourselves, so no clarity is lost. This can be a practical solution, as witnessed by the Taiwan's tw.* newsgroups, where ETen Big5 is the de facto standard and ETen's vendor-specific extensions (e.g. Japanese kana) are used to mirror Japanese newsgroups. > 1. CNS11643 Plane1&2 (here after just CNS) is registered in > ISO, which means everyone can get information about it > through official route. Big5 is not an official government standard, but nevertheless Big5 users can get reliable information about Big5 from many sources, for example the ETen manual. (ETen is an extremely popular Big5 operating environment. It is installed on millions of computers. It is easy to buy. It costs about $65 US, including the manual.) By contrast, ISO charges a lot money for their documents, and these documents are not readily available to the general public. On CCNET, one user asked for ISO-2022 specs and was told (by an ISO-2022 advocate) that the information is either not available or too expensive to buy. > 2. CNS has no variation and the set of characters are fixed. Not true. For example, ETen also supports CNS (in a non-default mode) with vendor-specific extensions. > 3. As for a character set, CNS is more well-defined than > Big5 (i.e. no dumplicated-encoding, meaningfull ordering). Not true. The common part of Big5 is just as meaningful as the common part of CNS. In fact it consists of almost the same character set, only in a somewhat different ordering, but not a less meaningful ordering. -- Steve From simpson@math.psu.edu Thu Jun 23 16:02:42 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA25097; Thu, 23 Jun 94 16:02:42 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id RAA02191; Thu, 23 Jun 1994 17:00:21 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 23 Jun 1994 17:00:19 -0400 Date: Thu, 23 Jun 1994 17:00:19 -0400 Message-Id: <199406232100.RAA19345@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: Re: HZ-EA [Re: Big5 to CNS conversion algorithms] In-Reply-To: <199406230127.VAA22019@leibniz.math.psu.edu> References: <199406222010.QAA13237@boole.math.psu.edu> <199406230127.VAA22019@leibniz.math.psu.edu> Reply-To: simpson@math.psu.edu Kenichi Handa writes: > Why I prefer CNS to Big5 on internet communication. > .... Why I prefer Big5 over CNS for all forms of electronic communication in Chinese. The main reason is that millions of people use Big5. The fonts and software are already installed on their computers. They know Big5 and use it every day. Virtually all of the Chinese software in Taiwan is based on Big5. Outside Taiwan, GB is also popular, but there is also a lot of Big5 software, including Chinese Windows, Chinese OS/2, Chinese MacOS, not to mention cxterm and Mule. By contrast, I don't know of one single person, in Taiwan or elsewhere, who uses CNS. The only software packages that I have heard of that support CNS are ETen (in a non-default mode which nobody uses), and Mule (but there are no fonts and nobody uses it). CNS fonts and software exist on only a few computers, and virtually all of these CNS computers also have Big5 fonts and software, and furthermore the users of the CNS computers always prepare documents in Big5 rather than in CNS. The fact that CNS is unpopular implies that CNS is useless for communication. -- Steve From simpson@math.psu.edu Thu Jun 23 16:12:57 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA25410; Thu, 23 Jun 94 16:12:57 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id RAA02344; Thu, 23 Jun 1994 17:10:45 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 23 Jun 1994 17:10:42 -0400 Date: Thu, 23 Jun 1994 17:10:42 -0400 Message-Id: <199406232110.RAA19359@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: Re: HZ+ for the rest of us In-Reply-To: <199406230139.VAA22075@leibniz.math.psu.edu> References: <199406222123.RAA13374@boole.math.psu.edu> <199406230139.VAA22075@leibniz.math.psu.edu> Reply-To: simpson@math.psu.edu I wrote: > > People like Ed Lai have argued that *it* *doesn't* *matter* what > > conversion method we use, because the end user doesn't care. I find > > this argument silly. I have consistently opposed this silly argument. Kenichi Handa writes: > Could you please explain why it's silly. Why is that particular argument silly? The reason is that, even though end users may not care about conversion methods, conversion methods can affect end users in important ways. Since conversion methods can affect users (not to mention programmers) in important ways, the choice of a good conversion method does indeed matter. The interesting thing is that the "it doesn't matter" argument has been used by Ed and others to argue for or against particular conversion methods. Doesn't this strike you as somewhat illogical? If it doesn't matter, then what's the argument about? -- Steve From simpson@math.psu.edu Thu Jun 23 16:25:01 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA25772; Thu, 23 Jun 94 16:25:01 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id RAA02434; Thu, 23 Jun 1994 17:22:46 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 23 Jun 1994 17:22:45 -0400 Date: Thu, 23 Jun 1994 17:22:45 -0400 Message-Id: <199406232122.RAA19395@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: Re: HZ+ for the rest of us In-Reply-To: <199406230139.VAA22075@leibniz.math.psu.edu> References: <199406222123.RAA13374@boole.math.psu.edu> <199406230139.VAA22075@leibniz.math.psu.edu> Reply-To: simpson@math.psu.edu I wrote: > > 1. We need a conversion method that is in fact capable of > > representing Big5 with guaranteed, complete, 100 percent accuracy > > (including vendor-specific extensions of Big5). Some of the > > conversion methods that have been proposed (e.g. Ed Lai's vague > > proposal to "use CNS codes as the exchange codes for Big5 codes") do > > not meet even this basic, minimal criterion. Kenichi replied: > HZ+S doesn't provide 100 percent accuracy between users of > different variants of Big5. True. However, 100 percent accuracy between users of all possible variants of Big5 would be an unreasonable requirement, don't you think? What HZ+S and similar proposals do provide is 100 percent accuracy among all users of the same Big5 variant, no matter which Big5 variant we are talking about, be it ETen-Big5 or Chinese-OS/2-Big5 or .... This kind of accuracy is, to say the least, a very desirable feature. Ed's proposal (and other proposals to support Big5 by means of CNS conversion) cannot provide even this much accuracy. > The conversion of Big5-ETen > <-> GB&CNS&JIS&private is not that hard task. I'll add > "APPENDIX" part to my proposal how to handle Big5-ETen with > HZ-EA. Please give me a time. I'll give you all the time you want, but I don't think you'll come up with a method based on CNS that works as well as HZ+S. -- Steve From simpson@math.psu.edu Thu Jun 23 16:31:27 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA26054; Thu, 23 Jun 94 16:31:27 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id RAA02497; Thu, 23 Jun 1994 17:29:17 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 23 Jun 1994 17:29:16 -0400 Date: Thu, 23 Jun 1994 17:29:16 -0400 Message-Id: <199406232129.RAA19404@boole.math.psu.edu> To: soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu Subject: vendor-specific GB In-Reply-To: <9406230323.AA20734@ifcss.org> References: <9406230323.AA20734@ifcss.org> Reply-To: simpson@math.psu.edu Ya-Gui writes: > As to vender specific extensions, there are maybe 2 dozen vender > specific extensions to GB, yet there has never been a discussion > about HZ supporting them. Interesting. I wasn't aware of these vendor-specific extensions of GB. What code ranges do they use? What characters are in those code ranges? > I think that may be a desirable situation for an HZ extension. Does it require an extension of HZ? If the code ranges are included in the 0xA1A1-0xFEFE range, then I would have imagined that HZ already supports them (similar to how HZ+ and HZ-2 already support vendor-specific extensions of Big5). Could you please clarify this? -- Steve From simpson@math.psu.edu Thu Jun 23 16:44:11 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA26416; Thu, 23 Jun 94 16:44:11 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id RAA02675; Thu, 23 Jun 1994 17:42:00 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 23 Jun 1994 17:41:58 -0400 Date: Thu, 23 Jun 1994 17:41:58 -0400 Message-Id: <199406232141.RAA19434@boole.math.psu.edu> To: soft-authors@ifcss.org, ccnet-l@uga.cc.uga.edu Subject: HZ-EA's Big5 support In-Reply-To: <199406230349.XAA22704@leibniz.math.psu.edu> References: <9406230320.AA18887@etlpost> <199406230349.XAA22704@leibniz.math.psu.edu> Reply-To: simpson@math.psu.edu Kenichi Handa's HZ-EA proposal lacks Big5 support. Kenichi proposes to remedy this lack, by means of a translation of ETen Big5 into CNS + GB + JIS, as follows: > >From my point of view, ETen-Big5 is not a single character > set but an encoding mechanism of multiple character sets, > CNS11642 Plane 1, CNS Plane 2, GB2312, and JISX0208. > > 0xC6C1 ==> 0x2726 of CNS11642 Plane 1 [ many lines omitted] > 0xC6D8 ==> 0x212F of JISX0208 [many more lines omitted] > 0xC6E7..0xC7F2 ==> 0x2421..0x2576 of JISX0208 or GB2312 .... > 0xC7F3..0xC875 ==> 0x2721..0x2771 of JISX0208 or GB2312 > 0xC876..0xC8D3 ==> ???? .... > 0xC940..0xC949 ==> 0x2121..0x212A of CNS11642 Plane 2 > 0xC94A ==> 0x4442 of CNS11642 Plane 1 > (duplicate 0xA461 of Big5) > 0xC94B..0xDDFB ==> 0x212B..0x4423 of CNS11642 Plane 2 > (code order is different) > 0xDDFC ==> 0x4176 of CNS11642 Plane 1 > (duplicate 0xDCD1 of Big5) > 0xDDFD..0xF9D5 ==> 0x4424..0x7244 of CNS11642 Plane 2 > (code order is different) ... > 0xF9D6 ==> 0x4665 of GB2312 or 0x386B of JISX0208 [many more lines omitted] ... > 0xF9DD..0xF9FE ==> ???? > (The following mapping is the alternative consideration of > this area. The reason why CNS11643 is chosen is that GB and > JIS don't have rounded corners.) > 0xF9DD ==> 0x233C of CNS11642 Plane 1 [many more lines omitted] > In the table above, (3-8),(10-11) are Big5-ETen specific > area among variants of Big5. > We should also consider just ignoring (8) and (11) to avoid > using private area in HZ-EA. I would like to point out that HZ+S and similar proposals support ETen Big5 and all other Big5 variants in a much, much simpler and more accurate way. There is no reason to go through all these contortions. -- Steve From simpson@math.psu.edu Thu Jun 23 17:16:20 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA27760; Thu, 23 Jun 94 17:16:20 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id SAA02998; Thu, 23 Jun 1994 18:14:05 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Thu, 23 Jun 1994 18:14:04 -0400 Date: Thu, 23 Jun 1994 18:14:04 -0400 Message-Id: <199406232214.SAA19503@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: the need for Big5 support In-Reply-To: <9406231711.AA29217@simd.stanford.edu> References: <9406230746.AA00365@etlken.etl.go.jp> <9406231711.AA29217@simd.stanford.edu> Reply-To: simpson@math.psu.edu Fung Fung Lee writes: > Well, say, something like EHZ-BIG5 is missing, then the BIG5 user > community will be "forced" to use a combination of CNS, GB and even > JIS (7-bit forms) to post their 7-bit messages, originally in one > code, namely BIG5. I can't imagine they will like or accept this > kind of arrangement. Thus, the BIG5 issue will still remain > unresolved, waiting for newer and better proposals. I completely agree. Big5 users won't stand for this, nor should they. It would be a waste of time to adopt something like HZ-EA, that doesn't support Big5. Big5 is real. Vendor-specific extensions of Big5 are real. Big5 variants won't go away just because some people don't like them. We can't evade this issue. We need HZ plus some form of 7-bit Big5, along the lines of my HZ+S or Fung's EHZ-Big5. Anything less is an attempt to evade the real problem. > Why not give BIG5 and CNS an equal chance as in EHZ, and let the users > decide what is (are) best for them? As everybody knows, I'm a Big5 fan and *not* a CNS fan. Nevertheless, I think Fung's idea of giving CNS an equal chance alongside Big5 has some merit. That's why my HZ+S proposal mentions CNS support also. Why aren't CNS fans willing to give Big5 an equal chance? Maybe because they know that CNS will lose in a fair fight? Fung writes: > For those who are concerned about vendor independent BIG5, I urge > them to read my EHZ proposal carefully. By adopting something like > EHZ-BIG5 that simply tries to do one-to-one code space mapping, we > can actually achieve vendor-independence of BIG5 in a way similar > to what HZ has achieved on GB. As pointed out by Yagui Wei, yes, > there are many vendor-dependent additions to 8-bit GB > implemenations. What Fung is saying about vendor-independence in EHZ-Big5 is equally true of my HZ+S proposal, for similar reasons. (Fung, if I am wrong on this, please correct me. I have read your EHZ proposal several times, but your reasoning is somewhat complicated, and it is conceivable that I missed some subtle point.) -- Steve From handa@etlken.etl.go.jp Thu Jun 23 20:41:25 1994 Received: from etlpost (etlpost.etl.go.jp) by ifcss.org (4.1/IFCSS-Mailer) id AA04680; Thu, 23 Jun 94 20:41:25 CDT Received: from etlpom.etl.go.jp by etlpost (5.67+1.6W/2.7W) id AA01136; Fri, 24 Jun 94 10:38:15 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA11795; Fri, 24 Jun 94 10:38:14 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA00940; Fri, 24 Jun 94 10:38:21 JST Date: Fri, 24 Jun 94 10:38:21 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9406240138.AA00940@etlken.etl.go.jp> To: lee@simd.stanford.edu Cc: Ricky.Yeung@Eng.Sun.COM, alt.chinese.computing@usenet.ifcss.org, ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: Fung Fung Lee's message of Thu, 23 Jun 1994 10:11:24 -0700 <9406231711.AA29217@simd.stanford.edu> Subject: Re: EHZ == HZ-EA ? On Thu, 23 Jun 1994 10:11:24 -0700, lee@simd.stanford.edu (Fung Fung Lee) said: > I am very disturbed by Mr. Handa's response to Dr. Ricky Yeung's message. > Dr. Yeung was the one who first invented and published the concept of his > EHZ (Extension to HZ) framework. As described in his draft proposal, he > has outlined the structure of how EHZ can extend HZ to handle CNS and > similarly other ISO-registered character sets. As clearly inidicated in > the title, Dr. Yeung was proposing a "framework", the major essence of > which is the special role played by "~{" to distinguish ASCII and non-ASCII > modes. It is significantly different from every other escape sequence > proposals that I have seen, including ISO-2022. Obviously, Mr. Handa > has borrowed every major piece of Dr. Yeung's EHZ proposal without > ever properly acknowledging or even referring to it. I'm VERY VERY SORRY for that. As I'm a new commer on this discussion, I didn't know where the idea came from. In addition, I thought the idea has already been common sense and didn't realize the necessity to pay my respects to a inventor. One more misunderstanding of mine is that I wrote my previous reply just after reading through your document entitled "EHZ: Extended HZ for GB, CNS and BIG5" roughly. I mixed up your EHZ (Extended HZ) and Dr. Yeung's original "framework". > In this spirit of respecting the original inventor of EHZ, I think the > proper way to label the JIS part would be to call it EHZ-JIS, which can > be straightforwardly deduced from Ricky's EHZ proposal anyway. Yes, I agree with you. >>2) HZ-EA doesn't include Big5. In HZ-EA, Big5 should be >>mapped to CNS, GB, or JIS. Although the mapping should be >>decided by each Big5 variant community, it would be better >>to attach a description about the recommended mapping to the >>proposal of HZ-EA. > Isn't it true that the mapping between BIG5 and CNS has already > been defined in Taiwan back in the 80's ? As for a kernel (common?) part of Big5, I think it's true. The problem arises only in extended area by vendors. > If ETen comes up with > a new version with additional characters, is this HZ-EA's job > to define the mapping? To make such a job apart from HZ?? is one of the reason of my proposing HZ-EA. > What you were really saying is that you don't want to see anything > like EHZ-BIG5. Yes. > Well, say, something like EHZ-BIG5 is missing, > then the BIG5 user community will be "forced" to use a combination > of CNS, GB and even JIS (7-bit forms) to post their 7-bit messages, > originally in one code, namely BIG5. I can't imagine they will like > or accept this kind of arrangement. Thus, the BIG5 issue will still > remain unresolved, waiting for newer and better proposals. If we don't have to care about vendor-dependent area of Big5, Big5 can be mapped to only one charater set CNS. But I heard that there's a big requirement for supporting also vendor-dependent area. Your wrote in your dract: ------------------------------------------------------------ Note that this is NOT an attempt to unify CNS and BIG5. For example, the 30 CNS numerals (212621-21263E) are present in one or the other of the above two vendor-defined ranges, depending on which vendors, such as KuoChiao and ETen. Such a unification would imply some kind of vendor affinity. This kind of bias is not desirable because it is not necessary at all in the first place. ------------------------------------------------------------ Don't we need to assure a KuoChiao user and an ETen user can exchange every character which both system support. Just defining a mapping of area<->area doesn't solve this problem. I beleive it is the best to define a mapping of each vendor-dependent character and already defined character in GB/CNS/JIS. If there's no such requirement, I think your proposal is the best choise. > Why not give BIG5 and CNS an equal chance as in EHZ, and let the users That's only because Big5 has several variants. > code space mapping, we can actually achieve vendor-independence of BIG5 > in a way similar to what HZ has achieved on GB. As pointed out by Yagui > Wei, yes, there are many vendor-dependent additions to 8-bit GB > implemenations. I didn't know about that. I must read the latest document to know how HZ handles vendor-dependent code of GB. Could you please tell me where to get the latest one or send it to me personally. --- Ken'ichi HANDA handa@etl.go.jp From CAI@neurophys.wisc.edu Thu Jun 23 23:33:30 1994 Return-Path: Received: from neurophys.wisc.edu (mve.neurophys.wisc.edu) by ifcss.org (4.1/IFCSS-Mailer) id AA10801; Thu, 23 Jun 94 23:33:30 CDT Received: from VMSmail by neurophys.wisc.edu; Thu, 23 Jun 94 23:31 CST Message-Id: <24062323310082@neurophys.wisc.edu> Date: Thu, 23 Jun 94 23:31 CST From: CAI@neurophys.wisc.edu Subject: Big5 and Vendor-specific Portions of Big5 To: soft-authors@ifcss.org X-Vms-To: IN%"soft-authors@ifcss.org" Big5 and Vendor-specific portions of Big5 There seems agreement that Big5 should be supported in some sense among the net, given the extreme popularity of Big5. However, there are some preference over the method of support: direct 7-bit mapping, using CNS to "encode" Big5,and using CNS, GB, JIS to "encode" big5. In principle, I'm not against the later two, but I have to repeat that the later two are in an disadvantagous position at present, since there is no specific and unique mapping presented so far. As for the vendor specific portion of Big5, I think first of all we have to accept it, any proposal should support it one way or the other. However, vendor specific portion should be discouraged on the network where your message reaches tens of thousands people around world, many of them do not have the same system as you have and thus could not read your message correctly. This does not contracdict with the previous statement that vendor specific portion should be supported. In the case of private mail or one has to use some vendor specific characters, one need such kind of support. In that case, the author can say something like "this file contains some XX vendor specific characters, you need XX to view them correctly", just like an ASCII file contains some Chinese or Japanese characters inside, but people would still get the main idea of the article even if they don't have XX. How to support vendor-specific portion? One approach is to try to map all those characters from all known vendors to other character sets that contains the characters. This fails when new vendors come with their own specific characters. The other is to keep the system open, i.e., just mapping the vendor-specific coding *spaces* to 7 bit, thus for each vendor, the conversion is 100% reversible. This is certainly a simpler approach and would work well in practice with the principle that vendor specific characters should be discouraged. If you want to let yourself understood by everybody, let your idea reach the whole audience, you should restrain yourself from using vendor specific characters in the first place. On the other hand, if everybody uses vendor specific characters, there must be a competition and finally a preferred vendor would be "picked". Before the network is set up, it doesn't matter wether you use ETen-specific characters or KC-specific characters to produce an document, since you exchange the hardcopies, not the electronic form. With current network, I'm sure users will sovle this problem themself, maybe after a certain period of chaos, be it by competition or by administration. Yidao CAI From Ricky.Yeung@Eng.Sun.COM Fri Jun 24 01:59:10 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA15085; Fri, 24 Jun 94 01:59:10 CDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA04641; Thu, 23 Jun 94 23:56:40 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA24643; Thu, 23 Jun 94 23:56:43 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA02295; Thu, 23 Jun 1994 23:57:55 +0800 Date: Thu, 23 Jun 1994 23:57:55 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406240657.AA02295@hk.Eng.Sun.COM> To: handa@etlken.etl.go.jp Cc: lee@simd.stanford.edu, Ricky.Yeung@Eng.Sun.COM, alt.chinese.computing@usenet.ifcss.org, ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: Kenichi Handa's message of Fri, 24 Jun 94 10:38:21 JST <9406240138.AA00940@etlken.etl.go.jp> Subject: Re: EHZ == HZ-EA ? Content-Length: 1933 Since your "HZ-EA proposal" is really a restricted subset of my EHZ framework, I think it would not be appropriate to label it as an HZ-xx proposal. Do you agree? Moreover, if your main point is to argue not to support Big5, I don't think we need a new term like "HZ-EA", since it would just create more confusion. -Ricky > Date: Fri, 24 Jun 94 10:38:21 JST > From: handa@etlken.etl.go.jp (Kenichi Handa) > > On Thu, 23 Jun 1994 10:11:24 -0700, > lee@simd.stanford.edu (Fung Fung Lee) said: > > I am very disturbed by Mr. Handa's response to Dr. Ricky Yeung's message. > > Dr. Yeung was the one who first invented and published the concept of his > > EHZ (Extension to HZ) framework. As described in his draft proposal, he > > has outlined the structure of how EHZ can extend HZ to handle CNS and > > similarly other ISO-registered character sets. As clearly inidicated in > > the title, Dr. Yeung was proposing a "framework", the major essence of > > which is the special role played by "~{" to distinguish ASCII and non-ASCII > > modes. It is significantly different from every other escape sequence > > proposals that I have seen, including ISO-2022. Obviously, Mr. Handa > > has borrowed every major piece of Dr. Yeung's EHZ proposal without > > ever properly acknowledging or even referring to it. > > I'm VERY VERY SORRY for that. As I'm a new commer on this > discussion, I didn't know where the idea came from. In > addition, I thought the idea has already been common sense > and didn't realize the necessity to pay my respects to a > inventor. > > One more misunderstanding of mine is that I wrote my > previous reply just after reading through your document > entitled "EHZ: Extended HZ for GB, CNS and BIG5" roughly. I > mixed up your EHZ (Extended HZ) and Dr. Yeung's original > "framework". From handa@etlken.etl.go.jp Fri Jun 24 02:57:53 1994 Received: from etlpost (etlpost.etl.go.jp) by ifcss.org (4.1/IFCSS-Mailer) id AA16862; Fri, 24 Jun 94 02:57:53 CDT Received: from etlpom.etl.go.jp by etlpost (5.67+1.6W/2.7W) id AA06221; Fri, 24 Jun 94 16:53:57 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA18444; Fri, 24 Jun 94 16:53:56 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA01411; Fri, 24 Jun 94 16:54:02 JST Date: Fri, 24 Jun 94 16:54:02 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9406240754.AA01411@etlken.etl.go.jp> To: Ricky.Yeung@Eng.Sun.COM Cc: lee@simd.stanford.edu, Ricky.Yeung@Eng.Sun.COM, alt.chinese.computing@usenet.ifcss.org, ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: Ricky Yeung's message of Thu, 23 Jun 1994 23:57:55 +0800 <9406240657.AA02295@hk.Eng.Sun.COM> Subject: Re: EHZ == HZ-EA ? On Thu, 23 Jun 1994 23:57:55 +0800, Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) said: > Since your "HZ-EA proposal" is really a restricted subset of my EHZ > framework, I think it would not be appropriate to label it as an HZ-xx > proposal. Do you agree? Yes, surely. > Moreover, if your main point is to argue not to support Big5, I don't > think we need a new term like "HZ-EA", since it would just create more > confusion. Then, do you have any idea how to label my proposal? EHZ-EA? But, since Dr. Lee wrote in his draft that: ------------------------------------------------------------ The subset of EHZ that deals only with ASCII and a non-ASCII code X can be referred as EHZ-X. EHZ-CNS and EHZ-BIG5 are the topics of this paper. ------------------------------------------------------------ EHZ-EA seems to be more confusing. My proposal is, in short: "Let's use only EHZ-GB, EHZ-CNS, and EHZ-JIS" Perhaps no label is necessary for such an breif proposal. By the way, I don't argue not to support Big5, but am proposing to support it by converting to/from CNS/GB/JIS. --- Ken'ichi HANDA handa@etl.go.jp From simpson@math.psu.edu Fri Jun 24 09:46:48 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA28167; Fri, 24 Jun 94 09:46:48 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id KAA08880; Fri, 24 Jun 1994 10:44:26 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 24 Jun 1994 10:44:24 -0400 Date: Fri, 24 Jun 1994 10:44:24 -0400 Message-Id: <199406241444.KAA21100@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: Big5 and Vendor-specific Portions of Big5 In-Reply-To: <24062323310082@neurophys.wisc.edu> References: <24062323310082@neurophys.wisc.edu> Reply-To: simpson@math.psu.edu I'd like to thank Yidao Cai for his thoughtful posting on how our proposed 7-bit standards can support vendor-specific Big5. Yidao has expressed my thoughts exactly, better than I myself expressed them. The key point: it is best to simply map all the common and vendor-specific codes into the 7-bit range, in a uniform, vendor-independent fashion. This is the approach taken in proposals such as my HZ+ and Fung's EHZ-Big5. Since Yidao's posting went only to soft-authors and not to ccnet-l, I will quote it here in its entirety. -- Steve CAI@neurophys.wisc.edu writes: > Big5 and Vendor-specific portions of Big5 > > There seems agreement that Big5 should be supported in some sense among > the net, given the extreme popularity of Big5. However, there are some > preference over the method of support: direct 7-bit mapping, using CNS to > "encode" Big5,and using CNS, GB, JIS to "encode" big5. In principle, I'm > not against the later two, but I have to repeat that the later two are in > an disadvantagous position at present, since there is no specific and > unique mapping presented so far. > > As for the vendor specific portion of Big5, I think first of all we have to > accept it, any proposal should support it one way or the other. However, > vendor specific portion should be discouraged on the network where your > message reaches tens of thousands people around world, many of them do not > have the same system as you have and thus could not read your message > correctly. This does not contracdict with the previous statement that > vendor specific portion should be supported. In the case of private mail > or one has to use some vendor specific characters, one need such kind of > support. In that case, the author can say something like "this file > contains some XX vendor specific characters, you need XX to view them > correctly", just like an ASCII file contains some Chinese or Japanese > characters inside, but people would still get the main idea of the article > even if they don't have XX. > > How to support vendor-specific portion? One approach is to try to map > all those characters from all known vendors to other character sets that > contains the characters. This fails when new vendors come with their own > specific characters. The other is to keep the system open, i.e., just > mapping the vendor-specific coding *spaces* to 7 bit, thus for each vendor, > the conversion is 100% reversible. This is certainly a simpler approach > and would work well in practice with the principle that vendor specific > characters should be discouraged. If you want to let yourself understood > by everybody, let your idea reach the whole audience, you should restrain > yourself from using vendor specific characters in the first place. On the > other hand, if everybody uses vendor specific characters, there must be a > competition and finally a preferred vendor would be "picked". Before the > network is set up, it doesn't matter wether you use ETen-specific characters > or KC-specific characters to produce an document, since you exchange the > hardcopies, not the electronic form. With current network, I'm sure users > will sovle this problem themself, maybe after a certain period of chaos, > be it by competition or by administration. > > Yidao CAI From simpson@math.psu.edu Fri Jun 24 11:00:16 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA01469; Fri, 24 Jun 94 11:00:16 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id LAA09511; Fri, 24 Jun 1994 11:58:03 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 24 Jun 1994 11:58:01 -0400 Date: Fri, 24 Jun 1994 11:58:01 -0400 Message-Id: <199406241558.LAA21220@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: follow-up to Ed's comments about EHZ In-Reply-To: <199406232307.TAA03754@leibniz.math.psu.edu> References: <199406232307.TAA03754@leibniz.math.psu.edu> Reply-To: simpson@math.psu.edu Edmund Lai writes: > Honestly I do not understand thing is supposed to work in EHZ. There is > an escape sequence to CNS, and there is an escape sequence to HZ+ like > encoding. How are they supposed to be used? > ... > I need a usage model since this cannot be infered from the spec. Except for the part about "two exchange code where the Chinese characters are the same" (see below), I think that Ed's questions about EHZ are legitimate. In a sense, EHZ is not one communication standard but a family of standards: EHZ-Big5, EHZ-CNS, EHZ-Big5-CNS, EHZ-Big5-JIS, EHZ-CNS-JIS, EHZ-Big5-CNS-JIS, etc. There is a legitimate question about which of these standards (if any) is to be presented as a consensus for 7-bit Chinese on the Internet, for use in chinese.* etc. There may also be questions about how the proposed standard is to be used. I'm not saying that these questions are difficult or impossible, but I do think that these questions reflect legitimate concerns. EHZ advocates should think about these questions and be prepared to answer them in due course. Given the opportunity to choose from among the various EHZ standards, I would choose EHZ-Big5. One of my reasons for this choice is that, in my opinion, the usage model for EHZ-Big5 is reasonably clear. By contrast, the usage model for EHZ-CNS is not so clear, because Big5 is popular and EHZ-CNS does not support Big5. Ed writes: > For me other than for testing purpose, it does not make sense to > have two exchange code that covers the same set of character > side by side in the same standard. This is not strictly correct. Big5 and CNS do not cover exactly the same character set. ETen Big5 covers some vendor-specific characters (e.g. ETen's Japanese kana) which CNS does not necessarily cover, or at least which are not covered by Big5->CNS translation. -- Steve From simpson@math.psu.edu Fri Jun 24 11:05:54 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA01715; Fri, 24 Jun 94 11:05:54 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id MAA09563; Fri, 24 Jun 1994 12:03:31 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 24 Jun 1994 12:03:29 -0400 Date: Fri, 24 Jun 1994 12:03:29 -0400 Message-Id: <199406241603.MAA21231@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: What's wrong with not offending anybody? In-Reply-To: <199406232307.TAA03754@leibniz.math.psu.edu> References: <199406232307.TAA03754@leibniz.math.psu.edu> Reply-To: simpson@math.psu.edu About EHZ, Edmund Lai writes: > out so that one can be picked or is it to include both to avoid > making a choice so that nobody is offended? Ed has a legitimate point. On the other hand, as I found out, not offending anybody may be a worthwhile goal. I think that my HZ+ (= HZ+S) got into trouble, not because of any lack of technical merit, but simply because I offended some people (Ed Lai, Yuan Jiang, Ricky Yeung, ...). If I had been more careful not to offend people, maybe "the great HZ+ debate" would be finished by now and we would have a consensus standard to present to the world. The actuality is that several people are angry and we don't have a consensus. -- Steve From simpson@math.psu.edu Fri Jun 24 12:20:53 1994 Return-Path: Received: from leibniz.math.psu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA05746; Fri, 24 Jun 94 12:20:53 CDT Received: from boole.math.psu.edu (simpson@boole.math.psu.edu [146.186.131.137]) by leibniz.math.psu.edu (8.6.4/8.5) with ESMTP id NAA09964; Fri, 24 Jun 1994 13:18:39 -0400 From: Stephen G Simpson Received: from localhost by boole.math.psu.edu id ; Fri, 24 Jun 1994 13:18:37 -0400 Date: Fri, 24 Jun 1994 13:18:37 -0400 Message-Id: <199406241718.NAA21371@boole.math.psu.edu> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: technical questions about EHZ In-Reply-To: <9406210559.AA13472@simd.stanford.edu> References: <9406210559.AA13472@simd.stanford.edu> Reply-To: simpson@math.psu.edu I have some small technical questions about EHZ. These questions are not intended as criticism, merely as questions which may help me to understand EHZ better. First, a couple of terminological questions. 1. Can we assume that EHZ-Big5, EHZ-CNS, EHZ-JIS etc. all include HZ/GB automatically? The reason I ask is that the spec says: > The subset of EHZ that deals only with ASCII and a non-ASCII code X > can be referred as EHZ-X. which would seem to say that EHZ-X contains only ASCII and X, not GB, but this is probably not what is intended. 2. What is the actual meaning of the term EHZ itself, at the present time? Is EHZ the union of EHZ-Big5, EHZ-CNS, EHZ-JIS, ... or is it the intersection of these? Next, a notational question. 3. In the following excerpt from the EHZ spec, what is the meaning of the prefixes 20, 21, 22? > EHZ-BIG5 BIG5 internal code (ETen, Chinese Windows 3.1, MacOS) > (2nd byte ranges: 40-7E, A1-FE) > u. 202121-20564A <-> 8140-A0FE 5024 user defined symbols > a. *212121-212540 <-> A140-A3BF 408 common symbols > d. 212940-21297E <-> A3C0-A3FE 63 vendor defined hanzi/symbols > e. 212A21-212E40 <-> C6A1-C8FE 408 vendor defined hanzi/symbols > g. *214421-217D4B <-> A440-C67E 5401 frequently-used hanzi > h. *222121-227246 <-> C940-F9D5 7652 less-frequently-used hanzi > i. 227247-227B32 <-> F9D6-FEFE 826 vendor defined hanzi/symbols > other code points unused Finally, some technical questions about the EHZ-Big5 mapping itself. 4. Fung's EHZ-Big5 mapping of Big5 into the graphical ASCII code space is given by the above table. This mapping looks similar to, but slightly more complicated than, the mapping used in my HZ+ proposal. My question is, what benefit is gained by the extra complication? Maybe I am missing some subtle point here. For comparison, my HZ+ mapping in Fung's notation would look something like u. 202121-20564A <-> 8140-A0FE 5024 user defined symbols g. 212121-21???? <-> A140-C8FE Big5 block 1 h. 222121-227B32 <-> C940-FEFE Big5 block 2 (Sorry, I am too lazy to fill in ???? here, but you get the idea.) 5. The EHZ proposal says: > For the two vendor-defined ranges (0xA3C0-0xA3FE, 0xC6A1-0xC8FE), > the BIG5 codes could not be mapped to successive EHZ-BIG5 codes, > because images of codes in the former range would overlap some > existing CNS symbol code range, which is undesirable, and images of Why is this overlap undesirable? CNS and Big5 are two different coding systems, so who cares if the code ranges overlap? 6. The EHZ proposal says: > codes in the later range would extend beyond the end of Plane 1 > (217D7E, 0x7D7E of Plane 1). This is not the case in my HZ+, so why would it be the case in EHZ-Big5? I guess I am missing some subtle point here. -- Steve From jiang Fri Jun 24 15:13:12 1994 Return-Path: Received: by ifcss.org (4.1/IFCSS-Mailer) id AA13529; Fri, 24 Jun 94 15:13:12 CDT Date: Fri, 24 Jun 94 15:13:12 CDT From: jiang (Yuan Jiang) Message-Id: <9406242013.AA13529@ifcss.org> To: CAI@neurophys.wisc.edu, soft-authors Subject: Re: Big5 and Vendor-specific Portions of Big5 I'm glad that discussion atmosphere is better now. I'm not anadvocate for CNS but am glad that the bad impression spread about CNS is fading. The "standard" Big5 is popular, and ET is popular. But how popular is ET's vendor-specific code? If only a small number of people use it, should an EHZ support it? Remember HZ and EHZ are just remedies for the current defficiency of the current systems. They are suppose to save the trouble of the mass. If only a small number of people who need to exchange files with vendor-specific code, they can always use FTP or uuencode or whatever. If EHZ supports the vendor-specific code the wrong way, it may save the trouble of a few humdres of users, but will cause trouble to thousands of KC users. HZ and EHZ should be the common demoninators, instead of the cutting-edge standard seters. I would suggest extend HZ to cover only the popular, "standard", stable, non-contraversion part of Big5. When picking the escape code, leave room for future expansion. Yuan From Ricky.Yeung@Eng.Sun.COM Fri Jun 24 15:55:47 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA15654; Fri, 24 Jun 94 15:55:47 CDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA04503; Fri, 24 Jun 94 13:53:31 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA15047; Fri, 24 Jun 94 13:53:41 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA04170; Fri, 24 Jun 1994 13:54:45 +0800 Date: Fri, 24 Jun 1994 13:54:45 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406242054.AA04170@hk.Eng.Sun.COM> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Fri, 24 Jun 1994 12:03:29 -0400 <199406241603.MAA21231@boole.math.psu.edu> Subject: Re: What's wrong with not offending anybody? Content-Length: 1420 > From: Stephen G Simpson > Date: Fri, 24 Jun 1994 12:03:29 -0400 > > About EHZ, Edmund Lai writes: > > out so that one can be picked or is it to include both to avoid > > making a choice so that nobody is offended? > > Ed has a legitimate point. On the other hand, as I found out, not > offending anybody may be a worthwhile goal. I think that my HZ+ (= > HZ+S) got into trouble, not because of any lack of technical merit, Speaking of (lack of) technical merit, the HZ+S proposal doesn't address the maximal ASCII transparency issue and it chooses not to support CNS. On the other hand, the EHZ framework closes the flood gate for the further erosion of ASCII transparency and opens the door for compromise (i.e. consensus) - namely it leaves the users to decide if they want to support Big5 and/or CNS instead of forcing a choice on them. Moreover, I think the EHZ-Big5 mapping is better than HZ+S, for example, it reflects the 3-part structure of the Big5 codes. -Ricky > but simply because I offended some people (Ed Lai, Yuan Jiang, Ricky > Yeung, ...). If I had been more careful not to offend people, maybe > "the great HZ+ debate" would be finished by now and we would have a > consensus standard to present to the world. The actuality is that > several people are angry and we don't have a consensus. > > -- Steve From Ricky.Yeung@Eng.Sun.COM Fri Jun 24 20:24:53 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA24425; Fri, 24 Jun 94 20:24:53 CDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA14672; Fri, 24 Jun 94 18:22:28 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA11081; Fri, 24 Jun 94 18:22:40 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA06187; Fri, 24 Jun 1994 18:23:42 +0800 Date: Fri, 24 Jun 1994 18:23:42 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406250123.AA06187@hk.Eng.Sun.COM> To: lee@simd.stanford.edu Cc: handa@etlken.etl.go.jp, Ricky.Yeung@Eng.Sun.COM, alt.chinese.computing@usenet.ifcss.org, ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: Fung Fung Lee's message of Thu, 23 Jun 1994 10:11:24 -0700 <9406231711.AA29217@simd.stanford.edu> Subject: Naming of EHZ Content-Length: 1166 I have no objection to the use of the term EHZ (Extended HZ) to denote my framework. In fact I like it so much that I've been using it myself. I also agree with Dr. Lee's labeling scheme, namely using EHZ-X to denote the individual mapping proposal. -Ricky > As for my own proposal EHZ (Extended HZ), I took the liberty to change > Ricky's "Extension to" to "Extended". (If Ricky has any objection, I'll > glad to discuss openly with him on this naming issue.) In my proposal, > I explicitly said that EHZ followed exactly his framework. What my > propsoal did was to fill out the simple details concerning CNS, and > the more complicated details concerning BIG5. EHZ-BIG5's 7-bit mapping > scheme is clearly different from every other BIG5 7-bit mapping proposals > previously posted. Strictly speaking, what cannot be straightforwardly > deduced from Ricky's EHZ proposal is the EHZ-BIG5 part. > In this spirit of respecting the original inventor of EHZ, I think the > proper way to label the JIS part would be to call it EHZ-JIS, which can > be straightforwardly deduced from Ricky's EHZ proposal anyway. From butta1@crsa.bu.edu Fri Jun 24 21:57:27 1994 Return-Path: Received: from crsa.bu.edu by ifcss.org (4.1/IFCSS-Mailer) id AA27000; Fri, 24 Jun 94 21:57:27 CDT Received: from localhost by crsa.bu.edu (8.6.4/Spike-2.1) id WAA08322; Fri, 24 Jun 1994 22:55:18 -0400 Date: Fri, 24 Jun 1994 22:55:17 -0400 (EDT) From: C I I Project Subject: Re: Big5 and Vendor-specific Portions of Big5 To: soft-authors@ifcss.org In-Reply-To: <9406242013.AA13529@ifcss.org> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII On Fri, 24 Jun 1994, Yuan Jiang wrote: > or whatever. If EHZ supports the vendor-specific code the wrong way, > it may save the trouble of a few humdres of users, but will cause trouble > to thousands of KC users. KC has ETen mode (this further indicates the popularity of ETen and recognization and support of ETen Big5 by a competing vendor, ie KC) > HZ and EHZ should be the common demoninators, instead of the cutting-edge > standard seters. I would suggest extend HZ to cover only the popular, > "standard", stable, non-contraversion part of Big5. When picking the > escape code, leave room for future expansion. another way is if a japanese discussion group ever gets created on chinese.* then those users can define a 7-bit transmission of the kana's in ETen BIG5, maybe as an add-on extension to EHZ, like the BIG5->GB/CNS/JIS mapping table provided by Handa, or some other methods. Nelson From jiang Sat Jun 25 20:23:04 1994 Return-Path: Received: by ifcss.org (4.1/IFCSS-Mailer) id AA06464; Sat, 25 Jun 94 20:23:04 CDT Date: Sat, 25 Jun 94 20:23:04 CDT From: jiang (Yuan Jiang) Message-Id: <9406260123.AA06464@ifcss.org> To: butta1@crsa.bu.edu, soft-authors Subject: Re: Big5 and Vendor-specific Portions of Big5 >> standard seters. I would suggest extend HZ to cover only the popular, >> "standard", stable, non-contraversion part of Big5. When picking the >> escape code, leave room for future expansion. >another way is if a japanese discussion group ever gets created on chinese.* >then those users can define a 7-bit transmission of the kana's in ETen >BIG5, maybe as an add-on extension to EHZ, like the BIG5->GB/CNS/JIS Japanese groups should be confined to fj.*. I certainly hope no Japanese or English groups created in chinese.* dierarchy. It defeats the purpose of chinese.* >Nelson From handa@etlken.etl.go.jp Mon Jun 27 02:51:26 1994 Received: from etlpost (etlpost.etl.go.jp) by ifcss.org (4.1/IFCSS-Mailer) id AA29510; Mon, 27 Jun 94 02:51:26 CDT Received: from etlpom.etl.go.jp by etlpost (5.67+1.6W/2.7W) id AA25485; Mon, 27 Jun 94 16:48:46 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA15782; Mon, 27 Jun 94 16:48:44 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA04642; Mon, 27 Jun 94 16:48:50 JST Date: Mon, 27 Jun 94 16:48:50 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9406270748.AA04642@etlken.etl.go.jp> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: HZ-EA encoder/decoder Here's an program for HZ-EA encoding/decoding. (I'll keep using the name HZ-EA for a while because I don't know any other suitable name. Accoding to the terminology of EHZ, HZ-EA means "Let's use only EHZ-GB, EHZ-CNS, EHZ-JIS, and plane 0 of EHZ-BIG5".) I have tested the program on only 32-bit UNIX machine, and don't know if it runs correctly on 16-bit machines (although I designed it cafefully so that it runs on such machines). Sorry for the poor README file because I have run out of my time for today. Actualy the encoder doesn't procude a correct HZ-EA sequence but it only converts a Big5 code (4 hex digits) to a character-set indicator (CNS11643_1, CNS11643_2, JIS, ...) and a code (4 hex digits). By giving an appropriate argument ot `make' command, programs for HZ+S, EHZ-BIG5, and HZ-EA-2 are also created. Please see README. What is HZ-EA-2? I'll explain it later. It is an compromizing plan between EHZ-BIG5 and HZ-EA. --- Ken'ichi HANDA handa@etl.go.jp ------------------------------------------------------------ #!/bin/sh # This is a shell archive (produced by shar 3.49) # To extract the files from this archive, save it to a file, remove # everything above the "!/bin/sh" line above, and type "sh file_name". # # made 06/27/1994 07:45 UTC by handa@etlken # Source directory /home/etlken/handa/BIG5 # # existing files will NOT be overwritten unless -c is specified # # This shar contains: # length mode name # ------ ---------- ------------------------------------------ # 1357 -rw-rw-r-- README # 8844 -rw-rw-r-- MAP # 849 -rw-rw-r-- Makefile # 4946 -rw-rw-r-- b2hz.c # 257 -rw-rw-r-- b5gen.c # # ============= README ============== if test -f 'README' -a X"$1" != X"-c"; then echo 'x - skipping README (File already exists)' else echo 'x - extracting README (Text)' sed 's/^X//' << 'SHAR_EOF' > 'README' && This package is a Public Domain Software. Anyone can freely copy, modify, distribute any files in this package. X FILES: X o README -- this file X o MAP -- mapping info between BIG5 and HZ code X Part of table for CNS_11643_1&2 is from X `b5cns' of Dr. Ross Paterson . X o Makefile X o b2hz.c -- converter between BIG5 and HZ code X o b5gen.c -- BIG5 code generator X INSTALL: X Just type `make', then the following files are created. X b2hz.h hz2b.h b2hz.o b2hz You can specify several options to `make' as follows: X /* To create map for HZ+S, EHZ-BIG5, HZ-EA, or HZ-EA-2, X specify 0, 1, 2, or 3 respectively for `m. X `n' defaults to 2 (HZ-EA). */ X % make METHOD=m X X /* Create map for a specifc vendor ETEN or KC, X specify 0 or 1 respectively for `v'. X `v' defaults to 0 (ETEN). */ X % make VENDOR=v X X /* Create map which resolve duplicated encoding. X In this case big5->hz->big5 wan't produce the X same code. */ X % make DUPLICATION=0 X X /* If you make your own MAP file, specify the name X of the file as `n'. */ X % make MAPFILE=n X VERIFY TEST: X Just type `make verify'. See Makefile what is done. X USAGE: X `b2ea' converts BIG5 code from stdin (4 hexa digits) to HZ code (1 digit for a character set and 4 hexa digits for a character code). The meaning of a digit for character set is described in the file `MAP'. X SHAR_EOF chmod 0664 README || echo 'restore of README failed' Wc_c="`wc -c < 'README'`" test 1357 -eq "$Wc_c" || echo 'README: original size 1357, current size' "$Wc_c" fi # ============= MAP ============== if test -f 'MAP' -a X"$1" != X"-c"; then echo 'x - skipping MAP (File already exists)' else echo 'x - extracting MAP (Text)' sed 's/^X//' << 'SHAR_EOF' > 'MAP' && #define METHOD_HZ_S 0 /* DEFINITIONS of ENCODING METHODS */ #define METHOD_EHZ_BIG5 1 #define METHOD_HZ_EA 2 #define METHOD_HZ_EA_2 3 /* ... */ X #ifndef METHOD #define METHOD METHOD_HZ_EA #endif X #define VENDOR_ETEN 0 /* DEFINITIONS of VENDORS */ #define VENDOR_KC 1 /* ... */ X #ifndef VENDOR #define VENDOR VENDOR_ETEN #endif X #define CNS11643_1 0 /* DEFINITIONS of CHARSETS */ #define CNS11643_2 1 #define BIG5_0 2 #define BIG5_1 3 #define BIG5_2 4 #define GB2312 5 #define JISX0208 6 #define KSC5601 7 X #ifndef PREF_CHARSET /* PREFERENCE of JISX0208 vs GB2312 */ #define PREF_CHARSET JISX0208 #endif X #ifndef DUPLICATION /* HOW TO HANDLE DUPLICATED CHARACTERS */ #define DUPLICATION 0 /* 0: RESOLVE, 1: MAP TO BIG5_0 */ #endif X /************************************************/ /* DEFINITIONS of MAPPING */ /* BIG5, FROM, TO, CHARACTER-SET, */ /* FROM and TO shows the range in CHARACTER-SET */ /************************************************/ X #if METHOD == METHOD_HZ_S X /*************************************************************************/ /* The map proposed in HZ+S by Stephen G Simpson . */ /*************************************************************************/ X /* USER DEFINED CHARACTERS */ X 0x8140, 0x2121, 0x564A, BIG5_0, /* BIG5 Plane 1 */ X 0xA140, 0x2121, 0x636C, BIG5_1, /* BIG5 Plane 2 */ X 0xC940, 0x2121, 0x7B32, BIG5_2, X #else /* METHOD != METHOD_HZ_S */ #if METHOD == METHOD_EHZ_BIG5 X /*************************************************************************/ /* The map proposed in EHZ-BIG5 by Fung Fung Lee . */ /*************************************************************************/ X /* USER DEFINED CHARACTERS */ X 0x8140, 0x2121, 0x564A, BIG5_0, /* COMMON SYMBOLS */ X 0xA140, 0x2121, 0x2540, BIG5_1, /* VENDOR DEPENDENT CHARACTERS */ X 0xA3C0, 0x2940, 0x297E, BIG5_1, /* FREQUENTRY USED CHINESE CHARACTERS */ X 0xA440, 0x4421, 0x7D4B, BIG5_1, /* VENDOR DEPENDENT CHARACTERS */ X 0xC6A1, 0x2A21, 0x2E40, BIG5_1, /* LESS-FREQUENTRY USED CHINESE CHARACTERS */ X 0xC940, 0x2121, 0x7246, BIG5_2, /* VENDOR DEPENDENT CHARACTERS */ X 0xF9D6, 0x7247, 0x7B32, BIG5_2, X #else /* METHOD != METHOD_HZ_S && METHOD != METHOD_EHZ_BIG5 */ #if METHOD >= METHOD_HZ_EA X /*************************************************************************/ /* The map proposed in HZ-EA by Kenichi Handa . */ /* It contains both HZ-EA and HZ-EA-2. */ /*************************************************************************/ X /* USER DEFINED CHARACTERS */ X 0x8140, 0x4021, 0x754A, BIG5_0, /* COMMON SYMBOLS */ X 0xA140, 0x2121, 0x234E, CNS11643_1, X 0xA2AF, 0x2421, 0x2570, CNS11643_1, /* VENDOR DEPENDENT CHARACTERS */ #if VENDOR == VENDOR_ETEN X 0xA3C0, 0x2121, 0x215F, BIG5_0, #else X 0xA3C0, 0x2121, 0x215F, BIG5_0, #endif /* FREQUENTRY USED CHINESE CHARACTERS */ X 0xA440, 0x4421, 0x5322, CNS11643_1, X 0xACFE, 0x5753, 0x5753, CNS11643_1, X 0xAD40, 0x5323, 0x5752, CNS11643_1, X 0xAFD0, 0x5754, 0x6B4F, CNS11643_1, X 0xBBC8, 0x6B51, 0x6F5B, CNS11643_1, X 0xBE52, 0x6B50, 0x6B50, CNS11643_1, X 0xBE53, 0x6F5C, 0x7543, CNS11643_1, X 0xC1AB, 0x7536, 0x7736, CNS11643_1, X 0xC2CB, 0x7535, 0x7535, CNS11643_1, X 0xC2CC, 0x7737, 0x782C, CNS11643_1, X 0xC361, 0x782E, 0x7863, CNS11643_1, X 0xC3B9, 0x7865, 0x7865, CNS11643_1, X 0xC3BA, 0x7864, 0x7864, CNS11643_1, X 0xC3BB, 0x7866, 0x7961, CNS11643_1, X 0xC456, 0x782D, 0x782D, CNS11643_1, X 0xC457, 0x7962, 0x7D4B, CNS11643_1, /* VENDOR DEPENDENT CHARACTERS */ #if (METHOD == METHOD_HZ_EA) && (VENDOR == VENDOR_ETEN) X /* VARIOUS NUMBERS (circled, parenthesized, lower-roman) */ X 0xC6A1, 0x2621, 0x263E, CNS11643_1, X /* CHINESE RADICALS */ X 0xC6BF, 0x2723, 0x2723, CNS11643_1, X 0xC6C0, 0x2724, 0x2724, CNS11643_1, X 0xC6C1, 0x2726, 0x2726, CNS11643_1, X 0xC6C2, 0x2728, 0x2728, CNS11643_1, X 0xC6C3, 0x272D, 0x272D, CNS11643_1, X 0xC6C4, 0x272E, 0x272E, CNS11643_1, X 0xC6C5, 0x272F, 0x272F, CNS11643_1, X 0xC6C6, 0x2734, 0x2734, CNS11643_1, X 0xC6C7, 0x2737, 0x2737, CNS11643_1, X 0xC6C8, 0x273A, 0x273A, CNS11643_1, X 0xC6C9, 0x273C, 0x273C, CNS11643_1, X 0xC6CA, 0x2742, 0x2742, CNS11643_1, X 0xC6CB, 0x2747, 0x2747, CNS11643_1, X 0xC6CC, 0x274E, 0x274E, CNS11643_1, X 0xC6CD, 0x2753, 0x2753, CNS11643_1, X 0xC6CE, 0x2754, 0x2754, CNS11643_1, X 0xC6CF, 0x2755, 0x2755, CNS11643_1, X 0xC6D0, 0x2759, 0x2759, CNS11643_1, X 0xC6D1, 0x275A, 0x275A, CNS11643_1, X 0xC6D2, 0x2761, 0x2761, CNS11643_1, X 0xC6D3, 0x2766, 0x2766, CNS11643_1, X 0xC6D4, 0x2829, 0x2829, CNS11643_1, X 0xC6D5, 0x282A, 0x282A, CNS11643_1, X 0xC6D6, 0x2863, 0x2863, CNS11643_1, X 0xC6D7, 0x286C, 0x286C, CNS11643_1, X /* JAPANESE SYMBOLS */ X 0xC6D8, 0x212F, 0x212F, JISX0208, X 0xC6D9, 0x2130, 0x2130, JISX0208, X 0xC6DA, 0x2133, 0x2133, JISX0208, X 0xC6DB, 0x2134, 0x2134, JISX0208, X 0xC6DC, 0x2135, 0x2135, JISX0208, X 0xC6DD, 0x2136, 0x2136, JISX0208, X 0xC6DE, 0x2137, 0x2137, JISX0208, X 0xC6DF, 0x2138, 0x2138, JISX0208, X 0xC6E0, 0x2139, 0x2139, JISX0208, X 0xC6E1, 0x213A, 0x213A, JISX0208, X 0xC6E2, 0x213B, 0x213B, JISX0208, X 0xC6E3, 0x213C, 0x213C, JISX0208, X 0xC6E4, 0x214E, 0x214E, JISX0208, X 0xC6E5, 0x214F, 0x214F, JISX0208, X 0xC6E6, 0x2176, 0x2176, JISX0208, X /* JAPANESE HIRAGANA */ X 0xC6E7, 0x2421, 0x2473, PREF_CHARSET, X /* JAPANESE KATAKANA */ X 0xC77B, 0x2521, 0x2576, PREF_CHARSET, X /* CYRILLIC UPPER */ X 0xC7F3, 0x2721, 0x2741, PREF_CHARSET, X /* CYRILLIC LOWER */ X 0xC855, 0x2751, 0x2771, PREF_CHARSET, X /* MISCELLANEOUS SYMBOLS */ X 0xC876, 0x2538, 0x2573, BIG5_0, X /* UNUSED AREA */ X 0xC8D4, 0x2574, 0x2640, BIG5_0, #else /* (METHOD == METHOD_HZ_EA_2) || (VENDOR != VENDOR_ETEN) */ X /* VENDOR DEPENDENT */ X 0xC6A1, 0x2221, 0x2640, BIG5_0, #endif /* (METHOD == METHOD_HZ_EA_2) || (VENDOR != VENDOR_ETEN) */ /* LEFF-FREQUENTRY USED CHINESE CHARACTERS */ X 0xC940, 0x2121, 0x212A, CNS11643_2, #if DUPLICATION == 0 X 0xC94A, 0x4442, 0x4442, CNS11643_1, #else X 0xC94A, 0x3F7D, 0x3F7D, BIG5_0, #endif X 0xC94B, 0x212B, 0x214B, CNS11643_2, X 0xC96C, 0x214D, 0x217C, CNS11643_2, X 0xC9BE, 0x214C, 0x214C, CNS11643_2, X 0xC9BF, 0x217D, 0x224C, CNS11643_2, X 0xC9ED, 0x224E, 0x2438, CNS11643_2, X 0xCAF7, 0x224D, 0x224D, CNS11643_2, X 0xCAF8, 0x2439, 0x376E, CNS11643_2, X 0xD6CC, 0x3E63, 0x3E63, CNS11643_2, X 0xD6CD, 0x3770, 0x387D, CNS11643_2, X 0xD77A, 0x3F6A, 0x3F6A, CNS11643_2, X 0xD77B, 0x387E, 0x3E62, CNS11643_2, X 0xDADF, 0x376F, 0x376F, CNS11643_2, X 0xDAE0, 0x3E64, 0x3F69, CNS11643_2, X 0xDBA7, 0x3F6B, 0x4423, CNS11643_2, #if DUPLICATION == 0 X 0xDDFC, 0x4176, 0x4176, CNS11643_2, #else X 0xDDFC, 0x3F7E, 0x3F7E, BIG5_0, #endif X 0xDDFD, 0x4424, 0x554A, CNS11643_2, X 0xE8A3, 0x554C, 0x5721, CNS11643_2, X 0xE976, 0x5723, 0x5A27, CNS11643_2, X 0xEB5B, 0x5A29, 0x5B3E, CNS11643_2, X 0xEBF1, 0x554B, 0x554B, CNS11643_2, X 0xEBF2, 0x5B3F, 0x5C69, CNS11643_2, X 0xECDE, 0x5722, 0x5722, CNS11643_2, X 0xECDF, 0x5C6A, 0x5D73, CNS11643_2, X 0xEDAA, 0x5D75, 0x6038, CNS11643_2, X 0xEEEB, 0x642F, 0x642F, CNS11643_2, X 0xEEEC, 0x6039, 0x6242, CNS11643_2, X 0xF056, 0x5D74, 0x5D74, CNS11643_2, X 0xF057, 0x6243, 0x6336, CNS11643_2, X 0xF0CB, 0x5A28, 0x5A28, CNS11643_2, X 0xF0CC, 0x6337, 0x642E, CNS11643_2, X 0xF163, 0x6430, 0x6437, CNS11643_2, X 0xF16B, 0x6761, 0x6761, CNS11643_2, X 0xF16C, 0x6438, 0x6572, CNS11643_2, X 0xF268, 0x6934, 0x6934, CNS11643_2, X 0xF269, 0x6573, 0x664C, CNS11643_2, X 0xF2C3, 0x664E, 0x6760, CNS11643_2, X 0xF375, 0x6762, 0x6933, CNS11643_2, X 0xF466, 0x6935, 0x6961, CNS11643_2, X 0xF4B5, 0x664D, 0x664D, CNS11643_2, X 0xF4B6, 0x6962, 0x6A4A, CNS11643_2, X 0xF4FD, 0x6A4C, 0x6C51, CNS11643_2, X 0xF663, 0x6A4B, 0x6A4B, CNS11643_2, X 0xF664, 0x6C52, 0x7165, CNS11643_2, X 0xF977, 0x7167, 0x7233, CNS11643_2, X 0xF9C4, 0x7166, 0x7166, CNS11643_2, X 0xF9C5, 0x7234, 0x7234, CNS11643_2, X 0xF9C6, 0x7240, 0x7240, CNS11643_2, X 0xF9C7, 0x7235, 0x723F, CNS11643_2, X 0xF9D2, 0x7241, 0x7244, CNS11643_2, /* VENDOR DEPENDENT CHARACTERS */ #if (METHOD == METHOD_HZ_EA_2) && (VENDOR != VENDOR_ETEN) X /* MISCELLANEOUS CHINESE CHARACTERS */ #if PREF_CHARSET == JISX0208 X 0xF9D6, 0x386B, 0x386B, JISX0208, X 0xF9D7, 0x6E78, 0x6E78, JISX0208, X 0xF9D8, 0x4E22, 0x4E22, JISX0208, X 0xF9D9, 0x5456, 0x5456, JISX0208, X 0xF9DA, 0x3931, 0x3931, JISX0208, X 0xF9DB, 0x3E51, 0x3E51, JISX0208, X 0xF9DC, 0x5546, 0x5546, JISX0208, #else /* PREFERED_CHARSET == GB2312 */ X 0xF9D6, 0x4665, 0x4665, GB2312, X 0xF9D7, 0x5062, 0x5062, GB2312, X 0xF9D8, 0x406F, 0x406F, GB2312, X 0xF9D9, 0x5899, 0x5899, GB2312, X 0xF9DA, 0x3A63, 0x3A63, GB2312, X 0xF9DB, 0x5731, 0x5731, GB2312, X 0xF9DC, 0x6635, 0x6635, GB2312, #endif /* PREFERED_CHARSET == GB2312 */ X /* RULED DOUBLE LINES */ X 0xF9DD, 0x2728, 0x2748, BIG5_0, X /* BLACK BOX */ X 0xF9FE, 0x2749, 0x2749, BIG5_0, X /* UNUSED AREA */ X 0xFA40, 0x274A, 0x2F6A, BIG5_0, #else /* (METHOD == METHOD_HZ_EA_2) || (VENDOR != VENDOR_ETEN) */ X 0xF9D6, 0x2721, 0x2F6A, BIG5_0, #endif /* (METHOD == METHOD_HZ_EA_2) || (VENDOR != VENDOR_ETEN) */ X #endif #endif #endif SHAR_EOF chmod 0664 MAP || echo 'restore of MAP failed' Wc_c="`wc -c < 'MAP'`" test 8844 -eq "$Wc_c" || echo 'MAP: original size 8844, current size' "$Wc_c" fi # ============= Makefile ============== if test -f 'Makefile' -a X"$1" != X"-c"; then echo 'x - skipping Makefile (File already exists)' else echo 'x - extracting Makefile (Text)' sed 's/^X//' << 'SHAR_EOF' > 'Makefile' && MAPFILE = MAP X METHOD = 2 VENDOR = 0 DUPLICATION = 1 X CC = cc CPP = cc -E CFLAGS = -DMETHOD=${METHOD} -DDUPLICATION=${DUPLICATION} -DVENDOR=${VENDOR} X all: b2hz.o b2hz X b2hz.h: ${MAPFILE} X ${CPP} ${CFLAGS} ${MAPFILE}\ X | sed -n 's/\(^ 0.*\)$$/\1 0xFFFF,/p' > b2hz.h X hz2b.h: ${MAPFILE} X ${CPP} ${CFLAGS} ${MAPFILE}\ X | sort +3 -4 +1 -3\ X | sed -n 's/\(^ 0.*\)$$/\1 0xFFFF,/p' > hz2b.h X b2hz.o: b2hz.h hz2b.h b2hz.c X ${CC} -c b2hz.c X b2hz: b2hz.h hz2b.h b2hz.c X ${CC} -DSTANDALONE -o b2hz b2hz.c X verify: b2hz b5gen X ./b5gen > b5code1 X ./b2hz < b5code1 > hzcode X ./b2hz -rev < hzcode > b5code2 X @if diff b5code1 b5code2 > verifylog; then\ X echo "VERIFY SUCCEEDED.";\ X else\ X echo "VERIFY FAILED. SEE THE FILE verifylog.";\ X fi X b5gen: X ${CC} -o b5gen b5gen.c X clean: X rm -f b2hz.h hz2b.h b2hz.o b2hz b5gen b5code1 b5code2 hzcode verifylog SHAR_EOF chmod 0664 Makefile || echo 'restore of Makefile failed' Wc_c="`wc -c < 'Makefile'`" test 849 -eq "$Wc_c" || echo 'Makefile: original size 849, current size' "$Wc_c" fi # ============= b2hz.c ============== if test -f 'b2hz.c' -a X"$1" != X"-c"; then echo 'x - skipping b2hz.c (File already exists)' else echo 'x - extracting b2hz.c (Text)' sed 's/^X//' << 'SHAR_EOF' > 'b2hz.c' && #define CNS11643_1 0 /* DEFINITIONS of CHARSETS */ #define CNS11643_2 1 #define BIG5_0 2 #define BIG5_1 3 #define BIG5_2 4 #define GB2312 5 #define JISX0208 6 #define KSC5601 7 #define PRIVATE 8 X #define N_CHARSET 9 /* NUMBER of CHARSETS */ X static unsigned int b2hz_map[][5] = { #include "b2hz.h" X 0xFFFF, 0, 0, N_CHARSET, 0 /* This entry is for anchoring. */ }; X static unsigned int hz2b_map[][5] = { #include "hz2b.h" X 0, 0xFFFF, 0, N_CHARSET, 0 /* This entry is for anchoring. */ }; X static int hz2b_idx[N_CHARSET][2] = {-1}; X static unsigned int * map_search (code, table, from, to, offset) X unsigned int code, *table; X register int from, to, offset; { X register int i; X register unsigned int c; X X i = (from + to) / 2; X c = table[i * 5 + offset]; X while (from < i) { X if (c < code) X from = i, i = (i + to) / 2; X else if (c > code) X to = i, i = (from + i) / 2; X else X break; X c = table[i * 5 + offset]; X } X X return (table + i * 5); } X /* I hope all constant calculations be resolved by a compiler. */ X #define B_ROW (0xFF - 0xA1 + 0x7F - 0x40) #define B_OFF_1 0x40 #define B_OFF_2 (0xA1 - 0x7F + 0x40) X #define B_PACK(bh,bl) \ X (((bh - 0xA1) * B_ROW) + (bl - (bl < 0x7F ? B_OFF_1 : B_OFF_2))) #define B_UNPACK(b,bh,bl) \ X (bh = (((b) / B_ROW) + 0xA1), \ X bl = (b) % B_ROW, \ X bl += (bl < (0x7F - 0x40)) ? B_OFF_1 : B_OFF_2) X #define HZ_ROW (0x7F - 0x21) #define HZ_OFF 0x21 X #define HZ_PACK(c) \ X (((c >> 8) - HZ_OFF) * HZ_ROW + (c & 0xFF) - HZ_OFF) #define HZ_UNPACK(c,ch,cl) \ X (ch = (c) / HZ_ROW + HZ_OFF, cl = (c) % HZ_ROW + HZ_OFF) X b2hz (bh, bl, cs, ch, cl) X unsigned int bh, bl, *cs, *ch, *cl; { X register unsigned int big5, c, *element; X X big5 = (bh << 8) | bl; X element = map_search (big5, b2hz_map, X 0, (sizeof (b2hz_map)) / (sizeof (b2hz_map[0])) - 1, X 0); X *cs = element[3]; X c = element[1]; X if (c == element[2]) { X /* one-to-one match */ X *ch = c >> 8; *cl = c & 0xFF; X } else { X /* range */ X if (element[4] == 0xFFFF) { X /* Set an offset value for later access. */ X register unsigned int b0, b1; X X b0 = element[0] >> 8, b1 = element[0] & 0xFF; X if (b0 < 0xA1) /* trick for 0x8140-0xA0FE */ X b0 += 0x20; X c = element[1]; X (int)(element[4]) = HZ_PACK (c) - B_PACK (b0, b1); X } X if (bh < 0xA1) /* trick for 0x8140-0xA0FE */ X bh += 0x20; X c = B_PACK (bh, bl) + (int)element[4]; X HZ_UNPACK (c, *ch, *cl); X } X return 0; } X hz2b (cs, ch, cl, bh, bl) X unsigned int cs, ch, cl, *bh, *bl; { X register unsigned int big5, c, *element; X int i, offset; X X if (cs >= N_CHARSET) X /* Invalide character set. */ X return -1; X X if (hz2b_idx[0][0] < 0) { X /* We must create index for each character-set range. */ X int current_cs; X X /* At first, make all index but the first invalid. */ X for (i = 1; i < N_CHARSET; i++) X hz2b_idx[i][0] = -1; X X /* Then, set actual index. */ X i = 0; X while (i < (sizeof (hz2b_map) / sizeof (hz2b_map[0]) - 1)) { X current_cs = hz2b_map[i][3]; X hz2b_idx[current_cs][0] = i++; X while (hz2b_map[i][3] == current_cs) i++; X hz2b_idx[current_cs][1] = i; X } X } X X if (hz2b_idx[cs][0] < 0) X /* Can't map any character in this character set. */ X return -1; X X c = (ch << 8) | cl; X element = map_search (c, hz2b_map, X hz2b_idx[cs][0], hz2b_idx[cs][1], X 1); X if (c > element[2]) { X /* out of range */ X return -1; X } else if (element[1] == element[2]) { X /* one-to-one match. */ X c = element[0]; X *bh = c >> 8; *bl = c & 0xFF; X } else { X /* range */ X if (element[4] == 0xFFFF) { X /* Set an offset value for later access. */ X register unsigned int b0, b1; X X b0 = element[0] >> 8, b1 = element[0] & 0xFF; X if (b0 < 0xA1) /* trick for 0x8140-0xA0FE */ X b0 += 0x20; X (int)element[4] = B_PACK (b0, b1) - HZ_PACK (element[1]); X } X c = HZ_PACK (c) + (int)element[4]; X B_UNPACK (c, *bh, *bl); X if (cs == BIG5_0 /* trick for 0x8140-0xA0FE */ X && ch >= (b2hz_map[0][1] >> 8)) X *bh -= 0x20; X } X return 0; } X #ifdef STANDALONE X #include X main(argc, argv) X int argc; X char **argv; { X unsigned int c, cs, ch, cl, bh, bl; X int rev = 0; X X if (argc >= 2) { X if (strcmp (argv[1], "-rev")) { X fprintf (stderr, "Usage: %s [-rev]\n", argv[0]); X exit (1); X } X rev = 1; X } X X if (rev) { X while (scanf ("%d %x", &cs, &c) > 0) { X ch = c >> 8, cl = c & 0x7F; X if (hz2b (cs, ch, cl, &bh, &bl) < 0) { X fprintf (stderr, "Invalide HZ character %d:%02x%02x\n", cs, ch, cl); X exit (1); X } X printf ("%02x%02x\n", bh, bl); X } X } else { X while (scanf ("%x", &c) > 0) { X bh = c >> 8, bl = c & 0xFF; X if (b2hz (bh, bl, &cs, &ch, &cl) < 0) { X fprintf (stderr, "Invalide Big5 char %04x\n", c); X exit (1); X } X printf ("%d %02x%02x\n", cs, ch, cl); X } X } X X exit (0); } #endif /* STANDALONE */ SHAR_EOF chmod 0664 b2hz.c || echo 'restore of b2hz.c failed' Wc_c="`wc -c < 'b2hz.c'`" test 4946 -eq "$Wc_c" || echo 'b2hz.c: original size 4946, current size' "$Wc_c" fi # ============= b5gen.c ============== if test -f 'b5gen.c' -a X"$1" != X"-c"; then echo 'x - skipping b5gen.c (File already exists)' else echo 'x - extracting b5gen.c (Text)' sed 's/^X//' << 'SHAR_EOF' > 'b5gen.c' && #include X main() { X unsigned int bh, bl; X X for (bh = 0x81; bh < 0xFF; bh++) { X for (bl = 0x40; bl < 0x7f; bl++) X printf ("%02x%02x\n", bh, bl); X for (bl = 0xA1; bl < 0xFF; bl++) X printf ("%02x%02x\n", bh, bl); X } X X exit (0); } SHAR_EOF chmod 0664 b5gen.c || echo 'restore of b5gen.c failed' Wc_c="`wc -c < 'b5gen.c'`" test 257 -eq "$Wc_c" || echo 'b5gen.c: original size 257, current size' "$Wc_c" fi exit 0 From handa@etlken.etl.go.jp Mon Jun 27 08:12:24 1994 Received: from etlpost (etlpost.etl.go.jp) by ifcss.org (4.1/IFCSS-Mailer) id AA06509; Mon, 27 Jun 94 08:12:24 CDT Received: from etlpom.etl.go.jp by etlpost (5.67+1.6W/2.7W) id AA28396; Mon, 27 Jun 94 22:10:08 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA19736; Mon, 27 Jun 94 22:10:07 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA04847; Mon, 27 Jun 94 22:10:13 JST Date: Mon, 27 Jun 94 22:10:13 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9406271310.AA04847@etlken.etl.go.jp> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Thu, 23 Jun 1994 16:43:30 -0400 <199406232043.QAA19317@boole.math.psu.edu> Subject: Re: HZ-EA [Re: Big5 to CNS conversion algorithms] On Thu, 23 Jun 1994 16:43:30 -0400, Stephen G Simpson said: > defined and readily available from many sources. Even if we want to > use vendor-specific extensions of Big5, we are free to label them as > such or to agree among ourselves, so no clarity is lost. This can be I think `free to label them' means `uncleaness'. > Big5 is not an official government standard, but nevertheless Big5 > users can get reliable information about Big5 from many sources, for > example the ETen manual. (ETen is an extremely popular Big5 operating > environment. It is installed on millions of computers. It is easy to > buy. It costs about $65 US, including the manual.) By contrast, ISO > charges a lot money for their documents, and these documents are not > readily available to the general public. On CCNET, one user asked for > ISO-2022 specs and was told (by an ISO-2022 advocate) that the > information is either not available or too expensive to buy. When new companies enters a market of Chinese computer/software, I think they'll select CNS as a default character set in the current situation of CNS being registerd in ISO. I heard on this list or somewhere else that unix vendors such as DEC, SUN, HP are supporting CNS. Is it correct? How about IBM's unix machines? I guess that their decision was mainly because CNS is official and meets ISO. We Japanse met the same situation. In Japanese PC world, MicroSofts and NEC's Japanese MS-DOS had the largest share like ETen in Taiwan, and all PC users were using MS-code (Shift-JIS encoding) (even now). Shift-JIS is an special encoding mechanism to represent ASCII, JISX0201-Kana, and JISX0208 and vendor-specific characters by 16bit. It's very similar to Big5. But, most of unix vendors (entered later in this market area) put a base of supporting Japanese on EUC encoding (8-bit representation of ISO2022). > > 2. CNS has no variation and the set of characters are fixed. > Not true. For example, ETen also supports CNS (in a non-default mode) > with vendor-specific extensions. In my mail, "CNS" means "CNS11643", and if a vendor adds some characters to it, that part is not CNS11643. > > 3. As for a character set, CNS is more well-defined than > > Big5 (i.e. no dumplicated-encoding, meaningfull ordering). > Not true. The common part of Big5 is just as meaningful as the common > part of CNS. In fact it consists of almost the same character set, > only in a somewhat different ordering, but not a less meaningful > ordering. Why do you refer only common part of Big5? Anyway, although it's difficult to discuss about meaning of character ordering, anyone can easily guess that Big5 and CNS put the base of ordering on stroke counts. In this aspect, Big5 puts several charactes in incorrect code point. In the above paragraph, please note that I don't intend to blame any Big5 users just because they are using it. I even think that Big5 (as an encoding mechanism) is an gode idea to support many Chinese characters if only 16bit can be used. --- Ken'ichi HANDA handa@etl.go.jp From Ricky.Yeung@Eng.Sun.COM Mon Jun 27 12:55:15 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA17086; Mon, 27 Jun 94 12:55:15 CDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AA02566; Mon, 27 Jun 94 10:52:32 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA17855; Mon, 27 Jun 94 10:53:11 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA11543; Mon, 27 Jun 1994 10:53:45 +0800 Date: Mon, 27 Jun 1994 10:53:45 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406271753.AA11543@hk.Eng.Sun.COM> To: handa@etlken.etl.go.jp Cc: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: Kenichi Handa's message of Mon, 27 Jun 94 16:48:50 JST <9406270748.AA04642@etlken.etl.go.jp> Subject: Naming. Restricted EHZ encoder/decoder (Re: HZ-EA encoder/decoder) Content-Length: 494 > Date: Mon, 27 Jun 94 16:48:50 JST > From: handa@etlken.etl.go.jp (Kenichi Handa) > > (I'll keep using the name HZ-EA for a while because I don't > know any other suitable name. Accoding to the terminology > of EHZ, HZ-EA means "Let's use only EHZ-GB, EHZ-CNS, > EHZ-JIS, and plane 0 of EHZ-BIG5".) How about Restricted EHZ? Or EHZ w/o Big5? A name like "HZ-EA" with no reflection of its relation to EHZ really misleads people and creates confusion. -Ricky From Ricky.Yeung@Eng.Sun.COM Wed Jun 29 01:55:14 1994 Return-Path: Received: from Sun.COM by ifcss.org (4.1/IFCSS-Mailer) id AA00549; Wed, 29 Jun 94 01:55:14 CDT Received: from Eng.Sun.COM (zigzag.Eng.Sun.COM) by Sun.COM (sun-barr.Sun.COM) id AB11803; Tue, 28 Jun 94 23:53:03 PDT Received: from hk.Eng.Sun.COM by Eng.Sun.COM (4.1/SMI-4.1) id AA13646; Tue, 28 Jun 94 23:54:00 PDT Received: by hk.Eng.Sun.COM (5.0/SMI-SVR4) id AA16739; Tue, 28 Jun 1994 23:54:16 +0800 Date: Tue, 28 Jun 1994 23:54:16 +0800 From: Ricky.Yeung@Eng.Sun.COM (Ricky Yeung) Message-Id: <9406290654.AA16739@hk.Eng.Sun.COM> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: Stephen G Simpson's message of Fri, 24 Jun 1994 13:18:37 -0400 <199406241718.NAA21371@boole.math.psu.edu> Subject: Re: technical questions about EHZ Content-Length: 3239 > From: Stephen G Simpson > Date: Fri, 24 Jun 1994 13:18:37 -0400 > > First, a couple of terminological questions. [...] Please see my "Naming of EHZ" message. > Next, a notational question. > > 3. In the following excerpt from the EHZ spec, what is the meaning of > the prefixes 20, 21, 22? >From Fung Lee's document: 'beyond the end of Plane 1 (217D7E, 0x7D7E of Plane 1). Hence, a mapping' It should be easy to infer the meaning of the prefixes: 20 - Plane 0 21 - Plane 1 22 - Plane 2 Aparently you got the meaning since you used it correctly in your mapping questions below. Fung Lee perhaps should revise his draft to make this point clearer. > Finally, some technical questions about the EHZ-Big5 mapping itself. I think you can easily find the answers to your mapping questions (with some simple arithmetics) if you did not quote Fung Lee's document in the way you did. >From Fung Lee's document: 'As for BIG5, the situation is somewhat tricky, because there is no pre-defined 7-bit mapping as in the case of CNS and GB. However, a natural starting point is to find "anchor code points" using the approximate correspondence between CNS and BIG5. Three such anchor points are the first member of the common symbols (the space symbol), the first member of the frequently-used hanzi, and the first member of the less-frequently-used hanzi. This "anchoring" principle says that each of these three members should share the same 7-bit code point in both EHZ-CNS and EHZ-BIG5. Successive members of the three above ranges in BIG5 are then assigned successive code points in EHZ-BIG5, as shown in Table 1. These three regions cover all the vendor-independent common parts of BIG5.' -Ricky > 4. Fung's EHZ-Big5 mapping of Big5 into the graphical ASCII code > space is given by the above table. This mapping looks similar to, but > slightly more complicated than, the mapping used in my HZ+ proposal. > My question is, what benefit is gained by the extra complication? > > Maybe I am missing some subtle point here. For comparison, my HZ+ > mapping in Fung's notation would look something like > > u. 202121-20564A <-> 8140-A0FE 5024 user defined symbols > g. 212121-21???? <-> A140-C8FE Big5 block 1 > h. 222121-227B32 <-> C940-FEFE Big5 block 2 > > (Sorry, I am too lazy to fill in ???? here, but you get the idea.) > > 5. The EHZ proposal says: > > For the two vendor-defined ranges (0xA3C0-0xA3FE, 0xC6A1-0xC8FE), > > the BIG5 codes could not be mapped to successive EHZ-BIG5 codes, > > because images of codes in the former range would overlap some > > existing CNS symbol code range, which is undesirable, and images of > > Why is this overlap undesirable? CNS and Big5 are two different > coding systems, so who cares if the code ranges overlap? > > 6. The EHZ proposal says: > > codes in the later range would extend beyond the end of Plane 1 > > (217D7E, 0x7D7E of Plane 1). > > This is not the case in my HZ+, so why would it be the case in > EHZ-Big5? I guess I am missing some subtle point here. > > -- Steve > > From handa@etlken.etl.go.jp Wed Jun 29 04:23:43 1994 Received: from etlpost (etlpost.etl.go.jp) by ifcss.org (4.1/IFCSS-Mailer) id AA04511; Wed, 29 Jun 94 04:23:43 CDT Received: from etlpom.etl.go.jp by etlpost (5.67+1.6W/2.7W) id AA20198; Wed, 29 Jun 94 18:21:12 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA18161; Wed, 29 Jun 94 18:21:11 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA06482; Wed, 29 Jun 94 18:21:18 JST Date: Wed, 29 Jun 94 18:21:18 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9406290921.AA06482@etlken.etl.go.jp> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org Subject: A compromise proposal I think that the most difficult point of HZ-EA to be accepted is that it forces even vendor-dependent characters to be mapped to CNS/GB/JIS as far as possible. This surely requires a lot of works. I still think that this is the best way, but there's an secondary best way of mapping all vendor-dependent/user-defined characters to some private area (like "Plane 0" of EHZ-BIG5) while common characters still being mapped to CNS. This is what I wrote as "HZ-EA-2" in my previous mail. Here's the comparison of mapping method among HZ+S, EHZ-BIG5, and HZ-EA-2. ---------------------------------------------------------------------- Big5-code area HZ+S EHZ-BIG5 HZ-EA-2 U: 8140-A0FE Big5-P0:[2121-] Big5-P0:[2121-] Private:[4021-] C1:A140-A3BF Big5-P1:[2121- Big5-P1:[2121-] CNS-P1: [2121-] V1:A3C0-A3FE Big5-P1: - Big5-P1:[2940-] Private:[2121-] C2:A440-C67E Big5-P1: - Big5-P1:[4421-] CNS-P1: [4421-] V2:C6A1-C8FE Big5-P1: -636C] Big5-P1:[2A21-] Private:[2221-] C3:C940-F9D5 Big5-P2:[2121- Big5-P2:[2121- CNS-P2: [2121-] V3:F9D6-FEFE Big5-P2: -7B32] Big5-P2: -7B32] Private:[2721-] ---------------------------------------------------------------------- [Pn means Plane n] [HZ-EA-2 will map C94A and DDFC to 7E7D and 7E7E or Private] With HZ-EA-2, vendor-dependent/user-defined characters are not assured to be exchanged between different vendors/users, but the other methods also can't assure it. The merit of HZ-EA-2 over HZ+S and EHZ-BIG5 is that it assures communication between CNS users and Big5 users (common part only). [However, HZ-EA-2 may appear stepping-back to those who approved HZ-EA.] As for designation sequences, I think something which follows "EHZ framework" is the best. I do want to know how my proposals are appearing to many people. Private mails to me are also very welcome [just saying "Nonsence!" is ok. :-)]. --- Ken'ichi HANDA handa@etl.go.jp From lee@simd.stanford.edu Wed Jun 29 12:01:28 1994 Return-Path: Received: from simd.stanford.edu by ifcss.org (4.1/IFCSS-Mailer) id AA16984; Wed, 29 Jun 94 12:01:28 CDT Received: by simd.stanford.edu (5.65/inc-1.0) id AA06809; Wed, 29 Jun 1994 09:58:37 -0700 Date: Wed, 29 Jun 1994 09:58:37 -0700 From: lee@simd.stanford.edu (Fung Fung Lee) Message-Id: <9406291658.AA06809@simd.stanford.edu> To: handa@etlken.etl.go.jp Cc: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: Kenichi Handa's message of Wed, 29 Jun 94 18:21:18 JST <9406290921.AA06482@etlken.etl.go.jp> Subject: RE: A compromise proposal [[Again, this message may be rejected by ccnet-l, since I don't subscribe.]] >Date: Wed, 29 Jun 94 18:21:18 JST >From: handa@etlken.etl.go.jp (Kenichi Handa) > >I think that the most difficult point of HZ-EA to be >accepted is that it forces even vendor-dependent characters >to be mapped to CNS/GB/JIS as far as possible. This surely >requires a lot of works. I still think that this is the I think the fact that your scheme implicitly denies an "independent" direct 7-bit mapping of BIG5 is perhaps sufficient reason for it NOT to be accepted widely. With the current 8-bit networking environment in Taiwan (where the majority of BIG5 users are), it is already difficult for them to appreciate the usefulness of any 7-bit scheme, let alone one that denies the legitimacy of their favorite 8-bit choice, namely BIG5. Therefore, I think a "direct" 7-bit BIG5-mapping is mandatory at this stage of history. Personally I find CNS is a cleaner design, and perhaps many BIG5 users will agree if they have time or care to compare CNS and BIG5. But in reality, even switching to CNS is desirable, it cannot be done overnight. Nor this process can be solved by EHZ, ISO or other 7-bit proposals. However, this process may be facilitated by the existence and acceptance of EHZ, because EHZ-CNS and EHZ-BIG5 provide a NON-mutually exclusive environment to experiment with both coding schemes even in the same newsgroup article. >best way, but there's an secondary best way of mapping all >vendor-dependent/user-defined characters to some private >area (like "Plane 0" of EHZ-BIG5) while common characters >still being mapped to CNS. This is what I wrote as >"HZ-EA-2" in my previous mail. Ideally, I would like to map all common and non-common code spaces in BIG5 to only 2 planes rather than 3 planes for obvious efficiency and complexity reasons. However, since there are too many to be mapped, we are forced to use 3. Since the code space 8140-A0FE is user defined in ETen, KC, Chiense Windows 3.1, MacOS BIG5 versions, I would bet their chances of getting used is close to zero. Therefore, they are ideally suited for Plane 0. The same cannot be said for other non-common parts of BIG5. Therefore, I still prefer EHZ-BIG5's mapping. >Here's the comparison of mapping method among HZ+S, >EHZ-BIG5, and HZ-EA-2. >---------------------------------------------------------------------- >Big5-code area HZ+S EHZ-BIG5 HZ-EA-2 >U: 8140-A0FE Big5-P0:[2121-] Big5-P0:[2121-] Private:[4021-] >C1:A140-A3BF Big5-P1:[2121- Big5-P1:[2121-] CNS-P1: [2121-] >V1:A3C0-A3FE Big5-P1: - Big5-P1:[2940-] Private:[2121-] >C2:A440-C67E Big5-P1: - Big5-P1:[4421-] CNS-P1: [4421-] >V2:C6A1-C8FE Big5-P1: -636C] Big5-P1:[2A21-] Private:[2221-] >C3:C940-F9D5 Big5-P2:[2121- Big5-P2:[2121- CNS-P2: [2121-] >V3:F9D6-FEFE Big5-P2: -7B32] Big5-P2: -7B32] Private:[2721-] >---------------------------------------------------------------------- >[Pn means Plane n] >[HZ-EA-2 will map C94A and DDFC to 7E7D and 7E7E or Private] >With HZ-EA-2, vendor-dependent/user-defined characters are >not assured to be exchanged between different vendors/users, >but the other methods also can't assure it. The merit of >HZ-EA-2 over HZ+S and EHZ-BIG5 is that it assures >communication between CNS users and Big5 users (common part >only). [However, HZ-EA-2 may appear stepping-back to >those who approved HZ-EA.] Diagree. As long as some uncommon code points are mapped to 7-bit, you can never be sure they WON'T be used. This is also true for GB<->HZ mapping. I think the 7-bit scheme should provide the flexibility that all code spaces can be mapped, in case all have to be used in some unforseen (who can say that they can forsee everything?) cases. At the same time, we need CONVENTIONs such as using common parts, maximum line lengths (80 vs. 132 chars per line), etc. Obviously, you don't want to turn every convention into part of a standard. >As for designation sequences, I think something which >follows "EHZ framework" is the best. Agree. >I do want to know how my proposals are appearing to many >people. Private mails to me are also very >welcome [just saying "Nonsence!" is ok. :-)]. > >--- >Ken'ichi HANDA >handa@etl.go.jp Fung Fung Lee From handa@etlken.etl.go.jp Thu Jun 30 06:19:27 1994 Received: from etlpost (etlpost.etl.go.jp) by ifcss.org (4.1/IFCSS-Mailer) id AA20127; Thu, 30 Jun 94 06:19:27 CDT Received: from etlpom.etl.go.jp by etlpost (5.67+1.6W/2.7W) id AA02666; Thu, 30 Jun 94 20:17:06 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA03578; Thu, 30 Jun 94 20:17:05 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA07123; Thu, 30 Jun 94 20:17:11 JST Date: Thu, 30 Jun 94 20:17:11 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9406301117.AA07123@etlken.etl.go.jp> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: Fung Fung Lee's message of Wed, 29 Jun 1994 09:58:37 -0700 <9406291658.AA06809@simd.stanford.edu> Subject: RE: A compromise proposal On Wed, 29 Jun 1994 09:58:37 -0700, lee@simd.stanford.edu (Fung Fung Lee) said: > 8-bit choice, namely BIG5. Therefore, I think a "direct" 7-bit > BIG5-mapping is mandatory at this stage of history. Personally The concept of "direct 7-bit BIG5-mapping" is not clear to me. Do you mean that HZ-EA-[-2] doesn't allow Big5 to be mapped to Big5 specific code area? Then, why is it important to have different code area for Big5 and CNS? Or, do you mean that change of ordering of some characters while mapping Big5 to CNS Planes is not accepted? Then, I don't know why it can't be accepted. Even in EHZ-BIG5, the ordering of common part and vendor part is changed. My plan just adds more re-ordering. > I find CNS is a cleaner design, and perhaps many BIG5 users will agree > if they have time or care to compare CNS and BIG5. But in reality, > even switching to CNS is desirable, it cannot be done overnight. > Nor this process can be solved by EHZ, ISO or other 7-bit proposals. At the user level, they don't have to shift from Big5 to CNS even if HZ-EA[-2] is accepted. > However, this process may be facilitated by the existence and acceptance > of EHZ, because EHZ-CNS and EHZ-BIG5 provide a NON-mutually exclusive > environment to experiment with both coding schemes even in the same > newsgroup article. If both EHZ-CNS and EHZ-BIG5 are used, the process won't be facilitated, because there seems to be no reason for Big5 users to use EHZ-CNS instead of EHZ-BIG5. > we are forced to use 3. Since the code space 8140-A0FE is user defined > in ETen, KC, Chiense Windows 3.1, MacOS BIG5 versions, I would bet > their chances of getting used is close to zero. Therefore, they are > ideally suited for Plane 0. The same cannot be said for other non-common > parts of BIG5. Therefore, I still prefer EHZ-BIG5's mapping. I don't know the importance of difference of vendor parts being mapped to Plane1&2 or to Plane0. >>With HZ-EA-2, vendor-dependent/user-defined characters are >>not assured to be exchanged between different vendors/users, >>but the other methods also can't assure it. The merit of >>HZ-EA-2 over HZ+S and EHZ-BIG5 is that it assures >>communication between CNS users and Big5 users (common part >>only). [However, HZ-EA-2 may appear stepping-back to >>those who approved HZ-EA.] > Diagree. As long as some uncommon code points are mapped to 7-bit, > you can never be sure they WON'T be used. Ah! My English was poor. I ment in the above paragraph that: < I think the 7-bit scheme should provide the flexibility > that all code spaces can be mapped, in case all have to be used in > some unforseen (who can say that they can forsee everything?) cases. HZ-EA[-2] do map all Big5 code spaces. > At the same time, we need CONVENTIONs such as using common parts, > maximum line lengths (80 vs. 132 chars per line), etc. "Using common parts" is more promoted by HZ-EA-2 than by EHZ-BIG5 because HZ-EA-2 maps all non-common parts (vender/user area) to some Private code area. But, please note that such a promotion is not the reason of proposing HZ-EA[-2]. > Obviously, > you don't want to turn every convention into part of a standard. Yes. That's why Private code area exists in HZ-EA[-2]. --- Ken'ichi HANDA handa@etl.go.jp From handa@etlken.etl.go.jp Thu Jun 30 07:06:14 1994 Received: from etlpost (etlpost.etl.go.jp) by ifcss.org (4.1/IFCSS-Mailer) id AA21070; Thu, 30 Jun 94 07:06:14 CDT Received: from etlpom.etl.go.jp by etlpost (5.67+1.6W/2.7W) id AA02946; Thu, 30 Jun 94 21:03:26 JST Received: by etlpom.etl.go.jp (4.1/6.4J.6-ETLpom.MASTER) id AA03965; Thu, 30 Jun 94 21:03:25 JST Received: by etlken.etl.go.jp (4.1/6.4J.6-ETL.SLAVE) id AA07147; Thu, 30 Jun 94 21:03:31 JST Date: Thu, 30 Jun 94 21:03:31 JST From: handa@etlken.etl.go.jp (Kenichi Handa) Return-Path: Message-Id: <9406301203.AA07147@etlken.etl.go.jp> To: ccnet-l@uga.cc.uga.edu, soft-authors@ifcss.org In-Reply-To: John Delacour's message of Wed, 29 Jun 1994 19:49:56 +0100 <9406292015.AA05103@sutgate.bitnetjp.ad.jp> Subject: Re: A compromise proposal On Wed, 29 Jun 1994 19:49:56 +0100, John Delacour said: > Ken'ichi Handa writes:- >>I think that the most difficult point of HZ-EA to be >>accepted is that it forces even vendor-dependent characters >>to be mapped to CNS/GB/JIS as far as possible. > Is this really such a big problem? I think this is surely a heavy load for soft-writers because vendors will add characters any time and softs which follow the doctrine of "mapping all characters to CNS/GB/JIS as far as possible" must be updated continuously. > I doubt it myself and besides I don't > think it should be allowed to retard the adoption of the best method of > encoding Big-5 for information interchange. I have seen no statistics > regarding the "vendor-specific" characters, such as:- how many of them are > there in ETEN or in other vendor's products? If it's a question of a few > hundred characters, then it can't be a lot of work. If we fix the mapping process at this moment and don't try further mapping in the future, we may be able to achieve the work. But, that means giving up of the doctrine of HZ-EA. > Besides, how important are the vendor-specific characters in the present > context? As far as I heard on this mailing list, it seems that many people think it's very important. I can't judge it because I'm not Big5 users. > Why should "second best" be considered if the best is shown by solid > arguments and logical reasoning to be the best? It would be a shame if > second best were accepted simply to keep everyone happy and so that nobody > loses face. It is such considerations that have bedevilled this debate > throughout its all too long duration. Personal feelings and a sense of > ownership should not enter the discussion; for the benefit of the end users > it is important that a spirit of cooperation should exist between the > experts and a willingness to relinquish untenable positions. My intention of proposing HZ-EA-2 is not because "keeping everyone happy ...". I proposed it after re-considering the works to be done for supporting full spec of HZ-EA. At first, creating a MAP as in my "EZ-EA encoder/decoder" seems not that difficult. But to keep updating such MAP in the future continuously seems very hard. It's not only a heavy load for soft-writers but also for a heavy load for users because they also have to keep updating their softs, even if `updating' means `just copying' the latest version from somewhere. > If you are talking of escape sequences, I think it would be best to > distinguish between sequences that designate two-byte characters and those > that designate single-byte characters (v.iso-registry.txt) Hmm, that's an idea. But, for the moment, I'm not considering any designation of single-byte characters except for "~}" of ASCII designation. > Character set types > ( ): 94-char set w/ 3-char sequence > (!): 94-char set w/ 4-char sequence > (-): 96-char set > ($): multiple-byte set > (%): non-2022 system with standard return > (/): non-2022 system without standard return What do you mean by the above table? > Without any > solid proof or working code to back up your ideas, it is hard to say for > certain whether your ideas are the best, but I think it's quite possible > that they are. The working code is in my mail entitled "HZ-EA encoder/decoder". It also contains HZ-EA-2 handler (and also HZ+S and EHZ-BIG5 handlers). > Are you going to be able to streamline the code to deal swiftly with large > files? HZ <-> GB is obviously as quick as lightning because of the > simplicity of the conversion but is speed going to be a big problem with > HZ-EA? My HZ-EA encoder/decoder is not as quick as lightning, but perhaps as quick as thunder. :-) Here's some implementation note about my program: ------------------------------------------------------------ IMPLEMENTATION NOTE: `b2ea' uses MAP file to find the mapping. MAP file is just an array of showing start and end of continuous code range to be mapped. When b2ea get a source code to be converted, it search for the table by `binary search' to find the range in which the code belongs. In the case of Big5-ETEN (of HZ-EA), the table length is 127, and hence, it requires, at worst case, 7 comparisons of 16bit integer against 16bit integer. In the case of HZ+S, it requires only 1 comparison. If we add some heuristics to this search process, it can be achieved more rapidly (actually this is done in Dr.Paterson's `b5cns'). In addition, when mapping many codes at once, making some statistics of codes will also help making the process more rapid. ------------------------------------------------------------ > Another question I've been meaning to ask for weeks is this:- > It's all very well having all these nice escape sequences but surely, if we > mix GB and JIS and Latin-1 and ASCII and Big-5 in one piece of text, it > will be necessary to provide signals in the original text to show what code > is being used... Do you mean something like MIME's Charset-contents field? --- Ken'ichi HANDA handa@etl.go.jp From pdcruze@orac.iinet.com.au Thu Jun 30 09:40:19 1994 Return-Path: Received: from uniwa.uwa.edu.au by ifcss.org (4.1/IFCSS-Mailer) id AA25712; Thu, 30 Jun 94 09:40:19 CDT Errors-To: postmaster@iinet.com.au Received: from iinet.com.au (classic.iinet.com.au [203.0.178.1]) by uniwa.uwa.edu.au (8.6.8/8.6.4) with SMTP id WAA27552 for ; Thu, 30 Jun 1994 22:38:02 +0800 Received: from orac by iinet.com.au with smtp (Smail3.1.28.1 #7) id m0qJNFd-0003hQC; Thu, 30 Jun 94 22:38 WST Message-Id: To: soft-authors@ifcss.org Subject: Looking for Chinese translators and volunteers X-Mailer: exmh version 1.3 4/7/94 Date: Thu, 30 Jun 1994 22:44:10 +0800 From: "Patrick D'Cruze" Errors-To: postmaster@iinet.com.au Hello, I am one of a number of volunteers working on developing a freely redistributable operating system for computers called Linux. Linux has progressed well over the years and has been very well received especially amongst English speaking countries. However, support within Linux for other languages is very poor. And this is also the case for support for Chinese languages. A small group has been formed to remedy this situation and to improve Linux's native Chinese support but unfortunately we are very short of volunteers. Hence we are seeking people who would like to assist us in our endeavours. We are trying to achieve two things: Firstly, modify software to be able to handle 16-bit Chinese characters and to be able to display them properly. And secondly, to translate documents and instructions into Chinese language versions. If anybody would like to assist in the translation of documents or who would like to assist in modifying software, we would be eager to hear from you. Please contact me directly if you would like to assist in anyway. Linux is a freely available operating system and it has grown due to the support of many volunteers contributing to it. With your help, we would like to make Linux as comfortable for Chinese people as it presently is for English-speaking people and share with all Chinese the benefits that Linux has to offer. Thank you for your time. Regards, Patrick D'Cruze Linux International pdcruze@orac.iinet.com.au From lee@simd.stanford.edu Thu Jun 30 22:15:37 1994 Return-Path: Received: from simd.stanford.edu by ifcss.org (4.1/IFCSS-Mailer) id AA21834; Thu, 30 Jun 94 22:15:37 CDT Received: by simd.stanford.edu (5.65/inc-1.0) id AA15863; Thu, 30 Jun 1994 20:12:54 -0700 Date: Thu, 30 Jun 1994 20:12:54 -0700 From: lee@simd.stanford.edu (Fung Fung Lee) Message-Id: <9407010312.AA15863@simd.stanford.edu> To: handa@etlken.etl.go.jp Cc: soft-authors@ifcss.org In-Reply-To: Kenichi Handa's message of Thu, 30 Jun 94 20:17:11 JST <9406301117.AA07123@etlken.etl.go.jp> Subject: RE: A compromise proposal >Date: Thu, 30 Jun 94 20:17:11 JST >From: handa@etlken.etl.go.jp (Kenichi Handa) > >The concept of "direct 7-bit BIG5-mapping" is not clear to >me. Do you mean that HZ-EA-[-2] doesn't allow Big5 to be >mapped to Big5 specific code area? Then, why is it >important to have different code area for Big5 and CNS? Direct 7-bit BIG5 mapping refers to something like EHZ-BIG5, HZ+, HZ-2, BETTY and almost every scheme proposed so far, including HZ-EA-2, but NOT HZ-EA (or in Ricky's term, a convention that proposes to use a restricted subset of EHZ). Even if everything in EHZ is accepted, there is nothing to prevent a user from choosing to follow Mr. Handa's suggestion to convert his/her BIG5 text files to mixed CNS/GB/JIS EHZ files. Put it the other way, EHZ-BIG5 simply allows him/her an alternative choice, which is more straightforward, and is much much more likely to be supported by more robust software because of the inherent simplicity of BIG5<->EHZ-BIG5 translation vs. the first alternative. I am glad to see that Mr. Handa is spending effort to write some conversion software based on the first approach. This is useful just for the sake of 8-bit code translation. >If both EHZ-CNS and EHZ-BIG5 are used, the process won't be >facilitated, because there seems to be no reason for Big5 >users to use EHZ-CNS instead of EHZ-BIG5. You are probably right. If you are right, why not just support EHZ-BIG5 and NO EHZ-CNS ? However, I'm not so sure you're right. Why not let the USERS decide what they want to use, EHZ-BIG5 or EHZ-CNS? Are you saying that we should force upon the users one and only one alternative? One of the beauty of EHZ framework is that if offers a lot of escape sequences without losing more ASCII transparency. There are rooms for experiments under this framework. (This is not true in any other previous proposals such as HZ+, HZ-2, etc.) If the BIG5/CNS is a difficult issue to be resolved among soft-authors, maybe it is just as if not less difficult among the larger chinese Internet community. Then why just to restrict to one when we are uncertain. If one of EHZ-BIG5 or EHZ-CNS turns out to be useless, it will wither naturally just like some ancient languages have withered in the past. Are you worried that if we give EHZ-BIG5 a chance, your restricted EHZ proposal will fail in the real world too quickly? >I don't know the importance of difference of vendor parts >being mapped to Plane1&2 or to Plane0. That's why I mentioned it. >HZ-EA[-2] do map all Big5 code spaces. Yes, I know. >"Using common parts" is more promoted by HZ-EA-2 than by >EHZ-BIG5 because HZ-EA-2 maps all non-common parts >(vender/user area) to some Private code area. But, please >note that such a promotion is not the reason of proposing >HZ-EA[-2]. Then what is the real reason? Fung Fung Lee