Subject: About conversion servers

From: [email-masked]
Date: Tue, 16 Oct 90 14:19:58 MDT

I posted a message about a Chinese conversion server that will be
released with X11R5.  There were some questions about conversion
servers and X11.  Here is more detailed information about conversion
servers and so on.

The Wnn and cWnn conversion servers are Unix based programs that run
on one machine and wait for other programs to make connections with
it, across a network or from the same machine.

There are two major problems that arise when dealing computationally
with ideographic writing sytems: (1) word and phrase boundries are
ambiguous because there aren't any spaces to tell when one word/phrase
starts or ends, and (2) homonyms, words/phrases that sound similar,
but use different characters.

One solution is to gather statistical data on the frequency that a
phrase/word occurs in a certain position in a sentence.  This, along
with other disambiguating methods, is the way the Wnn conversion
server works.

In Japanese, the process of generating text is often done three steps:
(1) type in a Roman representation(Romaji) (2) convert that to the
intermediate phonetic form(Kana) and (3) convert the Kana into Kanji.
It's the third stage that presents difficulties computationally.  This
is where the conversion server comes in.  Going from step 1 to step 2
is easy because there is a clear, unambiguous mapping between Romaji
and Kana.

Given a program, somewhere on the network, that has some Kana to
convert to Kanji, the program makes a connection to the conversion
server, and passes the Kana.  The conversion server then does some
frequency checking, some syntactic processing to try and determine
things like word/phrase boundries, maybe a little sematic processing,
and then it builds a list consisting of a phrase or word with it's
homonyms, and sends it back to the program.  It is then up to the
program to work with what was returned from the conversion server.

In Chinese, the problem is similar, but a little more difficult.
There is a phonetic representation(BoPoMoFo/ZhuYinFuHao), but it
doesn't contain(as far as I know), computationaly, similar amounts of
information that is available in Kana. There exist many methods to do
the Chinese conversion(hundreds I believe).  One of the most prevalent
is Pinyin input; a Roman phonetic representation.  The cWnn conversion
server will peform the same tasks as the Wnn server does, probably
taking Pinyin as input(cWnn might be capable of other kinds of
conversions, but I haven't seen it yet, so I don't know).

(Abrupt subject change)

What is X11R5?  X11 is a freely available windowing system developed
for personal computer(recent), workstation, mini, and mainframe
systems.  X11 is a great vehicle for developing programs to deal with
text processing of non-Latin based writing systems.  It runs on many
different kinds of machines and operating systems.

You will notice that I have avoided talking about anything really
substantial, like text processors and fonts for Chinese and Japanese.

I didn't want to write about those because people are probably tired of
reading what I have written so far, and it's another "can of worms"
entirely.

Hope this helps.