Subject: Pinyin: data structure and algorithm

From: Ng Liang Shing <[email-masked]>
Date: Thu, 26 Aug 1993 15:31:45 MYT

The Pinyin Input Method: Data structure and algorithm

Pinyin Input Method is by far the most popular, though not the most efficient,
Chinese character input method. Hence, it deserves a closer look by all
Chinese programmers.

Its structure is in sharp contrast to Pinxing input methods (pin-compose,
xing-shape; e.g. Wubizixing and Cangjie). Pinxing is both more efficient both
in use and in algorithm. I will discuss Pinyin here and Pinxing later.

An input procedure should take keyboard input and give Chinese characters
coded in one of Big5, Guobiao or Hz.

The procedure can be further broken into the following steps:
1. read key
2. parsing: convert key values into pinyin codes
3. lookup: find the characters matching the pinyin codes.
4. ask & confirm: let the user choose one among all homophones (characters
with the same pronunciation)
5. echo & store: display the chosen characters to the screen and store the
characters into a variable.

The crucial part is the Pinyin Dictionary. This is a lookup table of the
characters. I will describe its implementation in detail. For specific example
i.e. where to find a reusable pinyin dictionary files, please email to me.

A pinyin dictionary should have two essential parts:  the index and the list
of characters.

Each entry in the index is a pair of two numbers - the pinyin code and the
location of the homophones.  A typical implemetation would be :
pinyin code: one byte for consonant, one byte for vowel plus tone, two bytes
(to form one integer) for the location of the characters in the list.

The list of characters lists all Chinese characters in the order of the pinyin
codes. It is a continuous string of characters grouped together according to
the pinyin codes. Each group of homophones is pointed to by an entry in the
index.

As an example, consonants {b, c, ch, d, f,...} has the pinyin codes {86, 87,
88, 89, 8A, ....} (in hexadecimal notation). vowels { a1, a2, a3, a4, a5, ai1,
ai2, ai3, ai4 ai5,.....} are { 1, 2, 3, 4, 5, 6, 7, 8, 9, A,.....}.

An entry in the index could be
86 01 AA 01,
86=b, 01=a1, hence all characters can pronounced as ba1 can be
found at location number 01AA in the list of characters. (note: here I use the
familiar format of storing integer - low byte first, high byte next - to store
the number 01AA in the format AA 01)

The next entry in the index would be
86 04 B1 01.
This is because homophones ba2, ba3 does not exist. 86 04 is ba4,
the location is at 01B1. Note you can determine the number of homophones ba1
by taking the difference of 01B1 - 01AA.

Liangsheng WU
26 Aug 1993
--
Ng Liang Shing            |   This is an Internet node host inside
Sungai Petani, Malaysia   |   an old 286.  A few years back, this place
[email-masked]          |   was not more than a small town.