.c;SPELL .TITLE;Spelling Checker .AP .HL1;Summary SPELL is a system of checking for spelling errors in text files on the RSX-11M operating system. The program SPELL does the checking and optionally may make changes, while DICTMK formats a file for SPELL to use. It is written in FORTRAN-77 except for certain file access routines that are written in Macro. The program has been optimized as much as possible for speed. In non-interactive mode a file will be produced which contains all the words from the document that SPELL could not find in its dictionary. In interactive mode words can be replaced, added to the dictionary (DICTMK will need to be run again), or simply ignored. For an in-depth see the source code as it contains much comments. Automated extractions of comments (C+/C-) will tell the workings of all the routines. .hl1;Background The programs were written in FORTRAN-77 and were developed on a PDP-11/t55 with 6-RK05, running RSX-11M V3.2. The programs came from a desire on my part to have a system that could be used to check my submissions for the DECUS North Texas Local Users Group Newsletter. The only spelling checker I could find was the one included in the Software Tools section of the Spring 81 RSXSIG tape. As I didn't want to build software tools, I used the data base as a starting point for program development. As sequential reads were too slow for the upwards of 42,000 words in the data base, I used Marge Knox's (now FORTRAN-77). Random access - block i/o was decided on to minimize the time for disk i/o. Using Ralph Stammerjohn's Index to the past RSXSIG tapes, I was able to come up with Phil Cannon's routines for FCS file access to the FDB. This enabled me to build the dictionary (using DICTMK) using random access block i/o, set the size of the file, and to truncate to the last block used on exit. The prototype of this program would read the document to be checked and put each different word in a list to be checked by the dictionary and those misspelled written out to a file. This worked somewhat allright for small files, but the larger the file the more time that was spent checking against the list for each word to insure the word was not already included. Even small files yielded 500 different words to be checked. This was before the index for each letter was included on the average of 30 reads for each word. .hl1;Use To build SPELL copy DICT.SEQ to uic [3,3] or any UIC you want if changes are made to the source code. @DICTMK will produce the DICTMK.TSK and @SPELL will produce the SPELL.TSK. Run DICTMK to produce DICT.RAF (the random access file). Install SPELL or set up CCL to do it and pass command lines. To check a document for spelling, >SPE filename[/sw] will do it. If you are not using a VT52 type terminal, please make changes to SUBROUTINE DISPLA for the right escape sequences for clear screen. .hl1;DICTMK DICTMK is the dictionary maker for the spelling system. It is used to provide a random access block i/o dictionary with internal indexes for all the letters. Access to the File Descriptor Block is possible by the use of Phil Cannon's routines that use an internal FDB in a byte array. Thus truncate on close is possible using block i/o. DICTMK will also provide statistics on the words in the dictionary after it has built the dictionary. .hl2;Dictionary The dictionary that goes along with this program comes from the software tools section of the Spring 81 RSXSIG tape. It was DICT.;1. It contains more than 42,000 words. The shortest word is 'a' and the longest is 22 letters. From looking through it, there are some proper names and places. This may or may not be advantageous 3% of the total space, so deleting them may not buy you much space. .hl2;Adding words When spell runs and produces a list of words that you want to add to the dictionary, run DICTMK from where SPELL ran (that's where the file of words is. DICTMK will take the file and merge it into DICT.SEQ then produce another DICT.RAF. .hl1;SPELL SPELL is the spelling checker program. It uses the dictionary file and its own internal list, and checks the text for words not found in either. Several switches are available for the command line, and several commands are available in the interactive mode. .ts20 .HL2;Switches When invoking SPELL several switches are available. To invoke SPELL: .literal >SPE filename[switches] or >SPE SPE>filename[switches] .end literal The switches for the command line are as follows: .literal /Q process in quick mode (non interactive) /F file is FORTRAN source, process comments only /T terse comments, on interactive, display line and word only /N numbers are valid symbols in tokens /L[:filename] preload the add list with words that are correct if filename exists use this list, if not use 'SPELL.DCT' /S:x ignore words with 'x' or less letters .end literal .hl2;Modes There are two modes of operation for the spelling checker. Depending on the availability mode can be chosen. The interactive is the non-quick mode, so named since it takes longer to process a document in this mode. .hl3;Quick The quick mode provides for no interactive processing. It will scan the document and make a list of words that it didn't find in its dictionary or in its internal list. This is the mode used for long documents and on line checking. .hl3;Non-quick The non-quick mode (default) is interactive with the terminal and uses screen clearing. This screen clearing can be adjusted to be compatible with the terminal type. For adjustments see SUBROUTINE DISPLA. In this mode the document is scanned for number of lines, to provide the operator with a scale of how far he has to go. If the verbose command has been used (default), the full display line and the given word is displayed. .hl4;Display The terminal display in the verbose mode will display the spelling header line which holds the title and version number. Next is a short display of the commands available. Then a short list of the statistics of the spelling checker so far this session. Thus the operator can see how many lines have been processed, how many words, and the number of words added to each of the lists. Next is the line containing the word in question and then the word itself with a prompt. .hl4;Commands The commands available .literal # (where # is a number between 1-9) ignore words # or less in length A this word is correct, add to dictionary C this word is correct, do not add to dictionary E exchange words, but do not add to list L replace current line N next word, do not add to list T terse display V verbose display W wrong spelling, make correction to dictionary X exit, as if end of document was here .end literal .hl2;Adjustments Several adjustments can be made to this program by the interested user. Words can be added to the internal list (at the sacrifice of space). Overlays are possible. Adjustments to the display section are possible for other than VT52 type terminals. The location of the dictionary can modified to increase the speed of reads. .hl3;Overlay Two overlay descriptions are given if you need them. SPEED.ODL has been set up to provide the fastest running of the overlay. The most used portions are all in memory so no overlay reads are needed. SPACE.ODL has been set up to provide the smallest version of SPELL. Only those sections needed are in memory at one time. It will probably be very slow since many overlay reads are needed for each and every word checked. These overlays are not tested so don't blame me if they don't work. SPEED.ODL will not run faster than the non-overlayed SPELL. SPELL will run faster if adjustments are made to FNDNOR which fills the normally used words list. If more words are included internally a increase in speed will be likely but this takes space and this task is at the limit now. .hl3;Add words There is a little space left in the program to allow for the expansion of the internal list it checks first. The current words in this list were found in the "Book of lists" which references "American Heritage Word Frequency Book" of 1971. There are currently 12 words in this list in the order in which they occur in their usage. If it can be found a longer and more complete list is in "Computational Analysis of Present-Day American English" by H. Kucera, Brown University Press, 1967. From this it can be seen that with a list of 16 words 28 per cent of the words in a document can be found. If 64 words are used 43 percent can be found with a corresponding decrease English occur in a vocabulary of 134 words. .hl3;Display Adjustments can be made to SUBROUTINE DISPLA to provide the escape sequences for ANSI VT100 and others. Even fancier terminal i/o can be done. .hl3;Disk A great increase in speed can be made if the dictionary is kept on a solid state disk.