! wordfreq.tes - program to count word frequencies ! @I% wordfreq.tec - Count word frequencies File to use: % ! insert prompt ! W ! refresh display ! :@EI%erfile%"F F> ' ! iif enter filespec and open for read fails exit ! @I% Exact case match (y/n)? % ! prompt ! W ! refresh display ! :@EI%yesno% UX ! get response into X.num ! @I% % ! insert ! W ! refresh display ! 2ED ! next reset the symbol constituent buffer (for C++) change this if you like ! @^U$%ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_% -1^X ! set exact case match ! 1B ! select buffer 1 ! HK ! kill it ! 5W ! set window height for buffer 1 to 5 lines ! @I%*** List of word usage *** % ! insert to buffer 1 title line ! .U1 ! remember starting position for word search ! 0B ! go back to buffer 0 ! @^A%Building list of words and frequency of use... % < ! loop (buffer by buffer) ! :@_~^EC~; ! Yank Search to some symbol constituent ! QX"E H^V ' ! iif NOT exact case, lowcase entire buffer ! ZJ ! jump to end of buffer ! @I% % ! make shure buffer ends with non-symbol ! J ! back to start of buffer ! < ! loop to process each 100 "words" in buffer ! W ! refresh the display (every 100 words) ! 100< ! loop to process each "word" in buffer ! :@S~^EM^EC~; ! find symbol constituent(s), exit no more ! @^US%:S% ! S.str = :S ! 10:@^US%% ! :S ! ^Y:XS ! :Ssymbol constituent(s) ! :@^US% % ! :Ssymbol constituent(s) ! 27:@^US%% ! :Ssymbol constituent(s) ! 1B ! select buffer 1 (word list buffer) ! Q1J ! jump to starting positon for search ! MS"S ! if word in list ! %\$ ! incriment count of usage ! 0B ! select buffer 0 ! F< ! continue ! ' ! endif word in list ! ZJ ! jump to end of word list buffer ! GS ! get search command ! ^SC ! backup infront of command ! 3D ! delete the :S ! ZJ ! end of buffer ! -1D ! delete the ! @I%1 % ! insert "1" ! 0B ! back to buffer 0 ! > ! end 100 word loop ! -.; ! if .=0, search failed (no more words) exit! > ! end buffer by buffer loop ! HK ! kill buffer ! > ! end buffer loop ! @^A% Sorting entries by frequency of use... % 1B ! select list buffer ! Q1J ! position at 1st word ! L ! then to 2nd word ! < ! loop for 2nd to nth words ! W ! refresh display ! :@S% %; ! search word delimiter ("wordcount") ! .US ! remember ^ ! XS ! save into S.str count ! 0L ! to front of line with word ! .UP ! remember start of line ! -:@S~ ^EQS~"S ! if backwards search for same count succeeds ! QP-."E ! if remembered line already follows ! L ! advance a line ! F< ! continue ! ' ! end if line follows ! .UI ! remember the insertion pointer ! QPJ ! jump to start of line to move ! XS ! copy line to S.str ! K ! kill the line ! QIJ ! jump back to the insertion pointer ! GS ! insert the line ! -^S+QPJ ! jump to next line to process (QPJ L) ! F< ! continue ! ' ! endif backwards search for same count succeeds ! QSJ ! position in front of count ! \UC ! put count into C.num ! QPJ ! go to front of line ! XA ! copy line to A.str ! Q1J ! jump to after title line ! < ! loop ! :@S% %; ! find next space (count prefix) ! \-QC"G F< ' ! iif this count .gt. C.num continue ! 0L ! go to start of line ! .-QP"E ! if same line as one to move ! L ! advance a line ! 0; ! exit loop ! ' ! endif same line as one to move ! GA ! insert line to move ! QP-^SJ ! jump to line we moved (not deleted yet) ! K ! delete it ! 0; ! exit find insertion loop ! > ! end find insertion loop ! > ! end word loop ! @^A%Calculating statistics... % 20W ! open the window on buffer 1 to 20 lines ! 0UW ! zero count of unique words ! 0UT ! zero count of total words ! 0UL ! zero count of characters (sum ofword lengths) ! Q1J ! jump to search position ! < ! loop for each word ! :@S% %; ! find else exit loop ! %W$ ! incriment count of unique words ! \UC ! C.num = # of this unique word ! QC%T$ ! add # of this unique word to total words ! 0^Q-1+^S*QC%L$ ! 0^Q = number of bytes to start of line ! ! -1 = remove 1 byte for ! ! +^S = add -length of spann of count (from \)! ! result is lenght of this word ! ! *QC times count of this word ! ! %L add to length of all words ! ! $ throw out result of %L ! > ! end loop for each word ! Q1J ! to 2nd line of buffer ! @I%Total # words % QT\ ! insert T.num ! @I%, # unique words % QW\ ! insert W.num ! @I%, average word length % QL*100/QTUL ! L.num = ((total # bytes)*100)/(total # words) ! QL/100UC ! C.num = whole # bytes/word (truncated) ! QC\ ! insert whole number of bytes/word ! @I%.% ! insert "." ! QL-(QC*100)\ ! calculate and insert fractional # bytes/word ! @I% % ! insert a ! @^A%Finished. %