Subject: [SWIPL] parsing newline bug??

From: "Mike Maxwell" <[email-masked]>
Date: Wed, 14 Feb 2001 15:08:39 -0500

I've run into what appears to be a rather curious bug in parsing newlines.  

My editor (Visual Slickedit--nice editor!) allows me to save a file in either DOS format (with CR-LF for newlines) or Unix format (just LFs).  I have a file which the parser I wrote in Swi-Prolog parses OK if the file was saved in Unix format, but _not_ if it was saved in DOS format.

What appears to be happening when it parses the DOS format file is this: the parser chugs along until it gets 4096 bytes (Hex 1000) into the file, then it suddenly drops back to a position = 4096 - L, where L is the number of newlines it has read up to that point.

By putting a call to
    ...
    stream_property(Stream, position('$stream_position'(P, _, _))),
    write(P), 
    write(' ')
    ...
in the parsing code, I found that the parser thinks its byte position is L less than where it actually is, until it has read in 4096 bytes.  That is, the last byte position reported by the above code before the parser goes back L characters is not 4096, but position 3844, which happens to be on the 252nd line.  And 3844+252 = 4096.

As far as I can tell, the parser never reads a CR (ASCII 13), only LFs (ASCII 10).  That is, the [Ch] construct in a parser rule never returns any CRs.  Now since I have redefined 'C' as

    :- redefine_system_predicate('C'(_,_,_)).
         'C'(Stream/Offset, Char, Stream/Next) :-
         succ(Offset, Next),
         seek(Stream, Offset, bof, _),
         get0(Stream, Char).

--what that really means is that get0/3 never returns a CR; somehow when it reads an LF, it also reads the CR without returning it (and hence without advancng the counter).  get_byte has the same behavior.

This seems like a bug, at least it sure messes up my code!

                                         Mike Maxwell
                                         Summer Institute of Linguistics
                                         [email-masked]