From mike_maxwell@sil.org Wed Feb 14 21:08:42 2001
Received: from mail.link77.net ([208.145.81.125])
	by swi.psy.uva.nl (8.11.2/8.11.2) with ESMTP id f1EK8fZ09797
	for <prolog@swi.psy.uva.nl>; Wed, 14 Feb 2001 21:08:42 +0100 (MET)
Received: from [208.145.81.102] (HELO CCSMaxwell)
  by mail.link77.net (CommuniGate Pro SMTP 3.3.1)
  with ESMTP id 1158988 for prolog@swi.psy.uva.nl; Wed, 14 Feb 2001 15:08:43 -0500
Message-ID: <01f401c096c1$ec3f9ba0$665191d0@CCSMaxwell>
From: "Mike Maxwell" <mike_maxwell@sil.org>
To: <prolog@swi.psy.uva.nl>
Date: Wed, 14 Feb 2001 15:08:39 -0500
Organization: SIL
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.50.4133.2400
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by swi.psy.uva.nl id f1EK8fZ09797
Subject: [SWIPL] parsing newline bug??

I've run into what appears to be a rather curious bug in parsing newlines.  

My editor (Visual Slickedit--nice editor!) allows me to save a file in either DOS format (with CR-LF for newlines) or Unix format (just LFs).  I have a file which the parser I wrote in Swi-Prolog parses OK if the file was saved in Unix format, but _not_ if it was saved in DOS format.

What appears to be happening when it parses the DOS format file is this: the parser chugs along until it gets 4096 bytes (Hex 1000) into the file, then it suddenly drops back to a position = 4096 - L, where L is the number of newlines it has read up to that point.

By putting a call to
    ...
    stream_property(Stream, position('$stream_position'(P, _, _))),
    write(P), 
    write(' ')
    ...
in the parsing code, I found that the parser thinks its byte position is L less than where it actually is, until it has read in 4096 bytes.  That is, the last byte position reported by the above code before the parser goes back L characters is not 4096, but position 3844, which happens to be on the 252nd line.  And 3844+252 = 4096.

As far as I can tell, the parser never reads a CR (ASCII 13), only LFs (ASCII 10).  That is, the [Ch] construct in a parser rule never returns any CRs.  Now since I have redefined 'C' as

    :- redefine_system_predicate('C'(_,_,_)).
         'C'(Stream/Offset, Char, Stream/Next) :-
         succ(Offset, Next),
         seek(Stream, Offset, bof, _),
         get0(Stream, Char).

--what that really means is that get0/3 never returns a CR; somehow when it reads an LF, it also reads the CR without returning it (and hence without advancng the counter).  get_byte has the same behavior.

This seems like a bug, at least it sure messes up my code!

                                         Mike Maxwell
                                         Summer Institute of Linguistics
                                         Mike_Maxwell@sil.org

