Subject: RE: reading a text file ..

From: Daniel Gross <[email-masked]>
Date: Tue, 7 Mar 2000 21:03:25 -0800

Dear Richard,

I appreciate very much your response. I demonstrates to me how little I in 
fact know about the "spirit" of prolog. In fact i am asking myself whether 
it is a good idea to propose writing a prototype implementation of my phd 
research in prolog. Perhaps it is, but it will add to the challenge the 
proper use of prolog as a prototyping tool.

I was wondering if i could ask you the following. I dont have the book 
handy to check how to tokenize the input file so it could be used by the 
parser. Do you perhaps know of  some prolog library form where i could 
fetch myself that sample code ...

again appreciating very much your response

Daniel

-----Original Message-----
From:	Richard A. O'Keefe [SMTP:[email-masked]]
Sent:	Tuesday, March 07, 2000 2:55 PM
To:	[email-masked]; [email-masked]
Subject:	Re:  reading a text file ..

Daniel Gross wrote
that he wants to parse input that is made of a series of
fairly conventional tokens.

	In order to make my life easy (i thought) i read in line by line and
	translate each line into a prolog clause.

I cannot imagine why anyone would expect that to make life easy,
UNLESS end of line is itself a highly significant token, and not
terribly easy then, except perhaps for parsing old Dartmouth BASIC.

There are basically three sensible levels of reading for something
like this, regardless of what programming language you are using.

1.  Read one token at a time.
    This is how YACC-derived parsers work.

2.  If the grammar you are parsing is like
    <input> ::= <top level form>* <end of file>
    and each <top level form> ends with a token having no other use,
    then it is easy and natural and useful to read one <top level form>
    at a time, returning it as a list of tokens.
    This fits Prolog and Erlang quite well, where the token that
    ends a top level form is '. '.
    This is how the Prolog parser works.

3.  Read the _entire_ input as a list of tokens.
    This is how the Erlang parser works.  It's the obvious way to
    handle Pascal in Prolog.

You will find a Prolog tokeniser written in Prolog in the book
"The Craft of Prolog".  It should be easy to adapt to your problem.
An updated version of the tokeniser was in the Quintus library.

But the major point is WHY ON EARTH PUT THESE THINGS INTO CLAUSES?
If you want to parse something in Prolog, you want to use a DCG (although
you might possibly want a non-standard translation of DCGs).  And DCGs
don't want *clauses*, they want *lists*.

	I managed to do all the work just when i wish to translate Comma
	seperated (sic) lists of (sic) into lists.i found it quite
	complicated.  Is there a simple way to do that.  In general is
	there an easy way to parse such structures into prolog facts.

DON'T even THINK of trying to "parse such structures into Prolog facts".
What you want to do is parse them into TERMS, a very different kind of
animal indeed.

The DEC-10 Prolog parser was broadcast to the net many years ago.

It would be useful to know more precisely what you mean by
"Comma separated lists of".  I shall assume that you mean

<foo list> ::= <foo> [',' <foo list>]

Let's suppose that you have a nonterminal foo//1 that parses a <foo>
and returns its translation.  Then

foo_list([Foo|Foos]) -->
    foo(Foo),
    (   [','], foo_list(Foos)
    ;   {Foos = []}
    ).

It could hardly be simpler.
***IF*** your grammar is in fact LL(1), you may turn this into an
if-then-else:

foo_list([Foo|Foos]) -->
    foo(Foo),
    (   [','] -> foo_list(Foos)
    ;   {Foos = []}
    ).

but you would be surprised how often this turns out to be a bad idea.

Your top level will look like

    parse_file(File, AST) :-
	seeing(Old),
	see(File),
	read_all_tokens(Tokens),
	seen,
	see(Old),
	well_formed_input(AST, Tokens, []).
	
    well_formed_input(AST) -->
	...

Reading an entire file into memory sounds a bit scary, so here
are some numbers.  All of the *.pl files in the library/ directory
of SWI Prolog 2.9.10 come to

    198989  characters  (about 91 characters per clause, mean)
    177848  characters after stripping out comments and blank lines
      5634  lines
      2186  clauses	
     34646  tokens      (about 16 tokens per clause, mean)
        29  files

This is a rather higher number of tokens per line than you will be
dealing with.  But it's not an unreasonable number of tokens to hold
as a list.

Let's look at your examples.

	Token SerializedViewObject_0_1
	    IN SerializedObject
	    WITH
	        attribute, type
	             : "OME.GraphicView$GVERecord"
	        attribute
	            x : 323
	        attribute
	            y : 301
	        attribute
	            ID : 1
	END
	
It looks very much as though your input falls into the second category
above, with 'END' as a <top level form> terminator.  So let's assume that.

token_form(token(Name,In,With)) -->
	[id('Token'),id(Name)],
	in_part(In),
	with_part(With).

in_part(Names) --> [id('IN')], !, name_list(Names).
in_part([])    --> [].  % Include this iff an IN part is optional.

name_list([Name|Names]) -->
	[id(Name)],
	(   [','] -> name_list(Names)
	;   {Names = []}
	).

with_part(Atts) --> [id('WITH')], !, att_seq(Atts).
with_part([]) --> ['END'].  % include this iff a WITH part is optional.

att_seq([]) --> ['END'].
att_seq([Att|Atts]) --> one_att(Att), att_seq(Atts).

one_att(att(Names,OptName,Value)) -->
	name_list(Names),
	(   [id(X),':'] -> {OptName = id(X)}
	;   [':']       -> {OptName = no_id}
	),
	(   [string(S)] -> {Value = string(S)}
	;   [number(N)] -> {Value = number(S)}
	).

This hasn't been tested, but the intent was that this example would
turn into the *TERM*

	token('SerializedViewObject_0_1',
	      ['SerializedObject'],
	      [att(['attribute','type'],
		   no_id, string("OME.GraphicView$GVERecord")),
	       att(['attribute'], id('x'),  number(323)),
	       att(['attribute'], id('y'),  number(301)),
	       att(['attribute'], id('ID'), number(1))])

	SimpleClass IStarRoleElement
	    IN OMEInstantiableClass, IStarActorElementClass
	    ISA IStarActorElement
	    WITH
	        attribute, imagesize
	            height : 80
	        attribute, imagesize
	            width : 80
	        attribute, imagename
	             : "Role.gif"
	        attribute, name
	             : "Role"
	END
	
class(class(Name,In,Isa,With)) -->
    [id('SimpleClass'),id(Name)],
    in_part(In),
    isa_part(Isa),
    with_part(With).

isa_part(Names) --> [id('ISA')], !, name_list(Names).
isa_part([]) --> [].  % include only if the ISA part is optional.

I leave the translation of this example as an exercise for the reader.

Note that this is MUCH easier than any imaginable hack that reads lines
and stores them as clauses.

----------------
* To UNSUBSCRIBE, please use the HTML form at

    http://www.swi.psy.uva.nl/projects/SWI-Prolog/index.html#mailinglist

or send mail to [email-masked] using the Subject: 
"unsubscribe"
(without the quotes) and *no* message body.

** An ARCHIVE of this list is maintained at

    http://www.swi.psy.uva.nl/projects/SWI-Prolog/mailinglist/archive/