Subject: Re: swi jpl character coding

From: Paul Singleton <[email-masked]>
Date: Fri, 15 Dec 2000 18:11:14 +0000

Jan Wielemaker wrote:

> > Jan, how are atoms encoded in the prolog vm?  As UTF?

> No.  As yet it is encoded as an unsigned char * with length, so it can
> contain 0-bytes.  So it cannot represent characters outside the range
> 0..255, which you can use with any 8-bit encoding for which the lower
> 128 characters match the ASCII set.  It is planned to change this to
> use UTF-8, probably somewhere next year.

I much admire SWI-Prolog's support for atom "names" as unrestricted byte
strings (any length, any values), since this allows me to use atoms to
represent arbitrary external "blobs" of data, e.g. long fields from
databases, JPEGs, OLE objects, whatever (it may not always be appropriate,
but the liberty to do this is valuable).

I see no problems with the semantics of Prolog if atoms are arbitrary
external values, not necessarily text strings (although I expect the
theologians to come up with something unhelpful :-/

Indeed, I reckon it's neither necessary nor desirable that a Prolog
engine is somehow committed to a particular interpretation of its
atoms' names: as long as I have atom_bytes/2 to dynamically compose
and decompose atoms to/from their "names" I'd be happy, I think...

I realise that it's convenient to have a concrete source syntax for
Prolog :-) but as long as we can compile a list of clause representations
which represent a procedure (as suggested in another posting) we can
implement module loaders from any concrete syntax we may devise, including
graphical ones.

I realise also that a debugger/tracer will want to represent atoms in
some standard, compact way, but with the hooks etc. which Jan provides,
I assume we can implement a debugger's user interface outside the basic
engine?

Is it the case that, if we define atom names as UTF-8 representations
of UCS strings, then certain byte-string atom values will strictly be
illegal?

Is it feasible to localise the UTF-8-specific kernel code so that we
can avoid it if we want?

Is there any sense in my vision of a Prolog kernel which genuinely
doesn't care what external values its atoms represent?

Should I resist the temptation to move to a private scheme whereby
each atom is explicitly typed by an initial "magic number" in its
first few bytes?  I think this could be better than wrapping them
in constructors to denote their type, when the types are of concern
only to I/O not to inference...

Paul Singleton