Subject: Re: String Representation

From: "Richard A. O'Keefe" <[email-masked]>
Date: Wed, 17 May 2000 14:56:37 +1200 (NZST)

I wrote:
> Using a *single* atom to hold a string has the advantage that if there
> are several occurrences of exactly the same string, they will share the
> representation, so you can get less than 1 byte per character.
	
Paul Sephton <[email-masked]> wrote:
	Unless, of course, you are talking about single character atoms...
	
Not even then.  Suppose for argument's sake that the space cost of
an entry in the atom table is 16 + 4*ceiling(length/4) bytes, so that
a single character atom costs 20 bytes.

If there are more than 20 references to that atom, the average cost
of the representation is less than 1 byte per character.

With that space cost, a four-character string would only need 5 references
to go below one byte per character.

It's worth noting that the SNOBOL 4 programming language, whose whole
purpose was to process lots of strings in a pattern-matchy way, used
"unique strings".  That is, whenever it constructed a string, it looked
it up in a hash table to ensure that it had only one copy.

It's also worth noting that one of the things Dewar et al did in SPITBOL
(a SNOBOL compiler that generated considerably faster code) was to get
rid of the hash table...

It really all depends on what you are doing with strings.
If you are doing lots of splits and joins, lists of characters
are far more efficient than packed arrays of bytes, and of course than atoms.
But not _all_ applications of strings do that.