Subject: Re: Mysterious memory allocation

From: Jan Wielemaker <[email-masked]>
Date: Tue, 22 Feb 2000 12:57:20 +0100

On Mon, 21 Feb 2000, you wrote:
>
>
>> > May I forward your mail to another user who e-mailes me to tell
>> > he had a similar problem?
>> 
>> Sure, it's on the mailing list anyway
>> 	
>
>Achsooo, sorry, I oversaw it.
>
>So. Here is a short test. I still have the hope that you will
>finally tell me "this is a simple programming mistake, if you
>turn the predicate the other way it will work nicely."
>
>I attach the file with my source code to read terms in files
>at the first time, and then assert them (file_access.pl), +
>one sample subdirectory of the database I want to use.
>
>If I launch pl and consult 'file_access', I use 12160 'real' Kb
>in RAM (in /proc), and statistics yields this :

This number is a bit fake.  Basically, SWI-Prolog does an mmap for the
stack-limit for each if the stacks,  protected to be inaccessible.  If
MAP_RESERVE is provided, it will use this flag.  Otherwise it assumes
the OS will not actually allocate memory as long as you don't use
it.  This is the case on Linux (at least, they claim and on
2.2.x it appears true).

As stack is required, the system make pages one-by-one accessible.
After completion of a query or after a garbage collect, all unused
stack is returned to the system by unmapping and remapping it 
as an inaccessible area.

Sofar, this schema looks portable and working properly.  You could
test this using a program that produces garbage, but is otherwise
an endless loop.  It should run forever.  For example:

loop :-
	_X = [hello, world],
	loop.

If this runs forever, getting a bit smaller and bigger as garbage
is collected and produced, this mechanism is working fine on your
os.

Further, memory consumption on first execution is hard to judge.
Many things are created `lazy' (just-in-time).  If you look for
memory leaks, write a program that should not consume any
additional memory after completion and run it twice.  You will
normally see memory consumption on the first pass, but afterwards
the system should be stable.  Make sure it refers to the same
atoms and name/arity pairs, or just make sure about name/arity,
but run the program in a loop and see the atom-garbage collector
doing its work.

On my machine I couldn't find anything definitely wrong with your
examples (SuSE Linux 6.3, kernel 2.2.13), though the claims in /proc
are hard to interpret to me.

	Regards --- Jan