Storable - persistency for perl data structures
SYNOPSIS
use Storable;
store \%table, 'file';
$hashref = retrieve('file');
use Storable qw(nstore store_fd nstore_fd freeze thaw dclone);
# Network order
nstore \%table, 'file';
$hashref = retrieve('file'); # There is NO nretrieve()
# Storing to and retrieving from an already opened file
store_fd \@array, \*STDOUT;
nstore_fd \%table, \*STDOUT;
$aryref = retrieve_fd(\*SOCKET);
$hashref = retrieve_fd(\*SOCKET);
# Serializing to memory
$serialized = freeze \%table;
%table_clone = %{ thaw($serialized) };
# Deep (recursive) cloning
$cloneref = dclone($ref);
DESCRIPTION
The Storable package brings persistency to your perl data
structures containing SCALAR, ARRAY, HASH or REF objects,
i.e. anything that can be convenientely stored to disk and
retrieved at a later time.
It can be used in the regular procedural way by calling
`store' with a reference to the object to be stored, along
with the file name where the image should be written. The
routine returns `undef' for I/O problems or other internal
error, a true value otherwise. Serious errors are propa
gated as a `die' exception.
To retrieve data stored to disk, use `retrieve' with a
file name, and the objects stored into that file are
recreated into memory for you, a reference to the root
object being returned. In case an I/O error occurs while
reading, `undef' is returned instead. Other serious errors
are propagated via `die'.
Since storage is performed recursively, you might want to
stuff references to objects that share a lot of common
data into a single array or hash table, and then store
that object. That way, when you retrieve back the whole
thing, the objects will continue to share what they origi
nally shared.
an already opened file descriptor using the `store_fd'
routine, and retrieve from a file via `retrieve_fd'. Those
names aren't imported by default, so you will have to do
that explicitely if you need those routines. The file
descriptor you supply must be already opened, for read if
you're going to retrieve and for write if you wish to
store.
store_fd(\%table, *STDOUT) || die "can't store to stdout\n";
$hashref = retrieve_fd(*STDIN);
You can also store data in network order to allow easy
sharing across multiple platforms, or when storing on a
socket known to be remotely connected. The routines to
call have an initial `n' prefix for network, as in
`nstore' and `nstore_fd'. At retrieval time, your data
will be correctly restored so you don't have to know
whether you're restoring from native or network ordered
data.
When using `retrieve_fd', objects are retrieved in
sequence, one object (i.e. one recursive tree) per associ
ated `store_fd'.
If you're more from the object-oriented camp, you can
inherit from Storable and directly store your objects by
invoking `store' as a method. The fact that the root of
the to-be-stored tree is a blessed reference (i.e. an
object) is special-cased so that the retrieve does not
provide a reference to that object but rather the blessed
object reference itself. (Otherwise, you'd get a reference
to that blessed object).
MEMORY STORE
The Storable engine can also store data into a Perl scalar
instead, to later retrieve them. This is mainly used to
freeze a complex structure in some safe compact memory
place (where it can possibly be sent to another process
via some IPC, since freezing the structure also serializes
it in effect). Later on, and maybe somewhere else, you can
thaw the Perl scalar out and recreate the original complex
structure in memory.
Surprisingly, the routines to be called are named `freeze'
and `thaw'. If you wish to send out the frozen scalar to
another machine, use `nfreeze' instead to get a portable
image.
Note that freezing an object structure and immediately
thawing it actually achieves a deep cloning of that struc
ture. Storable provides you with a `dclone' interface
which does not create that intermediary scalar but instead
freezes the structure in some internal memory space and
SPEED
The heart of Storable is written in C for decent speed.
Extra low-level optimization have been made when manipu
lating perl internals, to sacrifice encapsulation for the
benefit of a greater speed.
Storage is now slightly slower than retrieval since the
former has to also store data in a hash table to keep
track of which objects have been stored already, whilst
the latter uses an array instead of a hash table.
On my HP 9000/712 machine running HPUX 9.03 and with perl
5.004, I can store 0.85 Mbyte/s and I can retrieve at 0.90
Mbytes/s, approximatively (CPU + system time). This was
measured with Benchmark and the Magic: The Gathering
database from Tom Christiansen (1.6 Mbytes on disk).
CANONICAL REPRESENTATION
Normally Storable stores elements of hashes in the order
they are stored internally by Perl, i.e. pseudo-randomly.
If you set `$Storable::canonical' to some `TRUE' value,
Storable will store hashes with the elements sorted by
their key. This allows you to compare data structures by
comparing their frozen representations (or even the com
pressed frozen representations), which can be useful for
creating lookup tables for complicated queries.
Canonical order does not imply network order, those are
two orthogonal settings.
WIZARDS ONLY
The `Storable::last_op_in_netorder()' predicate will tell
you whether network order was used in the last store or
retrieve operation. If you don't know how to use this,
just forget about it.
EXAMPLES
Here are some code samples showing a possible usage of
Storable:
use Storable qw(store retrieve freeze thaw dclone);
%color = ('Blue' => 0.1, 'Red' => 0.8, 'Black' => 0, 'White' => 1);
store(\%color, '/tmp/colors') or die "Can't store %a in /tmp/colors!\n";
$colref = retrieve('/tmp/colors');
die "Unable to retrieve from /tmp/colors!\n" unless defined $colref;
printf "Blue is still %lf\n", $colref->{'Blue'};
$colref2 = dclone(\%color);
printf "Serialization of %%color is %d bytes long.\n", length($str);
$colref3 = thaw($str);
which prints (on my machine):
Blue is still 0.100000
Serialization of %color is 102 bytes long.
WARNING
If you're using references as keys within your hash
tables, you're bound to disapointment when retrieving your
data. Indeed, Perl stringifies references used as hash
table keys. If you later wish to access the items via
another reference stringification (i.e. using the same
reference that was used for the key originally to record
the value into the hash table), it will work because both
references stringify to the same string.
It won't work across a `store' and `retrieve' operations
however, because the addresses in the retrieved objects,
which are part of the stringified references, will proba
bly differ from the original addresses. The topology of
your structure is preserved, but not hidden semantics like
those.
On platforms where it matters, be sure to call `binmode()'
on the descriptors that you pass to Storable functions.
Storing data canonically that contains large hashes can be
significantly slower than storing the same data normally,
as temprorary arrays to hold the keys for each hash have
to be allocated, populated, sorted and freed. Some tests
have shown a halving of the speed of storing -- the exact
penalty will depend on the complexity of your data. There
is no slowdown on retrieval.
BUGS
You can't store GLOB, CODE, FORMLINE, etc... If you can
define semantics for those operations, feel free to
enhance Storable so that it can deal with them.
The store functions will `croak' if they run into such
references unless you set `$Storable::forgive_me' to some
`TRUE' value. In that case, the fatal message is turned in
a warning and some meaningless string is stored instead.
Setting `$Storable::canonical' may not yield frozen
strings that compare equal due to possible stringification
of numbers. When the string version of a scalar exists, it
is the form stored, therefore if you happen to use your
numbers as strings between two freezing operations on the
same data structures, you will get different results.
the mercy of perl's internal redesign or structure
changes. If that bothers you, you can try convincing Larry
that what is used in Storable should be documented and
consistently kept in future revisions.
CREDITS
Thank you to (in chronological order):
Jarkko Hietaniemi <jhi@iki.fi>
Ulrich Pfeifer <pfeifer@charly.informatik.uni-dortmund.de>
Benjamin A. Holzman <benjamin.a.holzman@bender.com>
Andrew Ford <A.Ford@ford-mason.co.uk>
Gisle Aas <gisle@aas.no>
Jeff Gresham <gresham_jeffrey@jpmorgan.com>
Murray Nesbitt <murray@activestate.com>
for their bug reports, suggestions and contributions.
Benjamin Holzman contributed the tied variable support,
Andrew Ford contributed the canonical order for hashes,
and Gisle Aas fixed a few misunderstandings of mine
regarding the Perl internals, and optimized the emission
of "tags" in the output streams by simply counting the
objects instead of tagging them (leading to a binary
incompatibility for the Storable image starting at version
0.6--older images are of course still properly under
stood). Murray Nesbitt made Storable thread-safe.
TRANSLATIONS
There is a Japanese translation of this man page available
at http://mem
ber.nifty.ne.jp/hippo2000/perltips/storable.htm , courtesy
of Kawai, Takanori <kawai@nippon-rad.co.jp>.
AUTHOR
Raphael Manfredi <Raphael_Manfredi@pobox.com>
Man(1) output converted with
man2html