perlfaq3 - Programming Tools ($Revision: 1.38 $, $Date:
1999/05/23 16:08:30 $)
DESCRIPTION
This section of the FAQ answers questions related to pro
grammer tools and programming support.
How do I do (anything)?
Have you looked at CPAN (see the perlfaq2 manpage)? The
chances are that someone has already written a module that
can solve your problem. Have you read the appropriate man
pages? Here's a brief index:
Basics perldata, perlvar, perlsyn, perlop, perlsub
Execution perlrun, perldebug
Functions perlfunc
Objects perlref, perlmod, perlobj, perltie
Data Structures perlref, perllol, perldsc
Modules perlmod, perlmodlib, perlsub
Regexes perlre, perlfunc, perlop, perllocale
Moving to perl5 perltrap, perl
Linking w/C perlxstut, perlxs, perlcall, perlguts, perlembed
Various http://www.perl.com/CPAN/doc/FMTEYEWTK/index.html
(not a man-page but still useful)
A crude table of contents for the Perl man page set is
found in the perltoc manpage.
How can I use Perl interactively?
The typical approach uses the Perl debugger, described in
the perldebug(1) man page, on an ``empty'' program, like
this:
perl -de 42
Now just type in any legal Perl code, and it will be imme
diately evaluated. You can also examine the symbol table,
get stack backtraces, check variable values, set break
points, and other operations typically found in symbolic
debuggers.
Is there a Perl shell?
In general, no. The Shell.pm module (distributed with
Perl) makes Perl try commands which aren't part of the
Perl language as shell commands. perlsh from the source
distribution is simplistic and uninteresting, but may
still be what you want.
Have you tried `use warnings' or used `-w'? They enable
warnings for dubious practices.
Have you tried `use strict'? It prevents you from using
symbolic references, makes you predeclare any subroutines
that you call as bare words, and (probably most impor
tantly) forces you to predeclare your variables with `my'
or `our' or `use vars'.
Did you check the returns of each and every system call?
The operating system (and thus Perl) tells you whether
they worked or not, and if not why.
open(FH, "> /etc/cantwrite")
or die "Couldn't write to /etc/cantwrite: $!\n";
Did you read the perltrap manpage? It's full of gotchas
for old and new Perl programmers, and even has sections
for those of you who are upgrading from languages like awk
and C.
Have you tried the Perl debugger, described in the perlde
bug manpage? You can step through your program and see
what it's doing and thus work out why what it's doing
isn't what it should be doing.
How do I profile my Perl programs?
You should get the Devel::DProf module from CPAN, and also
use Benchmark.pm from the standard distribution. Bench
mark lets you time specific portions of your code, while
Devel::DProf gives detailed breakdowns of where your code
spends its time.
Here's a sample use of Benchmark:
use Benchmark;
@junk = `cat /etc/motd`;
$count = 10_000;
timethese($count, {
'map' => sub { my @a = @junk;
map { s/a/b/ } @a;
return @a
},
'for' => sub { my @a = @junk;
local $_;
for (@a) { s/a/b/ };
return @a },
});
be dependent on your hardware, operating system, and the
load on your machine):
Benchmark: timing 10000 iterations of for, map...
for: 4 secs ( 3.97 usr 0.01 sys = 3.98 cpu)
map: 6 secs ( 4.97 usr 0.00 sys = 4.97 cpu)
Be aware that a good benchmark is very hard to write. It
only tests the data you give it, and really proves little
about differing complexities of contrasting algorithms.
How do I cross-reference my Perl programs?
The B::Xref module, shipped with the new, alpha-release
Perl compiler (not the general distribution prior to the
5.005 release), can be used to generate cross-reference
reports for Perl programs.
perl -MO=Xref[,OPTIONS] scriptname.plx
Is there a pretty-printer (formatter) for Perl?
There is no program that will reformat Perl as much as
indent(1) does for C. The complex feedback between the
scanner and the parser (this feedback is what confuses the
vgrind and emacs programs) makes it challenging at best to
write a stand-alone Perl parser.
Of course, if you simply follow the guidelines in the
perlstyle manpage, you shouldn't need to reformat. The
habit of formatting your code as you write it will help
prevent bugs. Your editor can and should help you with
this. The perl-mode for emacs can provide a remarkable
amount of help with most (but not all) code, and even less
programmable editors can provide significant assistance.
Tom swears by the following settings in vi and its clones:
set ai sw=4
map! ^O {^M}^[O^T
Now put that in your .exrc file (replacing the caret char
acters with control characters) and away you go. In
insert mode, ^T is for indenting, ^D is for undenting, and
^O is for blockdenting -- as it were. If you haven't used
the last one, you're missing a lot. A more complete exam
ple, with comments, can be found at
http://www.perl.com/CPAN-
local/authors/id/TOMC/scripts/toms.exrc.gz
If you are used to using the vgrind program for printing
out nice code to a laser printer, you can take a stab at
this using
ing.vgrind.entry, but the results are not particularly
satisfying for sophisticated code.
The a2ps at http://www.infres.enst.fr/%7Edemaille/a2ps/
does lots of things related to generating nicely printed
output of documents.
Is there a ctags for Perl?
There's a simple one at
http://www.perl.com/CPAN/authors/id/TOMC/scripts/ptags.gz
which may do the trick. And if not, it's easy to hack
into what you want.
Is there an IDE or Windows Perl Editor?
If you're on Unix, you already have an IDE -- Unix itself.
This powerful IDE derives from its interoperability, flex
ibility, and configurability. If you really want to get a
feel for Unix-qua-IDE, the best thing to do is to find
some high-powered programmer whose native language is
Unix. Find someone who has been at this for many years,
and just sit back and watch them at work. They have cre
ated their own IDE, one that suits their own tastes and
aptitudes. Quietly observe them edit files, move them
around, compile them, debug them, test them, etc. The
entire development *is* integrated, like a top-of-the-line
German sports car: functional, powerful, and elegant. You
will be absolutely astonished at the speed and ease exhib
ited by the native speaker of Unix in his home territory.
The art and skill of a virtuoso can only be seen to be
believed. That is the path to mastery -- all these cob
bled little IDEs are expensive toys designed to sell a
flashy demo using cheap tricks, and being optimized for
immediate but shallow understanding rather than enduring
use, are but a dim palimpsest of real tools.
In short, you just have to learn the toolbox. However, if
you're not on Unix, then your vendor probably didn't
bother to provide you with a proper toolbox on the so-
called complete system that you forked out your hard-
earned cash on.
PerlBuilder (XXX URL to follow) is an integrated develop
ment environment for Windows that supports Perl develop
ment. Perl programs are just plain text, though, so you
could download emacs for Windows (???) or a vi clone (vim)
which runs on for win32
(http://www.cs.vu.nl/%7Etmgil/vi.html). If you're trans
ferring Windows files to Unix, be sure to transfer in
ASCII mode so the ends of lines are appropriately mangled.
For a complete version of Tom Christiansen's vi configura
tion file, see http://www.perl.com/CPAN/authors/Tom_Chris
tiansen/scripts/toms.exrc.gz, the standard benchmark file
for vi emulators. This runs best with nvi, the current
version of vi out of Berkeley, which incidentally can be
built with an embedded Perl interpreter -- see
http://www.perl.com/CPAN/src/misc.
Where can I get perl-mode for emacs?
Since Emacs version 19 patchlevel 22 or so, there have
been both a perl-mode.el and support for the Perl debugger
built in. These should come with the standard Emacs 19
distribution.
In the Perl source directory, you'll find a directory
called "emacs", which contains a cperl-mode that color-
codes keywords, provides context-sensitive help, and other
nifty things.
Note that the perl-mode of emacs will have fits with
`"main'foo"' (single quote), and mess up the indentation
and highlighting. You are probably using `"main::foo"' in
new Perl code anyway, so this shouldn't be an issue.
How can I use curses with Perl?
The Curses module from CPAN provides a dynamically load
able object module interface to a curses library. A small
demo can be found at the directory
http://www.perl.com/CPAN/authors/Tom_Chris
tiansen/scripts/rep; this program repeats a command and
updates the screen as needed, rendering rep ps axu similar
to top.
How can I use X or Tk with Perl?
Tk is a completely Perl-based, object-oriented interface
to the Tk toolkit that doesn't force you to use Tcl just
to get at Tk. Sx is an interface to the Athena Widget
set. Both are available from CPAN. See the directory
http://www.perl.com/CPAN/modules/by-cate
gory/08_User_Interfaces/
Invaluable for Perl/Tk programming are: the Perl/Tk FAQ at
http://w4.lns.cornell.edu/%7Epvhp/ptk/ptkTOC.html , the
Perl/Tk Reference Guide available at
http://www.perl.com/CPAN-local/authors/Stephen_O_Lidie/ ,
and the online manpages at http://www-
users.cs.umn.edu/%7Eamundson/perl/perltk/toc.html .
The http://www.perl.com/CPAN/authors/id/SKUNZ/perl
menu.v4.0.tar.gz module, which is curses-based, can help
with this.
What is undump?
See the next questions.
How can I make my Perl program run faster?
The best way to do this is to come up with a better algo
rithm. This can often make a dramatic difference. Chap
ter 8 in the Camel has some efficiency tips in it you
might want to look at. Jon Bentley's book ``Programming
Pearls'' (that's not a misspelling!) has some good tips
on optimization, too. Advice on benchmarking boils down
to: benchmark and profile to make sure you're optimizing
the right part, look for better algorithms instead of
microtuning your code, and when all else fails consider
just buying faster hardware.
A different approach is to autoload seldom-used Perl code.
See the AutoSplit and AutoLoader modules in the standard
distribution for that. Or you could locate the bottleneck
and think about writing just that part in C, the way we
used to take bottlenecks in C code and write them in
assembler. Similar to rewriting in C is the use of mod
ules that have critical sections written in C (for
instance, the PDL module from CPAN).
In some cases, it may be worth it to use the backend com
piler to produce byte code (saving compilation time) or
compile into C, which will certainly save compilation time
and sometimes a small amount (but not much) execution
time. See the question about compiling your Perl programs
for more on the compiler--the wins aren't as obvious as
you'd hope.
If you're currently linking your perl executable to a
shared libc.so, you can often gain a 10-25% performance
benefit by rebuilding it to link with a static libc.a
instead. This will make a bigger perl executable, but
your Perl programs (and programmers) may thank you for it.
See the INSTALL file in the source distribution for more
information.
Unsubstantiated reports allege that Perl interpreters that
use sfio outperform those that don't (for I/O intensive
applications). To try this, see the INSTALL file in the
source distribution, especially the ``Selecting File I/O
mechanisms'' section.
Perl program by storing the already-compiled form to disk.
This is no longer a viable option, as it only worked on a
few architectures, and wasn't a good solution anyway.
How can I make my Perl program take less memory?
When it comes to time-space tradeoffs, Perl nearly always
prefers to throw memory at a problem. Scalars in Perl use
more memory than strings in C, arrays take more than that,
and hashes use even more. While there's still a lot to be
done, recent releases have been addressing these issues.
For example, as of 5.004, duplicate hash keys are shared
amongst all hashes using them, so require no reallocation.
In some cases, using substr() or vec() to simulate arrays
can be highly beneficial. For example, an array of a
thousand booleans will take at least 20,000 bytes of
space, but it can be turned into one 125-byte bit vector
for a considerable memory savings. The standard Tie::Sub
strHash module can also help for certain types of data
structure. If you're working with specialist data struc
tures (matrices, for instance) modules that implement
these in C may use less memory than equivalent Perl mod
ules.
Another thing to try is learning whether your Perl was
compiled with the system malloc or with Perl's builtin
malloc. Whichever one it is, try using the other one and
see whether this makes a difference. Information about
malloc is in the INSTALL file in the source distribution.
You can find out whether you are using perl's malloc by
typing `perl -V:usemymalloc'.
Is it unsafe to return a pointer to local data?
No, Perl's garbage collection system takes care of this.
sub makeone {
my @a = ( 1 .. 10 );
return \@a;
}
for $i ( 1 .. 10 ) {
push @many, makeone();
}
print $many[4][5], "\n";
print "@many\n";
You can't. On most operating systems, memory allocated to
a program can never be returned to the system. That's why
long-running programs sometimes re-exec themselves. Some
operating systems (notably, FreeBSD and Linux) allegedly
reclaim large chunks of memory that is no longer used, but
it doesn't appear to happen with Perl (yet). The Mac
appears to be the only platform that will reliably
(albeit, slowly) return memory to the OS.
We've had reports that on Linux (Redhat 5.1) on Intel,
`undef $scalar' will return memory to the system, while on
Solaris 2.6 it won't. In general, try it yourself and
see.
However, judicious use of my() on your variables will help
make sure that they go out of scope so that Perl can free
up their storage for use in other parts of your program.
A global variable, of course, never goes out of scope, so
you can't get its space automatically reclaimed, although
undef()ing and/or delete()ing it will achieve the same
effect. In general, memory allocation and de-allocation
isn't something you can or should be worrying about much
in Perl, but even this capability (preallocation of data
types) is in the works.
How can I make my CGI script more efficient?
Beyond the normal measures described to make general Perl
programs faster or smaller, a CGI program has additional
issues. It may be run several times per second. Given
that each time it runs it will need to be re-compiled and
will often allocate a megabyte or more of system memory,
this can be a killer. Compiling into C isn't going to
help you because the process start-up overhead is where
the bottleneck is.
There are two popular ways to avoid this overhead. One
solution involves running the Apache HTTP server (avail
able from http://www.apache.org/) with either of the
mod_perl or mod_fastcgi plugin modules.
With mod_perl and the Apache::Registry module (distributed
with mod_perl), httpd will run with an embedded Perl
interpreter which pre-compiles your script and then exe
cutes it within the same address space without forking.
The Apache extension also gives Perl access to the inter
nal server API, so modules written in Perl can do just
about anything a module written in C can. For more on
mod_perl, see http://perl.apache.org/
With the FCGI module (from CPAN) and the mod_fastcgi mod
ule (available from http://www.fastcgi.com/) each of your
Both of these solutions can have far-reaching effects on
your system and on the way you write your CGI programs, so
investigate them with care.
See http://www.perl.com/CPAN/modules/by-cate
gory/15_World_Wide_Web_HTML_HTTP_CGI/ .
A non-free, commercial product, ``The Velocity Engine for
Perl'', (http://www.binevolve.com/ or
http://www.binevolve.com/velocigen/) might also be worth
looking at. It will allow you to increase the performance
of your Perl programs, up to 25 times faster than normal
CGI Perl by running in persistent Perl mode, or 4 to 5
times faster without any modification to your existing CGI
programs. Fully functional evaluation copies are available
from the web site.
How can I hide the source for my Perl program?
Delete it. :-) Seriously, there are a number of (mostly
unsatisfactory) solutions with varying levels of ``secu
rity''.
First of all, however, you can't take away read permis
sion, because the source code has to be readable in order
to be compiled and interpreted. (That doesn't mean that a
CGI script's source is readable by people on the web,
though, only by people with access to the filesystem) So
you have to leave the permissions at the socially friendly
0755 level.
Some people regard this as a security problem. If your
program does insecure things, and relies on people not
knowing how to exploit those insecurities, it is not
secure. It is often possible for someone to determine the
insecure things and exploit them without viewing the
source. Security through obscurity, the name for hiding
your bugs instead of fixing them, is little security
indeed.
You can try using encryption via source filters (Filter::*
from CPAN), but any decent programmer will be able to
decrypt it. You can try using the byte code compiler and
interpreter described below, but the curious might still
be able to de-compile it. You can try using the native-
code compiler described below, but crackers might be able
to disassemble it. These pose varying degrees of diffi
culty to people wanting to get at your code, but none can
definitively conceal it (this is true of every language,
not just Perl).
If you're concerned about people profiting from your code,
license will give you legal security. License your soft
ware and pepper it with threatening statements like ``This
is unpublished proprietary software of XYZ Corp. Your
access to it does not give you permission to use it blah
blah blah.'' We are not lawyers, of course, so you should
see a lawyer if you want to be sure your license's wording
will stand up in court.
How can I compile my Perl program into byte code or C?
Malcolm Beattie has written a multifunction backend com
piler, available from CPAN, that can do both these things.
It is included in the perl5.005 release, but is still con
sidered experimental. This means it's fun to play with if
you're a programmer but not really for people looking for
turn-key solutions.
Merely compiling into C does not in and of itself guaran
tee that your code will run very much faster. That's
because except for lucky cases where a lot of native type
inferencing is possible, the normal Perl run time system
is still present and so your program will take just as
long to run and be just as big. Most programs save little
more than compilation time, leaving execution no more than
10-30% faster. A few rare programs actually benefit sig
nificantly (like several times faster), but this takes
some tweaking of your code.
You'll probably be astonished to learn that the current
version of the compiler generates a compiled form of your
script whose executable is just as big as the original
perl executable, and then some. That's because as cur
rently written, all programs are prepared for a full
eval() statement. You can tremendously reduce this cost
by building a shared libperl.so library and linking
against that. See the INSTALL podfile in the Perl source
distribution for details. If you link your main perl
binary with this, it will make it minuscule. For example,
on one author's system, /usr/bin/perl is only 11k in size!
In general, the compiler will do nothing to make a Perl
program smaller, faster, more portable, or more secure.
In fact, it will usually hurt all of those. The exe
cutable will be bigger, your VM system may take longer to
load the whole thing, the binary is fragile and hard to
fix, and compilation never stopped software piracy in the
form of crackers, viruses, or bootleggers. The real
advantage of the compiler is merely packaging, and once
you see the size of what it makes (well, unless you use a
shared libperl.so), you'll probably want a complete Perl
install anyway.
You can't. Not yet, anyway. You can integrate Java and
Perl with the Perl Resource Kit from O'Reilly and Associ
ates. See http://www.oreilly.com/catalog/prkunix/ for
more information. The Java interface will be supported in
the core 5.6 release of Perl.
How can I get `#!perl' to work on [MS-DOS,NT,...]?
For OS/2 just use
extproc perl -S -your_switches
as the first line in `*.cmd' file (`-S' due to a bug in
cmd.exe's `extproc' handling). For DOS one should first
invent a corresponding batch file, and codify it in
`ALTERNATIVE_SHEBANG' (see the INSTALL file in the source
distribution for more information).
The Win95/NT installation, when using the ActiveState port
of Perl, will modify the Registry to associate the `.pl'
extension with the perl interpreter. If you install
another port, perhaps even building your own Win95/NT Perl
from the standard sources by using a Windows port of gcc
(e.g., with cygwin or mingw32), then you'll have to modify
the Registry yourself. In addition to associating `.pl'
with the interpreter, NT people can use: `SET
PATHEXT=%PATHEXT%;.PL' to let them run the program
`install-linux.pl' merely by typing `install-linux'.
Macintosh Perl programs will have the appropriate Creator
and Type, so that double-clicking them will invoke the
Perl application.
IMPORTANT!: Whatever you do, PLEASE don't get frustrated,
and just throw the perl interpreter into your cgi-bin
directory, in order to get your programs working for a web
server. This is an EXTREMELY big security risk. Take the
time to figure out how to do it correctly.
Can I write useful Perl programs on the command line?
Yes. Read the perlrun manpage for more information. Some
examples follow. (These assume standard Unix shell quot
ing rules.)
# sum first and last fields
perl -lane 'print $F[0] + $F[-1]' *
# identify text files
perl -le 'for(@ARGV) {print if -f && -T _}' *
perl -0777 -pe 's{/\*.*?\*/}{}gs' foo.c
# make file a month younger than today, defeating reaper daemons
perl -e '$X=24*60*60; utime(time(),time() + 30 * $X,@ARGV)' *
# find first unused uid
perl -le '$i++ while getpwuid($i); print $i'
# display reasonable manpath
echo $PATH | perl -nl -072 -e '
s![^/+]*$!man!&&-d&&!$s{$_}++&&push@m,$_;END{print"@m"}'
OK, the last one was actually an Obfuscated Perl Contest
entry. :-)
Why don't Perl one-liners work on my DOS/Mac/VMS system?
The problem is usually that the command interpreters on
those systems have rather different ideas about quoting
than the Unix shells under which the one-liners were cre
ated. On some systems, you may have to change single-
quotes to double ones, which you must NOT do on Unix or
Plan9 systems. You might also have to change a single %
to a %%.
For example:
# Unix
perl -e 'print "Hello world\n"'
# DOS, etc.
perl -e "print \"Hello world\n\""
# Mac
print "Hello world\n"
(then Run "Myscript" or Shift-Command-R)
# VMS
perl -e "print ""Hello world\n"""
The problem is that none of this is reliable: it depends
on the command interpreter. Under Unix, the first two
often work. Under DOS, it's entirely possible neither
works. If 4DOS was the command shell, you'd probably have
better luck like this:
perl -e "print <Ctrl-x>"Hello world\n<Ctrl-x>""
Under the Mac, it depends which environment you are using.
The MacPerl shell, or MPW, is much like Unix shells in its
support for several quoting variants, except that it makes
free use of the Mac's non-ASCII characters as control
characters.
'single quotes', and `backticks`, may make one-liners eas
ier to write.
There is no general solution to all of this. It is a
mess, pure and simple. Sucks to be away from Unix, huh?
:-)
[Some of this answer was contributed by Kenneth
Albanowski.]
Where can I learn about CGI or Web programming in Perl?
For modules, get the CGI or LWP modules from CPAN. For
textbooks, see the two especially dedicated to web stuff
in the question on books. For problems and questions
related to the web, like ``Why do I get 500 Errors'' or
``Why doesn't it run from the browser right when it runs
fine on the command line'', see these sources:
WWW Security FAQ
http://www.w3.org/Security/Faq/
Web FAQ
http://www.boutell.com/faq/
CGI FAQ
http://www.webthing.com/tutorials/cgifaq.html
HTTP Spec
http://www.w3.org/pub/WWW/Protocols/HTTP/
HTML Spec
http://www.w3.org/TR/REC-html40/
http://www.w3.org/pub/WWW/MarkUp/
CGI Spec
http://www.w3.org/CGI/
CGI Security FAQ
http://www.go2net.com/people/paulp/cgi-security/safe-cgi.txt
Where can I learn about object-oriented Perl programming?
A good place to start is the perltoot manpage, and you can
use the perlobj manpage and the perlbot manpage for refer
ence. Perltoot didn't come out until the 5.004 release,
but you can get a copy (in pod, html, or postscript) from
http://www.perl.com/CPAN/doc/FMTEYEWTK/ .
xsubpp]
If you want to call C from Perl, start with the perlxstut
manpage, moving on to the perlxs manpage, the xsubpp man
page, and the perlguts manpage. If you want to call Perl
from C, then read the perlembed manpage, the perlcall man
page, and the perlguts manpage. Don't forget that you can
learn a lot from looking at how the authors of existing
extension modules wrote their code and solved their prob
lems.
I've read perlembed, perlguts, etc., but I can't embed
perl in
my C program, what am I doing wrong?" my C program, what
am I doing wrong?" Download the ExtUtils::Embed kit from
CPAN and run `make test'. If the tests pass, read the
pods again and again and again. If they fail, see the
perlbug manpage and send a bug report with the output of
`make test TEST_VERBOSE=1' along with `perl -V'.
When I tried to run my script, I got this message. What
does it
mean?" mean?" A complete list of Perl's error messages
and warnings with explanatory text can be found in the
perldiag manpage. You can also use the splain program
(distributed with Perl) to explain the error messages:
perl program 2>diag.out
splain [-v] [-p] diag.out
or change your program to explain the messages for you:
use diagnostics;
or
use diagnostics -verbose;
What's MakeMaker?
This module (part of the standard Perl distribution) is
designed to write a Makefile for an extension module from
a Makefile.PL. For more information, see the ExtU
tils::MakeMaker manpage.
AUTHOR AND COPYRIGHT
Copyright (c) 1997-1999 Tom Christiansen and Nathan Tork
ington. All rights reserved.
When included as an integrated part of the Standard
otherwise), this works is covered under Perl's Artistic
License. For separate distributions of all or part of
this FAQ outside of that, see the perlfaq manpage.
Irrespective of its distribution, all code examples here
are in the public domain. You are permitted and encour
aged to use this code and any derivatives thereof in your
own programs for fun or for profit as you see fit. A sim
ple comment in the code giving credit to the FAQ would be
courteous but is not required.
Man(1) output converted with
man2html