perlhack - How to hack at the Perl internals


DESCRIPTION

       This document attempts to explain how Perl development
       takes place, and ends with some suggestions for people
       wanting to become bona fide porters.

       The perl5-porters mailing list is where the Perl standard
       distribution is maintained and developed.  The list can
       get anywhere from 10 to 150 messages a day, depending on
       the heatedness of the debate.  Most days there are two or
       three patches, extensions, features, or bugs being dis­
       cussed at a time.

       A searchable archive of the list is at:

           http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/

       The list is also archived under the usenet group name
       `perl.porters-gw' at:

           http://www.deja.com/

       List subscribers (the porters themselves) come in several
       flavours.  Some are quiet curious lurkers, who rarely
       pitch in and instead watch the ongoing development to
       ensure they're forewarned of new changes or features in
       Perl.  Some are representatives of vendors, who are there
       to make sure that Perl continues to compile and work on
       their platforms.  Some patch any reported bug that they
       know how to fix, some are actively patching their pet area
       (threads, Win32, the regexp engine), while others seem to
       do nothing but complain.  In other words, it's your usual
       mix of technical people.

       Over this group of porters presides Larry Wall.  He has
       the final word in what does and does not change in the
       Perl language.  Various releases of Perl are shepherded by
       a ``pumpking'', a porter responsible for gathering
       patches, deciding on a patch-by-patch feature-by-feature
       basis what will and will not go into the release.  For
       instance, Gurusamy Sarathy is the pumpking for the 5.6
       release of Perl.

       In addition, various people are pumpkings for different
       things.  For instance, Andy Dougherty and Jarkko
       Hietaniemi share the Configure pumpkin, and Tom Chris­
       tiansen is the documentation pumpking.

       Larry sees Perl development along the lines of the US gov­
       ernment: there's the Legislature (the porters), the Execu­
       tive branch (the pumpkings), and the Supreme Court
       (Larry).  The legislature can discuss and submit patches

       branch is free to veto them.  Rarely, the Supreme Court
       will side with the executive branch over the legislature,
       or the legislature over the executive branch.  Mostly,
       however, the legislature and the executive branch are sup­
       posed to get along and work out their differences without
       impeachment or court cases.

       You might sometimes see reference to Rule 1 and Rule 2.
       Larry's power as Supreme Court is expressed in The Rules:

       1   Larry is always by definition right about how Perl
           should behave.  This means he has final veto power on
           the core functionality.

       2   Larry is allowed to change his mind about any matter
           at a later date, regardless of whether he previously
           invoked Rule 1.

       Got that?  Larry is always right, even when he was wrong.
       It's rare to see either Rule exercised, but they are often
       alluded to.

       New features and extensions to the language are con­
       tentious, because the criteria used by the pumpkings,
       Larry, and other porters to decide which features should
       be implemented and incorporated are not codified in a few
       small design goals as with some other languages.  Instead,
       the heuristics are flexible and often difficult to fathom.
       Here is one person's list, roughly in decreasing order of
       importance, of heuristics that new features have to be
       weighed against:

       Does concept match the general goals of Perl?
           These haven't been written anywhere in stone, but one
           approximation is:

            1. Keep it fast, simple, and useful.
            2. Keep features/concepts as orthogonal as possible.
            3. No arbitrary limits (platforms, data sizes, cultures).
            4. Keep it open and exciting to use/patch/advocate Perl everywhere.
            5. Either assimilate new technologies, or build bridges to them.

       Where is the implementation?
           All the talk in the world is useless without an imple­
           mentation.  In almost every case, the person or people
           who argue for a new feature will be expected to be the
           ones who implement it.  Porters capable of coding new
           features have their own agendas, and are not available
           to implement your (possibly good) idea.

       Backwards compatibility
           It's a cardinal sin to break existing Perl programs.

           that emits warnings is not broken, while others say it
           is.  Adding keywords has the potential to break pro­
           grams, changing the meaning of existing token
           sequences or functions might break programs.

       Could it be a module instead?
           Perl 5 has extension mechanisms, modules and XS,
           specifically to avoid the need to keep changing the
           Perl interpreter.  You can write modules that export
           functions, you can give those functions prototypes so
           they can be called like built-in functions, you can
           even write XS code to mess with the runtime data
           structures of the Perl interpreter if you want to
           implement really complicated things.  If it can be
           done in a module instead of in the core, it's highly
           unlikely to be added.

       Is the feature generic enough?
           Is this something that only the submitter wants added
           to the language, or would it be broadly useful?  Some­
           times, instead of adding a feature with a tight focus,
           the porters might decide to wait until someone imple­
           ments the more generalized feature.  For instance,
           instead of implementing a ``delayed evaluation'' fea­
           ture, the porters are waiting for a macro system that
           would permit delayed evaluation and much more.

       Does it potentially introduce new bugs?
           Radical rewrites of large chunks of the Perl inter­
           preter have the potential to introduce new bugs.  The
           smaller and more localized the change, the better.

       Does it preclude other desirable features?
           A patch is likely to be rejected if it closes off
           future avenues of development.  For instance, a patch
           that placed a true and final interpretation on proto­
           types is likely to be rejected because there are still
           options for the future of prototypes that haven't been
           addressed.

       Is the implementation robust?
           Good patches (tight code, complete, correct) stand
           more chance of going in.  Sloppy or incorrect patches
           might be placed on the back burner until the pumpking
           has time to fix, or might be discarded altogether
           without further notice.

       Is the implementation generic enough to be portable?
           The worst patches make use of a system-specific fea­
           tures.  It's highly unlikely that nonportable addi­
           tions to the Perl language will be accepted.

           Patches without documentation are probably ill-thought
           out or incomplete.  Nothing can be added without docu­
           mentation, so submitting a patch for the appropriate
           manpages as well as the source code is always a good
           idea.  If appropriate, patches should add to the test
           suite as well.

       Is there another way to do it?
           Larry said ``Although the Perl Slogan is There's More
           Than One Way to Do It, I hesitate to make 10 ways to
           do something''.  This is a tricky heuristic to navi­
           gate, though--one man's essential addition is another
           man's pointless cruft.

       Does it create too much work?
           Work for the pumpking, work for Perl programmers, work
           for module authors, ...  Perl is supposed to be easy.

       Patches speak louder than words
           Working code is always preferred to pie-in-the-sky
           ideas.  A patch to add a feature stands a much higher
           chance of making it to the language than does a random
           feature request, no matter how fervently argued the
           request might be.  This ties into ``Will it be use­
           ful?'', as the fact that someone took the time to make
           the patch demonstrates a strong desire for the fea­
           ture.

       If you're on the list, you might hear the word ``core''
       bandied around.  It refers to the standard distribution.
       ``Hacking on the core'' means you're changing the C source
       code to the Perl interpreter.  ``A core module'' is one
       that ships with Perl.

       The source code to the Perl interpreter, in its different
       versions, is kept in a repository managed by a revision
       control system (which is currently the Perforce program,
       see http://perforce.com/).  The pumpkings and a few others
       have access to the repository to check in changes.  Peri­
       odically the pumpking for the development version of Perl
       will release a new version, so the rest of the porters can
       see what's changed.  The current state of the main trunk
       of repository, and patches that describe the individual
       changes that have happened since the last public release
       are available at this location:

           ftp://ftp.linux.activestate.com/pub/staff/gsar/APC/

       Selective parts are also visible via the rsync protocol.
       To get all the individual changes to the mainline since
       the last development release, use the following command:

           rsync -avuz rsync://ftp.linux.activestate.com/perl-diffs perl-diffs


           rsync -avuz rsync://ftp.linux.activestate.com/perl-current perl-current

       Needless to say, the source code in perl-current is usu­
       ally in a perpetual state of evolution.  You should expect
       it to be very buggy.  Do not use it for any purpose other
       than testing and development.

       Always submit patches to perl5-porters@perl.org.  This
       lets other porters review your patch, which catches a sur­
       prising number of errors in patches.  Either use the diff
       program (available in source code form from
       ftp://ftp.gnu.org/pub/gnu/), or use Johan Vromans'
       makepatch (available from CPAN/authors/id/JV/).  Unified
       diffs are preferred, but context diffs are accepted.  Do
       not send RCS-style diffs or diffs without context lines.
       More information is given in the Porting/patching.pod file
       in the Perl source distribution.  Please patch against the
       latest development version (e.g., if you're fixing a bug
       in the 5.005 track, patch against the latest 5.005_5x ver­
       sion).  Only patches that survive the heat of the develop­
       ment branch get applied to maintenance versions.

       Your patch should update the documentation and test suite.

       To report a bug in Perl, use the program perlbug which
       comes with Perl (if you can't get Perl to work, send mail
       to the address perlbug@perl.com or perlbug@perl.org).
       Reporting bugs through perlbug feeds into the automated
       bug-tracking system, access to which is provided through
       the web at http://bugs.perl.org/.  It often pays to check
       the archives of the perl5-porters mailing list to see
       whether the bug you're reporting has been reported before,
       and if so whether it was considered a bug.  See above for
       the location of the searchable archives.

       The CPAN testers (http://testers.cpan.org/) are a group of
       volunteers who test CPAN modules on a variety of plat­
       forms.  Perl Labs (http://labs.perl.org/) automatically
       tests Perl source releases on platforms and gives feedback
       to the CPAN testers mailing list.  Both efforts welcome
       volunteers.

       To become an active and patching Perl porter, you'll need
       to learn how Perl works on the inside.  Chip Salzenberg, a
       pumpking, has written articles on Perl internals for The
       Perl Journal (http://www.tpj.com/) which explain how vari­
       ous parts of the Perl interpreter work.  The `perlguts'
       manpage explains the internal data structures.  And, of
       course, the C source code (sometimes sparsely commented,
       sometimes commented well) is a great place to start (begin
       with `perl.c' and see where it goes from there).  A lot of
       the style of the Perl source is explained in the


       It is essential that you be comfortable using a good
       debugger (e.g. gdb, dbx) before you can patch perl.  Step­
       ping through perl as it executes a script is perhaps the
       best (if sometimes tedious) way to gain a precise under­
       standing of the overall architecture of the language.

       If you build a version of the Perl interpreter with `-DDE­
       BUGGING', Perl's -D command line flag will cause copious
       debugging information to be emitted (see the `perlrun'
       manpage).  If you build a version of Perl with compiler
       debugging information (e.g. with the C compiler's `-g'
       option instead of `-O') then you can step through the exe­
       cution of the interpreter with your favourite C symbolic
       debugger, setting breakpoints on particular functions.

       It's a good idea to read and lurk for a while before chip­
       ping in.  That way you'll get to see the dynamic of the
       conversations, learn the personalities of the players, and
       hopefully be better prepared to make a useful contribution
       when do you speak up.

       If after all this you still think you want to join the
       perl5-porters mailing list, send mail to perl5-porters-
       subscribe@perl.org.  To unsubscribe, send mail to
       perl5-porters-unsubscribe@perl.org.


AUTHOR

       This document was written by Nathan Torkington, and is
       maintained by the perl5-porters mailing list.


Man(1) output converted with man2html