GGI Project Developer tutorial

Author : Christoph Egger
Revision : 1.4
Revision date : 2004-03-02

Introduction

Every large project needs to adhere to some conventions to function effectively. They help maintain a clear sense of focus on the goals and progress of the project, keep trivial distractions down, and in general, coordinate folks. We try to be casual about this in the GGI Project so that nobody gets a chafing feeling, but there are some areas where more rigorous standards must be met (e.g. release cycles.)

These conventions are categorized below as Project-Management, Change-Control and Quality-Management.

Project Management

Those who participate in project management monitor and control the development process.

This means setting up milestones and monitoring the progress towards their completion, as well as ensuring that the Change- and Quality Management- processes are followed.

Change Control

It is usual to maintain multiple versions of files, components, or even of the whole project for an amount of time. This is impossible by hand, in particular when too many people are working on same pieces of software, independent of their tasks. This is what CVS is for.

Nonetheless, maintaining multiple versions can easily be a full-time job. It is important to have a clearly stated and well documented history to help reduce the workload.

ChangeLog

Thus, it is important when committing a patch that the log describes what the patch does as detailed as reasonably possible. Links to messages in a mail archive or a (web-)page are also acceptable, when they describe what the patch does.

Sometimes developers tend to describe specifically how they did what they did instead of what they did (on a broader level.) In order to keep different parts of patches related to a patch easily associatable, it is sometimes necessary for CVS logs to be corrected. See section Tips & Tricks below on how to do that with CVS.

Change control also takes up the job of merging sets of patches between different versions of the project, e.g. moving a bugfix patch out of the development branch and into the stable branch after it is determined that no impact of the patch will violate release cycle rules (e.g. no API changes have occured.)

Versioning

The common shape of a version number is <major>.<minor>.<patch>.

The increase of the patch number means a bugfix only release and happen only in a stable release circle.

The increase of the minor number means the arrival of a new stable release circle. This kind of new version means new features, new targets, tiny new API additions such as new flags, 100% source backward compatibility, but no API changes.

The increase of the major number means the same as the minor version bump plus API changes and backward compatibility is not guaranteed. Documentation (Release Notes) explains what users have to do to make their applications work again.

Quality Management

  • Code - short and easy to read
  • Code - short and easy to understand
  • Code - well structured and designed
  • Code - well tested & documented

Those are the goals of quality management.

Code - readability

Several people have several coding styles. But in a project it is important to have a common coding style. The GGI Project uses the K&R coding style formatted to less than 80 columns with tabs at 8 columns. Not everyone in the project likes K&R, but we find that it is the least annoying for everyone on average.

Code can be converted to K&R style easily with the GNU indent utility:

indent -kr -i8 <files>

Note, that only GNU indent has the -kr option. Other indent variants (i.e. BSD) don't have it.

We try to indent with tabs, not spaces, wherever possible. This allows developers who prefer a different tab indentation to define it in their viewer or editor to an other value. It isn't perfect (especially because they have to watch their total column length) but it does make it easier to read at least (for those that prefer tight indents, that is.)

Code - understanding

Code that is easy to understand is short. It prevents block cascades where possible. This includes using goto statements, which contrary to anything our computer science professors may have said, are not in any way evil.

Example:

BAD                                             GOOD

if (a) {                                        if (!a) {
        if (b) {                                        /* Alternative A */
                if (c) {                                return;
                        /* do it */             }
                } else {                        if (!b) {
                        /* Alternative C */             /* Alternative B */
                }                                       return;
        } else {                                }
                /* Alternative B */             if (!c) {
        }                                               /* Alternative C */
} else {                                                return;
        /* Alternative A */                     }
}                                               /* do it */
return;                                         return;

Understandable code avoids using constants that are not very obvious from their context. It uses #defines instead. Also self-describing function and variable names are very important.

Example:

BAD                                             GOOD

                                                /* blend register index */
                                                #define BLEND   7

                                                /* decal blending */
                                                #define DECAL   0x0f57

array1[7] = 0x0f57;                             registers[BLEND] = DECAL;

Understandable code hides bit mangling in macros or inline functions respectively. Bit mangling is not everyone's favourite. There are really lots of people you can piss off with it. But macros can be used by everyone and are much more maintainable rather than code spreading bit mangling code around.

Example:

BAD                                             GOOD

flag |= ALPHA << (BLEND_TYPE >> 1)              flag |= SETALPHABLENDING

Code - designing

We don't start hacking things up until we have taken some time to specify the goals of what we are trying to do. That is like saying "We know for sure we have to buy a tire for our car, so let's go down to the tire store and get that chore out of the way, then when we get back home all we have left to do is figure out what size tire we need."

We do some research first. There may be code already out there that does what is needed, and just needs a good polishing to make it portable (maintainable, flexible, efficient, etc.) At the very least, reading about and comparing solutions to related problems may point things out to us that lead us to develop a better solution ourselves. This is especially important in the GGI Project because we feel a good amount of the software out there should be consolidated using code from projects like (and including) ours instead of reinventing every little part. It would be hypocritical of us not to even look at other portability-focused projects when we need to implement something new.

Then, carefully and with an effort to predict the future, we try to think how, and when, and by whom, the code will be most directly used. We draw up an API specification which is easy to use but leaves some room for flexibility. This is the perfect time to write the first rough draft of the documentation for the API functions. The very process of documenting brings to the surface minor details that can help improve the API. We also make a point not to forget that sometimes comments in code are not enough -- the inner workings of some algorithms actually have to be explained in plain language, not just how to use the API.

Now that we've got a rough API and rough documentation, we almost certainly have much more than a rough idea. Now is the time for a little peer review. Peer review implies that there is something to be reviewed, so it is important that we have taken the idea and fleshed it out somewhat. Asking for "RFCs" about very vague ideas is an invitation to long meandering mailing list threads that go in circles. Having a proposed API and draft documentation ensures that everyone knows specifically what we are trying to achieve and why.

On more complex projects we sometimes hack up a quick-and-dirty prototype implementation to see if there are hidden problems such as chicken-egg library dependencies or design issues, etc. The GGI Project uses, and sometime abuses, a large number of inter-related dynamic library objects, many of which we do not control, and a prototype implementation, no matter how hasty, usually will bring any unpleasant surprises to the surface sooner rather than later. It will also allow us to begin to integrate the API and code into the portable GGI Project build system, which can be a very annoying and frustrating job to do if you leave it for last, because sometimes it can drag on as long as it took to write the code itself, and worse sometimes changes have to be made to code that was written without the build system in mind.

Along with the final implementation of the API we also write suitable demos and unit tests. As a last step we thoroughly comment the code internally.

A clean implementation centralizes functionality in components. Clean implementation always separates technical code from use cases. Use cases only use technical code, never implements it. Use cases only implement their algorithms. Examples for technical code are access to the hardware and platform-dependant things.

Code - testing

Testing is obviously mandatory. But how do you know if software got thoroughly tested? Simple math tells us that the amount of testing needed to prove code works when treating it as a "black box" goes up exponentially as the code size grows. In addition, in graphics environments, some things just cannot be tested, and in many cases, writing an automated test for something is not at all easy and the tests can take a huge amount of time.

We find it helpful to think of things in the terms of the following quality levels (named differently depending on the country):

  • code compiles and runs without error (lowest)
  • conditions in the code are tested (low)
  • all combinations of conditions in the code are tested (middle)
  • use cases work (high)
  • use cases in various (especially unusual) environments work (highest)

Simple automated tests of small sections of code can achieve level low. Complex automated tests and simple demo applications can achieve level middle to high. The highest quality level can only achieved by complex demos and real applications.

If the code is reasonably designed in the first place, then after automated testing has flushed out most of the bugs, there will only be a few bugs left. Those will be discovered and fixed in time, hopefully by another developer while he is testing his own code, but unfortunately, sometimes by a frustrated end-user.

Statistics

Statistics about the number of open bugs is a good measure about the software quality in general. If the current number of open bugs is above, equal or under the average gives you an orientation about the current quality.

Statistics about the number of support requests shows you how easy a software is to build, install and to use. If the current number of them is above the average you should consider to redesign or rework something respectively.

Note, the average of both statistics says nothing in the beginning, because comparing them to other projects is not logical -- the number of bugs varies with codebase size, user population, and many other factors, like whether you have a really pesky person on your mailing list. The project has to exist for a good amount of time until the average becomes really valuable and new sections of code that are horribly buggy stand out.

Tips & Tricks

Commiting patches

As mentioned above it is important to describe what the patch does. Sometimes you need multiple lines for that. It is possible to the -m option on the commandline, but it is much easier to define your favourite editor in the CVSEDITOR environment variable. It is invoked automatically at commit time.

Back out / Undo previous commit

Say you want to backout revisions 1.5-1.7 of foo.c. Simply do this:

cvs -z3 update -PAd -j 1.7 -j 1.5 foo.c

Note the reversed order of the revisions!

Change a commit log after the commit

Run:

cvs admin -m <revision>:<new log message> foo.c

That's all.

Importing a tree

First import your sources into CVS. Don't forget to choose a unique release-tag (i.e. use the current date):

cvs -z3 import -m "import xxx version a.b from zzz" <repository> <vendor> <release-tag>

where <release-tag> is <vendor>_<version>_<date>. Then resolve the conflicts with:

cvs -z3 <repository> checkout -j <release-tag>:YESTERDAY -j <release-tag> <module>

Make sure, everything compiles then commit the merges:

cvs -z3 commit -m "solve conflicts from previous import"

That's all.

Control multiple versions

Sometimes it happens, that a developer works on multiple tasks.

Usually this occures, when other developers send him/her their code for review. The problem that arises past here to test them without interfering with the versions in your own working tree.

The solution is to create a clean working tree for each task:

cvs -z3 checkout -d <library>-<task> <module>/<library>

Then we have as many fresh checked out working tree's as tasks. We can work on each task without worrying about mixing code from the other tasks.

After you finished a task and you committed it, simply perform a cvs -z3 update -PAd in the other working trees. This merges the latest CVS in.

Finally, you may remove the working tree of the finished tasks - as the work is in CVS.

Maintaining a branch

First create a branch and a branch base tag:

cvs tag -b branch_<xxx>_<yyy>
cvs tag branch_<xxx>_<yyy>_base

This simplifies branch merging later.

Merging patches

Say, you want to merge revisions 1.5-1.6 into the branch. Run:

cvs -z3 update -PAd -r branch_<xxx>_<yyy> -j 1.5 -j 1.6 foo.c

Solve conflicts, then commit it as a usual patch. Note, mention the revisions you pulled up in the log. If you modified the patch by hand apart from solving conflicts, also describe this in the commit log.

This simplifies the life as branch maintainer a lot, as next time you can lookup in the log messages what you merged last time.

Merge a branch into an other one

First create a tag that marks the end of the branch, so that you know that this branch got merged at any time later:

cvs tag branch_<xxx>_<yyy>_final

Then cd into the working tree containing the sources of the code that will recieve the changes.

Then merge the branch:

cvs -z3 update -PAd -r <otherbranchtag> -j branch_<xxx>_<yyy>_base -j branch_<xxx>_<yyy>_final

That's all.