Subject: Re: New source packaging format - requirements and proposal

From: Ian Jackson <[email-masked]>
Date: Thu, 4 Jul 96 15:15 BST

[email-masked] writes ("Re: New source packaging format - requirements and proposal"):
> Unfortunately, the current discussion seem to focus more on the design
> of the source package format than an analysis of our basic
> requirements.  I don't feel I can usefully contribute to the
> discussion as it's currently progressing.  Yet, I worry that the
> result will be hard for me to understand, and hard for me to adapt to.

Hmm.

> Here's my take on the packaging format:
> 
> The requirements have to be easy to understand.  Convenience is nice
> (I happen to be rather fond of convenience), but understandability
> is paramount.

I take it you mean that the _format_ has to be easy to understand ?

> In particular, since we already have a broad base of debian source
> packages, there's going to have to be a period of transition.

Right.

> A more uniform package format would allow us to automate various sorts
> of things on a larger scale.  This could be nice.

I think the chief problems I'm trying to solve are:

- There is not enough information in a source package to tell you what
it is, what it will generate, &c.  This is already causing problems
for eg the bug system.

- We need a way to rebuild a Debian source package without having to
re-download the original source.

- We need to be able to do automatic processing.

> A more flexible package format would allow us to bring quickly into
> debian a wide array of software.  This could be nice.

What kind of software do you think isn't catered for atm ?

> [Aside: I have one source package which I put together so that I could
> just type one command and it would check the original ftp site for a
> new version -- and, if present, would update all relevant files and
> build new packages for me.  It wouldn't have been hard to go one step
> further and build a wrapper that would automatically upload the result
> and mail out the change announcement, but I left that out for a couple
> of reasons.  I've not upgraded the package recently, because I'm not
> sure that it still meets the requirements of a debian package, and
> I've not taken the time to sort out what that means.  No one has filed
> a bug report on either side of this issue, so maybe it's not an issue
> to anyone else.]

This looks like something we could make optional.  If we define a
standard way for people to provide this functionality we can see if
it's useful :-).

> Often my worries are groundless -- but I do want to recommend that
> the requirements of the format be easy to document and understand.

Oh, you mean that there should be a good spec. for the format.

Yes, obviously.

Ian.

---

From: Ian Jackson <[email-masked]>
Date: Mon, 8 Jul 96 14:54 BST

Manoj Srivastava writes ("Re: Source packaging - alternatives"):
> 	Sometimes a diff file alone is not enough to change the
>  original sources to the debianized sources, permissions on
>  debian.rules is one that has been noted. I remember interim
>  "unofficial" version of perl being distributed as shell scripts that
>  performed a few actions (deleting/renaming files, etc), unshared to
>  the [atch file, and applied the patch.
> 
> 	Could this be useful? The dsc file could contain the
>  information just as Ian proposed, but say
>  % ./package.dsc 
>  to generate the debian sources (with proper permissions for
>  debian.rules) from the orig and diff files automatically.

Absolutely not.  As I said in my posting titled `New source packaging
format - requirements and proposal' (23rd June):

 7. Unpacking a source file must not require the user to trust it - ie,
 no parts of the source package must be `executed'.  For example, any
 diffs must be checked only to edit real files in the source package,
 and scripts containing arbitrary shell commands are right out.

Ian.

---

From: Ian Jackson <[email-masked]>
Date: Mon, 8 Jul 96 01:49 BST

Lars Wirzenius writes ("Re: New source packaging format - requirements and proposal "):
...
> One more idea that would be nice: PGP signing all packages (not just source).
> While the announcements are PGP signed by most people, detached signatures
> (foo.deb.pgp, foo.tar.gz.pgp, foo.diff.gz.pgp, or possibly embedded into the
> package format) would make it easier to check things automatically.

We already have room in the package format for this.  I need to think
a bit more about exactly how this should be formatted, but basically
one of the main reasons behind the new package format is to allow
packages to contain signatures (and certification chains and md5sums
and any other guff).

Below is a byzantine scheme Steve Early and I worked out for automatic
key distribution and verification of packages.

...
> Disagreements?  I know Ian prefers the single-file-special-format approach,
> and I confess I'm also attracted to it, but I'm not pushing in either
> direction.  If we for separate files, however, we'll have to make sure
> all files that belong together can be downloaded with a single, simple
> mget (especially since this is trivial to do).

I'm coming round to this view, as I wrote earlier.  Having one of the
files give the md5sums of all the others means that you can't get them
out of step.

Ian.

------- start of forwarded message (RFC 934 encapsulation) -------
Message-ID: <[email-masked]>
From: Stephen Early <[email-masked]>,
      Ian Jackson <[email-masked]>
To: [email-masked]
Subject: Package authentication proposal
Date: Tue, 20 Feb 96 04:17 GMT

This bit of text (written in NPML [no particular markup language])
describes a system for ensuring that the packages we distribute are
not tampered with between leaving the maintainer and arriving at the
machine where they are to be installed.

Steve Early           Ian Jackson
[email-masked]     [email-masked]

\section{Keys}

There are three master keys, held offline in geographically separate
locations.

There are a number (about three, but definitely no more than five) of
certification keys, held (perhaps online) by separate important
developers.

Each package maintainer has a key.

There is a 'validity' key, held online in dedicated secure
hardware. This could be a separate, non-networked machine connected to
a networked front-end machine (not necessarily dedicated) by a serial
cable. It need not be securely connected to the primary ftp site.

\section{Certification}

Each package is signed by the package maintainer.

The package maintainer's key is signed by the nearest certification key.

Each certification key must be signed by at least two of the master
keys; usually it will be signed by all three.

The validity key is signed by all three of the master keys. This
certificate includes the date of the last revocation of a validity
key.

When a package is received by the archive its details are forwarded to
the validity server's front-end. This posts an announcement on
debian-devel and submits the certification information to the validity
server.  This delays for 24 hours; if the maintainer has not
repudiated his signature after 24 hours then a certificate is applied
using the validity key and returned to the archive site. This
certificate binds the certification key, the package maintainer's key
and the checksum of the package as found in the package maintainer's
signature.

If a package needs to be inserted into the archive within 24 hours
(security bugfixes, for example) then verification of the package
checksum and status of the maintainer's key must be obtained through
external means. Each package maintainer must provide a telephone
number and hours at which they can be contacted. The validity server
maintainer must call the package maintainer (_not_ vice versa).

All of the certificates have timestamps, but apart from the timestamps
in the master keys' signature on the validity key these are not
used. Timestamps cannot generally be used as this would, for example,
make CD-ROMs go 'out of date'.

\section{Distribution of keys and certificates}

Each package contains its own signature from the package maintainer's
key, the certification on the package maintainer's key from the
certification key, the certification key itself, and the certificates
on the certification key from the master keys.  It also contains the
certificate from the validity key, and the validity key and its
certificate from the master keys. This will add around 4k to the size
of a package.

The master keys are contained in the dpkg package, which also contains
a list of revoked keys and packages (though since these revocations
cannot be guaranteed to arrive they are not required for the security
of the system).

\section{Revocation}

If a package is issued in error we either live with it or have the
maintainer revoke their key.

If a maintainer's key or a certification key is compromised or needs
to be changed, it is revoked with the validity server; a certifcation
key or the master keys, respectively, must sign the new key.

If a maintainer's key compromise goes undetected for 24 hours and a
bad package is issued the validity key revocation procedure is
followed.

If the validity key is compromised then an announcement is posted
widely, asking users to run `dpkg-auth --revoke-validity-key' and give
the date of the announcement.  This updates the system's idea of when
the most recent validity key revocation occurred, so that only
validity keys certified by the master keys after this date are
accepted.  (This data, held on the system in /var/lib/dpkg, is also
updated when a more recent last revocation timestamp is found in a
package---see above.)

If a single master key is compromised we widely announce a new version
of dpkg with the new master keys and revocations and ask users to
upgrade urgently.

If a single master key is lost we release a new version of dpkg with
the new master keys.

If several master keys are compromised we widely announce that this
has happened, and widely publish the MD5 checksum of the new version
of dpkg that users are required to install.

If several master keys are lost simultaneously we widely announce that
this has happened, and users must use dpkg with special options to
update to a dpkg version with new master keys.

\section{Caveat}

Note that this doesn't protect against package maintainers providing
bad packages (maliciously or through bad upstream source), but it does
ensure that every Debian user sees the same packages and that bad
packages can be traced.

------- end -------