Return-Path: mdw@cs.cornell.edu
Received: from kruuna.Helsinki.FI (kruuna.Helsinki.FI [128.214.4.112]) by keos.Helsinki.FI (8.6.4/H45) with ESMTP id FAA08388 for <wirzeniu@cs.Helsinki.FI>; Mon, 6 Dec 1993 05:07:49 +0200
Received: from thialfi.cs.cornell.edu (THIALFI.CS.CORNELL.EDU [128.84.254.220]) by kruuna.Helsinki.FI (8.6.4/8.6.4) with SMTP id FAA11574 for <wirzeniu@cc.helsinki.fi>; Mon, 6 Dec 1993 05:07:45 +0200
Received: from CLOYD.CS.CORNELL.EDU by thialfi.cs.cornell.edu (5.67/I-1.99E)
	id AA29258; Sun, 5 Dec 93 22:07:23 -0500
Received: from ROCKY.CS.CORNELL.EDU by cloyd.cs.cornell.edu (5.67/I-1.99D)
	id AA17146; Sun, 5 Dec 93 22:07:30 -0500
Message-Id: <9312060307.AA19818@rocky.cs.cornell.edu>
Received: by rocky.cs.cornell.edu (5.67/N-0.13)
	id AA19818; Sun, 5 Dec 93 22:07:19 -0500
From: mdw@cs.cornell.edu (Matt Welsh)
Date: Sun, 5 Dec 1993 22:07:19 EST
X-Mn-Key: DOC
X-Mailer: Mail User's Shell (7.2.0 10/31/90)
To: linux-activists@joker.cs.hut.fi
Subject: One True Format?
Cc: johnsonm@sunsite.unc.edu, wirzeniu@cc.helsinki.fi (Lasu),
        ijackson@nyx.cs.du.edu, okir@mathematik.th-darmstadt.de (Olaf Kirch)
Status: RO
X-Status: 

Straw Poll: The ``One True Format''

This ``poll'' is a solicitation for comments and ideas about my idea of 
producing a `One True Format' for software documentation. Feel free to 
forward this wherever appropriate; I may post this to USENET newsgroups 
such as comp.text if the response is encouraging. 

Currently, software documentation ranges over many categories, from texinfo
to man pages to printed documents produced with LaTeX, roff, and other tools.
SGML and HTML should be mentioned here, as well, although they aren't as
popular due to the lack of free tools to manage these formats.

Each of the primary documentation tools and languages used today had a 
different purpose in mind at its inception. TeX and LaTeX were meant primarily
or professional typesetting---printed, not online, material. nroff and 
variants are used for both printed and online documentation, but are 
procedural markup languages, and use a somewhat complex format. texinfo was
intended to fill the gap, by producing both printed and online documentation.
Unfortunately, unless one uses special tools, writing texinfo directly can
be time-consuming because it requires the author to do a great deal of work
in structuring the document. 

The technology exisits for there to be a single, simple format for software
documentation---both in printed and online form (hypertext as well as plain
ASCII). Unfortunately, there is no single markup language/documentation format
available to produce all of this. I would like to coordinate an effort to
produce a ``One True Format'' for software documentation. This ``OTF'' would
be mangled by a parser, or converted to other ``target formats'' such as 
TeX, Info, or roff source for formatting. That is, OTF may not be processed
directly into human-readable output; instead, it will be the starting point for
conversion to other well-known formats. More about this later.

Presently, documentors find themselves fumbling around with a large array of
various tools and conversion utilities to get ``just what they need'' in
terms of documentation format. It seems to make sense to start with a single,
well-defined and simple format, and convert to other formats as necessary 
from there. Instead of trying to hunt down your favourite latex2ascii or
texinfo2html or latex2html or html2foobarbaz, you can simply write your 
document in OTF, and produce LaTeX, Info, HTML, what have you---directly. 
OTF should be ``the'' format for writing software documentation. 

The goals:
	* To produce a SIMPLE, LOGICAL markup language for software 
	  documentation. The language must be easy for writers to pick up
	  and use, and not require the writer to do any more work than is 
	  necessary. (E.g., laying out menu items in an Info file.) The 
	  language must be a LOGICAL one, not a procedural one. That is,
	  writers concern themselves primarily with the logical layout of the
	  document---sections, subsections, cross-references, and the like.
	  Procedural details (such as how something looks on the page, or
	  where things are placed in the output) are handled mostly by 
	  the text formatting tools.

	* The language should include the bare minimum number of facilities
	  used in software documentation. Most software docs are limited to:
	  font changes, bulleted and numbered lists, tables, sectioning 
	  commands, cross-references, and "screen output" or "verbatim" text
	  display. There is little need, for example, for mathematical 
	  formulae in software documentation.

	* The OTF should include facilities for the writer to define macros,
	  much like the LaTeX \newcommand facility.

	* The OTF should allow the writer to include "literal" source text 
	  to be passed down untouched to the target format. More on this
	  later.

The major idea is to unify software documentation formats, and to produce
a format which is simple and direct enough that almost anyone can write with
it. I don't want to see OTF become another format which sits in the closet 
and gathers dust from lack of use---as I see it, the documentation 
community (at least, this part of it) is clamoring for a simple, reliable,
and straightforward markup language to produce software documentation with
little or no fuss. With OTF, hopefully, someone can be up and running,
writing docs with it in less than an hour of reading over a tutorial 
(simplicity comparable to HTML). 

Clearly, all of this should be possible. The only major question at this time
is what features the OTF should provide. I gave a list above, including 
sectioning commands, cross-references, and so on. In my experience, there
are very few instances where run-of-the-mill software docs require anything
more esoteric than the features listed above. I would like to hear your 
comments on the matter, to add or delete any items from that list. If we
can define well beforehand the features that the OTF should provide, we will
be in much better luck when trying to specify the format itself.

I expect that the OTF will only be converted to other formats for processing.
That is, there will be no "OTF formatter" itself, but instead a set of
conversion scripts to convert OTF to several "target formats" for processing:
	* TeX/LaTeX (printed documentation)
	* texinfo (or info) for printed docs/hypertext
	* roff (printed docs/plain ASCII)

and any other formats that we may see fit. The above three should be 
sufficient for now; actually, groff and others can produce .dvi and PostScript
for printed documentation just fine, and I see no reason to bother with TeX
unless there is a large call for it. The idea here is to get at least the
bare minimum implemented, and concentrate on "fancy" features when necessary.

Now, each of the above target formats provide analogues for the various
OTF features that I listed: sectioning commands, cross-references, and so on.
If OTF itself is an easy-to-parse format, conversion from OTF to any of the
above target formats should be quite simple. For example, if OTF looks 
something like this (just using LaTeX as a model for now):

	\section{Using ShoopWare v1.3}

it can easily be converted to another format using something as simple as a
Perl script (or lex/yacc if the parsing gets to be hairy). 

Now, because of the limited number of features that OTF will provide, most
documents written in OTF will have a common look-and-feel. For example, OTF
will provide a facility for bulleted lists, but everyone who uses the bulleted
list in OTF will get something that looks the same. As long as we can decide
upon what that bulleted list should look like, most everyone should be 
content with that look and feel. We should keep in mind, however, that the
point is to get information across, in a clear and consistent manner. 

However, for those of us who enjoy being creative, there will be various
ways for someone using OTF to customize the look-and-feel to their needs.
For one, the writer should be able to override any global parameters of
the conversion process (e.g., how much indentation before each item in a 
list? How much vertical space between paragraphs?) For example, I expect
that when using LaTeX as the target format, OTFtoLaTeX will use a LaTeX
style file "otf.sty" for setting up the page layout parameters. The user
can edit otf.sty (or substitute his or her own) to modify the look and feel
of OTF-generated LaTeX documents. 

In addition, OTF should provide a simple mechanism to allow the writer to 
insert unadultered target source in the OTF itself. For example, if the writer 
wants to include a bit of mathematics in the OTF document, they can write:
	
	\iflatex{
	  The running time of this algorithm is $O(n^2)$.
        }

The unprocessed line of LaTeX source will be included directly in the 
target document at this point, if the target format is LaTeX. In this way,
the OTF writer can override any of the silliness that OTF may impose on 
the target source.

In addition, I see no reason why the user could not modify the OTF conversion
scripts themselves to modify the behaviour of the conversion process; for
example, changing OTF bulleted lists to LaTeX numbered lists. OTF should be
simple enough to understand how this conversion will take place. 

As mentioned above, OTF is meant to be simple. The conversion scripts should
take care of almost all of the writer's needs, such as setting up 
page layout parameters, generating menus and hypertext links for Info files, 
and so on. All that the writer will have to concern himself with is the
content and structure of the document. 

Clearly, OTF is not the kind of thing that you would write a book or thesis
with. It's intended as a tool to generate online and printed documentation
with little unnecessary work by the writer. Things such as FAQs, HOWTO 
documents, and even manuals from the LDP could be written in OTF (although
the LDP manuals approach the definition of `book', they generally fall under
the same category as the other software documentation that OTF intends to 
handle). 

I imagine that 90% of software documentation could be written using the 
limited number of features that OTF will provide. And, because OTF will allow 
the writer to cheat if necessary, it should be able to handle the rest. 

I would like to receive feedback and questions from you on the above 
proposal. The purpose of this message is to shape and mold the goals and
technical details of OTF, and hopefully move towards implementation. Once
I know how people feel about the format, and what writers need out of a
software documentation markup language, I can write up a technical spec for
OTF itself, and start to write code (with help from volunteers, of course!). 
OTF is not just "my" idea or "my" format---I want this to be potentially 
useful to a wide array of programmers and writers. As I see it, most of the
work of producing software documentation can be done with the computer. 
After all, that's what they're made for.

Specifically, comments on the following would be helpful:
	* General comments on OTF itself... good idea/bad idea? What about
	  my outlined goals? Is this at all feasable? (I think that it is.
	  In fact, OTF is designed to be easy to write in AND easy to 
	  parse by the machine.) 

	* What kinds of features do you want to see in a software documentation
	  markup language? I've already suggested sectioning commands,
	  cross-references, lists, and so on. Anything else?

	* How should the markup language LOOK? I like using the LaTeX model,
	  because it is easy to parse AND easy for the author to understand. 
	  Of course, OTF will be significantly simpler than LaTeX. 

	* What kinds of target formats should we use? At first, I'd like to
	  aim for groff (it can produce printed docs as well as plain ASCII)
	  and texinfo. LaTeX should also be quite simple, as would be 
	  HTML/SGML.

	* Any other comments or questions are welcome.


Thanks for your time,
mdw
