Paul M. Foster (paulf@quillandmouse.com)
Tue, 11 Jan 2000 18:39:53 -0500 (EST)
On Tue, 11 Jan 2000, Kim Lester wrote:
>
> ODE Summary Report #1
> =====================
<snip of excellent summary>
Rather than quoting the prior post and responding point by point, I'll
just weigh in. No, I'm not associated with any of the doc groups
mentioned. Just a concerned Linux user. Those of you not in the Linux
world will hopefully not take offense if I couch things in terms of Linux.
First off, there are two types of ultimate end results here, as I see it.
First, there is the software that will sit on someone's computer and
hopefully be shipped with distros to assist the user in finding what he
wants. Second there is the "web repository" idea. Different thing that
might encompass not only man pages and such, but also email threads, etc.
Much more cumbersome. I presume the idea is the develop system(s) that
works for both areas, perhaps slightly different ones for each area. My
comments will relate to building such a system for local machines, running
on their machines.
Next, do we plan to have any humans do any indexing on existing documents,
or do we want the "indexer" program to be smart enough to figure all this
out?
What should our favored format be? Obviously, opinions vary. However,
since we're trying to index information, and since presentation should be
less important, I'd vote for XML. It is specifically designed to allow
indexing of information, and has incredible flexibility. The writers of
documentation needn't worry with what format we use, since we are
translating from their format into ours. Real issue is what will make our
indexing job the easiest? I'm inclined to think XML. And forget standards
for XML. You actually don't need a DTD for XML, but we can come up with
one if needed. After all, this is our internal format. Now, if you want
others to write to this format, you'll have to make the DTD simple.
How should the user access the information? I think the standard
technology most widely available today is the browser. It works fairly
well for this purpose and is pretty flexible. That way we are not building
yet another tool. That said, we still will need some sort of cgi or other
back end that will eat one format and turn it into another. This also
means that we will need to have a [thin] document server running on the
local machine.
FWIW, I'd request that we make things Lynx-friendly.
It might be instructive to look at the way documentation is handled for
the Debian Linux distribution. If you choose, you can install programs
that allow the search and retrieval of all documentation on the system
via Apache, Lynx and such. It is _similar_ to what we are talking about,
but extremely primitive. For instance, you really need to specify whether
you want to view information from the man files, info files or HOWTOs in
order to find what you want. Mostly, their solution is a backend that uses
off-the-shelf tools to convert man, info and other docs into HTML. As I
said, very primitive, but it works.
Do you want to be able to search the Red Hat package database as
well? Like, "what package is file X in?"
One thing sorely lacking in Linux distros, which was commented on in
previous posts, is the ability to ask, "What programs can I use to look at
my mail with?" And "what does X program in Y directory do?" Quite often,
these programs don't even have a doc file of any kind with them.
Is the plan to index things via some hierarchy, like the way man files are
laid out, or the way the Dewey Decimal System works? If so, then we much
exhaustively generate appropriate categories. Then the question is, can
our indexing technology be smart enough to figure out from the
documentation the categories into which the documentation falls? And what
if we further subdivide the content of a single man page? Does the indexer
do a good job with that as well? I'd love to see some suggestions on what
categories we should use.
As I see it, we're looking at a program or suite of programs that are
installed on a system, which then go out and index the documentation on
the system. Perhaps a cron job. And when the user asks about "foo" and
there is no documentation nor any such program on the computer, the cgi or
whatever says, "Sorry, you might want to search on the web...."
Do we plan to include things like the kernel docs in our indexing? That
is, do we plan to include some of the more arcane, less well-known
documentation sources in our indexing?
Mention was made earlier about categorization in XML being done in
comments. This defeats the purpose of putting it in XML. The tags,
attributes and values need to be part of the ultimate XML document in
order to be of value in indexing.
Language may not be that much of an issue. User defines an environment
variable that specifies his native language, and we serve it to him in
that language if it's available.
Don't know how much importance versioning has in all this. If the user is
using fetchmail 5.0, he probably will not care what fetchmail 4.2
did. He/she will only be interested in the documentation for what's on his
system. Maybe I misunderstand what is meant by versioning here.
Personally, I'd like to see all the docs distributed in ONE format and
skip all the goofy alternate formats. Nothing's worse than calling "man
foo" and finding out that GNU has decided they don't want to maintain the
man pages anymore, and that the real up-to-date stuff is in info. Argh. I
don't like the info format anyway. Just put it all in XML or SGML or
something, and let the user access all of it that way. And if they want to
type "man foo", then what they get is Lynx handing them an XML page for
what they asked for. Of course, I'd like to see most Linux docs
completely rewritten so that one doesn't need three PhDs from four
universities to understand it. Too radical, I know. </rant>
Also, if this catches on, and if the tools for creating docs in our
"native" format are good enough (and free), then we might find that coders
will being writing their docs in our format.
So as I see it,
1) We will work with existing doc formats, subset of which we must
define. (Vote: man, info, SGML, tex, text.)
2) We must define our native format (there's flame-bait!) (Vote: XML.)
3) We must build/find formatters, black boxes that go from format X to
our format.
4) We must define how much or how little work we wish to do in indexing.
Do we want humans going through all the docs, marking them up for the
indexer, or should the indexer be smart enough to figure it all out? Where
do we draw the line or is there one?
5) We must define the categorization system.
6) We must define and find/build the indexer.
7) We must define the user interface. (Vote: browser, specifically Lynx as
the lowest common denominator.)
8) We must define how the user may search. (Vote: text, program name,
keyword, topic/category, filename.)
9) We must prototype the system.
10) We must test it with live users.
11) We must convince other people to use/distribute it.
All these are separate projects.
I assume that we don't just intend to sit around and build standards. I
assume that we intend to actually do some coding at some point. Yes?
If we want this thing to take forever, we should be obsessive about
obtaining everyone's agreement. Otherwise, let's just build a better
mousetrap, get most people to agree, and let the rest do as they like
(they'll probably come around anyway).
I know, this will ruffle some feathers. But look, I head a local Linux
Users Group and own my own business, and I know what it's like to get
everyone to agree. Forget it. On the other hand, look at things like W3C,
where they don't ask a lot of people what they think. Yet their standards
stick. Democracy's overrated. Most open source software was developed (and
quite successfully) by single individuals who didn't really care if anyone
agreed with them or not. So let's have a tight group that can make
decisions and act quickly and decisively. Let's get as much agreement as
possible, but when the inevitable impasse results, we skip the agreement
part and just _decide_. Rightness and correctness have very little to do
with agreement anyway.
My 83 pesos worth.
Paul M. Foster
This archive was generated by hypermail 2.0b3 on Tue Jan 11 2000 - 17:34:44 EST