Re: [oswg-dis] Re: Open Document Environment (ODE)


Aaron Turner (aturner@linuxkb.org)
Mon, 10 Jan 2000 00:54:08 -0800 (PST)


On Sun, 9 Jan 2000, Deb Richardson wrote:

> Aaron Turner wrote:
>
> > Personally I would be very interested in getting the Free/Open/NetBSD and
> > other groups involved in such a project. I however also feel (rather
> > strongly) that content should be compartimentalized so that when I'm
> > looking for getting Apache to run on Linux I don't keep running into
> > FreeBSD centeric docs. Of course these compartments can overlap, but we
> > need to be able to split it up for the end user.
>
> This is where things get complicated. Who should document Apache, for
> example? The Apache Group? The LDP for Linux? The FreeBSD for
> FreeBSD? Etc. Logically, of course, Apache should be documented by the
> Apache group. Additional documentation can be written to further
> enhance existing docs -- enhancements by the LDP for Linux, by the
> FreeBSD folks for their OS, etc. There could even be separate
> expansions written for each separate Linux distribution -- "Apache on
> Red Hat", "Apache on Debian", etc.

In a perfect world that would be reasonable. And for large projects such
as Apache it probably is even feasible. However there are a huge number
of small projects where the documentation is more likely to come from an
interested third party. Trying to compartamentalize such docs without a
governing body of some sort is impossible. And there will be a lot of
such documents (short FAQ's) like this.
 
> Should all of these documents be stored in the same place and maintained
> by the same group? No. Apache docs belong with the Apache projects,
> LDP docs with the LDP, and so on.

Why? The licenses for the content definately allows me to take it, then
index it, and stick it on the same site with each other. Taken another
way there are advantages for Apache, thttpd, phttpd, and other web servers
to have all their documentation in the same place- especially since often
you're looking for a webserver to solve a problem, not just how to do
something in a specific web server.

> On the other hand, a centralized "card catalog" like that the Open
> Source Research Team (OSRT) is working to develop, should and could
> effectively index and cross reference all of these documents. The docs
> aren't in the same location, nor are they controlled or updated by the
> same group. Instead, all documentation projects are working together
> with the OSRT to develop and maintain the meta-information that goes
> into the card catalog. I really like where the OSRT is going with this,
> and am very interested in working with them to make it a reality.

There are definately advantages of a card catalog system. We at the Linux
KB Project thought of doing something more along those lines, but for
various reason decided on hosting content on our own servers. The biggest
reason for that was insuring the co-operation of each of the sites would
be critical or you'd end up with a database of broken links. Either way
is valid, depending on your ability to impliment it well- each have their
strenghts and pitfalls.

> The docs will already be individuated by OS, package, etc., simply by
> belonging to different projects and existing on different sites. What
> the OSRT card catalog promises is the ability to quickly and easily find
> the information you need through a centralized repository of
> meta-information that is cooperatively maintained by all these different
> projects. Simple is better. The OSRT is working on a simple but very
> powerful solution.

I took a quick look. One problem I have is the speed of the current
engine:

Query: apache and alias

Matching Record Count: 0
Total Retrieved: 0
Interpreted Query: apache:1 and alias:1
Total Database Records: 5329
Query Time: 26 seconds

Half a minute to query 5K records (2.5MB)? Ouch. I'm sure things can and
will be optimized, but it seems to me that a lot of work needs to be put
into the backend storage and retrieval. But overall I like where OSRT is
going- it's a perfect example of what happens when the content and the
retreival engine are well integrated.
 
> > Wouldn't using the LSB or LDP exclude the FreeBSD'ers as those are Linux
> > specific organization? I'm confused about the relationships between the
> > various groups as you propose.
>
> I specifically said that the LSB should be treated as the standards base
> for Linux. I didn't suggest that it be used for/by other projects. If
> the FreeBSD folks want to use the same standards, they can. If they
> don't, they don't have to. If there were only one Linux-related
> documentation project, then that documentation project would be the
> logical place to develop and promote Linux documentation standards. But
> there are multiple Linux-related documentation projects, just as there
> are multiple distributions, etc. Similarly, if there were only one
> Linux distribution, there would be no need for an LSB.

Ok, things are a bit more clear for me now.
 
> As far as I know, there is only one FreeBSD-related documentation
> project. (I could be wrong about this, and folks should correct me if
> I'm in error). As such, the FreeBSD DP is the logical organization for
> developing documentation standards for FreeBSD.
>
> Now, if we have open lines of communication between the various
> standards-developing organizations (ie: The LSB, the FreeBSD DP, the
> GNOME DP, etc), then these organizations can discuss chosen or proposed
> standards as a group. But we don't need another organization to do
> this...we don't need another committee. All we need is a mailing list
> that interested people can participate in. Simple is better.

As long as we both agree that the priniciple players from each group needs
to get together. I would prefer some kind of basic organization because
of the diversity of the people/groups involved. But a simple email list
is a good start and enough for now.

> > As long as you have an open standard, others can write tools for it. HTML
> > is such an example.
>
> HTML is "sort of" a standard.

yeah, yeah, i know- you just had to get all technical on me didn't ya. :)

[snip about SGML and Docbook]

> > For me at least, I've always been interested in a system where the content
> > and retrevial engine are very integrated. It still can and should be
> > open, but such a system has many benifits for the end user. Realize that
> > I'm not talking about so much about the markup of the document, but rather
> > the storage and retrieval of the document itself.
>
> Hm. Could you explain this a bit more? I think I know what you're
> getting at, but I want to make sure before I comment.

Writing good documentation is not enough. People who need it need to be
able to find it for it to be of any use. Simple search engines like Yahoo
and Altavista which can't do more than simple keyword or at best phrase
searching isn't enough. You need to be able to read a file and assign
context to the words- title, author, problem, answer, etc. That way when
someone does a search the results can be much more meaningful. The OSRT
is a good example of this. The LKBP will be another one, though not as
good as we have much less control over the format of the document.
  
> > I would assume though that it would be quite
> > possible for someone to develop a web form or the like that the writer can
> > fill out to create a DocBook compliant article for the small stuff.
>
> It's possible, but would be unnecessarily complicated. DocBook allows
> you to mark up far more than just "author", "title" and "paragraph".
> You can mark up filenames, directory names, screenshots and details,
> tables, tables of contents, index words, lists, commands, command-lines,
> command-line options, replaceable values, environment variables...and so
> on and so on.
>
> Creating a web-based editor for even the simplest of DocBook documents
> would be significantly complicated. Also, many people (myself included)
> don't care for web-based editing tools (it's my biggest gripe with Zope,
> actually). Finally, there are a number of editing tools already
> available that are being improved to better support DocBook, including
> psgml (emacs), and LyX. Again, simple is better, and in this case it
> would be unnecessary to reinvent the wheel.

As long as there is a variety of tools available that make it easy, then
okey.
 
> Mozilla is going to offer some very interesting possibilities for the
> creation of an editor and processing tools, as well. Mozilla is far
> more than just a web browser.

That would be prefereable to LyX or emacs- editor wars and all.

> > I've seen situations where the author chose not to pick a good license and
> > it ended up causing a lot of problems.
>
> Unfortunately, that's simply part of the game. Sometimes people will
> choose bad licenses. And they are (and must remain) free to do so.
> This whole thing is about freedom. We cannot lose sight of that.

We can't make anyone do anything. But if you get a large group of
documentation organizations who can all agree on a short list that will be
enough to sway most. One can strongly suggest and give preference to
to documents that agree to those terms. I personally don't mind making
"invalid licensed docs" 2nd class citizens for the greater good.

> > Also lack of standardized licenses prevents a centralized repository which
> > is desperately needed. You can't expect end users to look at 20 different
> > sites looking for one answer just because some dufus couldn't pick a good
> > license.
>
> We don't need a centralized repository, we need a centralized index.
> The Open Source Research Team is making very exciting progress in this
> area with their meta-information project.

A centeralized index is a positive move, but I'd like to see more. A big
reason for that is you can then do things like burn a CD of it and give it
to people who don't have cheap Internet access or the patience to download
it. Giving someone with limited or expensive Internet access an index
without the actual documents is only a cruel tease. :) Also web site
admins have this nasty habit of breaking links. Again a repository will
be less impacted by this than a index. Last, doing things like keyword
indexing for later searching is MUCH slower over the Internet at large
than if you have it local. Can you imagine the problems you'd have trying
to have something like Ht://Dig index 1,000's of documents over 100's of
sites?
  
> > I guess I don't know how to do the standardization required to pull this
> > off without creating a dedicated oversight committee to manage it.
>
> It's simply a matter of scope. We do not need a centralized
> repository. We do not need to standardize licenses. What we need to do
> is discuss the possibility of cooperatively developing a set of open
> standards for Open Source documentation. Within Linux, that can be the
> LSB. Within the Open Source community as a whole, all we need is a
> mailing list on which a sufficient number of different projects will
> actively discuss these issues.
>
> Another committee is unnecessary, and would only serve to further
> complicate the situation, both practically and politically.

Ah, now there's a good discussion- Just what is the scope of this?
Obviously my idea is much bigger than yours. I'm curious to hear about
what other people think. Just because something is more complicated
doesn't mean we shouldn't try because it's easier.
 
> > That's one reason why I'm against a distributed system. You can build
> > such a system in a monolithic manner. Making it distributed is
> > significantly more complicated.
>
> Not really. Open Source documentation already exists in a distributed
> system. All we need to do is develop a centralized index so people can
> easily find the information they're looking for. The OSRT
> (http://www.metalab.unc.edu/osrt) has already made significant progress
> towards making this a reality.

The current OS documentation while it is distributed is frankly a horrible
system. It's just a hodge-poge of websites linked together in a
half-hazzardly way dependant on third parties such as Yahoo to index them
as a whole. It doesn't scale nor does it provide the user a consistant
interface for finding information.
 
> > It's already unmanageable- do a search on Yahoo for some linux related
> > question- "Linux pop3 server" returns 13,000+ hits.
>
> Thus the need for the OSRT's card catalog. It's more than just a search
> engine. Their meta-information project promises to create a
> comprehensive and extremely powerful resource for open source
> documentation. As Paul mentioned, they're politically and
> technologically neutral, and they're working with open standards to
> produce some remarkable tools.

OSRT does address many of the issues that plauge the current system, but
in some ways I don't feel it goes far enough.

> Really, you should go check it out :)

I'm looking :)
 
> (Paul, maybe you could give us a rundown about the project? I'd also be
> interested in knowing how the meta-info project is progressing and if
> there's any way we (that's a rather general "we") can help).

That would be useful and interesting to me as well.

--
Aaron Turner, Core Developer       http://vodka.linuxkb.org/~aturner/
Linux Knowledge Base Organization  http://linuxkb.org/
Because world domination requires quality open documentation.
aka: aturner@vicinity.com, aturner@pobox.com, ion_beam_head@ashtech.net
The difference between `Unstable' and `Usable' is only two characters: NT



This archive was generated by hypermail 2.0b3 on Mon Jan 10 2000 - 03:54:51 EST