Subject: Search Engine

From: "Brian C. White" <[email-masked]>
Date: Fri, 19 Jul 1996 12:39:07 -0400

[ I posted this to debian-private, but never saw it come through. ]

My company has a search engine we have put together.  It is intended for
small to medium sized sites containing thousands of documents totalling
tens of megabytes, though it does scale quite well.  While not limited
to the Web in any way, it does work very well with it.

While it is a commercial product, it is free to non-profit and educational
instituions.  There will also be freely available evaluation versions.

Being somewhat of a Debian fan <grin>, I'd like to make it available to
search the Debian web site and its bug system.  We, of course, get some
free advertising out of it.

Version 1.0 is due to be released Monday and is in final testing right now.
Included is a nifty front-end to the query system and a fairly-functional
web crawler.

Would Debian like to make use of this?

                                        Brian
                               ( [email-masked] )

-------------------------------------------------------------------------------
    In theory, theory and practice are the same.  In practice, they're not.

---

From: "Brian C. White" <[email-masked]>
Date: Fri, 19 Jul 1996 18:27:48 -0400

> As you may know, there is a search engine in the docdebian package
> that Lars is building, and there is a glimpse-based search engine in
> the Debian WWW pages.  I think we ought to get coordinated on whatever
> we do (I don't have a stake in anything.)

I don't have specs for them, so I can't compare them for speed and the like.

> Does your search engine do these things:
>  Boolean operators, including AND NOT

Yes.  It uses "&" (and), "|" (or), and "!" (not).  Also includes
paretheses to prioritize certain operations.

>  Proximity matching, e.g., Susan NEAR Kleinmann

Yes.  Use '"Susan Kleinmann"' to match next to, or 'Susan 10 Kleinmann' to
search for within 10 words of each other.

(The web front-end makes easy selection of operators using words instead
 of symbols, if desired.)

>  Case (in)sensitivity

Yes.  "susan" to match case insensitive.  Any uppercase letters will force
an exact case match of the word.

>  substrings

Yes.  It supports the standard command line "globbing" characters: *,?,[]

>  Fielded searches

Yes, but you first have to break your document into fields and index each
field.  The results of individual queries on each field can then be combined.
This feature is still in alpha, but should be done soon.

>  phrases

Yes.  A phrase '"Susan Kleinmann"' is identical to 'Susan 1 Kleinmann'.

> The output format is also important (obviously).  And I think we'd
> need to build it into a form that allowed a user to zero-in on a
> particular dataset, like the mailing lists, the bug log, the
> (newly expanded!) FAQ, the files in /usr/doc, and the files in /usr/info,
> etc., etc.

It's a Perl extension library so you can format the results any way
you want.  The matches are also scored based on how well they matched.

We do have a really cool Web front end to it, though!  Come monday, it
will be up on our website for you to look at.

> I'm interested!

This is good!

                                        Brian
                               ( [email-masked] )