Dissertation Information for Eric Christian Van Vliet NAME: - Eric Christian Van Vliet
DEGREE:
- Ph.D.
DISCIPLINE:
- Library and Information Science
SCHOOL:
- University of California, Berkeley (USA) (1983)
ADVISORS: - Michael D. Cooper
COMMITTEE MEMBERS: - Patrick Wilson - James Alexander Reeds
MPACT Status: Fully Complete
Title: EVALUATION OF DOCUMENT RETRIEVAL RANKING ALGORITHMS
Abstract: This thesis presents a method of evaluating the ranking algorithms that determine the output of machine based bibliographic retrieval systems. The method is based on a general theoretical model that describes these rules in terms of how they weakly order a document collection. This model shows that certain ideal rules are upper bounds for large classes of ranking algorithms.
The evaluation methodology measures performance by a function that gives the expected number of relevant documents for every retrieval level. By using the statistical notion of expectation it is possible to calculate a value that reflects the performance of a weak ordering for each level of retrieval. Statistical formulas make it possible to summarize the range of values from a sample population of users.
Eleven different ranking algorithms are compared using data from the UKCIS retrieval experiment in 1972. The evaluation shows that several variations of the overlap retrieval rule, compared with randomly ordered retrieval, can increase the expected number of relevant documents by 40%. The performance of ideal rules, constructed for the purpose of determining an upper bound on retrieval, is yet 50% greater than overlap retrieval.
The methodology is inherently flexible. It can be extended to other methods of user assessment, e.g., partial relevance. Assigning a negative value to non-relevant documents models the effect of stopping rules.
The problem of evaluating document retrieval ranking algorithms is fundamental to information studies. It is adaptable to the more general setting of indexed retrieval and thus may be pertinent to other applications.
| |
MPACT Scores for Eric Christian Van Vliet A = 0
C = 0
A+C = 0
T = 0
G = 0
W = 0
TD = 0
TA = 0
calculated 2008-05-29 13:34:30
Advisors and Advisees Graph
generating graph, please reload |