INLS 740: Digital Library Evaluation
The IMLS/NISO Framework of Guidance for Building Good Digital Collections states:
Collections Principle 6: A good collection has mechanisms to supply usage data and other data that allows standardized measures of usefulness to be recorded.
Many such mechanisms exist. For this exercise we will look at two of them: automatically-collected weblog statistics and user comments.
The digital library that you will be building for this course will not be fully enough developed, nor used heavily enough, for evaluation data to be meaningful. We will therefore be looking at data from a well-established DL: Documenting the American South. DocSouth is the flagship project of the Carolina Digital Library and Archives, and contains several collections, with a wide range of types of materials, file formats, and presentation styles.
Imagine that you are the Head of DocSouth, or the Head of the CDLA, or the University Librarian. What would you want or need to know about DocSouth, in order for you to do your job? Develop a set of evaluation questions about DocSouth. For example:
- Which of DocSouth's collections receive the most / least use?
- What user communities make the greatest / least use of DocSouth's collections? For what are different user communities using the DocSouth collections?
- How to users navigate the DocSouth site? What are users browse paths? For what do users search?
- Given users' behaviors on the DocSouth site, what functionality or tools might be most appropriate to provide?
These are just suggestions: other evaluation questions of your invention are fair game.
Then, look at the Google Analytics data for DocSouth.
You may or may not be able to answer your evaluation questions with the data available to you, or you may be able to answer different questions. Part of this exercise is for you to identify the questions that you ideally want to be able to answer, and then the questions that are actually answerable given the data available to you. Given that gap, make recommendations about additional data that it would be useful to collect, to answer important but currently unanswerable questions.
This assignment will proceed in two stages: individually then collectively. First, prior to class, individually develop a set of evaluation questions, and a brief justification for each. In other words, why are these evaluation questions the best / most useful question to answer?
Second, in class, working groups will look at the dataset, identify which questions proposed by group members are answerable or not, identify which questions are worthwhile and why, attempt to answer those that the group thinks are worthwhile, and brainstorm about additional data that DocSouth should collect.
At the end of this class session, working groups will submit a brief report, including the following: the evaluation questions agreed upon by the group, data analyses, and recommendations for additional data collection; and the evaluation questions prepared by each member of the group.