i Social Semantics on the Web

Social Semantics on the Web

Harry Halpin, <H.Halpin@ed.ac.uk>
<harry@w3.org>

Textorized Parthenon

Social Semantics on the Web

What is the Semantic Web?

 Links between column headings Links Joining Across Different Kinds of Information using URIs. URIs not just to refer to web-pages, but to things.

There must be agreement from decentralized agents on what a URI means or denotes - i.e. its semantics.

Pat Hayes vs. Tim Berners-Lee

Social Meaning and RDF: "the meaning of an RDF document includes the social meaning, the formal meaning, and the social meaning of the formal entailments" so that "when an RDF graph is asserted in the Web, its publisher is saying something about their view of the world" and "such an assertion should be understood to carry the same social import and responsibilities as an assertion in any other format" RDF Concepts and Abstract Syntax Draft

Berners-Lee said "a single meaning is given to each URI" which is summarized by the slogan that a URI "identifies one thing."

Hayes retorted that "I'm not saying that the `unique identification' condition is an unattainable ideal: I'm saying that it doesn't make sense, that it isn't true, and that it could not possibly be true. I'm saying that it is crazy" since it goes against the "basic results in 20th century linguistic semantics"

Berners-Lee responded that "`we are not experimental philosophers, we are philosophical engineers" such that "we are not analysing a world, we are building it"

Two Positions

  1. Model-theoretic Position: the meaning of a URI is given by whatever model(s) satisfy the formal semantics of the Semantic Web languages.
    • "the Semantic Web languages would operate exactly unchanged if the identifiers in them were not URIs at all, and if the Web did not exist" (Hayes, In Defense of Ambiguity).
    • Corresponds to Russell's descriptivist theory of reference.
  2. URI Ownership Position: the meaning of a URI is whatever was intended by the owner.
    • The position held by Berners-Lee, seems to make sense trivially on the hypertext Web (a URI identifies the web-page it accesses!)
    • Corresponds to Kripke's causual theory of reference.
Halpin (2010) "Sense and Reference on the Web", Minds and Machines.

Frege and Meaning

Frege posits that the actual thing in the world is the referent and a name is a symbol that identifies a referent(s). The sense is the mode of presentation, a type of public, objective(?), knowledge about that private concept among a shared community. The third party of sense (meaning) mediates the reference relationship.Gottlieb Frege

Example:Hesperus has a sense ("the morning star") different from that of Phosphorus ("the evening star"), yet both have the same referent, the planet Venus.

Russell's Descriptivist Theory of Names

Bertrand Russell

Patches of sense-data known through direct acquaintance allow one to ground the atoms of logical statements or create descriptions that can form the basis for names.

The priority of reference over sense

Russell deals with referents in the past, or imaginary referents by having a name be a short-cut to a set of logical expressions.

Names are shorthand for logical expressions - so the name "Eiffel Tower" includes a host of facts about it such as "in Paris" and "completed in 1889", as well as an existential quantification!

The Model-Theoretic Position

Tarski removed the quaint Russellian sense-data epistemology, grounding out in just a mathematical model defined by extensional satisfaction, composition, and axioms.

Pat HAYES This formal semantics became the foundation of the Semantic Web via the RDF Formal Semantics specified by Pat Hayes.

Ambiguity is built in: The web of logical statements is the bearer of meaning, and whatever satisfies the model could be a referent.

A Third Position?

Ludwig Wittgenstein Before one reasons using logic, one must share a form-of-life so that one can agree on the assumptions.

This means that two entities must not only share the set of possible referents, but the Fregean "sense" of each referent. How can we formalize the notion of sense?

Answer: Use the computational traces of users in their everyday Web activity, with the end-goal of building formal theories of reference upon the "pretty-good-enough" statistical approximations of sense.

We hypothesise this philosophical notion of sense can be approximated by empirical statistics of natural language terms (language models) generated by the ordinary activity of information gathering- on the Web, such as tagging and searching.

How do we reach consensus on a sense?

A social semantics of sense can only be approximated via tagging if users agree on a macro-scale when tagging a resource.

Question:How can agents in a large decentralized agreement reach agreement on the sense of resources? Is it possible and how close is agreement? power-law tags

Tagging: Description being natural language terms added after a resource is discovered.

Plotting the infamous power-law for a singe resource, averaged over 500 "popular" resources from del.icio.us (Halpin, Robu, and Shepard, 2007, World Wide Web Conference ).

Distributions are rank-ordered frequencies of tags.

Using KL Divergence to determine stability

Kulber-Leibler Equation To determine if agreement is possible, we need to have some measure of agreement.

Kullback-Leibler Divergence: Information-Theoretic Measure to determine differences between two distributions.

Kulber-Leibler Equation

Collection i distributions over i time-steps (months in del.icio.us)

Choose as P the i-1 distribution the distribution, and Q as i distribution. When KL approximates zero, distribution has stopped changing. Tagging distributions stabilize, typically within a few weeks.

Under What Conditions

Is it Just Reinforcement?: Possibly tagging is a flawed example of decentralized agreement on information agreement, due to the presence of tag reinforcement mechanisms.

There are two main models of tagging:

Cattuto et. al: Their model uses a simple Simon-Yuill process, with a probability conditioned two parameters, either you either choose choose a new word from random or you re-inforce an existing word.

Dellshaft and Staab: Their model uses a multi-parameterized process, with a probability conditioned by the exponent of the power-law, either you either choose choose a new word from "topic distribution" or you re-inforce an existing word.

Both models say reinforcement is the answer...

Do we get power-laws without reinforcement?

Experiment (Halpin and Bollen, Web Intelligence 2009): 200 participants. Half get to see reinforced tags (feedback condition), and half do not get to see reinforced tags (no feedback condition). Over 11 URIs from easy-to-understand subject matter, chosen at random URIs given by del.icio.us from popular "lifestyle" tags.

Feedback Kulber-Leibler Equation

No Feedback Kulber-Leibler Equation

Agreement on Sense Is Possible

The "no feedback" condition is actually a power-law - and feedback simply shortens long tail.

People in a decentralized environment, even without feedback, will in general describe resources in the same manner using natural language terms, at least as regarding popular categories and general subject matters they are equally familiar with.

This is likely to be true because tags are grounded in the natural language, and as we use the same background knowledge (form-of-life) to describe web-sites.

Further experiments are done being done with the same experimental set-up looking at specialist knowledge in tagging.

Tags are like Search Terms

referent

However, tagging is just one kind of generation of terms. Search engines mediate access to a far greater number of resources than collaborative tagging systems.

Consider tagging to just be post-hoc search, with a much more sparse number of terms spread over a wider group of resources.

Social Semantics for Ontologies

Ontologies themselves can be thought of as having a model-theoretic structure built on top of an agreed upon sense.

So each ontological term (URI on the Semantic Web) should have a "sense" of natural language terms. This sense could then be used as a query expansion or term expansion technique to bootstrap tasks like information retrieval and ontology mapping respectively.

Can we also look at what ontologies are associated with what keywords using Semantic Web search engines like Sindice?

Can we use hypertext search terms and even decomposed hypertext documents as sources of sense? Can we have use ontologies themselves as sources of keywords?

To Test: See whether treating ontologies as a sense helps normal hypertext search, and see whether or not senses deduced from hypertext search can help the "semantic" search for ontologies?

An Algorithm

Halpin and Lavrenko, forthcoming, Journal of Web Semantics

  1. Given a term, retrieve a set of web-pages (Using Yahoo!).
  2. Given a term, retrieve a set of Semantic Web URIs and all triples (facts) associated with them (Using Falcon-S).
  3. Human searches through web-sites.
  4. For each web-page the human clicks on, strip out HTML and reduce to words.
    1. Extract all text and typed data from each RDF fact
    2. Decompose RDF into "a bag of words" with lemmatization and removal of words from end of URI.
    3. Match converted RDF to HTML using information retrieval text.
  5. Pick the URI with the best ranking score given by IR techniques.

Relevance Feedback

Have a human judge actually figure out what web-pages are relevant, and then use those to feed back and expand the query, in order to re-rank the results.

Experiment: Had 200 queries from previous work, retrieved top 10 hypertext Web (Yahoo!) results and top 10 (FALCON-S) results each judged by 3 judges for relevancy. Fliess's Kappa=0.5724$ (p < .05, 95% Confidence interval [0.5678,0.5771]), indicating the rejection of the null hypothesis and moderate agreement. referent

Results

referent

Results of Querying the Hypertext Web

referent

Results of Querying the Semantic Web

Finding Right IR Technique and Parameters

MAP Scores for Vector-space Model Parameters: Relevance Feedback From Hypertext to Semantic Web

referent

Finding the Right URI is now Acceptable!

Summary of Best MAP Scores: Relevance Feedback From Hypertext to Semantic Web

referent

Run it in reverse!

Now apply relevance feedback from Semantic Web search engines to the hypertext Web!

MAP for Language Model Parameters: Relevance Feedback From Hypertext to Semantic Web

referent

The Semantic Web can help Hypertext Search

Summary of Best MAP Scores: Relevance Feedback From Semantic Web to Hypertext

referent

What's Next?

yahoo

Is this a unified picture - a computational and approximate notion of sense?

Thesis: Finding and giving meaning to ontological terms (URIs) on the Semantic Web can be built out of the social semantics implicitly given by the searching and tagging behavior of ordinary users. With Henry Thompson, currently expanding this work to test the reliability of links in "Linked Data" and TREC-style evaluations of Semantic Web search engines.

However, the major drawback of this work has been an inability to correlate tags, users, search terms, and Semantic Web ontologies (as well as group membership and social networks of these users) on a large-scale. So off to Yahoo! Research for a few months...

Standards for the Social Web

w3c

One question would be how to boot-strap even more structured data?

While users produce these senses for free while searching, this knowledge is trapped and can only be reliably tracked by major search engines.
  1. What if there was a standardize way to pay for tasks over the web-browser?
    • Micropayments in the Browser? (Web Payments)
  2. What if we could more easily share this kind of social data?
    • The Federated Social Web Test-Cases (SWAT0)
  3. Can we make this social infrastructure secure?
    • WebID (certificates in URIs), JSON Web Tokens (certificate signed data)
The World Wide Web Consortium will begin standardization work with browser vendors in this area shortly...

Laying the foundations for wide-scale capture and sharing of social semantics by ordinary users in browsers, and hopefully building a statistical and social infrastructure for the Semantic Web.