The Web 2.0

The Web as a Totalizing System

Increasingly all data is on the Web: a universal information space

The "Web 2.0" is the use of the Web as a platform for applications, a universal computation space

What is the break point for the full realization of the Web as a platform for applications?

Data is Trapped Within HTML!

So we can build Web-based cross-platform applications using just Javascript and XML

Applications are only as good as the data they can access.

Unfortunately, most data is trapped in idiosyncratic HTML!

or in databases behind firewalls...

Liberate the data and put it in a common universal format for accessing by our Web 2.0 applications?

How to Get the Data!

Opposing Methodologies

1. Authors make their data explicit (The Semantic Web, microformats, databases)
2. Applications scrape the implicit data (Google and other search engines)

Peter Norvig: We deal with millions of Web masters who can't configure a server, can't write HTML. It's hard for them to go to the next step.

Tim Berners-Lee: I don't want statistics telling me how much money I have in my bank account!

The Semantic Web

Is a open world and universal space for machine-readable data.

To a computer, then, the web is a flat, boring world devoid of meaning...This is a pity, as in fact documents on the web describe real objects and imaginary concepts, and give particular relationships between them...Adding semantics to the web involves two things: allowing documents which have information in machine-readable forms, and allowing links to be created with relationship values.TimBL, WWW1994

The Semantic Web Stack

The World Wide Web Consortium

The World Wide Web Consortium (W3C) is an international consortium where Member organizations, a full-time staff, and the public work together to develop Web standards. W3C's mission is:

To lead the World Wide Web to its full potential by developing protocols and guidelines that ensure long-term growth for the Web.

And Duke is a member...

Sort of like HASTAC, but for Web companies and research institutions

Is the Semantic Web Just Artificial Intelligence?

Just when dynamicism and anti-represenationalism had won...

As we enter the age of embodiment in cognitive science, the Web is leading our lives to be increasingly disembodied.

Good Old Fashioned AI in the form of knowledge representation is back on the Web.

Brief History of AI

Dartmouth Proposal:The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.

A program has common sense if it automatically deduces for itself a sufficient wide class of immediate consequences of anything it is told and what it already knows

Knowledge Representation Languages were created to formalize human knowledge and so inject meaning into a machine

Knowledge Representation Breakdown

First Order Predicate Calculus: As in the foundations of mathematics, but it's undecidable and researchers wanted more "natural" mappings to human cognition.

Frames Knowledge to be composed of frames and slots that are filled in by various values of the slots (Minksy, 1975)

Semantic Networks: One common behavior that humans have attempting to represent knowledge is to begin to start drawing circles and lines between them. (Richens, 1956)

AI Winter

KRL The most flexible knowledge representation system ever was undecidable on even simple problems.

Winograd:Can we re-found artificial intelligence not on analytic philosophy but on hermeneutic philosophy?

Brachman-Smith Survey and Woods's "Whats in a Link?": AI had fragmented into communities with untranslatable knowledge representations.

Smith on Cyc Forget intelligence completely, in other words; take the project as one of constructing the world's largest hypertext system, with Cyc functioning as a radically improved (and active) counterpart for the Dewey decimal system. Such a system might facilitate what numerous projects are struggling to implement: reliable, content-based searching and indexing schemes for massive textual databases

Web Architecture 101

URIs (Uniform Resource Identifiers, TimBL wanted "universal" instead of "uniform".

Resource: whatever might be identified by a URI" such as "human beings, corporations, and bound books in a library" and even "abstract concepts"

Representation: data that encodes information about resource state (such as hypertext web-pages)

Link: a connection between two resources

Resource Description Framework

RDF(Resource Description Framework) is basically just a formal logic for describing a network of links and nodes, where links are called properties that connect subjects and objects:

http://www.ibiblio.org/hhalpin# foaf:knows http://www.w3.org/People/Connolly/#me.

It's semantic because RDF defines a formal semantics that gives the each RDF statements well-defined properties - i.e. it's a data model, not just a syntax.

Differences of Scale and Decentralized

How big is the SemWeb? Currently 5 million sources of Semantic Web data, containing over 363 million statements

How Big is the Web? Over 108 million distinct web sites (NetCraft, Feb 2007), consisting of over at least 8 billion web-pages with an estimated 29.7 billion web-pages.

Projects translating text (like Wikipedia) to the Semantic Web, exposing relational databases to the Semantic Web

Classical AI hoped that an elite of "wise old men" would create the "God Forthcoming" ontology of entire world.

The Semantic Web relies on people (and their applications) to in a decentralized manner have their programs map to knowledge representations with e domains of emerging consensus and domains of unending opposition.

Formal Semantics

Formal semantics accomplishes moving meaning from the realm of human interpretation by defining semantics in terms of a models defined using mathematics. The interpretation function maps from the syntax, a collection of physical marks, to the semantics, a mathematical model that stands in for the "real" world. In formal semantics, the interpretation function is a substitute for human interpretation

Mappings from model theory to useful domains done by humans.

Where's the Semantics in the Semantic Web?

It's about connecting the World Wide Web to the meaning of the wide world outside the web.

Logical semantics are not derivative once they are inside the machines, as the semantics of the machine operation is independent of the human and the machine once it in the model.

After all, what your employer finds about you on via Google can determine if you can get a job!

Further Advantages of Formal Semantics

The threat of undecidability: RDF is trivially decidable, and OWL (Web Ontology Language) consists of three languages, of which OWL Lite and OWL DL are described by the decidable description logic SHOIN(D) and SHIF(D) respectively.

Good Engineering: Specifications are made using formal semantics is because mathematics is the closest thing humans have to a precise lingua franca; formal semantics goes above the level of prose and can specify with certainty regardless of any situation the behavior of a program.

Suitable for Computers!: Due to their heritage in mathematical logic computers are ideal machines for following formal semantics. Due to their huge memory and limitless patience, in the world of pure formalism computers far outmatch humans in unerringly deducing the results of a formal semantics.

Knowledge Representation Problem

If knowledge representations are fundamentally stand-in surrogates for facets of the world, then how close is the surrogate to the real thing? What attributes of the original does it capture and make explicit, and which does it omit? Perfect fidelity is in general impossible, both in practice and in principle. It is impossible in principle because any thing other than the thing itself is necessarily different from the thing itself.

Knowledge representations are the exteriorization of our understanding of some portion of their world onto a digital substrate...the fuzzy and vague understanding interior to ``our head'' must be given digital flesh and so exteriorized into bits and bytes

Digitality by definition always forces one to abstract away from the infinite fuzziness of the analogue world, and so an ineluctable difference is always lost in translation from interior understanding to exterior knowledge representation due to the digitality of the external representation.

The Frame Problem

Discovered by Hayes and McCarthy: if we had a number of actions to be performed in a sequence, we would have quite a number of conditions to write down that certain actions do not change the value of certain fluents

Frames for common situations: a number of fluents are declared as attached to the frame. and the effect of an action is described by telling which fluents are changed, all others being presumed unchanged

Frames discretely portion out limited situations in the world, and these situations themselves change over time. Any change in the domain over time itself cannot be specified completely without changing the frame itself, and to do so leads one into an infinite regress of frame axioms.

The Frame Problem In Different Frames

Computational: How to deduce the consequences of any action without having to also explicitly search through all the elements in the knowledge representation are not effected by the action?

Semantic Web tries hard to avoid this by stating it does not provide any analysis of time-varying data and the slogan Cool URIs don't change

Philosophical: a problem is "isotrophic" when there is no bound to what could possibly be relevant to the problem at hand

Hamlet's Problem: When to stop thinking!

Bounded Rationality: Artificial intelligence is trying to create finite human-level intelligence, not the mind of God

Closed World Assumption

STRIPS: if you cannot prove something true to a collected of known facts, add the fact that it's not true to the database of known facts.

common sense law of inertia: that one should assume that when something changes, nothing else changes

Dynamic systems build time directly into the system.

Open World Assumption

You never know if you have all the facts on the Web

So nothing can be proven false, only shown to be true!

Would the SemWeb operate exactly the same if the identifiers in them were not URIs at all, and therefore if the Web did not exist?

URIs ground the program evermore data on the Web - but does it solve the Symbol-grounding Problem?

Harnad: Just grounding one symbol in merely a never-ending cycle of endless symbols, each without any genuine semantics.

Self-Describing (Follow Your Noise Principle): a URI is not just an identifier, but allows one to accessing more information about that identifier

Impact of the Open World Assumption

Can't use non-monotonic reasoning like default logic, where the results of adding new knowledge may lead to previous inferences being withdrawn.- so must use monotonic reasoning

Throws out the Law of the Excluded Middle: Every fact is either true or not true

And therefore all of classical Platonic mathematics!

Instead must develop intuitionistic or constructivist logic, a logic without a notion of truth.

Something is only true in relation to a proof of its existence. Similarly, one no longer has a notion of false, but only those facts for which one does not have proofs, and so one can imagine that any fact can still be true.

On the Web, you never have all the facts...but you can always get more

Semantic Mashups

The Semantic Web is a solution in search of a problem: Data Integration

URIs can be used as a universal (globally-unique) identifier, like a foreign key in a database

RDF is a natural format for combining up any sort of data with any other sort of data!, since the URI preserves the identity of the data.

Due to the open world assumption and being based on links, it is an extremely flexible data format that should be able to capture data from relational databases, microformats, and more...

Using the Semantic Web to get a date

My Calendar: http://www.ibiblio.org/hhalpin

Dan's Calendar: http://www.w3.org/People/Connolly/

Querying the Semantic Web

Use SPARQL (SPARQL Protocol And RDF Query Language), a SQL-like alternative for querying Semantic Web data.

FROM <http://www.ibiblio.org/hhalpin/calendar.rdf>
FROM <http://www.w3.org/2000/06/webdata/xslt?xslfile=http%3A%2F%2Fwww.w3.org%2F2003%2F11%2Frdf-in-xhtml-processor&xmlfile=http%3A%2F%2Fwww.w3.org%2FPeople%2FConnolly%2F>

WHERE
  {
        ?event1 a ical:Vevent;
              ical:summary ?summ1 ;
              ical:dtstart ?start1 ;
              ical:dtend ?stop1 ;
              ical:location ?loc1.
        ?event2 a ical:Vevent;
              ical:summary ?summ2 ;
              ical:dtstart ?start2;
              ical:dtend ?stop2;
              ical:location ?loc2.
      FILTER (  ?event1 != ?event2) .
      FILTER ( xs:string(?start1) = xs:string(?start2) ).
      FILTER ( xs:string(?stop1) = xs:string(?stop2) ).
      FILTER ( xs:string(?loc1) = xs:string(?loc2) ).
}

Try SPARQL Query Online with our query

Keeping It Down

Lately everyone is talking about embodiment...

Yet does this really work to explain the Web, and the current explosion of representations and accompanying feeling of disembodiment? Walking robots are hard, but not interesting from an augmentation standpoint - after all, we already walk.

The Semantic Web is Augmentation!

The Semantic Web is not about replicating human intelligence. The Semantic Web is about augmentation and so "keeping humans in the loop."

Man's population and gross product are increasing at a considerable rate, but the complexity of his problems grows still faster, and the urgency with which solutions must be found becomes steadily greater in response to the increased rate of activity and the increasingly global nature of that activity....

by "augmenting human intellect" we mean increasing the capability of a man to approach a complex problem situation, to gain comprehension to suit his particular needs, and to derive solutions to problems. Increased capability in this respect is taken to mean a mixture of the following: more-rapid comprehension, better comprehension, the possibility of gaining a useful degree of comprehension in a situation that previously was too complex, speedier solutions, better solutions, and the possibility of finding solutions to problems that before seemed insoluble.

What about Folksonomies?

Folksonomy is a neologism for a practice of collaborative categorization using freely chosen keywords. More colloquially, this refers to a group of people cooperating spontaneously to organize information into categories. In contrast to formal classification methods, this phenomenon typically only arises in non-hierarchical communities, such as public website

For an example, see my del.icio.us bookmarks

Our search for me on flickr

Folksonomies versus ontologies

Clay Shirky:A lot of users tagging things foobar are also tagging them frobnitz. I'll tell the user foobar and frobnitz are related." It's up to the user to decide whether or not that recommendation is useful -- del.icio.us has no idea what the tags mean. The tag overlap is in the system, but the tag semantics are in the users. This is not a way to inject linguistic meaning into the machine.

Is somehow folksonomy the an opposing challenger to the Semantic Web?

From taxonomy...

...to...

folksonomy...

Something about that distribution...

Are tags really that dynamic? Are they always chaotic?

Online tagging systems are complex systems: a large number of users, a lack of central coordination, non-linear dynamics.

Known to produce a power-law distributions.

The Power-Law

In Math:

y = cx^α

log y = α log(x) + log(c)

The Long Tail and the Giant Component - and it's scale free

Adding Them Up in Log Space

Folksonomies stabilize into coherent categorization schemes over time.

For more details see my upcoming WWW2007 paper

The Tag Commons Project

Categorization and Classification

Tagging is considered a categorization process in contrast to a pre-optimized classification process, as exemplified by expert-created Semantic Web ontologies.

According to Jacob:

• Categorization divides the world of experience into groups or categories whose members share some perceptible similarity within a given context. That this context may vary and with it the composition of the category is the very basis for both the flexibility and the power of cognitive categorization
• Classification as process involves the orderly and systematic assignment of each entity to one and only one class within a system of mutually exclusive and non-overlapping classes; it mandates consistent application of these principles within the framework of a prescribed ordering of reality

From the Extended Mind

What is the difference between this and an act of memory? The mind is therefore extended into the environment.

What if there are other people in the environment? Or knowledge representations? Imagine Otto has a knowledge representation-driven Semantic Web system helping him? Or del.ici.us generated bookmarks?

The evolution of digital technology is the logic of human's overcoming their own biological heritage. Far from replacing or replicating human intelligence, digital technology fundamentally complements human intelligence.

To Collective Intelligence

When one deprivileges the human subject as an transcendental individual, one opens the door for new subjects, subjects composed of what were formerly thought of as individuals. This new subjects are collective subjects, and the act of overcoming the artificial divisions between formerly individual subjects and creating new subjects is the act of collectivity. When this collectivity is put into practice, not as an abstract or utopian notion, but in confrontation against the concrete and practical problems that threaten these new subjects, collective intelligence is born.

The Web makes much more sense as a "a common space in which we could all interact", a technological and universalizing system of information that has the possibility of creating universal collective intelligence.