XML 2006

Social Semantic Mashups: Exploring Networks Using Microformats and GRDDL

Harry Halpin [School of Informatics, University of Edinburgh]

Introduction: What is the Web 2.0?

Recently, the slogan "The Web as a Platform" has been the rallying cry of the diverse group of technologies known as the "Web 2.0," yet there seems to be little consensus over what these terms actually mean. It's clear that the "Web 2.0" has something to do with Javascript finally being standardly implemented in most browsers, but the recent growth on the Web goes beyond AJAX, and other key technologies include microformats, social networking, and mash-ups. Is there any underlying coherent connection between these diverse technologies, and what role does XML and RDF have to play?

For XML, the case should be clear: the development of the XMLHttpRequest and the DOM have been the powerhouse of almost all new Web applications in general, and it's clear that on some level the "Web 2.0" is just harvesting the hard work that was put in to make the Web as a whole a more standardized phenoumenon. So, to understand the term "Web 2.0", it would be worthwhile to examine the roots of the "Web 1.0". Tim Berners-Lee has in general defined the role of the Web informally as a universal information space, and the growth of everyone putting up everything from personal profiles to reviews on the Web show that this definition is proving to be increasingly true.

In fact, if anything, the "Web 2.0" is best defined as the use of the Web not just to transmit information, but as a platform for applications, which as a corrolary also means putting data on the Web in a way that it is easily accessible for machines. Diverse technologies such as microformats and REST Web Services can all be thought of as different ways to make data easily accessible for appliation developers and their machines. If the "Web 1.0" is a universal information space, then the Web 2.0 is a universal computation space, where diverse applications and data, accessible in standardized formats, can be dynamically recombined in new mash-ups.

Data is Trapped Within HTML

What is the break point for the full realization of the Web as a platform? The answer is obvious: Although the Web is the world's largest ever body of data, very little of it is useful to applications simply because the data is trapped within HTML. This is a large problem for mash-ups, as pplications are only as good as the data they can access. To make things worse, the data is often not even trapped within well-formed XHTML, but behind idiosyncratic HTML. This is increasingly an absurd state of affairs, as the often sloppy HTML is increasingly generated automatically from databases behind firewalls. The solution should be straight-forward: Liberate the data and put it in a common universal format for accessing by our Web 2.0 applications.

Yet, how can we even put the current scruffy data out there on the Web in a format more amendable to our applications. There are two main options:

  1. Authors make their data explicit
  2. Applications scrape the implicit data
Current trends on the Web over the last decade have mostly involved spidering and scraping implicit data, and the success of Google and many other large web services stands witness to the fact that this approach is feasible. The W3C has advocated for precisely the opposite, that by use of standards such as XML and RDF, web masters cleanly separate the form from the content of their web pages. For indeed, there are systematic problems with scraping implicit data. To do statistical processing often requires huge computational resources outside the scope of many people and companies. Furthermore, even with the simplest Perl script scraper, one has to deal with the continual uncertainty that the scraper could have not scraped the data properly - which is often the case with real-world messy HTML. This means scraping fine-grained implicit data often means getting bad data, and this undermines attempts to build applications. Yet there seems to be little choice, for as Peter Norvig of Google put it in response to Tim Berners-Lee's presentation at AAAI 2006, "We deal with millions of Web masters who can't configure a server, can't write HTML. It's hard for them to go to the next step."

Microformats to the Rescue

Luckily, there is a new technology on the block that makes putting explicit on the Web nearly idiot-proof: microformats. Microformats are, as described on their web-page, are "designed for humans first and machines second, microformats are a set of simple, open data formats built upon existing and widely adopted standards." These microformats range across everything from personal data as stored in VCard to social networking information. In contrast to the Semantic Web of RDF, microformats are often called the lower-case semantic web as they are one method for making semantic data explicit while embedding it directly in HTML.

Shockingly, a large number of web sites are using microformats

  1. Upcoming.org
  2. LinkedIn
  3. Yedda
  4. Yahoo! Local
  5. Yahoo! Tech Reviews
There's even a Dreamweaver plug-in!

Microformats in general just "overload" the div and class elements, although there are variations in individual microformats. hCard is a serialization of the well-known and widely used vCard format for storing personal information as stored on business cards.

<span class="vevent">
<p><abbr class="dtstart" title="2006-12-05">December 5-</abbr> 
<abbr class="dtend" title="2006-12-07">7th</abbr>
</b> At <b><span class="summary"&rt;XML 2006</span></b> 
(<span class="location">Boston, MA USA</span>) 
for a presentation on "Social Semantic Mashups".</span>
XHTML Friends Network (XFN) is used to describe social networking. Again, it's a straightforward mapping, at least on the surface.
<li><a href="http://www.w3.org/People/Connolly/" rel="colleague met">Dan Connolly</a></li>
<li> <a href="http://seanmcgrath.blogspot.com/" rel="colleague met">Sean McGrath</a></li>
<li><a href="http://www.jclark.com/" rel="colleague">James Clark</a></li>

These examples barely scrape the surface. Microformats are centralized data formats for different types of data, often (nearly) isomorphic to already widely adopted non-Web standards:

Yet there are systematic problems with the "Web 2.0" phenoumenon. On a general scale, the Web 2.0 web services trap data in the service itself. Despite their "power to the people" rhetoric, many of these services that rely on user-created content imprisoning the data of the user "behind the garden wall." ??, where the user cannot import or export their data usually. breaking straw

Too many services replicate the same sort of data....what if you have a Friendster, a Myspace, and a Twtter account? Do you really want to to import - or worse - retype all of your "friends" out over and over again every time you join a new service?

The main problem with microformats is that they put your data into HTML, but you have no standard way to get the data out. Except really hairy XSLT style-sheets such as Suda's x2v.... A further problem is that they cannot be validated easily. For example, you can mix hCard and hCal.

Mixing Microformats
<div class="hreview">
   <span><span class="rating">5</span> out of 5 stars</span>
    <h4 class="summary">Crepes on Cole is awesome</h4>
   <span class="reviewer vcard">Reviewer: <span class="fn">Tantek</span> - 
    <abbr class="dtreviewed" title="20050418T2300-0700">April 18, 2005</abbr></span>
   <div class="description item vcard"><p>
   <span class="fn org">Crepes on Cole</span> is one of the best little 
   creperies in <span class="adr"><span class="locality">San Francisco</span></span>, far better than <span class="fn org">Crepes R Us"</span>, as you
   have read from my friend <span class="fn reviewer">John Fugue</span>'s review.
    <p>Visit date: <span>April 2005</span></p>
  <p>Food eaten: <span>Florentine crepe</span></p>

Semantic Mash-ups

The Semantic Web is a solution in search of a problem, and the problem of the hour is data integration. RDF(Resource Description Framework) is basically just a formal logic for describing a network of links and resources, where links are called properties that connect subjects and objects. In graph theory, this would correspond to edges connecting vertices.

A simple example of a single social link, using the FOAF?? vocabulary, in RDF can be stated (using N3, a concise and easy-to-read alternative to RDF/XML) as:

A Social Connection in RDF
http://www.ibiblio.org/hhalpin# foaf:knows http://www.w3.org/People/Connolly/#me.

One eternal question about the Semantic Web is "Where's the semantics?" Is it just a buzzword, or does it actually mean something? In fact, the use of the term "semantic" is intentional, for it's crucial to the entire project and differentiates it from vanilla XML. It's semantic because RDF defines a formal semantics that gives the each RDF statements well-defined properties - i.e. it's a data model, not just a syntax. However, it's note that XML does not have a data model per se, as one has clearly been defined for XQuery in general and XPath in particular. So, in fact, the Semantic Web is just once choice of a formal model. In fact, as pointed out by Jonathan Robie??, one could use XQuery functions to simulate the properties of RDF, although one would add that there is little reason to reinvent the wheel this as the hard work of making RDF reasoners and databases has already been done in many products, including upcoming ones by Oracle and IBM??. Furthermore, as RDF is formal semantics, not a concrete syntax, its data model can be used outside of serializing anything as XML. Furthermore, one great advantage of RDF is that unlike the XPath data model, the Semantic Web data model does not assume a closed world of XML infoitems, but an open world of loosely connected resources. The key to the open world and its loose connections is the use in the RDF of the URI.

The URI can be used as a universal (globally-unique) identifer, like a foreign key in a database, and so makes RDF a natural format for combining up any sort of data with any other sort of data!, since the URI preserves the identity of the data. In other words, while RDF may be a somewhat odd choice for a logical language due to its base on graphs (as opposed to say, formulae in first-order predicate logic), its base in a logic with even the most simple of properites such as sub-classing of resources allows us to use RDF, in combination with its brilliant use of a URI as a global key, for semantic mash-ups of any sort of data. As stated before, since RDF is based on an open-ended number of links (the open world assumption), it is an extremely flexible data format that should be able to capture data from relational databases, microformats, and more...an therefore, at least by technical merit it seems RDF is the perfect universal data format for the Web 2.0.

Since RDF is a simple data model for machine-readable semantics (albeit one with a terrifyinglyu complex serialization in XML!) and microformats a simple and easy-to-use way to put machine-readable data in HTML, rather suprisignly RDF and microformats are compatible. Furthermore, they are a marriage made in heaven, since RDF complements the weaknesses of microformats, since by assigning a formal semantics and URIs to microformat data, microformat data can be combined and mashed-up. The first step of this of course is modelling each microformat in RDF. So, for each popular microformat vocabulary people are in the midst of manufacturing an isomorphic RDF model, called a micromodel. These are listed below:







Document/Content Metadata

Dublin Core


usable (Sep 2005 testing status)


Social Networks










Calendars and Events

RDF Calendar




Opinions, Ratings and Reviews

RDF Review


in progress



Creative Commons




Tags, Keywords, Categories

Tag Ontology


Liberating Data with GRDDL

However, the missing component in the our ability to make mash-ups of microformat based data is a method of getting data out of HTML. This is large problem for microformats in general: Microformats let us put machine-readable data into HTML, but there's no way to just ditch the HTML and get the machine-readable data out! Except, now there is a solution, going by the rather odd acronym of GRDDL.

GRDDL (Gleaning Resource Descriptions from Dialects of Languages) is a simple and low-cost way to boostrap RDF out of XML and in particular XHTML data by explicitly labelling transformations from RDF to XML. In other words, GRDDL is just a transform from XML to RDF. GRDDL was originally an informal yet useful method used by Dominique Hazael-Massieux?? and Dan Connolly??, and is now in process to become a W3C Recommendation.??

GRDDL consists of three components:

  1. GRDDL-aware agent: a GRDDL-aware agent is a software agent able to identify the GRDDL transformations and run them to extract RDF.
  2. Source Document: an XML document which references at least one GRDDL transformation for a GRDDL-aware agent to use to extract RDF from it.
  3. GRDDL Transformation: a GRDDL transformation is an algorithm which, when applied to a compliant source document, allows a GRDDL-aware agent to extract RDF from this document.

We'll illustrate a simple GRDDL example that allows us to get my calendar in RDF Calendar format ?? from my homepage, which marks-up my calendar inline with the hCal microformat. Once we get the data out, we can use this data for useful purposes, such as booking a date with a friend!

Adding GRDDL to HTML: Before
<html xmlns="http://www.w3.org/1999/xhtml">
<link rel="stylesheet" type="text/css" href="homepage.css"/>
<link rel="schema.DC" href="http://purl.org/dc/elements/1.1/"/>
<meta name="ICBM" content="055.94270, -003.18599"/>
<meta name="DC.title" content="Harry Halpin's Homepage"/>
<meta name="DC.creator" content="Harry Halpin"/>
<meta name="DC.language" content="en"/>

<title>Harry Halpin's Homepage</title>
Adding GRDDL to HTML: After

<html xmlns="http://www.w3.org/1999/xhtml">
<head profile="http://www.w3.org/2003/g/data-view">
<link rel="stylesheet" type="text/css" href="homepage.css"/>
<link rel="schema.DC" href="http://purl.org/dc/elements/1.1/"/>
<link rel="transformation" href="http://www.w3.org/2002/12/cal/glean-hcal.xsl"/>
<meta name="ICBM" content="055.94270, -003.18599"/>
<meta name="DC.title" content="Harry Halpin's Homepage"/>
<meta name="DC.creator" content="Harry Halpin"/>
<meta name="DC.language" content="en"/>

<title>Harry Halpin's Homepage</title>

Until now, all our examples have been of extracting RDF from microformats in XHTML; yet the definition of GRDDL is much more general, and GRDDL allows us to get RDF from arbitrary XML documents. After all, XHTML is just a dialect of XML, although we allow the XHTML a bit of syntactic sugar in order to make GRDDL fit in with HTML conventions such as use of the head element. Howver, GRDDL can easily just be attached to the root document of any XML document. This is actually quite useful for practical applications. For example, instead of serving both a XML and RDF version of your data, you can serve an XML version of your data and convert to RDF on the fly on the client-side, saving space on the server and off-loading the processing to the client.

Adding GRDDL to HTML: After

<etc> ... </etc>

GRDDL can also be used on HTML and XML documents that have not been explicitly marked up for GRDDL at all through the use of profile URIs and namespace documents. In other words, GRDDL can work without transformationlinks by having the author of the vocabulary deploy a GRDDL transformation at the namespace document if an XML document (including a RDDL document??), or if an HTML document, at a profile URI. Indeed, this is a very plausible road to widespread deployment of the Semantic Web, as it requires no effort at all on either the authors of documents or the consumers of data from the documents. All it requires is a small bit of effort from the maintainers of XML vocabularies and microformats, both of which are fairly educated audiences familiar with XSLT, the main language, although not the only one, used to transform XML to RDF.

Adding GRDDL to a Namepsace RDF document.
 <rdf:Description rdf:about="http://www.w3.org/2004/01/rdxh/p3q-ns-example">

However, there are systematic problems with microformats. For example, URIs are simply missing from microformats. This leads to a host of problems, because without the use of namespaces, you can't mix microformats well as pointed out earlier. Furthermore, since URIs are missing from microformats, one cannot use simple microformats to embed arbitrary RDF statements into HTML. The ease-of-use of microformats comes at a price: You are restricted to using an already deployed microformat vocabulary! However, one can embed arbitrary RDF into HTML through one of two methods.

  1. Right now use: Embedded RDF. which is a GRDDL-enabled way of adding RDF to arbritary XML
  2. For the future, watch: RDFa, a future W3C Recommendation to create a way of embedding RDF in HTML (Semantic Web Deployment Working Group).

Here's a simple example from the VCard Ontology?? that allows us to embed into a table the OWL mappings and connections between the VCard ontology and two other popular RDF vocabularies for social networking and people, FOAF?? and the W3C PIM Vocabulary??.

Arbitrary RDF in HTML using Embedded RDF
<tr id="v.email">
<a href="http://www.w3.org/2006/vcard/ns#email">v:email
<a rel="owl.equivalentProperty" href="http://xmlns.com/foaf/0.1/mbox">
<a rel="owl.equivalentProperty" 
con:emailAddress</a></td> </tr>

Embedded RDF is very useful in a number of contexts. In our previous example, we only showed how to add a GRDDL transformation to a RDF Schema namespace document. However, many namespace documents are not in RDF natively. Using Embedded RDF, you can easily add a transformation to a RDDL namespace document by simply adding a single embedded triple. All you need is to get the object of the RDF property for a dataview:namespaceTransformation. In fact, you can even use this technique on XML Schema namespace documents by virtue of their annotations.

GRDDL-enabled XML Schema Documents

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
            data-view:transformation="http://www.w3.org/2003/g/embeddedRDF.xsl" >
    <xsd:element name="Order" type="OrderType">
      <xsd:documentation>This element is the root element.</xsd:documentation>
      <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
	<rdf:Description rdf:about="http://www.w3.org/2003/g/po-ex">
	      rdf:resource="grokPO.xsl" />

Use-Case: Getting a Date

One question to ask is: Now that I have my data in RDF, what can I do with it? After all, why bother using GRDDL to transform your XML and microformat data into RDF if there's no advantage over just tranforming your data to another standard, such as vCard or iCal? However, once you have your data in RDF you can easily mash it up and then query over it, using a RDF query language such as SPARQL??. Here's a simple use case: Chime Ogbuji and myself are trying to meet-up. Are we in Boston together next week? Yet our calendars use different formats: I store mine in microformats, while Chime natively stores his in RDF.

My Calendar: http://www.ibiblio.org/hhalpin

Chime's Calendar: http://metacognition.info/mycal.rdf

Calendar RDF
Using SPARQL to match cleanars
PREFIX foaf: <http://www.w3.org/2002/12/cal/icaltzd#>
FROM <http://www.ibiblio.org/hhalpin>
FROM <http://metacognition.info/mycal.rdf>
  	?evt1 a ical:Vevent;
  	      ical:dtstart ?start1;
  	      ical:dtend   ?stop1;
  	      ical:url ?url1.
  	?evt2 a ical:Vevent.
  	      ical:dtstart ?start2;
  	      ical:dtend   ?stop2;
          ical:url ?url2.
	FILTER (?start1 = ?start2 && ?stop1 = ?stop2). }

The output of this query is exactly what I need: When myself and Chime are in town together, which happens to be for the International Semantic Web Conference and XML 2006.

Exploring Social Networks

From Myspace?? to Facebook??, everywhere social networks are surging in popularity. This trend shows no sign of slowing down, as over ?? of entering college freshman have Facebook accounts ??FredStutzman??. However, as noted on other sites all of these social networking sites are in effect one-trick ponies that trap their users within their "walled garden"??, prevening their users from having friends on other services. So, over times these "closed world" servioces fall victim to the carpicious preferences of users and to their own inability to provide for their technical demand, just as Friendster gave way to Myspace and beyond. Currently, there are two main semi-standardized open vocabularies for social networking, a lightweight microformat called XFN (XHTML Friends Network, and the FOAF (Friend ofa Friend) based on RDF. Both of these open formats allow developers to make their own tools to visualize these networks. A sample of some different tools to search and visualize social networks are below:

Microformat Style

Using Rubhub

RDF Style

Using Foaf Explorer

FOAF Searching

Is Pretty Easy


Social networks are just one examples of a network. Networks exist everywhere, from food webs in ecology to computers on the Internet. Networks can be formalized using graph theory, a fairly young branch of mathematics began in the 1950s. Graphs are composed of vertices and edges, where vertices, often called "nodes" are connected to each other by edges, also called "links".

There are numerous questions one can then ask about a network. Is the network cyclic, allowing for circular paths to exist in the graph, or is the network acyclic and so disallow any circularity? Edges are of two types: directed and undirected. Myspace is undirected, while FOAF is directed. In a social network of friends, if the edges are undirected, then if you are a friend with someone, then they are a friend of yours. If the edges are directed, then you may be a friend with someone but they may not know you. The number of edges connected to the vertex is its degree.In a directed network, these are divided among in-degrees (how many people know you) and out-degrees (How many people do know you).

To view at a glance and manipulate these networks, we have to have a compact way of summarizing them mathematically. Imagine that the number of links between people can be represented by an adjancency matrix where each person's connections are represented by a column, and whose connected to a person is represented by a row. To represent the connections mathematically, in the adjancency matrix 1 represents "I Know You"" and 0 represents "I Don't Know You"

. A Matrix SocialNet

As anyone who has seen the use of social networking sites can attest, there are few things more important to users than knowing how many friends they have. However, the raw number of connections is not too useful. For example, one could have a large number of friends that are each isolated and not too influential, or you can have a small number of highly connected and therefore influential friends. The question is: given a list of your friends and their connections in an adjancy matrix, can you give me a number that tells me how influential you are? One answer is the Eigenvector centrality. This is the leading eigenvalue of the adjanency matrix, so my centraility is the sum of influential people I know. ??

xi= λ- 1jAij

λx approximates Ax

There are some weaknesses in measure of centrality. First of all, what if I know lots of people but no-one knows me? ?? Another variant on this problem would is what if everyone I know has a reputation of zero? The way to get around this problem is for everyone to get some free centrality.

D Matrix

xi = α ∑j Aijxj + β


Strangely enough, the Google Pageranking algorithm is just a variation of these ways of measuring centrality in networks, since it famously brings up towrds the top of its search list web-pages that are highly linked to (i.e. "central").

D = ∑ j I * kj is number of outgoing links

x = αAD-1x + β

α = . 85
β = . 15

Trusted Review Recommendations using GRDDL-powered Mashups

RDF For Reviews

There exists micromodels for hReview that has a GRDDL transformation.

This allows us to make reviews of the Boston Sheraton in a hotel and map to RDF

And reviews of restaurants like Ma Soba

  1. item (given a URI!)
  2. Summary
  3. Reviewer
  4. Rating
  5. tags
A Review of a Restaurant


<h2 class="summary">Really Enjoyed It</h2> <abbr class="dtreviewed" title="20061207T1756">Dec 7, 2006</abbr> by <span class="reviewer vcard"> <span class="fn">http://www.theryanking.com</span> </span> <img alt="photo of 'Ma Soba'" src="http://www.masobaboston.com/images/IMGP1372_2.jpg" class="photo" /> <div class="item"> <a class="fn url" href="http://www.masobaboston.com/">Ma Soba</a> </div> <blockquote class="description"> <p> <abbr title="5" class="rating">★★★★★</abbr> I had a vegetarian option, which included an excellent side of seafood salad, which I highy recommend. The sushi was obviously very fresh and it makes me wonder if I should start eating raw fish and forsake my vegetarian ways. </p> </blockquote> </div>
Identity Consolidation

There's so many identities to choose from!

  1. My ClaimID Account
  2. Links HTML
  3. My FOAF account
  4. http://www.ibiblio.org/hhalpin#me

Use Identity Consolidation by converting XFN rel="me" to...well...what?

Are you (a URI, as in http://www.ibiblio.org/hhalpin#me) the same (owl:sameAs) http://www.ibiblio.org/hhalpin?

Trusted Decentralized Recommendations SocialNet

Now myself and Chime can use my trusted friend Ryan King's review of Ma Soba

  1. Convert XFN to FOAF
            <rdf:Description about="http://www.theryanking.com">
    <rdf:Description rdf:about="http://gmpg.org/xfn/11#friend">
            <rdfs:subPropertyOf rdf:resource="http://xmlns.com/foaf/0.1/knows">
    <rdf:Description rdf:about="http://gmpg.org/xfn/11#colleague">
            <rdfs:subPropertyOf rdf:resource="http://xmlns.com/foaf/0.1/knows">
  2. Remap the network and look for reviews in Boston
  3. Found Ma Soba! Five Stars!
Giant Components Social Network Visualization

A giant component, a residue of the small world effect, arises once the distribution gets of certain size - and it stays stable

A few people know everyone so your friends are more likely to be more outgoing than you are!

The Power-Law Individual Distribution

In Math:

y = cxα

log y = α log(x) + log(c)

The Long Tail and the Giant Component - and it's scale free

Adding Them Up in Log Space

Now you can see clearly...

Cumulative Distribution

Six Degrees of Tim Berners-Lee


1 * 100: One Step

1 * 100 * 100: Two Steps

1 * 100 * 100 * 100 * 100 * 100: Five Steps = 10 Billion!

Greater than the total number of people in the world...so you are without a doubt six degrees of Tim Berners-Lee - as well as everyone else!