Archive for the ‘XQuery’ Category

The future of XHTML - and of XML?

Monday, July 6th, 2009

I was concerned late last week when I read that the XHTML Working Group will not be allowed to finish its work. The announcement indicates that this is being done to speed the work of HTML5 - but it also raises important questions about how XML will work with data on the Web.

http://www.w3.org/News/2009#item119

2009-07-02: Today the Director announces that when the XHTML 2 Working Group charter expires as scheduled at the end of 2009, the charter will not be renewed. By doing so, and by increasing resources in the HTML Working Group, W3C hopes to accelerate the progress of HTML 5 and clarify W3C’s position regarding the future of HTML. A FAQ answers questions about the future of deliverables of the XHTML 2 Working Group, and the status of various discussions related to HTML. Learn more about the HTML Activity.

Originally, HTML was based on SGML. XML was designed to make SGML easier to parse, so that it would be more accessible to software. From the beginning, XML was designed to support documents meant for humans to read, and data designed for programs. In modern software, there is no firm boundary between the two - data is often transformed on the fly for humans to read, and software often mixes human-readable content with data.

By representing HTML as an XML vocabulary, XHTML addresses HTML parsing issues, difficulty combining HTML with XML data, and extensibility issues:

  • An XML parser can not parse arbitrary HTML, and parsers which do are extremely complex. Tools like Tag Soup parse arbitrary HTML and convert it into XHTML, so that even legacy content in odd dialects of HTML can be processed by XML tools.
  • XML vocabularies like SVG (a graphics format) and MathML (a mathematical notation format) need to play nicely with HTML. XHTML lets them be combined with HTML using the same mechanisms all XML tools use to combine vocabularies.
  • HTML has become quite large, and XHTML provides a modularization of HTML, using XML mechanisms, that makes it easier to maintain and extend.

Typically, Working Groups are rechartered if there is active interest in their work and they are close to their goal, as the XHTML Working Group is. Another year of effort would probably be sufficient to finish the XHTML specifications. And the XHTML people are intimately familiar with XML and namespace issues, plus the issues that arise with XML and HTML software.

The FAQ that the announcement points to concerns me. Some of the work of the XHTML Working Group will not be finished, and there is even a suggestion that HTML5 will not handle namespaces correctly, and that XML vocabularies will not be able to mix freely with HTML5!

Consider this issue, to which the FAQ points:

http://www.w3.org/html/wg/tracker/issues/41

The HTML5 specification does not have a mechanism to allow decentralized parties to create their own languages, typically XML languages, and exchange them in HTML5 text/html serializations. This would allow languages such as SVG, MathML, FBML and a host of others to be included. At one point, an editors version of the HTML5 specification contained a subset and reformulation of SVG and MathML. Tim Berners-Lee described this incorporation of SVG and MathML without namespaces as horrific and the issue raiser [Dave Orchard] completely concurs with the him.

This issue limits the ability of non-HTML5 working groups to define languages as the languages must be “brought into” the HTML5 language. This dramatically increases the scope of HTML5 and decreases the ability to modularize development of orthogonal languages.

In the end, the problem could result in the text/html serialization rules becoming the standard serialization rules for XML languages, replacing XML itself. This could occur if every decentralized language has a choice between the XML serialization, the text/html serialization or both. In many cases, the language may choose the text/html serialization.

I find this last paragraph alarming. XML was designed specifically to allow decentralized parties to create their own languages, and to combine them cleanly using namespaces. By using the mechanisms defined in XML, XHTML inherits this ability, an ability that HTML5 lacks. One proposed solution is to absorb XML entirely into the HTML5 specification. I agree with Tim and Dave that incorporating SVG and MathML without namespaces is horrific, but the obvious solution is for HTML5 to support namespaces and XML cleanly - just as XHTML does.

I would like to see the XHTML Working Group finish their work, and I would like to see the result used as the XML serialization of HTML5. Legacy HTML ensures that there will always be large amounts of HTML content that is difficult to parse without tools like Tag Soup. But XHTML gives us a very sane XML vocabulary to convert such content into, and makes HTML accessible to the wide variety of XML tools that make up much of the infrastructure of the Web.

XQJ (XQuery API for Java) is now an official JSR!

Thursday, July 2nd, 2009

The Expert Group for for JSR-000225 XQuery API for Java
has (finally!!!) completed its work and published XQJ as a Final Release.

You can get the XQJ specification at either of the following URLs:

XML and XQuery (Second Edition) - Proposed Edited Recommendations for review by 31 May

Thursday, April 23rd, 2009

We have now released Proposed Edited Recommendations of the XQuery and XSLT specifications. These fix known bugs and problems, and are out for community review. Some of these changes affect conformance. If these documents pass public review, they will become W3C Recommendations.

These documents were published 21 April 2009, and the review period ends 31 May 2009:

Using the XML Exchange

Thursday, March 26th, 2009

The XML Exchange allows routing decisions to be made using XQuery. It is used primarily for two things:

  • XML Content-based Routing - a queue can be bound to an exchange using an XQuery that identifies “interesting” messages. This is particularly useful when the message publisher does not know what information a message consumer may use to decide whether to route the message, since the publisher and consumer do not need to agree on a set of headers.
  • Queries on Headers - the XML Exchange can query headers using XQuery whether or not the message content is in XML, so this provides query capability analogous to Java JMS Selectors. And the same query may use data from both headers and message content.

In this post, I will focus on XML content-based routing. Suppose a message publisher sends weather report data to an XML Exchange using the routing key “weather”. It might use the National Weather Service’s XML format - here’s an example of a report in this format:

http://www.weather.gov/xml/current_obs/KRDU.xml

Here’s a simpler version that we will use in our sample program:

<weather>
    <station>Raleigh-Durham International Airport (KRDU)</station>
    <wind_speed_mph>16</wind_speed_mph>
    <temperature_f>70</temperature_f>
    <dewpoint>35</dewpoint>
</weather>

Now I want to write a message consumer that tells me when there is good weather for sailing. In this post I will show how to route messages to a queue - the complete program is one of the Apache Qpid examples, and is covered in Red Hat’s MRG Messaging Tutorial.

In my little sailboat, I don’t like wind above 20 mph, and it’s a lot more fun when there’s at least 7 mph wind. Rain isn’t a big issue, but it’s a little nicer when it’s not raining, so I’d like to be 5 degrees above dewpoint or so. And I don’t usually sail when it’s colder than 50 degrees, which makes me a bit of a wimp. Here is an XQuery that tests a message to see if it meets these criteria:

   let $w := ./weather
   return $w/station = 'Raleigh-Durham International Airport (KRDU)'
      and $w/temperature_f > 50
      and $w/temperature_f - $w/dewpoint > 5
      and $w/wind_speed_mph > 7
      and $w/wind_speed_mph < 20

Now I can take that query and bind it to the XML Exchange, asking that it be used to decide whether to route messages with the routing key “weather”. In Python, that looks like this:

session.queue_declare(queue="message_queue")

binding = {}
binding["xquery"] = """
   let $w := ./weather
   return $w/station = 'Raleigh-Durham International Airport (KRDU)'
      and $w/temperature_f > 50
      and $w/temperature_f - $w/dewpoint > 5
      and $w/wind_speed_mph > 7
      and $w/wind_speed_mph < 20 """

session.exchange_bind(exchange="xml", queue="message_queue", binding_key="weather", arguments=binding)

As you can see, the XML Exchange allows message consumers to use sophisticated queries to decide which messages are interesting, and the message producer does not need to do anything special to provide the data used for these routing decisions. And it’s very simple to use!

XML Exchange Patent

Wednesday, March 18th, 2009

I seem to have achieved my 15 minutes of fame as the Inventor in a patent application for the XML Exchange filed by Red Hat. Some people are upset to see Red Hat file a patent application related to open standards and open source software.

Red Hat works hard to make it easier to write innovative software despite the toxic software patent environment. We all would like to see this fixed (are you listening, President Obama?) In the meantime, defensive patents are a necessary evil.

Here’s an excerpt from a statement on this controversy from Rob Tiller, Vice President and Assistant General Counsel of Intellectual Property at Red Hat:

http://www.press.redhat.com/2009/03/17/discouraging-software-patent-lawsuits/:

Red Hat has worked hard to address the problems of our patent system. We believe there are serious problems with the existing system, particularly as it affects free and open source software. In just the last few months, we have supported new patent reform legislation, submitted a brief in the Bilski case arguing against patenting of software, and created an innovative patent settlement in the FireStar case that gave broad protection to the open source community. We are also proud of our work in helping establish the Open Invention Network, supporting the Peer-to-Patent program, and developing our Patent Promise.

We have also worked to build a patent portfolio. As we have explained many times, the purpose of this portfolio is defensive. This means that it is designed to discourage patent lawsuits by giving us the ability to retaliate against potential patent aggressors by asserting counterclaims as a defense. We believe it is important to have such a portfolio, because of the threats of companies that are hostile to FOSS and that have amassed large stockpiles of overbroad patents.

We have made a public commitment to this purely defensive approach with our Patent Promise, which is at http://www.redhat.com/legal/patent_policy.html. This promise is binding. It demonstrates our determination to use our patent portfolio only in the defense of free software.

In the next few days, I’ll write a post that describes the XML Exchange and shows how to use it in programs.

Ken Holman teaches “Practical Transformation Using XSLT, XQuery and XPath”

Friday, March 6th, 2009

Ken Holman is an excellent teacher who is probably best known for his XSLT and XPath courses, which he has been offering for years. He has been in the SGML and XML communities for an extremely long time, and people know him as an interesting person and a really nice guy.

He has now added XQuery to his XSLT / XPath course:

http://www.cranesoftwrights.com/training/ptuxq/ptuxqsyl.htm

Video excerpts of his courses and excerpts of his training materials are available here:

http://www.cranesoftwrights.com/training/index.htm

There’s also a schedule of future training events, and he apparently licenses people to train using his materials.