QpidComponents.org

July 14th, 2009

We have launched a site called QpidComponents.org for Apache Qpid components that can not be developed as part of Apache Qpid per se, generally because they use software components that use open source licenses other than the Apache license.

Currently, these components include persistence and management tools.

See http:://QpidComponents.org!

The future of XHTML - and of XML?

July 6th, 2009

I was concerned late last week when I read that the XHTML Working Group will not be allowed to finish its work. The announcement indicates that this is being done to speed the work of HTML5 - but it also raises important questions about how XML will work with data on the Web.

http://www.w3.org/News/2009#item119

2009-07-02: Today the Director announces that when the XHTML 2 Working Group charter expires as scheduled at the end of 2009, the charter will not be renewed. By doing so, and by increasing resources in the HTML Working Group, W3C hopes to accelerate the progress of HTML 5 and clarify W3C’s position regarding the future of HTML. A FAQ answers questions about the future of deliverables of the XHTML 2 Working Group, and the status of various discussions related to HTML. Learn more about the HTML Activity.

Originally, HTML was based on SGML. XML was designed to make SGML easier to parse, so that it would be more accessible to software. From the beginning, XML was designed to support documents meant for humans to read, and data designed for programs. In modern software, there is no firm boundary between the two - data is often transformed on the fly for humans to read, and software often mixes human-readable content with data.

By representing HTML as an XML vocabulary, XHTML addresses HTML parsing issues, difficulty combining HTML with XML data, and extensibility issues:

  • An XML parser can not parse arbitrary HTML, and parsers which do are extremely complex. Tools like Tag Soup parse arbitrary HTML and convert it into XHTML, so that even legacy content in odd dialects of HTML can be processed by XML tools.
  • XML vocabularies like SVG (a graphics format) and MathML (a mathematical notation format) need to play nicely with HTML. XHTML lets them be combined with HTML using the same mechanisms all XML tools use to combine vocabularies.
  • HTML has become quite large, and XHTML provides a modularization of HTML, using XML mechanisms, that makes it easier to maintain and extend.

Typically, Working Groups are rechartered if there is active interest in their work and they are close to their goal, as the XHTML Working Group is. Another year of effort would probably be sufficient to finish the XHTML specifications. And the XHTML people are intimately familiar with XML and namespace issues, plus the issues that arise with XML and HTML software.

The FAQ that the announcement points to concerns me. Some of the work of the XHTML Working Group will not be finished, and there is even a suggestion that HTML5 will not handle namespaces correctly, and that XML vocabularies will not be able to mix freely with HTML5!

Consider this issue, to which the FAQ points:

http://www.w3.org/html/wg/tracker/issues/41

The HTML5 specification does not have a mechanism to allow decentralized parties to create their own languages, typically XML languages, and exchange them in HTML5 text/html serializations. This would allow languages such as SVG, MathML, FBML and a host of others to be included. At one point, an editors version of the HTML5 specification contained a subset and reformulation of SVG and MathML. Tim Berners-Lee described this incorporation of SVG and MathML without namespaces as horrific and the issue raiser [Dave Orchard] completely concurs with the him.

This issue limits the ability of non-HTML5 working groups to define languages as the languages must be “brought into” the HTML5 language. This dramatically increases the scope of HTML5 and decreases the ability to modularize development of orthogonal languages.

In the end, the problem could result in the text/html serialization rules becoming the standard serialization rules for XML languages, replacing XML itself. This could occur if every decentralized language has a choice between the XML serialization, the text/html serialization or both. In many cases, the language may choose the text/html serialization.

I find this last paragraph alarming. XML was designed specifically to allow decentralized parties to create their own languages, and to combine them cleanly using namespaces. By using the mechanisms defined in XML, XHTML inherits this ability, an ability that HTML5 lacks. One proposed solution is to absorb XML entirely into the HTML5 specification. I agree with Tim and Dave that incorporating SVG and MathML without namespaces is horrific, but the obvious solution is for HTML5 to support namespaces and XML cleanly - just as XHTML does.

I would like to see the XHTML Working Group finish their work, and I would like to see the result used as the XML serialization of HTML5. Legacy HTML ensures that there will always be large amounts of HTML content that is difficult to parse without tools like Tag Soup. But XHTML gives us a very sane XML vocabulary to convert such content into, and makes HTML accessible to the wide variety of XML tools that make up much of the infrastructure of the Web.

XQJ (XQuery API for Java) is now an official JSR!

July 2nd, 2009

The Expert Group for for JSR-000225 XQuery API for Java
has (finally!!!) completed its work and published XQJ as a Final Release.

You can get the XQJ specification at either of the following URLs:

Missing Balisage this year …

June 29th, 2009

I often attend the Balisage XML conference in Montreal, the best “geek-fest” gathering for XML techies:

http://www.balisage.net/

Balisage will be meeting from August 11 — 14, 2009, in Montréal, Canada. On August 10th, Mike Kay is doing a workshop on Processing XML Efficiently that looks especially interesting.

I hope my friends at Balisage will excuse my absence this year. I will be on my honeymoon.

MRG Messaging: A Programmer’s Overview

June 26th, 2009

I am going to be giving a presentation on MRG Messaging for programmers at the Red Hat Summit in Chicago, Thursday, September 3, from 1:30 to 2:30 p.m.

You can see the schedule here:

http://www.redhat.com/promo/summit/2009/agenda/tracks/

Here’s the blurb for the talk:

MRG Messaging: A Programmer’s Overview
Jonathan Robie, Software Engineer, Red Hat

Red Hat Enterprise MRG is an open source messaging system based on Advanced Message Queuing Protocol (AMPQ), an open standard for enterprise messaging. Red Hat Enterprise MRG allows users to write applications for simple, high-speed messaging in multiple languages (Java JMS, C++, Python, Ruby) and it allows users to run the applications on multiple platforms (Linux, Unix, Macintosh, or Windows). Red Hat Enterprise MRG provides guaranteed delivery, speed, and security.

This presentation serves as a programmer’s overview of Red Hat Enterprise MRG and is designed to teach programmers what they need to know to get started writing messaging applications. This presentation will feature:

  • Overviews of Red Hat Enterprise MRG, Apache Qpid, C++, and Python Messaging APIs
  • Discussion of the AMQP messaging standard and Apache Qpid
  • API summary for C++ and Python
  • Description of how to use Red Hat Enterprise MRG with Java JMS
  • Demonstration of how to write applications based on common messaging paradigms (e.g., direct, fanout, publish-subscribe, request-response, and XML content-based routing)
  • Demonstration of how to write Java JMS programs that use Red Hat Enterprise MRG
  • Demonstration of how to implement persistence, messaging transactions, and failover for clients using a high-availability cluster
  • Demonstration of the tools that can be used to manage the messaging server, view of the configuration, and track the state of messages in the system

Apache Qpid 0.5 is released!

May 27th, 2009

Apache Qpid 0.5 is now released! Download it here:

http://qpid.apache.org/download.html

Here is the text of the announcement:

http://mail-archives.apache.org/mod_mbox/qpid-users/200905.mbox/browser

The Apache Qpid community is pleased to announce the release of Apache Qpid 0.5

Apache Qpid (http://qpid.apache.org) is a cross platform enterprise messaging solution which implements the Advanced Message Queueing Protocol (http://www.amqp.org). It provides brokers written in C++ and Java and clients in C++, Java (including a JMS implementation), .Net, Python, and Ruby.

New features included in this release are:

C++ Broker

  • [QPID-1567] - Queue replication (asynchronous) between two sites
  • [QPID-1669] - Client connection management in the qpid-cluster CLI utility
  • [QPID-1673] - Dynamic Library Build on Windows (DLL)

Java Broker

  • [QPID-1583] - IP White/Black lists for virtual hosts
  • [QPID-1648] - Enable live reconfiguration of Log4J settings for
    the Java broker via JMX
  • [QPID-1699] - Reload security section in configuration files through JMX

C++ Client

  • [QPID-1673] - Dynamic Library Build on Windows (DLL)

Java Client

Ruby Client

Java Management : JMX Console

  • [QPID-1500] - Mac OS X Build
  • [QPID-1648] - Enable live reconfiguration of Log4J settings for the Java broker via JMX
  • [QPID-1691] - Linux x86-64 and Solaris builds

Java Management : QMan

It is available to download from:

http://qpid.apache.org/download.html

Complete release notes are available here:

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310520&styleName=Html&version=12313597

Tim Fox: AMQP “of strategic importance” for JBoss

April 23rd, 2009

Tim Fox has announced that

JBoss Messaging 2.x will implement AMQP, as we consider this to be of strategic importance, along with JMS, and of course the full set of enterprise messaging functionality that you’d expect in any serious messaging broker.

And also that JBoss will be licensed under Apache 2.0:

JBoss Messaging will also be relicensing it’s code under the ASL 2.0 licence (moving away from LGPL) in order to be usable by the widest range of interested parties.

XML and XQuery (Second Edition) - Proposed Edited Recommendations for review by 31 May

April 23rd, 2009

We have now released Proposed Edited Recommendations of the XQuery and XSLT specifications. These fix known bugs and problems, and are out for community review. Some of these changes affect conformance. If these documents pass public review, they will become W3C Recommendations.

These documents were published 21 April 2009, and the review period ends 31 May 2009:

Broker Federation in Apache Qpid

April 8th, 2009

In Apache Qpid, Federation is a feature that allows messages to be routed to other brokers automatically. Suppose a business receives requests at one central location, but needs to route them to regional servers or servers in local offices, based on information contained in the message headers. Federation allows this kind of routing to be performed by configuring the brokers, with no need for programming. And the rules used for routing can be changed dynamically as servers change, responsibilities change, at different times of day, etc.

Here are some use cases for federation (taken from http://qpid.apache.org/using-broker-federation.html):

The Qpid C++ messaging broker supports broker federation, a mechanism by which large messaging networks can be built using multiple brokers. Some scenarios in which federation is useful:

  • Connecting disparate locations across a wide area network. In this case full connectivity across the enterprise can be achieved while keeping local message traffic isolated to a single location.
  • Departmental brokers that have a policy which controls the flow of inter-departmental message traffic.
  • Scaling of capacity for expensive broker operations. High-function exchanges like the XML exchange can be replicated to scale performance.
  • Co-Resident brokers Some applications benefit from having a broker co-resident with the client. This is particularly true if the client produces data that must be delivered reliably but connectivity to the consumer(s) is non-reliable. In this case, a co-resident broker provides queueing and durablilty not available in the client alone.
  • Bridging disjoint IP networks. Message brokers can be configured to allow message connectivity between networks where there is no IP connectivity. For example, an isolated, private IP network can have messaging connectivity to brokers in other outside IP networks.

For details on setting up Federation, see the description of qpid-route in the Apache Qpid article, “Using Broker Federation“.

Using the XML Exchange

March 26th, 2009

The XML Exchange allows routing decisions to be made using XQuery. It is used primarily for two things:

  • XML Content-based Routing - a queue can be bound to an exchange using an XQuery that identifies “interesting” messages. This is particularly useful when the message publisher does not know what information a message consumer may use to decide whether to route the message, since the publisher and consumer do not need to agree on a set of headers.
  • Queries on Headers - the XML Exchange can query headers using XQuery whether or not the message content is in XML, so this provides query capability analogous to Java JMS Selectors. And the same query may use data from both headers and message content.

In this post, I will focus on XML content-based routing. Suppose a message publisher sends weather report data to an XML Exchange using the routing key “weather”. It might use the National Weather Service’s XML format - here’s an example of a report in this format:

http://www.weather.gov/xml/current_obs/KRDU.xml

Here’s a simpler version that we will use in our sample program:

<weather>
    <station>Raleigh-Durham International Airport (KRDU)</station>
    <wind_speed_mph>16</wind_speed_mph>
    <temperature_f>70</temperature_f>
    <dewpoint>35</dewpoint>
</weather>

Now I want to write a message consumer that tells me when there is good weather for sailing. In this post I will show how to route messages to a queue - the complete program is one of the Apache Qpid examples, and is covered in Red Hat’s MRG Messaging Tutorial.

In my little sailboat, I don’t like wind above 20 mph, and it’s a lot more fun when there’s at least 7 mph wind. Rain isn’t a big issue, but it’s a little nicer when it’s not raining, so I’d like to be 5 degrees above dewpoint or so. And I don’t usually sail when it’s colder than 50 degrees, which makes me a bit of a wimp. Here is an XQuery that tests a message to see if it meets these criteria:

   let $w := ./weather
   return $w/station = 'Raleigh-Durham International Airport (KRDU)'
      and $w/temperature_f > 50
      and $w/temperature_f - $w/dewpoint > 5
      and $w/wind_speed_mph > 7
      and $w/wind_speed_mph < 20

Now I can take that query and bind it to the XML Exchange, asking that it be used to decide whether to route messages with the routing key “weather”. In Python, that looks like this:

session.queue_declare(queue="message_queue")

binding = {}
binding["xquery"] = """
   let $w := ./weather
   return $w/station = 'Raleigh-Durham International Airport (KRDU)'
      and $w/temperature_f > 50
      and $w/temperature_f - $w/dewpoint > 5
      and $w/wind_speed_mph > 7
      and $w/wind_speed_mph < 20 """

session.exchange_bind(exchange="xml", queue="message_queue", binding_key="weather", arguments=binding)

As you can see, the XML Exchange allows message consumers to use sophisticated queries to decide which messages are interesting, and the message producer does not need to do anything special to provide the data used for these routing decisions. And it’s very simple to use!