Subject: RE: INCITS-V1: Procesing models and XML

From: "Mason, James David (MXM)" <[email-masked]>
Date: Tue, 05 Jun 2007 16:20:05 -0400

What Patrick has come across has to do with an underlying model that I
noticed long ago in Word. I've been using Word since Version 1.0 came
out long ago, so I'm used to the model and know how to cope with its
quirks (most of the time). But you have to think like Word to get the
most out of it.

The simplest document model, one that underlies the earliest batch
formatters like Runoff and troff, is just sticking commands in a file.
Each takes effect when it's inserted and stays in effect until something
overrides it. That was also the model behind the earliest stand-alone
word processors, and it's largely still the model behind WordPerfect. 

Programmable batch formatters, like troff and TeX and IBM's Script made
it possible to write macros and thus to implement generic coding. That's
the environment that gave birth to SGML and XML -- and to the model that
Patrick associates with ODF.

Word does something entirely different. Word came to Microsoft from a
program orignially developed at Xerox PARC. PARC was very much into
object-oriented programming, like Smalltalk, and Word partakes of that
spirit. It's also related to stack-oriented, postfix-operator
programming languages. Patrick mentions "sections": in Word, a section
is just a fairly high-level container to which properties, like margins
or page numbering, can be attached. So think of text entry as shoving
bytes on a stack. Then at some point you put in an operator that pops
the stack and applies itself to everything that comes off. Nesting of
sections is somewhat limited; they tend to pop the stack. 

I've spent a lot of time working with folks who were long-term users of
WordPerfect and who were getting frustrated making the transition to
Word. All of them were used to running WordPerfect in split-screen
"reveal codes" mode. That's easy when a file is linear and text and
codes just came one thing after another. They were always looking for a
similar mode in Word and couldn't understand that Word didn't have any
such linear structure. They were also used to seeing margins set first,
rather than as a postfix property of a whole section after the section
content was complete, and they couldn't understand how Word was doing
things.

Another aspect of Word's data structure, I've come to recognize, is that
a lot of it is implicit. All the data necessary to create the rendition
of a Word file in Word itself is in the file, but it's not easily
extracted. It becomes evident only when the file is instantiated in the
program, which knows how to deal with the implicit parts. 

In XML we're used to clearly marked (tagged) containment and
hierarchies. We think, for example, of having a <sect1> that contains a
<title> and a bunch of <para> elements. If we have a <sect2>, it's
clearly delimited inside the <sect1>. That's the sort of thing we see in
applications like DocBook (Norm, correct me if you need to). 

Of course there have been unstructured document models, like HTML, that
tagged only locally, and you could put <H1> and <P> and <H2> in the file
in any crazy order. Word appears to be like that: you can put in text
and apply styles in any crazy order, so you can have the first paragraph
of a document in "Normal" style, then one in "Heading2", one in
"NumberedList" and then one in "Heading1". But just as a Web browser
attempts to recover from any crazy codes you throw at it, Word attempts
to build -- in memory -- a structure from the styles. You can see this
structure if you put Word into "Outline View" rather than "Layout View"
or "Normal View". This structure acts pretty much like the hierarchical
structure with proper containment that we expect from the usual XML
application. But it is, so far as I can tell, something that happens
only when the file is in memory, and it's driven entirely from the
styles. (By default, it works with Word's predefined Heading1, Heading2,
etc., styles, though you can map user-generated styles to act like
those.) 

What Word is doing, in effect, is acting as if it's seeing a file with
lots of markup minimization: A "Heading1" style acts as both the start
tag for a chunk of document and the end tag for the previous chunk, if
any. Another way of looking at it from an XML perspective is that you
can work on the DOM view but the SAX view doesn't exist.

I haven't had time to do extensive analysis of OOXML, but I gather from
the things that Patrick has told me that it appears to lack conventional
XML structure. That would make sense if the data in the interchange file
was expecting to be interpreted by a piece of software (i.e., Word) that
has the capability to instantiate an implicit structure just from seeing
styles attached to pieces of data. I'd love for someone from Microsoft
to elucidate the matter.

Jim Mason

(For what it's worth, if my surmises are true, I think that what it
would imply would be that OOXML is not by itself a sufficient definition
of an interchange format: it would depent on something outside the data
stream to complete the structure. Whether that would be satisfied by a
mapping to a binary structure is beyond where I've speculated so far.)

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of Patrick Durusau
Sent: Monday, June 04, 2007 7:12 PM
To: [email-masked]
Subject: INCITS-V1: Procesing models and XML

Greetings!

I have been thinking about processing models and XML generally and XML
based office formats in particular over the weekend. (Little else to do
while waiting on parsers and documentation generators.)

When I say "XML based office formats" I immediately think in terms of
document models. That is to say that an XML format enforces a particular
model for some given class of documents.

But, I have come to think that view of XML implies something else. That
is it also implies a processing model in addition to the document model.

Although I have long argued that XML is an *interchange* syntax and has
nothing to do with how an XML file is processed after receipt, that is
sadly (in my view) not really true for most XML based applications. That
is to say that well-formedness and validity are measures to insure
interchange and not how the document will be processed by some
particular application.

Let's take a fairly neutral illustration to make that point:

In OOXML, section information appears at the end of a "section" with
additional formatting information. If you look at the XML source for the
Primer (Part 3) you will find that it has only two sections for an
almost 500 page document.

In ODF, I can replicate the same information using styles but then that
information is stored in a stylesheet.

That is to say the informational content is the same but the logic of
the applications to use that information is completely different.

In OOXML, the application is expecting the "section" information to
appear at the end of the text stream of markup.

In ODF, the application is expecting the "section" information in a
stylesheet and applies it to the content with that as an associated
style.

What I am saying poorly is that what OOXML represents is XML to support
a different processing model (indeed even a different document model)
than what we are accustomed to seeing in XML.

That is not to say that OOXML isn't being processed as XML, it certainly
is, but the underlying document model supports a particular processing
model for that content. A different processing model in markup would not
work with applications with that processing model.

I am still searching for a way to describe the XML document model in
OOXML. Arbitrary really doesn't work because it is not wholly
unconstrained. Granular doesn't work either as other schemas have
granular markup in addition to more traditional structures. This really
isn't an issue for V1 but from a theoretical pespective I would find it
helpful to be able to locate it within a class of schemas that share
common characteristics. (Other than being used by MS. ;-) An interesting
characteristic but not one that helps with theoretical analysis.)

Hope everyone is at the start of a great week!

Patrick

PS: Joseph responded offlist to point out that I was eliding over the
"3" in several schema file names in my quest to build the ur schema. ;-)
Auto-correction by your eyes is a nice feature but sometimes, like all
features, it can be a bug. ;-)

--
Patrick Durusau
[email-masked]
Chair, V1 - Text Processing: Office and Publishing Systems Interface
Co-Editor, ISO 13250, Topic Maps -- Reference Model Member, Text
Encoding Initiative Board of Directors, 2003-2005

Topic Maps: Human, not artificial, intelligence at work!