Date: Mon, 30 Sep 1996 17:44:01 -0700 To: papresco@calum.csclub.uwaterloo.ca From: Ralph Ferris Subject: Re: Will XML go beyond SGML Users? Cc: ralph@fsc.fujitsu.com, w3c-sgml-wg@w3.org You wrote: >As I understood it, it was our goal that all browsers should support XML >just as they do HTML. Since even Internet Explorer cannot open either RTF or >Word files, Internet Explorer 3.0 can open any ActiveX enabled control, including Word. Word (or Excel, or whatever) will then take over the entire client viewing area, adding its own toolbar, i.e., there is seemless integration within the original window. It's also worth noting that URLs are now recognized in non-HTML documents (some mail systems support this feature as well). So Word users do not need to convert their documents in order to post them to the Web. Conversely, what user's think of as the browser has been de-coupled from the formats it reads. An ActiveX control can be created to support any document markup. So a new markup could be created and popularized, not by creating a dedicated browser that supports it, but by supplying an ActiveX control that supports it. XML could be introduced to the market in this way. Regards, Ralph E. Ferris Project Manager, Electronic Publications Fujitsu Software Corporation Date: Mon, 30 Sep 1996 19:13:34 -0900 To: paul@arbortext.com (Paul Grosso) From: "W. Eliot Kimber" Subject: Re: Current Status of Discussion on RE/RS Handling Cc: w3c-sgml-wg@w3.org In following the subsequent discussion, I realized that my response to Bob was flawed in that I forgot that white space following end tags and preceding start tags is *not* ignored. Darn. I've corrected my example below, using "@" for spaces. E. At 07:54 AM 9/30/96 -0900, W. Eliot Kimber wrote: >>I'm curious to understand how the proposal would address the white-space >>in the following snippet. >> >> >> WFMD-AA, WAFE MAIN DRAWER >> 1.0000 >> >> >> >> 6.0000 >> >> >> 8.0000 >> >> >> 36.0000 >> >>... >> > >Using NSGMLS style output (and ignoring attributes): >(ROOT >(DS >-WFMD-AA, WAFE MAIN DRAWER >)DS -@@@ >(Q >-1.0000 >)Q -\n@@ >(DWG >(USES >(REF >(Q >-6.0000 >)Q >)REF -@@@@@ >(REF >(Q >-8.0000 >)Q >)REF -@@@@@ >(REF >(Q >-36.0000 >)Q >)REF -@[...] >)ROOT >There is *no* uncollapsed space in the example because, unless I've gotten >it horribly wrong, all the white space falls under rule 1. I did get it horribly wrong. My appologies [why can't we just end this madness! Just do what James says and nobody gets hurt...] Cheers, E. -- W. Eliot Kimber (kimber@passage.com) Senior SGML Consultant and HyTime Specialist Passage Systems, Inc., (512)339-1400 10596 N. Tantau Ave., Cupertino, CA 95014-3535 (408) 366-0300, (408) 366-0320 (fax) 2608 Pinewood Terrace, Austin, TX 78757 (512) 339-1400 (fone/fax) http://www.passage.com (work) http://www.drmacro.com (home) "If I never had existed, would you still remember me?..." --Austin Lounge Lizards, "1984 Blues" Date: Mon, 30 Sep 96 20:53:08 CDT To: Charles@sgmlsource.com From: Robert Streich Subject: Re: More on RE/RS Cc: lee@sq.com, dgd@cs.bu.edu, w3c-sgml-wg@w3.org At 01:06 PM 9/30/96 GMT, Charles F. Goldfarb wrote: >Could you picture a trivial variant of SGMLNORM (XMLNORM?) that brackets all the >data and solves the RE/RS/whitespace problem properly? Sure I could picture it. It's always in the back of my mind as an option should XML turn out too strange. With OmniMark in my back pocket and the tremendous potential of Jade round the corner, I feel strangely secure. (By the way, shouldn't we all chip in and take out an insurance policy on James?) I had such a thing in mind way back when I suggested "client/server" XML as an option. Didn't get much interest, though. I kind'a wish it were a viable option though since it would ease the tension between easy to type and easy to parse. You could make the client parser *very* simple. The server parser could be more complex, but not much more so. Perhaps I shouldn't have used "client/server" as the name. I really see it to be more like elisp with a byte compiler only instead of compiled, you have a very easy to parse, but still human-readable version for machine to machine communications (or agent to agent if you prefer). bob Robert Streich streich@slb.com Schlumberger voice: 1 512 331 3318 Austin Research fax: 1 512 331 3760 Date: Mon, 30 Sep 96 20:52:56 CDT To: Charles@sgmlsource.com From: Robert Streich Subject: Re: questions about entities and entity declarations Cc: paul@arbortext.com (Paul Grosso), w3c-sgml-wg@w3.org At 01:02 PM 9/30/96 GMT, Charles F. Goldfarb wrote: >I don't see much difference between a character number and an alphabetic symbol. >Both ultimately have to be mapped to some character semantic, and thence to a >glyph. But that mapping should occur in the style sheet. But you can't make the mapping if all you have is a number. In the 8859-1 world, you have to select the font first. If you only have a number you can only assume the current font. Since that usually restricts you to ISO Latin1, you can't even get an emdash into the text, let alone math symbols. >If you want to use symbolics, use the existing ISO character entity sets and >remove the SDATA keyword, since the entity text isn't system-specific data. (I >originally intended those declarations to be templates for real system data, but >no one uses them that way, I've been told.) Some tools do use the replacement text to good effect (even your favorite-- Author/Editor). I like it 'cause it gives me an opportunity to make some changes as necessary. Other tools just use them as symbolic names and make the mapping elsewhere (forcing me to go hunt it down). >But I'm happy to leave the resolution of character set issues to those more >passionate about them than I. I only proposed eliminating internal SDATA to >simplify entity declarations and because, unlike external SDATA entities, there >is no standardized way to specify a notation. Of course, we could adopt the >convention that I (and others) proposed earlier for PIs, but that would still >leave us with a special kind of internal entity, and I think that XML should >have all its internal entities resolved. I don't have a clue how character set issues will affect this. In my current systems, it's completely disconnected because the glyph for a character depends on the current font. If anyone could assure me that this will be a non-issue, then I'll be happy (as long as I have their home address for insurance...). bob Robert Streich streich@slb.com Schlumberger voice: 1 512 331 3318 Austin Research fax: 1 512 331 3760 Date: Mon, 30 Sep 96 21:09:45 CDT To: "W. Eliot Kimber" From: Robert Streich Subject: Re: Current Status of Discussion on RE/RS Handling Cc: w3c-sgml-wg@w3.org At 12:48 PM 9/30/96 -0900, W. Eliot Kimber wrote: >At 12:44 PM 9/30/96 CDT, Paul Grosso wrote: > >[... example omitted, see Paul's message ] > >>a. If 1 and 2 above are the only rules, then does that mean that the >>space and RE between the title and first paragraph are significant? >> >>b. Given the clarification of rule 2, then all of paragraph one's REs are >>collapsed to a single space, but paragraph two contains " @ @ @ @". Then, >>I assume rule 2 says each @ gets "collapsed" into a single space resulting >>in " ". Am I understanding this correctly? > >That's the way I interpret it: If you don't want spaces, don't put them in. So, correct me if I'm wrong, but with the proposed set of rules: "If you don't want spaces, don't put them in." With the other proposal: "If you don't want newlines, don't put 'em in." (I'm not ignoring the other proposals just juxtaposing these two.) I don't see a substantive difference other than that I lose the ability to do verbatim formatting with the former. bob Robert Streich streich@slb.com Schlumberger voice: 1 512 331 3318 Austin Research fax: 1 512 331 3760 From: Charles@sgmlsource.com (Charles F. Goldfarb) To: "David G. Durand" (David G. Durand) Cc: w3c-sgml-wg@w3.org Subject: Re: RS/RE: basic questions Date: Tue, 01 Oct 1996 03:19:59 GMT On Mon, 30 Sep 1996 12:10:33 -0400, "David G. Durand" (David G. Durand) wrote: >So you beg the question in another way. The issue is _why_ we should treat >the RE as markup rather than data. Since there is no _requirement_ to >include an RE after a tag, why _must_ it not be significant. Because the tag isn't part of the data. Therefore, any RE whose presence depends on where a tag is positioned can't possibly be part of the data either. >I've shown how >you can trivially add line RE and whitespace as needed to pretty-print your >text without the RE ignoring rule. >

>This is a paragraph.

It's not quite as pretty, or as natural, as:

This is a paragraph.

or even:

"This is a paragraph."

> > Other than pretty printing, why should that information not be >significant? I would appreciate a user-level justification, not a another >"true information" comment that pre-supposes that whitespace after markup >must be insignificant. The requirement that it be insignificant is the >issue at question. I don't see that the rule gains us anything of >importance, but I could be convinced, given an argument. O.K. I'll try again. A user wants to capture a particular piece of data. He knows what it is. He knows whether there are REs in it. He takes the following steps (which you can follow on your own text editor): 1. He opens a text editor and types in the data, putting the REs where he wants them. Now is the time for all good men to come to the aid of the party. 2. He now wants to identify that data as a "p" element, so he enters tags, like this:

Now is the time for all good men to come to the aid of the party.

3. He decides that the structure would look clearer during editing if he put the tags on their own input lines, like this:

Now is the time for all good men to come to the aid of the party.

In all of these steps, he hasn't changed one character of the data. So why should the application see a single RE in step 2 and three REs in step 3? The data has only ever had a single RE. > I keep thinking about a bug I was told about where a database was driven >by the ESIS, and later spat out the markup without a CR after every tag. >For some elements with several REs at the beginning they would thus lose >one _significant_ RE on every check-in/check-out cycle. They had to put an >RE after every tag to make things work right (or they would have had to >hack their parser to inform the application of the "non-significant" >whitespace). The operant word is "bug". If you are regenerating SGML from a database and the data of an element starts with a significant RE, you obviously need to insert an insignificant RE after the start-tag. In other cases you don't have to, but it is always safe to do so. Both algorithms are pretty simple, so a good system will allow the user to choose between them. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- From: Charles@sgmlsource.com (Charles F. Goldfarb) To: gtn@ebt.com (Gavin Nicol) Cc: jjc@jclark.com, lee@sq.com, w3c-sgml-wg@w3.org Subject: Re: Current Status of Discussion on RE/RS Handling Date: Tue, 01 Oct 1996 03:20:02 GMT On Mon, 30 Sep 1996 09:08:39 -0400, gtn@ebt.com (Gavin Nicol) wrote: > >Then who is to define *how* an entity manager is to recognise these >"lines"? It varies by the file system (or more generally, storage manager). IBM mainframes use a length field, DOS uses CR/LF, and so on. There is a good description of this in the FSI spec. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- From: Charles@sgmlsource.com (Charles F. Goldfarb) To: "David G. Durand" (David G. Durand) Cc: w3c-sgml-wg@w3.org Subject: Re: Element content the real issue?... Date: Tue, 01 Oct 1996 03:20:09 GMT On Mon, 30 Sep 1996 12:10:42 -0400, "David G. Durand" (David G. Durand) wrote: > We basically cannot afford to process element and non-element content >differently with regard to whitespace or anything else. > > ==> So we can't allow any ignored whitespace anywhere without resorting >to quoting, because of the non-DTD parsing requirement. Exactly. So having recognized the unavoidable fact, why do you keep trying to avoid it? I can understand trying to ameliorate its impact, but let us first face up to the truth: this is the only way we can go. > > My application of his theories says that we should change as little as >possible from HTML (the market leader), while adding the minimum we can >manage to get the most useful new functionality. This conclusion contradicts the inescapable fact. His earlier quote says we should keep things familiar, so I offer this premise: 1. Most people will create XML with a program that hides the real data format, in which case the "niceness" and "familiarity" of the format are non-issues. 2. People who will create XML with "dumb" editors or mode-assisted (dull?) editors also have written programs. To them, quoting data is quite familiar and macros can be written to assist with it. >I must say that I don't >see the point of targeting only the SGML community, because they already >have SGML. The SGML community needs a lean, mean conforming delivery format, which XML can be. It will be confusing enough to the market to have SGML and XML. Having a third "version" of SGML is at least one too many. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- From: lee@sq.com Date: Mon, 30 Sep 96 23:12:34 EDT To: jenglish@crl.com, w3c-sgml-wg@w3.org Subject: Re: Element content the real issue?... Joe English > [...] How about the following > as a heuristic to distinguish element content from mixed content: > > 3. If the only data appearing between two tags is a sequence of > lexical SEPCHARs (including RS and RE), then it is deemed > insignificant.

That doesn't work.

^-- you lose this space. If you want to inspect the entire element to see if it contains anything except spaces and sub-elements, you're in for a lot of lookahead (consider in a well-formed RFC 1822 document!). And in any case, just because my paragraph only contains individual emphasised words does not mean that the spaces (or record ends) are insignificant.

That doesn't work.

should be the same, right? I don't think any white-space should be discarded by the parser. Lee From: Charles@sgmlsource.com (Charles F. Goldfarb) To: "W. Eliot Kimber" Cc: paul@arbortext.com (Paul Grosso), w3c-sgml-wg@w3.org Subject: Re: Current Status of Discussion on RE/RS Handling Date: Tue, 01 Oct 1996 03:20:15 GMT On Mon, 30 Sep 1996 19:13:34 -0900, "W. Eliot Kimber" wrote: >>There is *no* uncollapsed space in the example because, unless I've gotten >>it horribly wrong, all the white space falls under rule 1. >I did get it horribly wrong. My appologies [why can't we just end this >madness! Just do what James says and nobody gets hurt...] Right. Delimit all data and be done with it! -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- From: Charles@sgmlsource.com (Charles F. Goldfarb) To: Paul Prescod Cc: w3c-sgml-wg@w3.org Subject: Re: Combine RE/RS proposals? Date: Tue, 01 Oct 1996 03:20:07 GMT On Mon, 30 Sep 1996 09:51:09 -0400, Paul Prescod wrote: >Could we combine proposals? Could we (in XML) define characters to begin a >section of text guaranteed not to have markup within, where spaces and >newlines should be passed to the application unmolested? In other words, a >verbatim delimiter? At that point it might be best to bite the rest of the bullet and always delimit data. Look at the advantages: 1. No need to access a DTD to determine if an element is mixed content. 2. All REs and RSs are ignored -- full stop! (See, I'm internationalized :-) 3. Really easy parsing. 4. Much simpler to explain. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- Date: Mon, 30 Sep 1996 22:25:22 -0500 From: Len Bullard To: Ralph Ferris CC: papresco@calum.csclub.uwaterloo.ca, w3c-sgml-wg@w3.org Subject: Re: Will XML go beyond SGML Users? Ralph Ferris wrote: > > You wrote: > > >As I understood it, it was our goal that all browsers should support XML > >just as they do HTML. Since even Internet Explorer cannot open either RTF or > >Word files, > > Internet Explorer 3.0 can open any ActiveX enabled control, including Word. > Word (or Excel, or whatever) will then take over the entire client viewing > area, adding its own toolbar, i.e., there is seemless integration within the > original window. It's also worth noting that URLs are now recognized in > non-HTML documents (some mail systems support this feature as well). And Word can now open Web documents. Sure has improved printing. ;-) >So Word > users do not need to convert their documents in order to post them to the > Web. Conversely, what user's think of as the browser has been de-coupled > from the formats it reads. An ActiveX control can be created to support any > document markup. So a new markup could be created and popularized, not by > creating a dedicated browser that supports it, but by supplying an ActiveX > control that supports it. XML could be introduced to the market in this way. Right. Welcome to framework design: to the notations, and beyond! The trick is, an XML viewer that just renders XML isn't as interesting an XML-handler that understands/inteprets other semantics of the XML elements. IOW, not just AN active-X, but lots of them, some dedicated to very simple XML aggregates. XML, IMO, allows us to send property values where some properties are complex as could be the case for a nest. Anyway, the Active X way is one way. It is a solid commercial way. VRML (which is actually an object language), so far is having a devil of a time with the issue of that interface. Best we leave it for the next phase. Yes, XML will go beyond SGML users, but because we are letting it go beyond SGML. We will find a lot of the same users. Heck, that's us. len From: lee@sq.com Date: Mon, 30 Sep 96 23:53:07 EDT To: Charles@sgmlsource.com, papresco@calum.csclub.uwaterloo.ca Subject: Re: Combine RE/RS proposals? Cc: w3c-sgml-wg@w3.org > 1. No need to access a DTD to determine if an element is mixed content. > 2. All REs and RSs are ignored -- full stop! (See, I'm internationalized :-) > 3. Really easy parsing. > 4. Much simpler to explain. Will the obvious \" work as expected? Does "string concatenation" work?

"this is line one " "this is line two " "this is line three"

if so, it can probably be made to fly. If not, I think it looks odd enough that you might as well but the bullet and use RTF or TeX syntax, \p{this is line one this is line two this is line three} with no hint of backwards compatibility. Then there is no problem. But it all seems like overkill to me. If you could have a top-level inclusion for #PCDATA, we would always be in mixed content, and the collapse-as-per-mixed-content rules would work fine in both SGML and XML. So how about adding the possibility of +(#PCDATA) in SGML? Lee To: w3c-sgml-wg@w3.org Subject: Re: Element content the real issue?... Date: Mon, 30 Sep 1996 23:45:48 -0700 From: Joe English lee@sq.com wrote: > Joe English > > [...] How about the following > > as a heuristic to distinguish element content from mixed content: > > 3. If the only data appearing between two tags is a sequence of > > lexical SEPCHARs (including RS and RE), then it is deemed > > insignificant. > >

That doesn't work.

> ^-- you lose this space. Yes: that's an example of a >> [...] "true" pseudoelement[...] >> that contain[s] nothing but lexical whitespace >> [... and would] have to be escaped or entered as references Are there any other cases where this heuristic fails? --Joe English jenglish@crl.com Date: Tue, 01 Oct 1996 09:52:17 +0000 To: Michael Sperberg-McQueen From: James Clark Subject: Re: Current Status of Discussion on RE/RS Handling Cc: W3C SGML Working Group At 17:45 30/09/96 CDT, Michael Sperberg-McQueen wrote: >Can anyone name a widely used >production text processor that has no equivalent to the \obeylines of >TeX or the XMP of the GML starter set? Microsoft Word: in RTF CR/LF is ignored unless quoted with a backslash. James Date: Tue, 01 Oct 1996 04:53:27 -0400 To: Ralph Ferris , papresco@calum.csclub.uwaterloo.ca From: Paul Prescod Subject: Re: Will XML go beyond SGML Users? Cc: ralph@fsc.fujitsu.com, w3c-sgml-wg@w3.org At 05:44 PM 9/30/96 -0700, Ralph Ferris wrote: >You wrote: > >>As I understood it, it was our goal that all browsers should support XML >>just as they do HTML. Since even Internet Explorer cannot open either RTF or >>Word files, > >Internet Explorer 3.0 can open any ActiveX enabled control, including Word. >Word (or Excel, or whatever) will then take over the entire client viewing >area, adding its own toolbar, i.e., there is seemless integration within the >original window. But you must download the Active-X or buy the product to do this. Over a 28.8 modem, a "Word Viewer" is going to take a while to download. Microsoft has explicitly chosen NOT to ship this with their product, so they obviously are NOT pushing Word's format (or RTF) as a Microsoft-shoves-it-down-your-throat-standard. Which is why I disagree with Charles that we could not achieve greater support for it. It's also worth noting that URLs are now recognized in >non-HTML documents (some mail systems support this feature as well). So Word >users do not need to convert their documents in order to post them to the >Web. Conversely, what user's think of as the browser has been de-coupled >from the formats it reads. An ActiveX control can be created to support any >document markup. So a new markup could be created and popularized, not by >creating a dedicated browser that supports it, but by supplying an ActiveX >control that supports it. XML could be introduced to the market in this way. If necessary. For all that Microsoft, Sun, Java tries to make components seem "native" native still seems more native. Native browser support would be better. Paul Prescod Date: Tue, 01 Oct 1996 04:53:32 -0400 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: Current Status of Discussion on RE/RS Handling At 07:13 PM 9/30/96 -0900, W. Eliot Kimber wrote: >In following the subsequent discussion, I realized that my response to Bob >was flawed in that I forgot that white space following end tags and >preceding start tags is *not* ignored. It was also my intuition that whitespace should be handled as it traditionally is in SGML and HTML (especially in "element content" where superflurous whitespace is very common) Joe English's rewording of my idea seems to reflect that intuition: 3. If the only data appearing between two tags is a sequence of lexical SEPCHARs (including RS and RE), then it is deemed insignificant. What do you think? This single, simple rule seems to cover the places where I would want to use whitespace as formatting. Paul Prescod Date: Tue, 01 Oct 1996 05:47:41 -0400 To: lee@sq.com, Charles@sgmlsource.com, papresco@calum.csclub.uwaterloo.ca From: Paul Prescod Subject: Re: Combine RE/RS proposals? Cc: w3c-sgml-wg@w3.org At 11:53 PM 9/30/96 EDT, lee@sq.com wrote: >If not, I think it looks odd enough that you might as well but the >bullet and use RTF or TeX syntax, >\p{this is line one >this is line two >this is line three} >with no hint of backwards compatibility. But this looks funny: \p{this is} \em{line} {one} Since the goal is to delimit the start and end of the significant text content, this would be required. >If you could have a top-level inclusion for #PCDATA, >we would always be in mixed content, and the collapse-as-per-mixed-content >rules would work fine in both SGML and XML. Every table manipulation program in the world would have to manually strip the spaces interpreted as content between table rows (or users would have to know to not put spaces in). Paul Prescod From: Charles@sgmlsource.com (Charles F. Goldfarb) To: lee@sq.com Cc: papresco@calum.csclub.uwaterloo.ca, w3c-sgml-wg@w3.org Subject: Re: Combine RE/RS proposals? Date: Tue, 01 Oct 1996 10:12:56 GMT On Mon, 30 Sep 96 23:53:07 EDT, lee@sq.com wrote: >> 1. No need to access a DTD to determine if an element is mixed content. >> 2. All REs and RSs are ignored -- full stop! (See, I'm internationalized :-) >> 3. Really easy parsing. >> 4. Much simpler to explain. > >Will the obvious \" work as expected? Yes. From an SGML standpoint it would just be a shortref resolving to ". > >Does "string concatenation" work? >

> "this is line one " > "this is line two " > "this is line three" >

Yes. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- From: Charles@sgmlsource.com (Charles F. Goldfarb) To: masonjd@ornl.gov (James David Mason) Cc: Rick Jelliffe , w3c-sgml-wg@w3.org Subject: Re: Make DTDs optional? Date: Tue, 01 Oct 1996 10:12:54 GMT On Mon, 30 Sep 1996 13:49:24 -0500, masonjd@ornl.gov (James David Mason) wrote: > >Charles and I wasted plenty of time dealing with ODA, and I'd hate to >replicate that experience. But we did learn some things from it, like what >led to architectural forms. It would be a great mistake if, while trying to >simplify building applications on something that was related to SGML, we >wound up getting so convoluted as ODA became (so convoluted it was >unimplementable). However, their original idea was implicit logical >structures, things that could be drived from tracing levels of nesting. I >think that's what I'm hearing here. As Jim says, ODA failed when it tried to become a native archiving/manipulation format for editors. That screwed up everything, including its usefulness for interchange. Its original interchange representation captured the structure without requiring a generic part (like our DTD). It was therefore very appealing because of its combination of structural richness and parsing simplicity. >I'm one of the folks who believes in mandatory DTDs, but that's because most >of the applications I deal with fall into what Rick calls a >"archiving/modelling/manipulation/open-system format". Maybe there is a >valid case for implicit structure in a "temporary/application/closed-system >format". If we go that route, we really do need to keep things simple and >avoid overspecifying. And we have to accept the idea that it can only be read-only. Or at least that you could only modify the data, not the structure. (Now that Jim has dug up those painful memories, I seem to recall that ODA also made the distinction between editing the structure and editing the data content.) -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- From: Charles@sgmlsource.com (Charles F. Goldfarb) To: Rick Jelliffe Cc: w3c-sgml-wg@w3.org Subject: Re: Make DTDs optional? Date: Tue, 01 Oct 1996 10:12:51 GMT On Tue, 1 Oct 1996 01:54:12 +1000 (EST), Rick Jelliffe wrote: >On Mon, 30 Sep 1996, Charles F. Goldfarb wrote: > >> 3. They won't get that support unless we make XML absurdly easy to implement. >> That means a delivery form of an SGML instance with no tricky parsing,no SGML >> declaration, and no DTD. > >I agree 100% with #3. I don't think the ease which someone with a text >editor can produce junk should be too influential on XML. > >But I think there may be a different potential user base than existing >SGML applications. The plug-in/compound-document market may be large >(VBX, Java applets, Netscape plugins, OpenDoc, OLE {whatever it is called >this week}, etc) and could do with some standard text formats for the >small input documents these use. I think it will be the same user community -- those who know the value of SGML -- but this "XML as delivery form of SGML" will give them a bunch of new opportunities, such as you describe. Those opportunities, in turn, will extend that user base. > > >Is this really more what XML should be about: a markup language for >presenting documents in the form required by the application? (Which >would correspond to the normalised form of an SGML document when parsed >against archetectural forms that model the XML application.) In other >words, a temporary/application/closed-system format rather than a >archiving/modelling/manipulation/open-system format like SGML. Yes. Yes. Yes. That should be our slogan: Archive in SGML, Deliver in XML. It's a damn sight better than today's: Archive in SGML, Deliver in HTML. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- From: Charles@sgmlsource.com (Charles F. Goldfarb) To: Paul Prescod Cc: Ralph Ferris , papresco@calum.csclub.uwaterloo.ca, ralph@fsc.fujitsu.com, w3c-sgml-wg@w3.org Subject: Re: Will XML go beyond SGML Users? Date: Tue, 01 Oct 1996 10:12:59 GMT On Tue, 01 Oct 1996 04:53:27 -0400, Paul Prescod wrote: >At 05:44 PM 9/30/96 -0700, Ralph Ferris wrote: >>You wrote: >> >>>As I understood it, it was our goal that all browsers should support XML >>>just as they do HTML. Since even Internet Explorer cannot open either RTF or >>>Word files, >> >>Internet Explorer 3.0 can open any ActiveX enabled control, including Word. >>Word (or Excel, or whatever) will then take over the entire client viewing >>area, adding its own toolbar, i.e., there is seemless integration within the >>original window. > >But you must download the Active-X or buy the product to do this. Over a >28.8 modem, a "Word Viewer" is going to take a while to download. Microsoft >has explicitly chosen NOT to ship this with their product, so they obviously >are NOT pushing Word's format (or RTF) as a >Microsoft-shoves-it-down-your-throat-standard. Which is why I disagree with >Charles that we could not achieve greater support for it. In Windows 97, the browser will be the Windows Explorer (not Internet Explorer). URLs will be another "document type" (icon, for the lexically impaired) that you click on. Whether you are looking at a directory view or at the content of a document won't matter. The format of the document you click on won't matter much because there'll be viewers for most all of them. In other words, URLs will be like UNCs are today, just another way of addressing files. Word/RTF will be the dominant document format just as it is in Win95 today. All they need to do is compress it with JPEG to make it slim enough for modems, but the corporate intranets that we are targeting aren't restricted to modem speeds. But even if Microsoft wanted some "SGML-like" thing, they would just create their own, add it to their software, and it would be a standard. They are on this committee to monitor it and learn from it, not to adopt its results. This doesn't make them bad people -- just smart businessmen. We should be equally smart and define realistic objectives for ourselves. >It's also worth noting that URLs are now recognized in >>non-HTML documents (some mail systems support this feature as well). So Word >>users do not need to convert their documents in order to post them to the >>Web. Conversely, what user's think of as the browser has been de-coupled >>from the formats it reads. An ActiveX control can be created to support any >>document markup. So a new markup could be created and popularized, not by >>creating a dedicated browser that supports it, but by supplying an ActiveX >>control that supports it. XML could be introduced to the market in this way. > >If necessary. For all that Microsoft, Sun, Java tries to make components >seem "native" native still seems more native. Native browser support would >be better. There is no such thing as a "native" file format for a browser unless it is badly written. The URL chaser is just like an SGML entity manager. HTML, today, is just the notation of the document entity. It doesn't have to remain so in the future and probably won't. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- To: Paul Prescod From: Martin Bryan Subject: Re: Make DTDs optional? Cc: w3c-sgml-wg@w3.org Date: Tue, 1 Oct 1996 11:45:10 +0100 >Here's what I think the XML would look like: > >
This is section 1 >Please see > Not if EMPTY elements are banned! (The word "would" needs to be changed to "should".) ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ To: Paul Prescod , w3c-sgml-wg@w3.org From: Martin Bryan Subject: Re: Combine RE/RS proposals? Date: Tue, 1 Oct 1996 11:45:13 +0100 > Could we (in XML) define characters to begin a >section of text guaranteed not to have markup within, where spaces and >newlines should be passed to the application unmolested? > ><TITLE>Here is some sample output</TITLE> ><" > ABCD 1234 > EFGH 5678 > IJKL 9012 >"> > > >I know that duplication of features makes the parser writer's job harder, >but Robert has a reasonable point about usability. This would have my full support. ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ Date: Tue, 1 Oct 1996 08:45:42 -0400 From: gtn@ebt.com (Gavin Nicol) To: streich@slb.com CC: Charles@sgmlsource.com, paul@arbortext.com, w3c-sgml-wg@w3.org Subject: Re: questions about entities and entity declarations >I don't have a clue how character set issues will affect this. In my current >systems, it's completely disconnected because the glyph for a character >depends on the current font. If anyone could assure me that this will be >a non-issue, then I'll be happy (as long as I have their home address for >insurance...). If you have a fixed coded character set (seems to be consensus here) and a character code, you should be able to select a reasonable glyph image without any other information. Things like the private use area in UNICODE would be a bit difficult without at least some font information. Date: Tue, 1 Oct 1996 08:43:07 -0400 From: gtn@ebt.com (Gavin Nicol) To: streich@slb.com CC: Charles@sgmlsource.com, lee@sq.com, dgd@cs.bu.edu, w3c-sgml-wg@w3.org Subject: Re: More on RE/RS >>Could you picture a trivial variant of SGMLNORM (XMLNORM?) that brackets >all the >>data and solves the RE/RS/whitespace problem properly? > >Sure I could picture it. It's always in the back of my mind as an option >should XML turn out too strange. With OmniMark in my back pocket and the >tremendous potential of Jade round the corner, I feel strangely secure. >(By the way, shouldn't we all chip in and take out an insurance policy >on James?) Can you imagine a trivial variant of XMLNORM that tranforms an XML instance that obeys the "No insignificant RE" to an SGML document that would have exactly the same post parse representation? Date: Tue, 1 Oct 1996 08:50:37 -0400 From: gtn@ebt.com (Gavin Nicol) To: Charles@sgmlsource.com CC: jjc@jclark.com, lee@sq.com, w3c-sgml-wg@w3.org Subject: Re: Current Status of Discussion on RE/RS Handling >> >>Then who is to define *how* an entity manager is to recognise these >>"lines"? > >It varies by the file system (or more generally, storage manager). IBM >mainframes use a length field, DOS uses CR/LF, and so on. There is a good >description of this in the FSI spec. Most operating systems I have seen deal with bytes or blocks, and any other structure is imposed through system libraries, which are linked into programs which generally run in user-space. Date: Tue, 1 Oct 1996 09:09:05 -0400 From: gtn@ebt.com (Gavin Nicol) To: Charles@sgmlsource.com CC: dgd@cs.bu.edu, w3c-sgml-wg@w3.org Subject: Re: RS/RE: basic questions >>So you beg the question in another way. The issue is _why_ we should treat >>the RE as markup rather than data. Since there is no _requirement_ to >>include an RE after a tag, why _must_ it not be significant. > >Because the tag isn't part of the data. Therefore, any RE whose >presence depends on where a tag is positioned can't possibly be part >of the data either. Which is another way of saying the say thing. How can you know for certain that the RE depends on the position of the tag, *and* that that was what the author intended? >>

>>This is a paragraph.

> >It's not quite as pretty, or as natural, as: > >

>This is a paragraph. >

Subjective opinion. >3. He decides that the structure would look clearer during editing if >he put the tags on their own input lines, like this: > >

>Now is the time for all good men >to come to the aid of the party. >

> >In all of these steps, he hasn't changed one character of the >data. So why should the application see a single RE in step 2 and >three REs in step 3? The data has only ever had a single RE. It is equally logical to say that in delimiting a block of character data the delimiters must be hard agaist the beginning and end of the block they delimit. For example, using your own example: 1) An author writes the following: As the thunderous roar of applause... 2) He then decides to mark it up using the FROGLESPECK markup language which uses quotes to delimit groups of characters: "As the thunderous roar of applause..." 3) He then decides it'd "look clearer" if the quotes where on lines by themselves: " As the thunderous roar of applause... " -- Would you argue that in FROGLESPECK these should equate to the same thing, or would you feel that syntax *requires* the quotes to be exactly at the start and end of each character block. Ask the same of a LISP programmer, and a C programmer. There is no *technical* reason for this, though I agree that there may be aesthetic reasons for it. I'm starting to think seriously now that XML should just be LISP (I was half joking before). At least it doesn't confluge syntax and semantic interpretation. (p "This is data in a paragraph" (emph "and here is an emphsised block" ) ) To: Charles@sgmlsource.com, lee@sq.com From: Martin Bryan Subject: Re: Combine RE/RS proposals? Cc: papresco@calum.csclub.uwaterloo.ca, w3c-sgml-wg@w3.org Date: Tue, 1 Oct 1996 17:06:07 +0100 Charles F. Goldfarb wrote: >> 1. No need to access a DTD to determine if an element is mixed content. >>> 2. All REs and RSs are ignored -- full stop! (See, I'm internationalized :-) >>> 3. Really easy parsing. >>> 4. Much simpler to explain. And later >>Does "string concatenation" work? >>

>> "this is line one " >> "this is line two " >> "this is line three" >>

> >Yes. but he still fails to suggest how to avoid mistakes in the following:

"In the last message Peter said "Why do it this way?". I would suggest the following: " " X="23"" "Y="33" " "as a better alternative"

Do I really have to a) escape every quote in my text (or use an entity reference for it) b) enter required RE's as part of the quoted string? If I have to do either of these I guarantee that half the documents I write would be invalid XML if I made the mistake of trying to use it as an authoring environment. ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ From: Paul Prescod Subject: Re: Combine RE/RS proposals? To: Charles@sgmlsource.com Date: Tue, 1 Oct 1996 14:59:08 -0400 (EDT) Cc: lee@sq.com, w3c-sgml-wg@w3.org > On Mon, 30 Sep 96 23:53:07 EDT, lee@sq.com wrote: > >Will the obvious \" work as expected? > > Yes. From an SGML standpoint it would just be a shortref resolving to ". > > > > >Does "string concatenation" work? > >

> > "this is line one " > > "this is line two " > > "this is line three" > >

> > Yes. Could I use an alternate syntax that uses actual tags:

This is some Text and some of it is Emphasized It's ugly, but I can teach my SGML editor to output it. I'm leary about teaching it to output shortrefs. Paul Prescod From: Paul Prescod Subject: Re: Make DTDs optional? To: Charles@sgmlsource.com Date: Tue, 1 Oct 1996 14:56:35 -0400 (EDT) Cc: masonjd@ornl.gov, ricko@allette.com.au, w3c-sgml-wg@w3.org > And we have to accept the idea that it can only be read-only. Or at least that > you could only modify the data, not the structure. (Now that Jim has dug up > those painful memories, I seem to recall that ODA also made the distinction > between editing the structure and editing the data content.) If the only level of structure you are interested in is the level required to attach a stylesheet to the document, isn't it reasonable to edit the structure without recourse to the DTD? Paul Prescod Date: Tue, 1 Oct 1996 15:34:42 -0400 To: w3c-sgml-wg@w3.org From: "David G. Durand" (David G. Durand) Subject: Re: Element content the real issue?... At 11:20 PM 9/30/96, Charles F. Goldfarb wrote: >On Mon, 30 Sep 1996 12:10:42 -0400, "David G. Durand" (David G. >Durand) wrote: > >> We basically cannot afford to process element and non-element content >>differently with regard to whitespace or anything else. >> >> ==> So we can't allow any ignored whitespace anywhere without resorting >>to quoting, because of the non-DTD parsing requirement. > >Exactly. So having recognized the unavoidable fact, why do you keep trying to >avoid it? I can understand trying to ameliorate its impact, but let us first >face up to the truth: this is the only way we can go. Becuase the unavoidable fact is not that we need quoting, but rather the following: Ignored whitespace _and_ no DTD _and_ compatibility with SGML implies quoted data. That means that we can avoid quoted data by eliminating ignorable whitespace. I'd rather give up ignored whitespace than start quoting. And we _do_ have that choice, since it makes XML more restrictive than SGML. I'm still waiting for a reason why we _must_ ignore newlines (but not trailing spaces) after a tag start. It is sometimes convenient, but is not essential. >> >> My application of his theories says that we should change as little as >>possible from HTML (the market leader), while adding the minimum we can >>manage to get the most useful new functionality. > >This conclusion contradicts the inescapable fact. Only if you make keeping the ability add arbitrary whitespace around tags a hard requirement. I say we should give up a cosmetic convenience that already causes trouble in HTML (e.g. in , whitespace in as a set of table row objects, data in between is not meaningful. If it's non- whitespace, issue an error, but if it's whitespace, ignore it. (See below on why there might be whitespace there after parsing.) -=-=-=- I envision three types of parsing of XML documents. a) SGML parsing. This will require a DTD, even if a FRED-generated or ANY one, per 8879. b) XML parsing with DTD. For the purposes of this discussion, the only relevant part of the DTD is whether a given content model is mixed or element. c) XML parsing without DTD. In (a): o Whitespace is ignored in element content. o All whitespace (including CR and LF) is preserved in mixed content. o After application conventions are applied, data should be identical to (b) and (c). In (b), parsing should be the same as (a); the data should be identical to (a) *before* application conventions are applied to either one. In (c): o Assume mixed content for all elements. o All whitespace is preserved. o After application conventions are applied, data should be identical to (a) and (b). - element-content space is all leading or trailing in that element, or - is between elements and is normalized away. - Most occurences of element content involve line-breaks for the children when styled, anyway, but that should not make a difference. - Potentially damaging whitespace (like the table-row problem in Navigator) is eliminated when formatting; whitespace that doesn't make sense in the context of a certain flow object is ignored. Comments, please? If you disagree with (1), please concentrate on that; there's no point in arguing with the other points if that doesn't fly. If you agree with (1), does the rest of this make sense? Is it workable? -Chris -- http://www.ebt.com +1.401.421.9550 +1.401.521.2030 One Richmond Square, Providence, RI 02906 USA" NDATA SGML.Geek> From: "Christopher R. Maden" Date: Wed, 2 Oct 1996 22:26:11 GMT To: w3c-sgml-wg@w3.org Subject: Re: RS/RE: basic questions [Joe English] > The question is which of these options is the least unpalatable. > Personally, I find a solution that lets me author in SGML and run > instances through a simple normalizer to be far more palatable than > one that requires me to add black magic to my SGML declaration and > all of my DTDs before I can publish XML. This is one thing I didn't address in my most recent message. It requires black magic to your SGML declaration, but one declaration is used for all XML documents. It doesn't require black magic to your DTD. It does require a little black magic in entity managers 'round the world, but... -Chris To: w3c-sgml-wg@w3.org Subject: Re: RS/RE: basic questions Date: Wed, 02 Oct 1996 15:41:01 -0700 From: Joe English Paul Prescod wrote: > At 02:36 PM 10/2/96 -0700, Joe English wrote: > >The question is which of these options is the least > >unpalatable. Personally, I find a solution that lets > >me author in SGML and run instances through a simple > >normalizer to be far more palatable than one that > >requires me to add black magic to my SGML declaration > >and all of my DTDs before I can publish XML. > > So you are supporting _which_ proposal? > I would prefer "strip lexical whitespace between tags" if it can be made to work -- I think it can, BTW, if XML producers are willing to use an XML validator and always supply a DTD. "No whitespace in element content" is my second choice. As for RS/RE, it's a toss-up between "RE delenda est" and "stick with 8879". "RE delenda est" makes life easier for XML implementors but would require a retrofit for SGML entity managers; "stick with 8879" doesn't break any current SGML tools but is One More Thing To Implement for XML parsers. "Quote all data" is at the very bottom of my list, and will remain there until somebody figures out a way to implement it that doesn't require major surgery on existing DTDs. --Joe English jenglish@crl.com From: lee@sq.com Date: Wed, 2 Oct 96 18:55:29 EDT To: w3c-sgml-wg@w3.org, jenglish@crl.com Subject: Re: Shortrefs fatally flawed > BTW, do many parsers support extending the short > reference delimiter set? I know SP does, and SGMLS > does not. No SoftQuad product supports short references (or datatag) today. There are no plans, as far as I know, to add that functionality, although I wouldn't necessarily know if there were. SoftQuad Explorer and Panorama do not support minimisation; if there are no EMPTY elements (and no CONREF), Panorama does not need a DTD; the parser it uses is not a validating SGML parser. It is not sensitive to RS/RE and/or white space -- and displays tables just fine :-) Lee -- Liam Quin, SoftQuad Inc. | lq-text freely available Unix text retrieval lee@sq.com +1 416 544-9000 | FAQs: Metafont fonts, OPEN LOOK UI, OpenWindows http://www.softquad.com/ | xfonttool (Unix xfontsel in XView) SGML - HTML - Other Things | The barefoot programmer Date: Wed, 02 Oct 1996 18:27:43 -0500 From: Len Bullard To: Bill Smith CC: w3c-sgml-wg@w3.org Subject: Re: Acceptance of XML Bill Smith wrote: > > Len Bullard wrote: > > > This is eminently sensible. But that can be viewed as > > "We already have HTML, why do we need this?". I don't wish to be > > tendentious, but XML is not the natural evolution of HTML. HTML is > > in a species of DTDs which some of us have created prior to HTML. > > Not everyone does need it, just like not everyone needs tables, or applets, or frames. What > many people do need is an extensible tag set. They don't recognize this per se but they do > recognize that HTML is too restrictive for their application. In due course, the "HTML > community" will respond to this need by allowing arbitrary structure and semantic > information. Some are not so far from this already and I suspect that the response will be > sooner rather than later. > > If an initial XML is in place early enough and isn't overly complex, it could be used by > the HTML community rather than their reinventing the wheel. This is hardly guaranteed but > we gain little by ignoring that community. Let's not repeat our mistake with HTTP. > > Perhaps a small point but I believe that I said "allow the Web community to view XML as the > natural evolution of HTML". I wouldn't mind if HTML became extinct through natural > selection. However, it would be unfortunate if HTML can evolve fast enough to make XML > unimportant. No disagreement. I do believe HTML will live on regardless of what is designed here. There is too much code supporting it. On this, I side with Dr. Goldfarb. They won't walk away from a tactical advantage. So, someone has to get to higher ground. Since this group contains people influential in the development of HTML, there is nothing that can be devised here that cannot be renamed, redone, and readied for HTML with or without acknowledgement. I don't suggest the user base is unimportant. The habits of HTML are very much the habits of SGML authors in most cases. So, trying to prevent "Shocking the Monkey" goes for all of them. HTML is just one more example of "keep it familiar". I agree with James Clark; there are lots of HTML browsers out there already. Can we please get over it? Fear kills the mind. Lack of it sometimes kills the body. One way or another, someone isn't going to be happy. So far, of all of the suggestions I have seen and from my limited understanding, James seems to have the best one. len bullard Date: Wed, 2 Oct 1996 16:27:55 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org CC: bosak@atlantic-83.eng.sun.com Subject: Re: PNG Gets a Big Write-Up [Paul Prescod:] | The "New" PNG (Portable Network Graphics) Spec got a nice write-up in | C|Net. I hope that when the time comes, a few months from now, we can | a) get our message across as clearly, b) offer as many easy to | understand advantages over HTML as PNG does over GIF and c) have the | same support from the W3C in taking our message to the world. | | I'm a little concerned about that last point. Has anyone seen Dan | Connolly around here. =) He's been critical of SGML in the past and I | would hate to see us do a bunch of work on XML and not have W3Cs full | backing in promoting it to vendors. I'm sure he has ideas about | internationalization and entity referencing. (he's been critical of | SGML's entity syntax) Dan is a major supporter of SGML on the Web. It is because of his untiring efforts that there is a DTD for HTML. He is the SGML ERB's official liaison with W3C and monitors this effort as best he can given the demands of all the other W3C activities that he's responsible for. The fact that he has not been uncritically accepting of every feature of ISO 8879 does not mean that he and the W3C are not supportive of this effort. Jon Date: Wed, 02 Oct 1996 20:12:20 -0400 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: PNG Gets a Big Write-Up At 04:27 PM 10/2/96 -0700, Jon Bosak wrote: >Dan is a major supporter of SGML on the Web. It is because of his >untiring efforts that there is a DTD for HTML. He is the SGML ERB's >official liaison with W3C and monitors this effort as best he can >given the demands of all the other W3C activities that he's >responsible for. The fact that he has not been uncritically accepting >of every feature of ISO 8879 does not mean that he and the W3C are not >supportive of this effort. I didn't mean to imply that I didn't think he was supportive of the effort. I just hope he will be supportive of the finished _product_. We are making some fairly fundamental decisions here and he is going to have to try to sell it to vendors. As so, it would be useful to have his input. For instance, one decision or the other on internationalization or linking might be inconsistent with W3C's directions. Paul Prescod Date: Wed, 02 Oct 1996 20:12:08 -0400 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: RS/RE: Yet Another Proposal At 10:23 PM 10/2/96 GMT, Christopher R. Maden wrote: > - Potentially damaging whitespace (like the table-row problem in > Navigator) is eliminated when formatting; whitespace that doesn't > make sense in the context of a certain flow object is ignored. We're back to the True Information problem. Let's ignore tables, for a second, and look at something else. I don't want to give the impression that I'm obsessed with tables. ... ...
... Okay, an SGML application knows that the newlines are insignificant because of the DTD. Now what about a formatter? It will, by default, leave the newlines in (because the central tenet of this proposal is that newlines are significant). So how do you turn them off? Okay, maybe with a stylesheet. Now your document looks okay in an SGML editor and in Netscape (presuming that SGML editors handle the RS/RE remapping hack). Now you want to convert it to RTF. Okay, maybe your conversion program has a stylesheet language that allows you to strip out newlines. Now you want to put it in an "XML database". Each element will be stored individually in the database, for later retrieval alone. How does the database determine which newlines go in the database, and which are "formatting". I guess you need some other "style sheet thing" (or a DTD). The problem is not immediate/practical. It is long-term/theoretical. I think that long-term degredation of your data will occur if you depend on "application conventions" (like "table smarts") to determine what is the real information and what is formatting. Therefore, the only safe way to encode this bibliography in the proposed markup language is with no insignificant newlines. In this case, the author can either have convenient editing or unambiguous true content. If we go with this proposal, we should be clear on that and encourage users to sacrifice convenience in favour of rigour. I could accept this, but would rather go the opposite way, like most markup languages (I think) and make newlines and tabs insignificant unless you declare them to be so (in some form of verbatim section). In most SGML documents, this is what authors intend most of the time. Unlike most markup languages, however I would proclaim that space characters are significant outside of markup(as Liam Quin said to me: "I kinda need the spaces between words." =) ). Certain kinds of formatting would have to be done with tabs and comments intead of spaces, and authors would have to be careful to put a space at the end of each line if they don't want their words concatenated. Most editors do this for you automatically. On both Windows and the Mac, the standard text editor widgets Do the Right Thing. Paul Prescod From: lee@sq.com Date: Wed, 2 Oct 96 22:44:32 EDT To: papresco@calum.csclub.uwaterloo.ca, w3c-sgml-wg@w3.org Subject: Re: RS/RE: basic questions There is a proposal near the end of this article that is very like Paul P's, but I think retains more backward compatibility with existing SGML tools. Paul Prescod proposed: > #1. RE's are not signficant, except within verbatim elements <" "> (which > can only contain data content, no markup). > > #2. RE's within verbatim elements ARE significant (i.e. the parser passes > them to the application). > > #3. RE's between words (i.e not immediately following or preceding markup) > are significant. You also need the newline to be significant between these two lines. I don't quite understand this verbatim markup -- how do you do it in SGML products that don't allow you to change the RCS beyond increasing NAMELEN? There's no point in practice in being compatible with 8879 -- instead, XML has to be compatible with actual tools. Probably the most widespread tools that read SGML are HoTMetaL, Panorama, Adept, and A/E, with NSGMLS and SGMLS and Omnimark being the most widespread on the `next layer down' (conceptually, I don't mean to deprecate them!). I don't list DynaText because the viewer is the widespread part, and it reads a compiled form. As far as I know, only the tools in the 2nd group I mentioned support shortref or datatag or making " a name start character (is that what's going on there with <">...??). InContext, Microstar's editor (Far and Wide? I don't mean Near & Far), SoftQuad Explorer, LivePage, and many many other tools that are perhaps less widespread than the ``biggies'' (e.g because newer) also don't, as far as I know, support such things. Correct my privately if I am wrong; I will gladly post a summary. In fact, perhaps this is the sort of information that SGML OPEN could mantain?? At any rate, if most of the most widely deployed SGML tools won't support XML because it needs features that they don't implement, the SGML compatibility of XML buys little or nothing, I think. I know that RS/RE is an issue for James in SP (and hence in nsgmls); SoftQuad can cope with making all whitespace significant, and the heck with RS/RE, as our Fearless Leader used to say. Again, if there are any other software writers represented here whose current shipping software could not cope with making all whitespace significant, and could not supply a script or patch or upgrade or sidegrade or whatever within, say, six to twelve months, send me mail and I will post a summary early next week. On the other hand, let me know if you _could_ cope with this: * any sequence of whitespace characters is equivalent to a single blank * there is no distinction between spaces and newlines * an application that processes XML may reduce whitespace to a canonical format, or to an equivalent format, or may leave the whitespace untouched. In this world, multiple spaces and newlines would have to be quoted in some way. A verbatim element might be acceptable, although it would need to be syntactically distinguished in a way that was still acceptable to most existing SGML software (e.g. no changing name start characters). For example, suppose all verbatim elements began their names with xml.nf. e.g.: In this listing, multiple spaces are not compressed to single blanks as they are elsewhere, and newlines and spaces are distinct. If we can all live with this, and there is no implementation problem, we can stop talking about it and move on... Lee Date: Wed, 02 Oct 1996 23:10:29 -0400 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: RS/RE: basic questions At 10:44 PM 10/2/96 EDT, you wrote: >You also need the newline to be significant between these >two lines. I don't think so. In fact, I have since thought that I should take out my newline-becomes-space rule altogether. Most text editors seem to put spaces at the end of lines before they word-wrap. Touch-typists will put them in without thinking about it. So under what circumstances are these newlines between words without spaces going to arise? I think that because we cannot SEE the spaces at the end of lines, we presume they are not there. Some editors wrap visually without inserting newlines at all, so those aren't a problem. >There's no point in practice in being compatible with 8879 -- instead, XML >has to be compatible with actual tools. Probably the most widespread >tools that read SGML are HoTMetaL, Panorama, Adept, and A/E, with NSGMLS >and SGMLS and Omnimark being the most widespread on the `next layer down' >(conceptually, I don't mean to deprecate them!). I don't list DynaText >because the viewer is the widespread part, and it reads a compiled form. >As far as I know, only the tools in the 2nd group I mentioned support >shortref or datatag or making " a name start character (is that what's >going on there with <">...??). Well, the shortref is just a syntactic hack to avoid namespace collision. If we don't want to break tools, we could call the element
.

>InContext, Microstar's editor (Far and Wide?  I don't mean Near & Far),

Near and Far author.

>SoftQuad Explorer, LivePage, and many many other tools that are perhaps
>less widespread than the ``biggies'' (e.g because newer) also don't, as
>far as I know, support such things.  Correct my privately if I am wrong;
>I will gladly post a summary.  In fact, perhaps this is the sort of
>information that SGML OPEN could mantain??

InContext is based on SGMLS and will support anything SGMLS did.

>At any rate, if most of the most widely deployed SGML tools won't support
>XML because it needs features that they don't implement, the SGML 
>compatibility of XML buys little or nothing, I think.

Agreed.

>Again, if there are any other software writers represented here whose
>current shipping software could not cope with making all whitespace
>significant, and could not supply a script or patch or upgrade or
>sidegrade or whatever within, say, six to twelve months, send me mail
>and I will post a summary early next week.

According to my understanding, the only way to make all whitespace
significant (i.e. to pass all whitespace to the application) is to do that
SGML DECL RE remapping hack. So a LOT of software would have to be changed
(i.e. almost all of the products you mentioned) if you are right that they
do not support SGML DECL tricks.

>On the other hand, let me know if you _could_ cope with this:
>
>* any sequence of whitespace characters is equivalent to a single blank
>* there is no distinction between spaces and newlines
>* an application that processes XML may reduce whitespace to a canonical
>  format, or to an equivalent format, or may leave the whitespace
>  untouched.

Are whitespace characters in element context illegal according to this
scheme? Or are they just passed on to the XML application which must figure
out to discard them? I think that that is still the biggest issue.

If they are illegal, then your markup will be very "terse" (i.e. no
formatting newlines). If they are passed on, then XML applications will
sometimes be working with different data than SGML applications are.

 Paul Prescod



From: lee@sq.com
Date: Thu, 3 Oct 96 00:10:27 EDT
To: papresco@calum.csclub.uwaterloo.ca, w3c-sgml-wg@w3.org
Subject: Re: RS/RE: basic questions

Paul Prescod 
>> You also need the newline to be significant between these
>> two lines.
> 
> Most text editors seem to put spaces
> at the end of lines before they word-wrap.

They do?  None of the ones I use do.

> Touch-typists will put them in without thinking about it.
Not the touch typists I know!  Why on earth would they do that?

Note also that mail software often strips trailing spaces, as users
of older versions of uudecode may sadly attest.

[...]
> Well, the shortref is just a syntactic hack to avoid namespace collision. If
> we don't want to break tools, we could call the element 
.

Well, I'd rather not wire in a name -- hence the prefix.

> >InContext, Microstar's editor (Far and Wide?  I don't mean Near & Far),
> Near and Far author.
Thanks, sorry about that.

[...]

>> At any rate, if most of the most widely deployed SGML tools won't support
>> XML because it needs features that they don't implement, the SGML 
>> compatibility of XML buys little or nothing, I think.
> 
> Agreed.

> According to my understanding, the only way to make all whitespace
> significant (i.e. to pass all whitespace to the application) is to do that
> SGML DECL RE remapping hack. So a LOT of software would have to be changed
> (i.e. almost all of the products you mentioned) if you are right that they
> do not support SGML DECL tricks.

That's what I am trying to find out.  I know that it is not difficult
for Author/Editor, as our parser already does that, on the theory that
an editor oughtn't to go removing whitespace.  The 5 Unicorns theory,
if you will, although since Author/Editor discards minimisation information,
it doesn't actually earn that many Unicorns on the CFG Purity test.

> Are whitespace characters in element context illegal according to this
> scheme? Or are they just passed on to the XML application which must figure
> out to discard them?
If an XML program has a parser and and an application in the SGML
sense, the application handles whitespace in this proposal.

> If they are illegal, then your markup will be very "terse" (i.e. no
> formatting newlines). If they are passed on, then XML applications will
> sometimes be working with different data than SGML applications are.

Of course, an XML-aware SGML program won't have a problem, and an
SGML-aware XML editor could easily give a choice of Save options.

Lee



Date: Thu, 03 Oct 1996 00:31:26 -0400
To: lee@sq.com, papresco@calum.csclub.uwaterloo.ca, w3c-sgml-wg@w3.org
From: Paul Prescod 
Subject: Re: RS/RE: basic questions

At 12:10 AM 10/3/96 EDT, lee@sq.com wrote:
>Paul Prescod 
>>> You also need the newline to be significant between these
>>> two lines.
>> 
>> Most text editors seem to put spaces
>> at the end of lines before they word-wrap.
>
>They do?  None of the ones I use do.

Well, try notepad. Type until the end of a line and watch it wrap. Then go
back to the line before. There's a space there.

>> Touch-typists will put them in without thinking about it.
>Not the touch typists I know!  Why on earth would they do that?

If you're typing away, you are putting spaces between every word, right? So
why would the ones at the ends of lines be magcially different unless you
are very wedded to the concept that your computer is really a typewriter and
that you should manually hit Return at the end of each line.

>Note also that mail software often strips trailing spaces, as users
>of older versions of uudecode may sadly attest.

That's true. That would be a problem according to my scheme.

>> If they are illegal, then your markup will be very "terse" (i.e. no
>> formatting newlines). If they are passed on, then XML applications will
>> sometimes be working with different data than SGML applications are.
>
>Of course, an XML-aware SGML program won't have a problem, and an
>SGML-aware XML editor could easily give a choice of Save options.

Wouldn't the SGML specification _require_ a validating parser to report that
a document had data characters between element content elements? Does this
mean that your application should do a preprocess before feeding the data to
a validating parser? If A/E does this okay, it will be because it does NOT
support the SGML DECL hacks that make this proposal possible. 

Since newlines are demoted to space characters, and space characters are
essentially ordinary data characters, my obssesive table would be basically
equivalent to this:

can screw you up). Otherwise, we can shun quoting by outlawing whitespace ignoration everywhere. A \ before newline convention is the _most_ intrusive solution that I can see working, and I am skeptical about acceptance of that. >His earlier quote says we >should keep things familiar, so I offer this premise: > >1. Most people will create XML with a program that hides the real data format, >in which case the "niceness" and "familiarity" of the format are non-issues. This was the plan for HTML. Tim Berner-Lee originally thought that HTML would be the machine-language of the web, and that "nice" editors would be the only thing to mess with HTML. He was _wrong_, because it was impossible to distribute uniform, friendly platform-neutral editing tools to the entire market (and some people don't want them anyway). We should learn from that failure that the _most likely_ authoring tool for XML will be a text-editor (maybe with some extra assistance grafted on), as it is for much SGML, and most HTML. Tools are nice, but the coverage of user needs and multiple platforms is always spotty. Even the graphic designers where I work use plain-text HTML editors, despite a completely fine-arts background. They need the visibility and control. >2. People who will create XML with "dumb" editors or mode-assisted (dull?) >editors also have written programs. To them, quoting data is quite familiar and >macros can be written to assist with it. I believe the claims above to be false. Most will _not_ be programmers. HTML is should be our user-model here. Most will _not_ be able to write macros in a programming editor. Macros and specialized tools will become available only when and _if_ there is enough acceptance of XML markup that there is an existing market for those tools. Even for the putative "technical" audience, I am doubtful of the convenience of quoting syntax. Quoting is a source of a significant fraction of all programming language syntax errors. I think it a particularly poor solution for a non-programming user-base. And that \" is a constant irritant, as well. >>I must say that I don't >>see the point of targeting only the SGML community, because they already >>have SGML. > >The SGML community needs a lean, mean conforming delivery format, which XML can >be. It will be confusing enough to the market to have SGML and XML. Having a >third "version" of SGML is at least one too many. If we target only the SGML market, there will be no platforms to deliver to. Standards are a dime a dozen on the Internet, and to break out of our community we need to bear the rest of the world in mind. I am not convinced (as you may have gathered) that SGML-conformance in every detail is that important. I believe that I have proposed nothing (yet) that will endanger that. The third "version" comment seems a non-sequitur to me. No-one has made such a suggestion. -- David RE delenda est. (granted, I'm no Cicero) --------------------------------------------+-------------------------- David Durand dgd@cs.bu.edu | david@dynamicDiagrams.com Boston University Computer Science | Dynamic Diagrams http://www.cs.bu.edu/students/grads/dgd/ | http://dynamicDiagrams.com/ Date: Tue, 1 Oct 1996 15:34:49 -0400 To: w3c-sgml-wg@w3.org From: "David G. Durand" (David G. Durand) Subject: Re: Combine RE/RS proposals? The list in the following may help to clarify things.. At 11:20 PM 9/30/96, Charles F. Goldfarb wrote: >At that point it might be best to bite the rest of the bullet and always >delimit >data. Look at the advantages: > >1. No need to access a DTD to determine if an element is mixed content. >2. All REs and RSs are ignored -- full stop! (See, I'm internationalized :-) >3. Really easy parsing. >4. Much simpler to explain. Or, by contrast: ================ It might be best to bite the rest of the bullet and never delimit data. Look at the advantages: 1. No need to access a DTD to determine if an element is mixed content. 2. No REs, RSs, or other whitespace are ignored. 3. Really easy parsing. 4. Much simpler to explain. 5. No quotes to get wrong -- ever. -- David RE delenda est. --------------------------------------------+-------------------------- David Durand dgd@cs.bu.edu | david@dynamicDiagrams.com Boston University Computer Science | Dynamic Diagrams http://www.cs.bu.edu/students/grads/dgd/ | http://dynamicDiagrams.com/ Date: Tue, 1 Oct 1996 14:04:41 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org Subject: Addition to the WG The following person has been added to the WG: Sarah G.E. Tourville, SAGRELTO sagrelto@nando.net Jon To: w3c-sgml-wg@w3.org Subject: Delimited pseudoelements Date: Tue, 01 Oct 1996 14:35:30 -0700 From: Joe English Regarding the proposal to require that all data content in XML be delimited: As I understand it, this proposal attempts to solve two problems: that of distinguishing element content from mixed content, and that of RS/RE processing. Since ISO 8879:1986 has no PELO/PELC ("pseudoelement open"/ "pseudoelement close") delimiter roles, the issue of SGML compatibility must be addressed. The following options are available for processing an XML document as SGML: 1a) use a native XML parser to build a grove or ESIS stream; 1b) use an XML-to-SGML translator; 1c) wait for ISO 8879:199X (and tools that support it); 1d) use SHORTREF tricks in the DTD; 1e) something else that I've missed. (1a), (1b), and to a lesser extent (1c) would severely limit XML's utility. If XML can't be directly processed with existing SGML tools, there is little point in putting an SGML-like notation on the Web. If a separate XML parser is required, we'd be better off using a completely new syntax that is *significantly* easier to parse than something SGMLoid; Lisp S-expressions might be a good choice. (1d) would allow XML instances to be parsed as SGML, but introduces the new problem of DTD incompatibility. (I'm still operating under the assumption that *some* XML applications will need to examine the DTD.) We can get around this by stating in the XML spec something to the effect of "Ignore and declarations; they're black magic intended for consumption by SGML parsers." Inelegant, but workable. It is unclear (to me) what the SHORTREF black magic should be. If I understand correctly, something like the following was proposed: With this scheme, an XML parser will *not* produce the same output as an SGML parser: the latter will construct a grove with all PEL nodes inside "wrapper" EL nodes that will be absent in the grove constructed by the former. Presumably this can be handled by an application convention on the SGML side that "pel" elements are to be ignored, or with another rule for XML that the PELO/PELC delimiters are equivalent to "pel" start- and end- tags -- in other words, requiring XML to interpret (or act as if it interprets) the short references after all. This scheme doesn't solve all of the RS/RE weirdness: " This data contains two delimited REs that are discarded by SGML parsers. " Maybe: &#RE;" > " > would do the trick. OK... so I've convinced myself at least that option (1d) is workable from a technical standpoint (though it excludes SGML parsers that don't support extending the short reference delimiter set, or don't support SHORTREF at all). Is there a better solution that I've missed? This leaves the problem in the hands of XML producers. It seems to me that the best way to produce XML will be to author in "full" SGML and down-translate. Ideally the down-translation will be a simple normalization process, and with any luck most "native" SGML editors already save files in a form that will be suitably normalized for consumption as XML. The only restriction on XML-able DTDs so far is that they cannot contain EMPTY elements or #CONREF attributes (unless we can come up with a way for DTD-less parsers to handle those too). However, very few (if any) existing DTDs utilize the short reference tricks required by option (1d); if these are required to make a DTD XML-able, it places a significant burden on XML producers. Do we expect them to use XML-able DTDs for all data in its "native" format? Is there a way to automatically convert existing DTDs to add the required SHORTREF declarations and perform #PCDATA substitutions? Do we require that providers maintain two copies of all their DTDs, one for use with their local SGML tools and one for publication as XML? I'm not sure how to address this issue. from mixed content is best solved by a markup convention forbidding SEPCHARs in element content. The cost of this solution -- that XML input cannot be formatted for editing the way most users would like -- is less than the cost of adopting option (1d). I think that the problem of RS/RE processing is best handled simply by keeping the rules from ISO 8879. If the rules can be explained in 14 lines of prose and implemented with a 4-state DFA, then RS/RE processing is at most a minor annoyance; it's *not* a major problem. I think that "RE delenda est" is the other best way to handle the RS/RE problem. Moreover, if we adopt the markup convention above then Charles' objections concerning the ontogeny of individual record ends are moot: if authors are forbidden to use whitespace to format element content, why should data content be any different? ]]> --Joe English jenglish@crl.com From: Paul Prescod Subject: Re: Will XML go beyond SGML Users? To: Charles@sgmlsource.com Date: Tue, 1 Oct 1996 19:48:24 -0400 (EDT) Cc: ralph@fsc.fujitsu.com, w3c-sgml-wg@w3.org > But even if Microsoft wanted some "SGML-like" thing, they would just create > their own, add it to their software, and it would be a standard. They are on > this committee to monitor it and learn from it, not to adopt its results. This > doesn't make them bad people -- just smart businessmen. We should be equally > smart and define realistic objectives for ourselves. I'm not claiming that microsoft WILL adopt our results. I am saying that at in the past, they HAVE adopted standards when it was expedient to do so. Recent examples are HTML and TCP/IP. Older ones are ASCII, Basic, C++, SQL. We shouldn't design XML presuming that it will be the universal document format of the future, but we shouldn't make the HTML mistake of presuming that it will NOT be. A "universal format" cannot require every "object transmission" to refer to a DTD that may or may not exist. Just as in SGML, sometimes the DTD will be implicit. Just as with HTML, sometimes that laxness will allow people to shoot themselves in the foot. Oh, well. > There is no such thing as a "native" file format for a browser unless it is > badly written. The URL chaser is just like an SGML entity manager. HTML, today, > is just the notation of the document entity. It doesn't have to remain so in the > future and probably won't. Unfortunately, browsers _are_ poorly written. That's why I hope that XML support is "native." I don't know much about Active-X, but OLE has some MAJOR PROBLEMS that confound theoretical expectations. Paul Prescod From: Paul Prescod Subject: New RS/RE Proposal To: w3c-sgml-wg@w3.org Date: Tue, 1 Oct 1996 19:37:14 -0400 (EDT) Well, everyone is probably tired of RS/RE proposals. Let me highlight the costs and benefits of mine (presented below), so you can decide whether to read on: * handles RS/RE in tables and other elements that would usually be considered "element content" in the usual way (RS/RE is not significant) * handles other whitespace in a predictable way (always significant) * handles verbatim elements * only three simple rules (and one implied simple rule) * believed to be compatible with the all SGML parsers and the great majority of SGML tool output. * not too weird looking to SGML and HTML users * most documents will require NO extra delimiters, nor will they have to remove insignificant RS/REs inserted by traditional SGML tools (other insignificant whitespace that is inserted will have to be removed) * only a few habits/macros have to change Here are my proposed rules: #1. All REs are insignificant unless they occur within verbatim sections or between non-whitespace data characters (i.e. between words). #2. Verbatim sections begin with <" and end with "> and may contain only data characters. REs within verbatim sections are significant. #3. REs between non-whitespace data characters (i.e. between words) are collapsed to a single space. (implied rule #4.) All other (non RS/RE) whitespace (outside of markup) is significant. Examples: I believe that under these rules that these fragments will behave as a typical author would intend them to. RS/RE ignored by rule 1 =======================
abcdefghijk...

Isn't the sky blue?

RS/RE becomes space by rule 2 =============================

This is a long sentence and my text editor is going to put a newline in for word wrap. Good thing XML knows what I mean!

Verbatim Content: ================
<"
Column 1       Column 2       Column 3      Column 4
=======        ========       ========      ========
12345          12345          12345         12345            
12345          12345          12345         12345            
12345          12345          12345         12345            
">

Significant Whitespace:
=======================

This is a paragraph with a bunch of space in it.

All spaces are retained. Significant Whitespace workarounds: ================================== ~ ~ This is the only major downside... Rationale: =========== This proposal gets rid of the SGML RS/RE compatibility problem by making all RS/REs insignificant except those explicitly asked for (as per Charles' proposal). It does not, however, require ALL mixed content to be delimited. Only the mixed content that requires the significant RE's must be delimited. In other words, since insignificant RE's are the norm in SGML document, RE's will be insignificant by default. Other whitespace cannot be handled so easily, because whitespace is used in significant contexts so often. I've taken the opposite stand to Charles on this issue. Since meaningful whitespace is the NORM, make meaningful whitespace the default. There are various tricks to put in insignificant whitespace. We can play SGML declaration games to make them easier if we want. Verbatim text is allowed through the verbatim delimiter as per Charles' proposal. Since it would be used less frequently than in Charles' proposal, a different shortref would be preferred. Paul Prescod Date: Tue, 1 Oct 1996 19:22:31 -0700 (PDT) From: Bill Smith Subject: Acceptance of XML To: w3c-sgml-wg@w3.org I've been following the discussion on the various topics and am concerned that in dealing with interesting details, we are ignoring the even more interesting "big issue" - accpetance. SGML is an important international standard, has garnered wide support, and is required for many applications. The flexibility inherent in SGML makes it ideal for a class of applications that cannot be supported by lesser markup languages. HTML is an obvious lesser language. SGML is superior and I support it. HTML, brain-damaged though it may be, has in short order dramatically eclipsed SGML in terms of number of adoptees/users. Functionality has improved to the point that some (none on this list) believe that HTML is, or will be the one-size-fits-all document markup. Clearly this is not the case and HTML is bound to "hit the wall" - in fact it already has on several occasions. Left to their own devices, HTML hackers (perjorative term) will invent new markup to solve specific problems/limitations with the then current HTML standard. Eventually, someone in the "HTML community" will realize that an elegant (and not especially difficult) solution to tag proliferation/escalation is a combination of arbitrary structure and semantic information. The "preferred" solution will build on an HTML parser and will result in an HTML-like parser capable of handling arbitrary structure - no DTD or an optional, minimal one required. It will use CSS or extentions to CSS to add semantic information to the "new" (and old) HTML tags. In short, this solution will permit incremental improvement from the then current state of HTML to something considerably more flexible and long-lived. It will not be SGML nor will it contain any of SGML's "complexities" but rather will be streamlined for delivery - a one-way operation. Assuming that this is HTML's natural evolution, than I would argue that the initial design of XML should follow this general model. It will be well-understood by tool developers and explaining it to authors will be considerably less onerous than describing RS/RE rules or other SGML esoterica. XML will be accepted as an extension to a well-established, widely-used, and popular standard. An XML that relies too heavily on SGML mechanisms or is viewed as overly complex vis a vis HTML will be ignored. I am strongly in favor of designing initial XML such that it adds minimally to the complexity of HTML. Arbitrary structure and semantic information are sufficient. Marked sections, processing instructions, text entities, complex RS/RE rules, and the like are not required and will delay or prevent accpetance. We should avoid them. ISO 8879 conformance is not required although I see no reason to gratuitously preclude it. I have taken some care to indicate that this would be an initial design for XML. XML should be extensible and we should expect that it will evolve - perhaps rapidly. The Web community will expect (demand) no less. As the HTML, no XML community develops new requirements, solutions can be rapidly uncovered and presented using SGML as a warehouse. This group, or a follow-on would be viewed as responding to needs rather than dictating a complex, difficult to implement standard. If we adopt this approach, one might ask what is SGML's role other than as an intellectual property repository? Beyond the idea warehouse, it will continue to be used in many applications because the full expressive power of SGML is frequently required. Further, SGML engines will be used to convert SGML document fragments (or complete documents) into XML documents (objects) for delivery to clients. These transformations and and one-way deliveries would avoid many of the round-trip SGML-XML-SGML problems that have been discussed here. In short, SGML is used where required and where the complexity can be managed - on the server side. XML is used for less "rigorous" applications or where complexity cannot be tolerated - on the client side. I've made my plea. I'm sure it will be heard - and I hope some agree with me. If not, I fear that we will succeed in having interesting discussions but fail in practical acceptance of XML. Please let's make XML as simple as possible - but no simpler. Thanks for your time. Bill Date: Tue, 01 Oct 1996 23:29:01 -0400 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: PNG Gets a Big Write-Up http://www.news.com/News/Item/0,4,4040,00.html The "New" PNG (Portable Network Graphics) Spec got a nice write-up in C|Net. I hope that when the time comes, a few months from now, we can a) get our message across as clearly, b) offer as many easy to understand advantages over HTML as PNG does over GIF and c) have the same support from the W3C in taking our message to the world. I'm a little concerned about that last point. Has anyone seen Dan Connolly around here. =) He's been critical of SGML in the past and I would hate to see us do a bunch of work on XML and not have W3Cs full backing in promoting it to vendors. I'm sure he has ideas about internationalization and entity referencing. (he's been critical of SGML's entity syntax) Paul Prescod Date: Wed, 2 Oct 1996 08:36:13 -0400 From: gtn@ebt.com (Gavin Nicol) To: papresco@calum.csclub.uwaterloo.ca CC: Charles@sgmlsource.com, ralph@fsc.fujitsu.com, w3c-sgml-wg@w3.org Subject: Re: Will XML go beyond SGML Users? >Unfortunately, browsers _are_ poorly written. That's why I hope that XML >support is "native." I don't know much about Active-X, but OLE has some >MAJOR PROBLEMS that confound theoretical expectations. Active-X is just as bad because it's just a variation on the same old bogosity. At least now CORBA and OpenDoc are getting looked at seriously. Date: Wed, 2 Oct 1996 11:09:19 -0400 To: w3c-sgml-wg@w3.org From: "David G. Durand" (David G. Durand) Subject: Acceptance of XML (Seconding Bill Smith's posting) At 7:22 PM 10/1/96, Bill Smith wrote: >I've been following the discussion on the various topics and am concerned that >in dealing with interesting details, we are ignoring the even more interesting >"big issue" - acceptance. >.... >HTML is an obvious lesser language. SGML is superior and I support it. Hear Hear to both comments. >I am strongly in favor of designing initial XML such that it adds minimally to >the complexity of HTML. Arbitrary structure and semantic information are >sufficient. Marked sections, processing instructions, text entities, complex >RS/RE rules, and the like are not required and will delay or prevent >acceptance. >We should avoid them. ISO 8879 conformance is not required although I see no >reason to gratuitously preclude it. I think that we can add complexity to HTML when it gives a lot of power. For instance, I think some forms of entity add enough real power in document management that it is worht the price. And the price need not be paid by the peson who is not using entities. Easy to implement, useful features that can be ignored if not wanted are not the problem. Intrusive features that change current expectations are a problem, I agree with you about the conformance issue, although that is not the way the charter is written. I think that any compaitibility required complexity or change to common (HTML, I'm afraid) practice needs to have a visible, easy to understand payoff in _user power_. SGML compatibility does inf act give a lot of power, but it's neither visible nor easy to understand. >XML should be extensible and we should expect that it will evolve - perhaps >rapidly. The Web community will expect (demand) no less. As the HTML, no XML >community develops new requirements, solutions can be rapidly uncovered and >presented using SGML as a warehouse. This group, or a follow-on would be >viewed >as responding to needs rather than dictating a complex, difficult to implement >standard. This is a good argument for some kind of instance-like DTD syntax. It is easier to add new features (even meta-features) in the guise of "adding a new tag." I'm talking salesmanship, not technical reality here, so don't jump down my throat about this. Let's keep end-user benefit in mind. A completely 8879-incompatible syntax that allowed for extensible markup languages and (optional) document grammars would still improve the state of the web 10,000-fold, or more! Reality dictates, however that it look and feel like HTML, no matter how divergent the underlying semantics really are. Look at the success of C-like languages for anything-at-all versus any new syntax you can think of. People want to feel that they are leveraging their knowledge, not learning something new. -- David RE delenda est. --------------------------------------------+-------------------------- David Durand dgd@cs.bu.edu | david@dynamicDiagrams.com Boston University Computer Science | Dynamic Diagrams http://www.cs.bu.edu/students/grads/dgd/ | http://dynamicDiagrams.com/ Date: Wed, 2 Oct 1996 11:09:14 -0400 To: w3c-sgml-wg@w3.org From: "David G. Durand" (David G. Durand) Subject: Shortrefs fatally flawed I just realized that we are seriously hosed if we endorse the use of shortrefs and some pseudo-element to implement an SGML-compatible a quoting syntax for XML. Say, we define a pseudo element , for the sake of argument. Now we need to document a restriction that no element may be called , or else we will generate a terribly wrong ESIS. We are also likely to make the process of creating content models extremely difficult, due to the name conflict, and content model cooking required in the conversion to an SGML DTD. If we avoid this by making the element visible in the XML application stream, then we need to explain the special case of '"' serving as a tag delimiter. This is bad for two reasons: 1. It complicates the user model a lot. 2. People will then wonder why they can't define their _own_ abbreviations. -- David RE delenda est. --------------------------------------------+-------------------------- David Durand dgd@cs.bu.edu | david@dynamicDiagrams.com Boston University Computer Science | Dynamic Diagrams http://www.cs.bu.edu/students/grads/dgd/ | http://dynamicDiagrams.com/ Date: Wed, 02 Oct 1996 09:54:46 -0900 To: "David G. Durand" (David G. Durand), w3c-sgml-wg@w3.org From: "W. Eliot Kimber" Subject: Re: Shortrefs fatally flawed At 11:09 AM 10/2/96 -0400, David G. Durand" (David G. Durand wrote: > I just realized that we are seriously hosed if we endorse the use of >shortrefs and some pseudo-element to implement an SGML-compatible a quoting >syntax for XML. Say, we define a pseudo element , for the sake of >argument. Now we need to document a restriction that no element may be >called , or else we will generate a terribly wrong ESIS. We are also >likely to make the process of creating content models extremely difficult, >due to the name conflict, and content model cooking required in the >conversion to an SGML DTD. I think we can pick a name that is highly likely not to be used. We could also use some sort of psuedo SGML declaration to rename both the psuedo-element element type and the delimiter pair used to quote data, e.g.: [This is quoted data] Note also that "\" *can* be used with the RCS as a shortref delimiter, you just have to use a numeric character reference in the SGML Declaration, e.g.: DELIM GENERAL SGMLREF SHORTREF SGMLREF "\<" "\&" "\[" "\]" "\"" "\'" "\\" These shortrefs provide "escapes" for the common delimters, e.g. \<, \&, \[, \], \\, etc. Cheers, E. -- W. Eliot Kimber (kimber@passage.com) Senior SGML Consultant and HyTime Specialist Passage Systems, Inc., (512)339-1400 10596 N. Tantau Ave., Cupertino, CA 95014-3535 (408) 366-0300, (408) 366-0320 (fax) 2608 Pinewood Terrace, Austin, TX 78757 (512) 339-1400 (fone/fax) http://www.passage.com (work) http://www.drmacro.com (home) "If I never had existed, would you still remember me?..." --Austin Lounge Lizards, "1984 Blues" >Received: by chuckd.allette.com.au.allette.com.au id AA14320 (5.65/IDA-1.5 for w3c-sgml-wg@w3.org); Thu, 3 Oct 1996 01:40:53 +1000 Date: Thu, 3 Oct 1996 01:40:53 +1000 (EST) From: Rick Jelliffe To: Gavin Nicol Cc: w3c-sgml-wg@w3.org Subject: Re: RS/RE: basic questions On Tue, 1 Oct 1996, Gavin Nicol wrote: > I'm starting to think seriously now that XML should just be LISP (I > was half joking before). At least it doesn't confluge syntax and > semantic interpretation. There is also the Interleaf option, of saying that all RE or RS are ignored (by the application if not the parser): that all of them are just there for nice formatting of the source code (assunming content models like ( #PCDATA | an.element) are out of XML and RE/RS aren't used for content model navigation). At the other extreme, there could also be something like the old Macintosh convention, so that RS/RE is shortreffed to

or the paragraph tag (or "<>" even). In otherwords, force RE/RS to have a definite meaning largely precluding its use willy-nilly inside mixed content elements. (It means that XML might have trouble in editors that handle wraparound by inserting linebreaks, but that would be bearable.) Rick Jelliffe http://www.allette.com.au/allette/ricko email: ricko@allette.com.au ================================================================ Allette Systems http://www.allette.com.au email: info@allette.com.au 10/91 York St, 2000, phone: +61 2 9262 4777 Sydney, Australia fax: +61 2 9262 4774 ================================================================ Date: Wed, 02 Oct 1996 10:59:42 -0500 From: Len Bullard To: Bill Smith CC: w3c-sgml-wg@w3.org Subject: Re: Acceptance of XML Bill Smith wrote: > > I've been following the discussion on the various topics and am concerned that > in dealing with interesting details, we are ignoring the even more interesting > "big issue" - accpetance. > > SGML is an important international standard, has garnered wide support, and is > required for many applications. The flexibility inherent in SGML makes it ideal > for a class of applications that cannot be supported by lesser markup languages. > HTML is an obvious lesser language. SGML is superior and I support it. > > HTML, brain-damaged though it may be, has in short order dramatically eclipsed > SGML in terms of number of adoptees/users. At the cost of an SGML hypermedia tool three years ago, that is not surprising. I'm not precisely disagreeing with you. However, HTML is not the only contender for position in the Internet community. Other languages, forms of markup, are also being developed and deployed. VRML is, for example, an object language. While deployed as a 3D graphics language, it like most hypermedia languages is also an interface. Extensions to it to better handle text have already been proposed. There will be more. HTML is, simply, the one most of us use daily and are familiar with. While making life better for the HTML user is a worthy goal, adopting a streamlined version of SGML does not depend on making a super-HTML. That this may be the result, will be worthwhile. But there are other markets some extant, some to become, that will also benefit >from this work, so, what I believe we should have as our goal, is the best product we can create for the environment, the Internet, now and and becoming, not the fix for HTML. HTML was adopted dramatically because it was free, easy, and put on the Internet. The real technical advantage is HTTP; not the markup. Where we who were already providing excellent support for SGML hypertext did not succeed is in recognizing the ubiquity of a simple transport protocol. Had we done that, we would be further along, IMO. Where we will gain acceptance is in ensuring that our clients have more smarts, not less, and that our language does not require overhauling the protocol engines, and the framework libraries for display and interapplication communication. Why? Because all of the other Internet notations depend on them. HTML users are able to change to more powerful SGML paradigms, but the other languages will not. In short, the advantage is in the interoperability within the framework because under current designs, any notation can be a hub. While I have no objection to supporting HTML users, and I see these users as the largest SGML authoring base to harvest, we should not compromise the XML design to do this. We should ensure that tools are easy to build for XML, and these tools in turn, the XML applications, will take on the task of harvesting the HTML community. Len Bullard Lockheed Martin Date: Wed, 02 Oct 1996 12:36:47 -0400 To: "David G. Durand" (David G. Durand), w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: Shortrefs fatally flawed At 11:09 AM 10/2/96 -0400, David G. Durand" (David G. Durand wrote: > I just realized that we are seriously hosed if we endorse the use of >shortrefs and some pseudo-element to implement an SGML-compatible a quoting >syntax for XML. Say, we define a pseudo element , for the sake of >argument. Now we need to document a restriction that no element may be >called , or else we will generate a terribly wrong ESIS. That doesn't seem like a big deal to me. I can't create an element called FULL_NAME either. =) (okay, okay, in the RCS) >We are also >likely to make the process of creating content models extremely difficult, >due to the name conflict, What's so difficult about avoiding a name conflict? >and content model cooking required in the >conversion to an SGML DTD. Again, I don't understand the difficulty. Paul Prescod Date: Wed, 2 Oct 1996 12:45:27 -0400 To: "W. Eliot Kimber" , w3c-sgml-wg@w3.org From: "David G. Durand" (David G. Durand) Subject: Re: Shortrefs fatally flawed At 9:54 AM 10/2/96, W. Eliot Kimber wrote:> >I think we can pick a name that is highly likely not to be used. It must still be a special name, and explicitly documented, and (possibly) part of the application interface (where explaining XYZZY elements will be, at the least, picturesque). And if I generate tags by some automated process (say for distinct formatting styles in a document coversion product) I might come up with the "unlikely name" as a result of an algorithm. (base-26 encoded integers, anyone). This is a fragile, arbitrary, and ugly-to-describe "solution". > We could >also use some sort of psuedo SGML declaration to rename both the >psuedo-element element type and the delimiter pair used to quote data, e.g.: > > > > [This is quoted data] > Again, explaining the Pseudo-type is hard -- it's meaningless in terms of a document user's needs. Just a bit of complication that must be adapted to, not a new source of capability. Redefining the delimiters makes the lexical analyzer hard to implement in LEX, which is the standard tool for lexical analysis. And it' raises the issue of checking that the delimiters don't conflict with anything that might be legitimate in some other syntactic context. >Note also that "\" *can* be used with the RCS as a shortref delimiter, you >just have to use a numeric character reference in the SGML Declaration, e.g.: > > DELIM GENERAL SGMLREF > SHORTREF SGMLREF "\<" > "\&" > "\[" > "\]" > "\"" > "\'" > "\\" > >These shortrefs provide "escapes" for the common delimters, e.g. \<, \&, >\[, \], \\, etc. This is a fine solution to the scaping problem, as it has _no_ impact on the XML definition that's visible to the user, and enables a convenient and widely familiar quoting model to be used instead of the uglier and less-familiar SGML ones (although entity references are pretty familiar nowadays, thanks to HTML). -- David RE delenda est. --------------------------------------------+-------------------------- David Durand dgd@cs.bu.edu | david@dynamicDiagrams.com Boston University Computer Science | Dynamic Diagrams http://www.cs.bu.edu/students/grads/dgd/ | http://dynamicDiagrams.com/ Date: Wed, 2 Oct 1996 13:44:08 -0400 To: w3c-sgml-wg@w3.org From: "David G. Durand" (David G. Durand) Subject: Content models Main points of this note: 1. I was wrong earlier, SGML content models are strictly less-expressive than full regular expressions 2. Since the differences are a proven cause of confusion, we should just use regular expressions (the standard literature offers cookbook solutions for easier and slower as well as harder but faster faster implementations. Content-model verification is unlikely to be the bottleneck in a parser anyway. Because of 1, 2 implies that some XML content models will be stronger than their translations to SGML content models. As Michael reminded me in a private note, SGML content models are _not_ equivalent to DFAs. They are strictly less-powerful. He also gave a counterexample: ((a,b)*, a?) This has a deterministic parsing automaton, but no non-ambiguous RE (in the sense of SGML). As Michael reminded me, Anne Brueggemann-Klein's papers on SGML content models describe the relevant theory and prove some useful results. On re-reading rather shan skimming the papers, I felt better because I finally had a clear definition of ambiguity, and an algorithm to test it, but it reinforced my earlier comment: >If you haven't guessed, I'd rather eliminate the ambiguity rules and be >able to use a rich and well-understood theory instead of having to help >invent a new theory that only allows things we might rather not allow >anyway. The Brueggemann-Klein papers (I have tech reports, but they may have made it to a journal by now) and the key literature they cite are relatively obscure, compared to the stnadard RE algorithms. It's particularly bad that the results are not in the standard books like the Dragon book. If there's one book that everyone (even undergrads) already knows they have to have if they're going to write a parser, that is it. Michael added: >Doing the instance-to-model match would make perfect sense if you >expected application actions to be defined on 'the third X token of >the content model'. Maybe some members of WG8 expected that to >happen. But I've never seen software that allowed such specifications, >and I have a hard time imagining users who would want it, either. Since people _don't_ do it now, that's an argument that we needn't worry about what they might do, if they did have it. -- David RE delenda est. --------------------------------------------+-------------------------- David Durand dgd@cs.bu.edu | david@dynamicDiagrams.com Boston University Computer Science | Dynamic Diagrams http://www.cs.bu.edu/students/grads/dgd/ | http://dynamicDiagrams.com/ Date: Wed, 2 Oct 1996 13:38:02 -0400 From: gtn@ebt.com (Gavin Nicol) To: dgd@cs.bu.edu CC: kimber@passage.com, w3c-sgml-wg@w3.org Subject: Re: Shortrefs fatally flawed >>I think we can pick a name that is highly likely not to be used. >It must still be a special name, and explicitly documented, and (possibly) >part of the application interface (where explaining XYZZY elements will be, >at the least, picturesque). > >And if I generate tags by some automated process (say for distinct >formatting styles in a document coversion product) I might come up with the >"unlikely name" as a result of an algorithm. (base-26 encoded integers, >anyone). > >This is a fragile, arbitrary, and ugly-to-describe "solution". Agreed. I cannot see why getting rid of RE/RS is such a problem. The rule is then simply "If you put them in, they're there". So far, I haven't seen a good *technical* justification for it except that of SGML tool compatability.... and now we're talking of using features of SGML that are not widely implemented, or of converters, I can't see why that is any better than the "no RE" proposal. In the long run, we're just describing an attributed tree (or a list of lists). Why is it so hard to describe it? The syntax should be dead simple, and the rest just semantic interpretation of the data structure built from parsing the description. Even the node names are often irrelevant except as an aid to later semantic interpretation, and we even have a framework for decoupling that link as well. I guess you could say that I'm thinking about people thinking about burgling a house.... Date: Wed, 2 Oct 1996 13:44:04 -0400 To: w3c-sgml-wg@w3.org From: "David G. Durand" (David G. Durand) Subject: Re: RS/RE: basic questions At 1:40 AM 10/3/96, Rick Jelliffe wrote: >There is also the Interleaf option, of saying that all RE or RS are >ignored (by the application if not the parser): that all of them are just >there for nice formatting of the source code (assunming content models >like ( #PCDATA | an.element) are out of XML and RE/RS aren't used for >content model navigation). I don't think this works, because the SGML behaviour also allows space and tab (SEPCHAR) characters that are ignored in element context. So just bundling RS/RE doesn't enable all the formatting that we used to have in SGML source. Since this is the sole justification for the rules in the first place, we aren't really gaining that much. by preserving RE/RS but loosing the spaces. To return to Paul Prescod's table examples, we also want whitespace for when tables nest:

Workaround NumberWorkaround Description
1Put space after GI for indentation.
2Space after previous element's GI.
3Use traditional comments
4Use some form of SGMLDECL comment.
....
cell1cell2
cell3cell4
Not a nested cell
But all the RS/RE folding proposals fail to allow that because of the element content rules. (unless we want to make some strange rules about inter-tag strings of whitespace). My argument against quoting is that SGML compatibility should not be _more_ important than user utility (and familiarity is a significant component of utility for most busy people who aren't toolsmiths). So we really need to bite the bullet, lump all whitespace together, make it all significant, and be done with it. >At the other extreme, there could also be something like the old Macintosh >convention, so that RS/RE is shortreffed to

or the paragraph tag (or >"<>" even). In otherwords, force RE/RS to have a definite meaning largely >precluding its use willy-nilly inside mixed content elements. (It means >that XML might have trouble in editors that handle wraparound by inserting >linebreaks, but that would be bearable.) This will break my ability to choose my own DTD (or force some form of minimization_, and still disallows whitespace in element content -- never mind that we can't tell if we're in element or mixed content when parsing without a DTD. -- David RE delenda est. --------------------------------------------+-------------------------- David Durand dgd@cs.bu.edu | david@dynamicDiagrams.com Boston University Computer Science | Dynamic Diagrams http://www.cs.bu.edu/students/grads/dgd/ | http://dynamicDiagrams.com/ Date: Wed, 2 Oct 1996 13:43:57 -0400 To: w3c-sgml-wg@w3.org From: "David G. Durand" (David G. Durand) Subject: DTD issues I'm tired of RE, so I'm going to address something more interesting (though probably less important in terms of ultimate acceptance). I agree with the general move to DTD-less processing, but I think that we should make a requirement on all XML parsers: that they be _capable_ of creating an XML DTD given a DTD-less instance. I thought at one time that creating the DTD should be a requirement, but that's obviously retarded for search and browsing applications. I can see a few scenarios for why this is a good idea: 1. If I'm browsing and I find document that I want to use as a template for a project of my own, getting a DTD for that document might be a good first step for my authoring process. Since we can't guarantee that the DTD exists, we should be able to make on on demand if it doesn't or we can't get it. 2. If I'm authoring a light-weight document (like a letter), I frequently want to use generic markup, but there is not a suitable DTD handy, nor the time to try to create one from scratch. I'd like XML to give me at least the flexibility that Word styles gave me for evolving a document type, when I still used it as a writing tool. (Now I just use HTML for my own writing unless it's a heavy duty project -- this is not exactly a step forwards). 3. It's an interesting and potentially productive way to get imput to DTD definition projects: explain tag syntax and encourage people to "work up some exaple documents." The resulting DTDs can help to find potential problems, and also help to train the users in how to help develop DTDs. Keith Schafer of OCLC described this apporoach at an SGML conference session on FRED. We should let tools compete on how they create this DTD (ANY, ANY, ANY is the degenerate case), as there is no _right_ answer. Even a degenerate set of ANY, ANY, ANY models gives me an element list as a starting point. I think this also addresses the authoring question in the best way possible. At any point, an user can ask to have a DTD made if they don't have one (and maybe if they way to update the one they have to allow something they've just decided to so). That DTD can me used for further editing, changed, inspected in hopes of some kind of enlightenment, or saved in a library for future use. Here we don't force users to do what is good for them, but make it easy for them to do it, once they realize what's up. -- David --------------------------------------------+-------------------------- David Durand dgd@cs.bu.edu | david@dynamicDiagrams.com Boston University Computer Science | Dynamic Diagrams http://www.cs.bu.edu/students/grads/dgd/ | http://dynamicDiagrams.com/ Date: Wed, 2 Oct 1996 13:58:27 -0400 To: w3c-sgml-wg@w3.org From: "David G. Durand" (David G. Durand) Subject: Re: Shortrefs fatally flawed At 12:36 PM 10/2/96, Paul Prescod wrote: >At 11:09 AM 10/2/96 -0400, David G. Durand" (David G. Durand wrote: >> I just realized that we are seriously hosed if we endorse the use of >>shortrefs and some pseudo-element to implement an SGML-compatible a quoting >>syntax for XML. Say, we define a pseudo element , for the sake of >>argument. Now we need to document a restriction that no element may be >>called , or else we will generate a terribly wrong ESIS. > >That doesn't seem like a big deal to me. I can't create an element called >FULL_NAME either. =) (okay, okay, in the RCS) Yes, but how would feel about a compiler that said you can't use the variable name "i" because it's needed to keep FORTRAN compatibility. It's bad because it's an ad-hoc restriction with no internal justification in terms of the structure of XML. (We're not using the RCS for names anyway, if I understood the internationalization thread). I suppose we add some XML non-NAMECHAR to the SGML delcaration, and use that in the element name. Since that only makes the converted SGML ugly, I don't have a problem with it. >>We are also >>likely to make the process of creating content models extremely difficult, >>due to the name conflict, > >What's so difficult about avoiding a name conflict? explaining it so it doesn't sound like a hack. A task made harder, of course, because it is a hack. SGML has enough rules that say "something simple, BUT NOT some random special case". This is a classic design problem. For an SGML example, try to make a SHORTREF containing the character 'B' sometime. -- David RE delenda est. --------------------------------------------+-------------------------- David Durand dgd@cs.bu.edu | david@dynamicDiagrams.com Boston University Computer Science | Dynamic Diagrams http://www.cs.bu.edu/students/grads/dgd/ | http://dynamicDiagrams.com/ Date: Wed, 2 Oct 1996 11:02:20 -0700 (PDT) From: Bill Smith Subject: Re: Acceptance of XML To: w3c-sgml-wg@w3.org Len Bullard wrote: > I'm not precisely disagreeing with you. However, HTML is not the only > contender for position in the Internet community. Other languages, > forms > of markup, are also being developed and deployed. (snip) Agreed. XML will need to coexist and compete in this changing arena. By adopting an incremental approach, XML appears responsive and up to date. A static standard will be viewed as out-of-date and uninteresting. > There will be more. HTML is, simply, the > one most of us use daily and are familiar with. While making life > better for the HTML user is a worthy goal, adopting a streamlined > version of SGML does not depend on making a super-HTML. That > this may be the result, will be worthwhile. But there are > other markets some extant, some to become, that will also benefit > from this work, so, what I believe we should have as our goal, is > the best product we can create for the environment, the Internet, > now and and becoming, not the fix for HTML. I'm not suggesting a super-HTML from a technical perspective. Acceptance of XML will depend on it's technical merits, ease of implementation, ease of use, and positioning with respect to other "offerings" - notably HTML. Allow the Web community to view XML as the natural evolution of HTML and it will be accpeted - provided it isn't too difficult to explain or implement. By taking an incremental approach to deploying an XML standard, we get the immediate benefit of positioning it as the natural evolution of HTML. We should not ignore other markets or applications but we should carefully weigh inclusion of SGML features lest we shift XML's perceived center of gravity too far from HTML. Over time, the shift can and will occur but we should allow XML to evolve just as HTML, HTTP, and other Web standards evolve. Bill Date: Wed, 2 Oct 1996 11:24:55 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org cc: bosak@atlantic-83.eng.sun.com Subject: Addition to the list Added to the list: John Lavagnino, Martin Hensel/Brown U. john_lavagnino@brown.edu Jon Date: Wed, 2 Oct 1996 11:31:10 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org cc: bosak@atlantic-83.eng.sun.com Subject: XML schedule of work The SGML ERB has determined the following schedule of work for the "Generic SGML on the Web" activity: October 2 through November 13, 1996 Creation of the basic XML specification. Questions will be considered in batches, with each batch allowed approximately one week of discussion in the WG before it is formally voted on by the ERB. Note that October 6 is the cutoff for unstructured WG discussion of 8879 features and access to the "voting booth". November 17, 1996 WG meeting the evening preceding SGML 96 in Boston. November 19, 1996 Presentation of the basic XML specification at SGML 96 in Boston. December 1996 through March 1997 Specification of linking and addressing mechanisms, hyperlink behavior, and entity management for XML. April 1997 Presentation of the full XML specification at the Sixth Annual World Wide Web conference in Santa Clara, California. May 1997 through October 1997 Completion of dsssl-o for Web applications and final refinements to the XML specification in light of early implementation experience. November 1997 Presentation of the final XML specification at SGML 97. Jon ---------------------------------------------------------------------- Jon Bosak, Online Information Technology Architect, Sun Microsystems ---------------------------------------------------------------------- 2550 Garcia Ave., MPK17-101, | Best is he that inuents, Mountain View, California 94043 | the next he that followes Davenport Group::SGML Open::ANSI X3V1 | forth and eekes out a good ::ISO/IEC JTC1/SC18/WG8::W3C SGML ERB | inuention. ---------------------------------------------------------------------- Date: Wed, 02 Oct 1996 12:35:45 -0900 To: w3c-sgml-wg@w3.org From: "W. Eliot Kimber" Subject: Re: RS/RE: basic questions At 01:44 PM 10/2/96 -0400, David G. Durand" (David G. Durand wrote: >My argument against quoting is that SGML compatibility should not be _more_ >important than user utility (and familiarity is a significant component of >utility for most busy people who aren't toolsmiths). David has, I think, crisply defined the key to this issue. Either XML is completely compatible with SGML or it isn't. A review of the ERB's stated principles shows that SGML compatibility gets higher priority over ease of entry. Whether this priority is the correct one or whether it should apply in this case is still open for debate, but I think it's clear, as James pointed out what seems like years ago that the only real solution to the RE problem that preserves SGML compatibility is to eliminate mixed content, which means quoting data. I certainly agree that quoting data will make authoring *by hand* more difficult (I hated it the first time I tried it), and we do have to be sensitive to the marketing implications of requiring it, but I feel very strongly that the cost of not having SGML compatibility in this case is much greater than the cost of authoring. Cheers, E. -- W. Eliot Kimber (kimber@passage.com) Senior SGML Consultant and HyTime Specialist Passage Systems, Inc., (512)339-1400 10596 N. Tantau Ave., Cupertino, CA 95014-3535 (408) 366-0300, (408) 366-0320 (fax) 2608 Pinewood Terrace, Austin, TX 78757 (512) 339-1400 (fone/fax) http://www.passage.com (work) http://www.drmacro.com (home) "If I never had existed, would you still remember me?..." --Austin Lounge Lizards, "1984 Blues" Date: Wed, 02 Oct 1996 11:48:42 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Derived DTDs At 01:43 PM 10/2/96 -0400, David G. Durand" (David G. Durand wrote: > I agree with the general move to DTD-less processing, but I think that we >should make a requirement on all XML parsers: that they be _capable_ of >creating an XML DTD given a DTD-less instance. I agree that this would be a desirable facility, and furthermore a desirable ground on which implementors could compete, but if it's a good idea the market will do it, and if it isn't they won't, and I don't think the XML standard will be made better by saying they should do this; anyone can trivially comply with a vacuous content model, so the normative effect is essentially zero. For what it's worth, and even though FRED is cool, this is one road I've been way, way down, and people should be warned that while extracting a DTD >from a de-facto parse tree is pretty trivial, turning it into a *good* DTD (e.g. figuring out where to put the parentheses and Kleene operators) is not just a hard problem, it is monstrously, horribly, intractable. When I was doing this a few years back I couldn't find any theory in the CS literature on derived-grammar simplification, so maybe someone has cut through this Gordian knot, but kids, don't try to do this at home. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Wed, 02 Oct 1996 14:27:28 -0500 From: Len Bullard To: Bill Smith CC: w3c-sgml-wg@w3.org Subject: Re: Acceptance of XML Bill Smith wrote: > > By taking an incremental approach to deploying an XML standard, we get the immediate > benefit of positioning it as the natural evolution of HTML. We should not ignore other > markets or applications but we should carefully weigh inclusion of SGML features lest we > shift XML's perceived center of gravity too far from HTML. Over time, the shift can and > will occur but we should allow XML to evolve just as HTML, HTTP, and other Web standards > evolve. > > Bill This is eminently sensible. But that can be viewed as "We already have HTML, why do we need this?". I don't wish to be tendentious, but XML is not the natural evolution of HTML. HTML is in a species of DTDs which some of us have created prior to HTML. HTML's market position notwithstanding, it is not a new technique or evolutionary product. The evolutionary limits of such species prompt efforts such as this one. XML evolves out of the practical experience the members of this group have with respect to many species of SGML applications. This effort attempts to create a new model of SGML that will allow these species to evolve by improving their shared *genetic heritage* such that it enables adaptation to the target environment (the Internet) using the forces that drive that environment (Internet application users and vendors). XML reduces ecotonal tension by improving the agreements at the overlaps. Ok: so much for metaphors of information ecologies. :-) But, if one wants so *sell* it as such, (the heir to HTML) I've no problem with that until it shows up in official project documents. What marketeers tell customers is somewhat like what we tell a prospect on a first date. No one believes it, but if it leads to a mutually rewarding experience, it is worth the trouble. Fibbing in court is another matter. len bullard Date: Wed, 02 Oct 1996 15:10:56 -0400 To: "David G. Durand" (David G. Durand), w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: DTD issues At 01:43 PM 10/2/96 -0400, David G. Durand" (David G. Durand wrote: > I agree with the general move to DTD-less processing, but I think that we >should make a requirement on all XML parsers: that they be _capable_ of >creating an XML DTD given a DTD-less instance. I thought at one time that >creating the DTD should be a requirement, but that's obviously retarded for >search and browsing applications. Somehow I can't see Netscape adding a "create DTD" menu item to their menu bar. And if it is only an internal feature, then what is the point? I agree with all of the benefits of being able to create a DTD for a document, but I think that we should just ask James Clarke to make that an XSP feature, or SoftQuad to add it to XPanorama. Besides, fred already exists and you could probably convert SP error messages into a DTD with a Perl script. =) Perhaps the standardization of features of XML software should be a different process than the standardization of the language itself. Paul Prescod Date: Wed, 02 Oct 1996 15:10:52 -0400 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: Shortrefs fatally flawed At 01:58 PM 10/2/96 -0400, David G. Durand" (David G. Durand wrote: >Yes, but how would feel about a compiler that said you can't use the >variable name "i" because it's needed to keep FORTRAN compatibility. It's >bad because it's an ad-hoc restriction with no internal justification in >terms of the structure of XML. (We're not using the RCS for names anyway, >if I understood the internationalization thread). I suppose we add some XML >non-NAMECHAR to the SGML delcaration, and use that in the element name. >Since that only makes the converted SGML ugly, I don't have a problem with >it. So can we create an element named <">? Paul Prescod Date: Wed, 2 Oct 1996 15:41:59 -0400 To: "W. Eliot Kimber" , w3c-sgml-wg@w3.org From: "David G. Durand" (David G. Durand) Subject: Re: RS/RE: basic questions At 12:35 PM 10/2/96, W. Eliot Kimber wrote: >At 01:44 PM 10/2/96 -0400, David G. Durand" (David G. Durand wrote: > >>My argument against quoting is that SGML compatibility should not be _more_ >>important than user utility (and familiarity is a significant component of >>utility for most busy people who aren't toolsmiths). > >David has, I think, crisply defined the key to this issue. Either XML is >completely compatible with SGML or it isn't. A review of the ERB's stated >principles shows that SGML compatibility gets higher priority over ease of >entry. Well, in that case the principles are wrong. If we make the solution upalatable to the majority of internet users, we have not advanced from the past. I am not arguing that hand entry is important because it saves typing -- If that were the reason, I would agree with you. But Bill Smith's and others' comments about acceptance cut more deeply than technical quirks of 8879. XML is the SGML community's last chance create something good and _popular_ that implements the basic abstractions of content markup. We know a lot about the semantic task of describing text structures. We also know that the mainframe-era syntax of SGML has hobbled it badly in achieving wide acceptance, despite clear and obvious technical, philosophical and business advantages. Why preserve those liabilities when making a pitch to a more naive, short-term-oriented, community. There are a lot of "PERL-based lifeforms" out there, to use Erik Naggum's resonant phrase. We don't need to pander to them, but we offend them at our own peril. We would be better off using my entity manager tricks and requiring declarations for element content if you want to use non-significant whitespace. If we have to quote, then let's use shortref, NET and the like to make a syntax like LISP -- if we're going to be unfamiliar, we might as well use the simplest possible syntax. > >I certainly agree that quoting data will make authoring *by hand* more >difficult (I hated it the first time I tried it), and we do have to be >sensitive to the marketing implications of requiring it, but I feel very >strongly that the cost of not having SGML compatibility in this case is >much greater than the cost of authoring. If XML tools are cheap and easy to write, preserving our investment in SGML editors etc. should be a minor factor -- we can just write conversion scripts. Just why is compatibility at this level so important? If we need to preserve the minutiae of whitespace handling in XML then we are doing a strict subset of SGML without any possibility of relaxing the requirement. If we add onto that the use of a huge pile of arcane SGML features so that the resulting syntax does not even capitalize on the acceptance that SGML already has (through HTML) we are dooming ourselves to a tiny nich within our same old insular community. The internet is different and we must adapt or die. HTML paragraphs with explicit style attachment let me do (slightly crippled) generic markup on top of HTML. That kind of alternative will drown us unless we can leverage the stuff that's already out there. -- David RE delenda est. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ http://www.dynamicdiagrams.com/services_map_main.html Date: Wed, 02 Oct 1996 15:38:59 -0400 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: RS/RE: basic questions At 01:40 AM 10/3/96 +1000, Rick Jelliffe wrote: >At the other extreme, there could also be something like the old Macintosh >convention, so that RS/RE is shortreffed to

or the paragraph tag (or >"<>" even). In otherwords, force RE/RS to have a definite meaning largely >precluding its use willy-nilly inside mixed content elements. (It means >that XML might have trouble in editors that handle wraparound by inserting >linebreaks, but that would be bearable.) I'm not convinced it is bearable. Most people editing by hand are likely to encounter the wordbreak problem. If we aren't worried about wordbreak, I made a compromise proposal yesterday that can be expressed in two lines: #1. RE's are not signficant, except within verbatim elements <" "> (which can only contain data content, no markup). #2. RE's within verbatim elements ARE significant (i.e. the parser passes them to the application). The third rule is just to handle the word wrap problem: #3. RE's between words (i.e not immediately following or preceding markup) are significant. There are two implied rules: #1.Whitespace is always significant (which makes table formatting a pain, but it's livable) #2.Those who want to always delimit their data are welcome to. Those that don't want to don't have to. Paul Prescod Date: Wed, 2 Oct 1996 16:07:56 -0400 To: w3c-sgml-wg@w3.org From: "David G. Durand" (David G. Durand) Subject: Re: Shortrefs fatally flawed At 3:10 PM 10/2/96, Paul Prescod wrote: >At 01:58 PM 10/2/96 -0400, David G. Durand" (David G. Durand wrote: >>Yes, but how would feel about a compiler that said you can't use the >>variable name "i" because it's needed to keep FORTRAN compatibility. It's >>bad because it's an ad-hoc restriction with no internal justification in >>terms of the structure of XML. (We're not using the RCS for names anyway, >>if I understood the internationalization thread). I suppose we add some XML >>non-NAMECHAR to the SGML delcaration, and use that in the element name. >>Since that only makes the converted SGML ugly, I don't have a problem with >>it. > >So can we create an element named <">? > > Paul Prescod Sure. Now the only problem is that if we implement quoting no-one will use XML. But, the language will be well-defined, and will be inputable to SGML software with the proper hacked declaration, and no other modifications. Of course no SGML editor that I'm aware of will be able to produce XML output. And since people will have to escape '"' in their documents anyway, there won't be any collisions with '<"' sequences turning '<' into a delimiter in context, in a valid XML document. (I think) -- David RE delenda est. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ http://www.dynamicdiagrams.com/services_map_main.html Date: Wed, 02 Oct 96 14:29:14 CDT From: Michael Sperberg-McQueen Subject: Re: RS/RE: basic questions To: W3C SGML Working Group On Wed, 2 Oct 1996 14:37:19 -0400 Eliot Kimber said: >At 01:44 PM 10/2/96 -0400, David G. Durand" (David G. Durand wrote: > >>My argument against quoting is that SGML compatibility should not >>be _more_ important than user utility (and familiarity is a >>significant component of utility for most busy people who aren't >>toolsmiths). > >David has, I think, crisply defined the key to this issue. Either Agreed. Except that he concedes too much. Familiarity is important for getting the toolsmiths on board, too. >XML is completely compatible with SGML or it isn't. A review of the >ERB's stated principles shows that SGML compatibility gets higher >priority over ease of entry. Yes. Ease of entry, however, is not quite what David mentions, and not quite the same as user utility and user acceptance. So I don't think the goals statement actually provides an unambiguous answer to this question, even if we believe that we need absolutely full compatibility or nothing. (I for one think almost-compatible is a lot better than nothing! XML with incompatible RE rules is, for example, much better than recommending that the world switch to Word Perfect binary format....) >Whether this priority is the correct one or whether it should apply >in this case is still open for debate, but I think it's clear, as >James pointed out what seems like years ago that the only real >solution to the RE problem that preserves SGML compatibility is to >eliminate mixed content, which means quoting data. I don't think that was quite what James said. Also, I don't think it's quite true beyond all cavil or doubt. James pointed out that prohibiting certain things in mixed content (things like subelements, comments, and PIs) would render the rules trivial. He also pointed out several other possible approaches which would preserve strict compatibility, as well as some that would come close. Quoting is one approach to trying to make the agonizingly painful restrictions on mixed content less agonizing and less painful. It is an extraordinarily clever use of shortref, and my hat is off to Charles for imagining it. The dean of SGML hackers (in the non-pejorative sense!) can still show the rest of us a thing or two. But I expect that it would be the death of XML to require it. It would be far better, I think, to adopt James's compromise solution of white-space stripping and RE-merging, which also achieves SGML compatibility in all non-pathological cases, or to treat RE as we treat any other white space (significant outside of angle brackets, a separator inside of angle brackets, -- roughly speaking), which breaks SGML compatibility in most non-pathological cases, but is very simple to use and understand, and which would *never* make a difference in any SGML application I've ever used in real life. (I.e. all the application code I actually work with in practice is already written to assume it's got to recover when it's passed an unwanted RE or two. Full disclosure requires that I admit I do have some code that does terrible things when confronted with leading blanks at times it doesn't expect them. But that's not relevant here, because 8879 doesn't deal with that. James's compromise solution, to its credit, does.) SGML already *has* delimiters between markup and data. Do we really need a second set of delimiters for *white space*, for heaven's sake? Say it ain't so! >I certainly agree that quoting data will make authoring *by hand* >more difficult (I hated it the first time I tried it), and we do >have to be sensitive to the marketing implications of requiring it, >but I feel very strongly that the cost of not having SGML >compatibility in this case is much greater than the cost of >authoring. We agree that the tradeoff involves authoring vs. strict compatibility, but I think it's more than authoring. The closer we come to making XML documents look familiar to today's users of SGML and SGML applications, including HTML, the greater the acceptance of XML not only among users but also among toolsmiths. (It's for this reason that I think Bill Lindsey's very smart NET tricks, though like Charles's quoting proposal they are a tour de force of ingenuity, are not the politically savvy route for XML to take. I read Scheme books in the evenings, and in the mornings, with a sigh, write code in C.) We seem to disagree primarily in assessing the relative costs. It's an empirical question, but it's going to be hard to test both possibilities, since if we don't get it right the first time, no one in their right mind will pay attention to us the second time. In this case, I think the cost of the quote proposal, like the NET tricks proposal, is massive resistance (= massive indifference) on the part of tool makers. We need a workable solution to the RE-simplicity / 8879-conformance tradeoff. There are several on the table I could live with; the white-space-stripping + RE-merger proposal that came out of the ERB last week seems the best to me, but I'm not dogmatic about it. I *am* dogmatic on the proposals to forbid comments and PIs inside of mixed content: I'd rather live with our current situation (full 8879 or nothing) than explain to incredulous users why comments are not allowed everywhere. And if I'm right about the user reaction to the quoting proposal, adopting it will mean we do continue to live with the current situation. Quoting, in short, seems to me to fail the Stoopid test. And failing the Stoopid test is not the way to call users and toolsmiths to the banner of XML as a Better Way for a Better Net. Michael Sperberg-McQueen &disclaimer; &serious-disclaimer; &really-serious-disclaimer; &no-really-i-mean-it-disclaimer; Date: Wed, 2 Oct 1996 13:54:08 -0700 (PDT) From: Bill Smith Subject: Re: Acceptance of XML To: w3c-sgml-wg@w3.org Len Bullard wrote: > This is eminently sensible. But that can be viewed as > "We already have HTML, why do we need this?". I don't wish to be > tendentious, but XML is not the natural evolution of HTML. HTML is > in a species of DTDs which some of us have created prior to HTML. Not everyone does need it, just like not everyone needs tables, or applets, or frames. What many people do need is an extensible tag set. They don't recognize this per se but they do recognize that HTML is too restrictive for their application. In due course, the "HTML community" will respond to this need by allowing arbitrary structure and semantic information. Some are not so far from this already and I suspect that the response will be sooner rather than later. If an initial XML is in place early enough and isn't overly complex, it could be used by the HTML community rather than their reinventing the wheel. This is hardly guaranteed but we gain little by ignoring that community. Let's not repeat our mistake with HTTP. Perhaps a small point but I believe that I said "allow the Web community to view XML as the natural evolution of HTML". I wouldn't mind if HTML became extinct through natural selection. However, it would be unfortunate if HTML can evolve fast enough to make XML unimportant. Bill Date: Wed, 02 Oct 1996 16:52:16 -0400 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: Shortrefs fatally flawed At 04:07 PM 10/2/96 -0400, David G. Durand" (David G. Durand wrote: >At 3:10 PM 10/2/96, Paul Prescod wrote: >>So can we create an element named <">? > >Sure. Now the only problem is that if we implement quoting no-one will use >XML. We can use whatever delimiter we like best. It can be <">, or " , or <" or >But, the language will be well-defined, and will be inputable to SGML >software with the proper hacked declaration, and no other modifications. Of >course no SGML editor that I'm aware of will be able to produce XML output. If <"> is a valid element, most SGML editors should produce it. If not, they will certainly not have a problem with <#T> or <#TEXT>. And you won't have to escape the quote character. Paul Prescod From: Charles@sgmlsource.com (Charles F. Goldfarb) To: "David G. Durand" (David G. Durand) Cc: w3c-sgml-wg@w3.org Subject: Re: Combine RE/RS proposals? Date: Wed, 02 Oct 1996 21:06:02 GMT On Tue, 1 Oct 1996 15:34:49 -0400, "David G. Durand" (David G. Durand) wrote: >The list in the following may help to clarify things.. > >At 11:20 PM 9/30/96, Charles F. Goldfarb wrote: >>At that point it might be best to bite the rest of the bullet and always >>delimit >>data. Look at the advantages: >> >>1. No need to access a DTD to determine if an element is mixed content. >>2. All REs and RSs are ignored -- full stop! (See, I'm internationalized :-) >>3. Really easy parsing. >>4. Much simpler to explain. 100% compatible with 8879. > >Or, by contrast: >================ > >It might be best to bite the rest of the bullet and never delimit >data. Look at the advantages: > >1. No need to access a DTD to determine if an element is mixed content. >2. No REs, RSs, or other whitespace are ignored. >3. Really easy parsing. >4. Much simpler to explain. >5. No quotes to get wrong -- ever. Incompatible with 8879. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- To: w3c-sgml-wg@w3.org Subject: Re: Shortrefs fatally flawed Date: Wed, 02 Oct 1996 13:47:52 -0700 From: Joe English W. Eliot Kimber wrote: > Note also that "\" *can* be used with the RCS as a shortref delimiter, you > just have to use a numeric character reference in the SGML Declaration, e.g.: > > DELIM GENERAL SGMLREF > SHORTREF SGMLREF "\<" > "\&" > [...] ... but if you do that it's no longer the Reference Concrete Syntax, no? BTW, do many parsers support extending the short reference delimiter set? I know SP does, and SGMLS does not. --Joe English jenglish@crl.com Date: Wed, 02 Oct 1996 14:40:58 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: RS/RE: basic questions At this point I'd rather write Word Perfect macros than read another 10 postings about RS/RE. But I must admit that the weight of argument is inclining me to a view best expressed as: >RE delenda est. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 To: w3c-sgml-wg@w3.org Subject: Re: RS/RE: basic questions Date: Wed, 02 Oct 1996 14:36:16 -0700 From: Joe English "W. Eliot Kimber" wrote: > [...] but I think it's clear, as James pointed out > what seems like years ago that the only real solution to the RE problem > that preserves SGML compatibility is to eliminate mixed content, which > means quoting data. This is not the *only* solution. Another is to forbid whitespace in element content. Another is to require that pseudoelements which contain nothing but whitespace be "escaped" with entity references. Still another (which nobody's mentioned yet for some reason :-) is to forbid *element* content, so that all whitespace becomes significant. The question is which of these options is the least unpalatable. Personally, I find a solution that lets me author in SGML and run instances through a simple normalizer to be far more palatable than one that requires me to add black magic to my SGML declaration and all of my DTDs before I can publish XML. --Joe English jenglish@crl.com Date: Wed, 02 Oct 1996 17:50:23 -0400 To: Joe English , w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: RS/RE: basic questions At 02:36 PM 10/2/96 -0700, Joe English wrote: > >This is not the *only* solution. Another is to >forbid whitespace in element content. Yes! >Another >is to require that pseudoelements which contain >nothing but whitespace be "escaped" with entity >references. I suggested this, but I am not convinced it is "understandable" enough. >Still another (which nobody's mentioned >yet for some reason :-) is to forbid *element* content, >so that all whitespace becomes significant. Lee Quin suggested that. >The question is which of these options is the least >unpalatable. Personally, I find a solution that lets >me author in SGML and run instances through a simple >normalizer to be far more palatable than one that >requires me to add black magic to my SGML declaration >and all of my DTDs before I can publish XML. So you are supporting _which_ proposal? Paul Prescod From: "Christopher R. Maden" Date: Wed, 2 Oct 1996 22:23:18 GMT To: w3c-sgml-wg@w3.org Subject: RS/RE: Yet Another Proposal Here's a proposal, constructed with help from Gavin Nicol and David Durand: 1) All XML entities are single records or part of records, with no record boundary characters. This is the hardest part of the proposal. I do not believe that it is incompatible with 8879, though I am open to arguments. Here's just about everything 8879 has about records: 4.252 record: A division of an SGML entity, bounded by a record start and a record end character, normally corresponding to an input line on a text entry device. NOTES 1 It is called a "record" rather than a "line" to distinguish it from the output lines created by a text formatter. 2 An SGML entity could consist of many records, a single record, or text with no record boundary characters at all (which can be thought of as being part of a record or without records, depending on whether record boundary characters occur elsewhere in the document). 4.253 record boundary (character): The record start (RS) or record end (RE) character. 4.254 record end: A _function character_ ([54]), assigned by the concrete syntax, that represents the end of a record. 4.255 record start: A _function character_ ([54]), assigned by the concrete syntax, that represents the start of a record. It is absolutely clear that record boundaries are something that *can* occur in entities. It is clear from note 2 to 4.252 that entities need not have record boundaries. It is *not* clear that carriage return or newline sequences need be turned into record boundary characters by an entity manager. If it is acceptable to the working group and ERB, this makes XML easy to implement, for both SGML-based and non-SGML based solutions; easy to generate, either by hand, from an editor, or from a script of some sort; and easy to understand. 2) A mechanism is defined for identifying this record non-creation to SGML systems. Currently, SGML has no rules (as far as I can see) for identifying records within system objects. If that's correct, then a mechanism is needed to determine what record-creation rules should be used for XML. I propose using the APPINFO declaration in the SGML declaration. APPINFO XML should suffice. It's true that no SGML parser currently treats entities as single records. However, I (and others) don't think it would be too difficult to change that. We could be wrong. 3) The RE and RS characters are defined to non-occuring code points. James Clark interpreted the definitions of these characters to be the signals that the entity manager should use when communicating with the parser. Given (1), the record boundaries will not be occurring, but the parser must not mistake code points 10 and 13 in data for record boundaries. 4) Code points 10 and 13 are defined as SEPCHAR. This allows them to occur in element content without error. 5) Whitespace handling is defined as an application convention. 5.1) In verbatim-styled content elements, all whitespace is preserved. 5.2) In non-verbatim content elements, leading and trailing whitespace is stripped. Intermediate whitespace is normalized to a single space before formatting. 5.3) In certain special elements, whitespace is eliminated. For 5.3, it doesn't need to involve special element names or parsing rules, just a formatting convention. The example I'm thinking of is a CR between table rows. Clearly, when formatting

bbbb
b...
I use the character b to indicate that it is not fair to expect a validating parser to just ignore arbitrary (whether they be space or 'b') data characters between cells in element content. In the same way, an application written to the XML spec has no good reason to expect that an author is going to throw random newlines at them in places where there should be no data (element content). If we are intent on allowing users to markup documents with newlines between elements in element content, then we MUST make those newlines insignificant (i.e. eaten by the parser). I suggest that we just make all newlines outside of verbatim elements insignificant and be done with it. But now I'm sounding like a broken record. Paul Prescod Date: Wed, 2 Oct 1996 22:42:09 -0600 From: "William D. Lindsey" To: Joe English Cc: w3c-sgml-wg@w3.org Subject: Re: RS/RE: basic questions <199610022136.AA20270@mail.crl.com> Joe English writes: > > "W. Eliot Kimber" wrote: > > > [...] but I think it's clear, as James pointed out > > what seems like years ago that the only real solution to the RE problem > > that preserves SGML compatibility is to eliminate mixed content, which > > means quoting data. > > > This is not the *only* solution. Another is to > forbid whitespace in element content. Another > is to require that pseudoelements which contain > nothing but whitespace be "escaped" with entity > references. Still another (which nobody's mentioned > yet for some reason :-) is to forbid *element* content, > so that all whitespace becomes significant. Actually, it has been mentioned, by Paul Prescod, I think. Doesn't your rule forbidding white space in element content amount to the same thing to a DTD-less XML parser? It won't know where data isn't allowed. I toyed with the idea myself for a while -- figuring we'd just leave it to the application (through style sheets or whatever) to ignore those SSEPs it wasn't interested in. When I realized I had just invented a new kind of comment, I got badly frightened by the implications. cell 1cell 2 Go ahead, tell me some clever hacker won't start informing the readers of comp.infosystems.authoring.htXml that this is where you put your special, secret codes. -Bill -- William D. Lindsey blindsey@bdmtech.com +1 (303) 672-8954 Date: Thu, 03 Oct 1996 01:12:31 -0400 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: RS/RE: basic questions We're thinking about what if we disallowed element content altogether. At 10:42 PM 10/2/96 -0600, William D. Lindsey wrote: >Actually, it has been mentioned, by Paul Prescod, I think. >Doesn't your rule forbidding white space in element content >amount to the same thing to a DTD-less XML parser? It won't >know where data isn't allowed. No, but the application will. Parsers are just front ends for applications. The application should say "data isn't allowed between these elements" and choke. Paul Prescod From: Charles@sgmlsource.com (Charles F. Goldfarb) To: Paul Prescod Cc: lee@sq.com, papresco@calum.csclub.uwaterloo.ca, w3c-sgml-wg@w3.org Subject: Re: RS/RE: basic questions Date: Thu, 03 Oct 1996 09:58:48 GMT On Thu, 03 Oct 1996 00:31:26 -0400, Paul Prescod wrote: >If we are intent on allowing users to markup documents with newlines between >elements in element content, then we MUST make those newlines insignificant >(i.e. eaten by the parser). I suggest that we just make all newlines outside >of verbatim elements insignificant and be done with it. But now I'm sounding >like a broken record. But at least it's playing music! -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- Date: Thu, 03 Oct 1996 13:24:05 +0100 From: Lou Burnard To: W3C-SGML-WG@w3.org CC: lou@vax.ox.ac.uk Subject: Re: acceptance of XML From: OXVAXD::LOU "Lou Burnard" 3-OCT-1996 13:23:05.54 To: MX%"cbullard@HiWAAY.net" CC: LOU Subj: Re: Acceptance of XML >I don't suggest the user base is unimportant. The habits of HTML are >very much the habits of SGML authors in most cases. So, trying to most cases of SGML authors or most cases of habit? Either way I vigorously dissent. HTML habits are for the most part inimical and in opposition to most habits of anyone worthy the name of "SGML author". HTML focuses attention on how to render fragmentary pieces of digital data. It actively discourages attention on how to organize such fragments into higher level structures, oither than by the tiresome procedure of tying everything together with explicit links. As far as XML is concerned, I think the way to sell it is to demonstrate (with some real software) that (a) you can do everything you can already do with HTML with XML (b) you can do a whole lot more besides with XML The way *not* to sell it, is to focus on (a) to the exclusion of (b). Pardon me for stating the obvious, but I don't think I'm the only one on this list guilty of that personality disorder. Lou Date: Thu, 3 Oct 1996 08:53:40 -0400 From: gtn@ebt.com (Gavin Nicol) To: papresco@calum.csclub.uwaterloo.ca CC: w3c-sgml-wg@w3.org Subject: Re: PNG Gets a Big Write-Up >For instance, one decision or the other on internationalization or >linking might be inconsistent with W3C's directions. I doubt I18N will be inconsistent with the W3C's position (one reason I took the stand I did). Date: Thu, 3 Oct 1996 09:26:48 -0400 From: gtn@ebt.com (Gavin Nicol) To: papresco@calum.csclub.uwaterloo.ca CC: w3c-sgml-wg@w3.org Subject: Re: RS/RE: basic questions >According to my understanding, the only way to make all whitespace >significant (i.e. to pass all whitespace to the application) is to do that >SGML DECL RE remapping hack. So a LOT of software would have to be changed >(i.e. almost all of the products you mentioned) if you are right that they >do not support SGML DECL tricks. As others have pointed out, the problem is really one for entity managers, as they are the things that actually put in RE and RS. In most SGML parsers I've seen, the entity manager actually doesn't even do this (ie. they don't *really* detect, and then add RE or RS), and instead, the parser is hardwired into thinking that CRLF are the things to look for. So the thing about "SGML compatability" is something of a red herring, because if you stick to the letter of the law (so to speak), many/most SGML applications don't conform anyway (good indication of a broken specification). In addition, having a parser perform removal of what the author might have intended to be significant whitespace (one can never be sure what the author's intentions were) to me seems to be at least as bad as leaving extra whitespace in. I would prefer to just leave it all in, and let applications process it. I think this will result in considerably *less* degradation of data of time as whatever occurs between a start and end tag would then always be the canonical content. So, my final proposal: 1) To say that RE handling follows the rules outlined by the ERB meeting *if* they occur. 2) That the RE and RS pair be defined to some private use area code so that there can be no confusion with CRLF. 3) That we strongly recommend that XML entity managers adopt a stream view of storage objects such that they do not recognise record boundaries (common application view provided by most system libraries on most modern OS's). Note that (3) is a *recommendation*, not a requirement. This would then let the market decide which way the decision should fall. To be frank, I think doing RE+RS and I18N *correctly* to be less important that actually *doing* XML. I have every faith that common usage will eradicate any bad decisions we make now (by either killing XML or changing it). As such, I will reduce my participation in discussions regarding both topics. Date: Thu, 03 Oct 1996 09:23:59 -0500 From: Len Bullard To: Lou Burnard CC: W3C-SGML-WG@w3.org Subject: Re: acceptance of XML Lou Burnard wrote: > > >I don't suggest the user base is unimportant. The habits of HTML are > >very much the habits of SGML authors in most cases. So, trying to > > most cases of SGML authors or most cases of habit? Either way I vigorously > dissent. HTML habits are for the most part inimical and in opposition to most > habits of anyone worthy the name of "SGML author". HTML focuses attention on > how to render fragmentary pieces of digital data. It actively discourages > attention on how to organize such fragments into higher level structures, > oither than by the tiresome procedure of tying everything together with > explicit links. Ok, but I consider the latter design, not authoring. So, a hair split because in every other way, I agree. It is the typing habits I am referring to that lead to exacerbated debates on white space. We keep trying to preserve the ability to put whitespace anywhere, and that complicates things. I would prefer to explicitly mark up everything, disallow inclusions and exclusions, etc. Quoting pseudo-elements isn't something I care for because folks will blow that one. Don't believe they don't hack the ASCII and that an editor or application can handle all of that. That won't happen. But they can be taught pretty easily to use markup more rigorously and that habit is just as bad in many SGML applications as it is in HTML. > As far as XML is concerned, I think the way to sell it is to demonstrate (with > some real software) that (a) you can do everything you can already do with HTML > with XML (b) you can do a whole lot more besides with XML I completely agree. > The way *not* to sell it, is to focus on (a) to the exclusion of (b). Agreed. We have to show them why that "whole lot more" is worth doing and accept that for some, it isn't. The CTS debate on Affording SGML is in some ways, a "drop the religious mystical fascination with SGML information" and get to the benefits vs the cost per application. Too often, I think, we have oversold the difficulty and undersold the smarts of the customer. I seem to be in the minority on this one because it is a sea change even for me, but requring the customer to rethink and redo every part of their pubs process to accomodate SGML quirks, or the preferences of the consultant, is the way to keep SGML out of the mainstream. Adjusting the expectations of both to the realities of the tools and the existing processes, then offering a path to evolve as needed is more realistic. This is the problem with IETMs vs ETMs and Java vs HTML: we can't afford programmers as authors. Further, they tend to be lousy at it. SGML is the middle ground, the interface. That is the benefit. Don't shock the monkey, and stay off the peels. Anyway, I think the list moderator will justifiably curtail this thread soon. We should debate these issues off list, or back on CTS. I apologize for moving the rumble onto a focused working group. > Pardon me for stating the obvious, but I don't think I'm the only one on this > list guilty of that personality disorder. We all be weird, Lou. len From: "Christopher R. Maden" Date: Thu, 3 Oct 1996 14:06:25 GMT To: w3c-sgml-wg@w3.org Subject: Re: RS/RE: Yet Another Proposal [Paul Prescod] > Okay, an SGML application knows that the newlines are insignificant > because of the DTD. Now what about a formatter? It will, by default, > leave the newlines in (because the central tenet of this proposal is > that newlines are significant). So how do you turn them off? The newlines are really there, as "true data". They are not *displayed*, per application convention dictating whitespace normalization after parsing (e.g., in the DSSSL grove). > Okay, maybe with a stylesheet. Now your document looks okay in an > SGML editor and in Netscape (presuming that SGML editors handle the > RS/RE remapping hack). Now you want to convert it to RTF. Okay, > maybe your conversion program has a stylesheet language that allows > you to strip out newlines. RTF is a presentation format. To map the XML into RTF, a stylesheet must be invoked; the stylesheet engine for XML normalizes the whitespace. > Now you want to put it in an "XML database". Each element will be > stored individually in the database, for later retrieval alone. How > does the database determine which newlines go in the database, and > which are "formatting". I guess you need some other "style sheet > thing" (or a DTD). The newlines go into the database. All of them. They are part of the data. > The problem is not immediate/practical. It is > long-term/theoretical. I think that long-term degredation of your > data will occur if you depend on "application conventions" (like > "table smarts") to determine what is the real information and what > is formatting. Therefore, the only safe way to encode this > bibliography in the proposed markup language is with no > insignificant newlines. I probably should have phrased the normalization rules better. o All new lines are data, except those known to be in element content (by virtue of SEPCHAR). o When formatting XML (for display, transformation into RTF, or printing): - In verbatim-styled blocks, preserve all whitespace. - In non-verbatim content blocks, eliminate leading and trailing whitespace, and normalize internal whitespace. - Whitespace between blocks is ignored. ("Block" meaning a paragraph flow object, table flow object, table part (row, entry, etc.), figure flow object, or other object for which intermediate spacing is a stylesheet function, not a content function.) > In this case, the author can either have convenient editing or > unambiguous true content. If we go with this proposal, we should be > clear on that and encourage users to sacrifice convenience in favour > of rigour. The problem is that true content, without one hack or another, is different between an SGML parse and an XML parse. Quoting is going to make XML unusable, IMO. By making *all* newlines data, handling is unambiguous. An SGML (or XML with DTD) parse will not be ESIS- identical to an XML parse without DTD, but after application conventions are applied, the result will be identical. Isn't that what matters? > I could accept this, but would rather go the opposite way, like most > markup languages (I think) and make newlines and tabs insignificant > unless you declare them to be so (in some form of verbatim > section). In most SGML documents, this is what authors intend most > of the time. True. But I see the alternatives this way: 1) Implement a shortref-based hack that won't work in most current SGML systems and complicates the markup, for a reason that won't be explainable to most users or implementers. 2) Define a simple application convention that won't work in most current SGML systems, simplifies markup, and is easy to explain. > Unlike most markup languages, however I would proclaim that space > characters are significant outside of markup(as Liam Quin said to > me: "I kinda need the spaces between words." =) ). Certain kinds of > formatting would have to be done with tabs and comments intead of > spaces, and authors would have to be careful to put a space at the > end of each line if they don't want their words concatenated. Most > editors do this for you automatically. On both Windows and the Mac, > the standard text editor widgets Do the Right Thing. They do? I've never had Notepad, Write, Wordpad, SimpleText, or Claris Works insert spaces at the end of lines for me. I think I would be upset if they did when I wasn't editing XML. And I don't see them implementing an XML mode any time soon. -Chris -- http://www.ebt.com +1.401.421.9550 +1.401.521.2030 One Richmond Square, Providence, RI 02906 USA" NDATA SGML.Geek> From: "Christopher R. Maden" Date: Thu, 3 Oct 1996 14:32:43 GMT To: w3c-sgml-wg@w3.org Subject: Re: RS/RE: Yet Another Proposal [me] > The newlines go into the database. All of them. They are part of the > data. ... except those known to be in element content, if the parse was done with a DTD. -Chris Date: Thu, 3 Oct 96 09:57:25 CDT From: paul@arbortext.com (Paul Grosso) To: w3c-sgml-wg@w3.org Subject: Re: RS/RE: basic questions > From: lee@sq.com > Date: Wed, 2 Oct 96 22:44:32 EDT > > There's no point in practice in being compatible with 8879 -- instead, XML > has to be compatible with actual tools. Probably the most widespread > tools that read SGML are HoTMetaL, Panorama, Adept, and A/E, with NSGMLS > and SGMLS and Omnimark being the most widespread on the `next layer down' > (conceptually, I don't mean to deprecate them!). I don't list DynaText > because the viewer is the widespread part, and it reads a compiled form. > As far as I know, only the tools in the 2nd group I mentioned support > shortref or datatag or making " a name start character (is that what's > going on there with <">...??). > > Again, if there are any other software writers represented here whose > current shipping software could not cope with making all whitespace > significant, and could not supply a script or patch or upgrade or > sidegrade or whatever within, say, six to twelve months, send me mail > and I will post a summary early next week. Adept handles shortrefs on input, though it will expand ("normalize") them upon input (and therefore never write them out, though it would be trivial to add an "outputfilter" that remapped certain tags to some other string of characters). Adept uses a conforming 8879 parser (except when it notices that the file is one it has previously written out in which case it uses a stripped down parser which, however, implements the same RE rules as 8879), so the 8879-insignificant REs are already gone by the time the application gets control. Therefore, Adept as written would not cope with making whitespace significant that isn't significant per 8879. Adept also ensures when it writes out a file that it introduces line breaks only where they are (1) insignificant per 8879 (and carefully places two REs when necessary to ensure that one is significant) or (2) in data content that is not indicated (by the style sheet) as a verbatim element (and the RE is treated as a space by the application when rereading the file). Adept introduces line breaks because most users want to be able to (a) print or (b) "ascii" email or (c) processes via something like sed or (d) edit in something like vi, and (a) and (b) require lines generally less than 80 characters long and (c) and (d) require lines less than 256 character long. However, the user can set the "target output record length" in Adept to anything between 40 and a very large number, and if they set it to, say, 30000, 8879-significant line breaks will not be introduced. The above is presented in the interest of factuality, and is not intended as any statement of opinion on this bordering-on-counter-productive and rapidly-approaching-ludicrous discussion on record ends. I'm reminded that the role of the work group participants is one of input-givers to the ERB, I consider that the ERB has been overwhelmed with input on this topic, and I plan to allow the ERB to incorporate the input into the proposal that Tim and Michael are developing without further input from me on this topic. Date: Thu, 3 Oct 1996 16:18:13 -0400 From: gtn@ebt.com (Gavin Nicol) To: papresco@calum.csclub.uwaterloo.ca CC: crm@ebt.com, w3c-sgml-wg@w3.org Subject: Re: RS/RE: Yet Another Proposal >> The newlines go into the database. All of them. They are part of the >> data. > >The spaces between my table cells cannot be considered part of the True >Data. "True data" is what I have in my head, before I sit down at the >computer. How can you claim to speak for every author in the world? By your definition of true data, you, I and the fellow next door will all disagree (I just checked). If you don't want the whitespace, don't put it in. >They have no meaning in ANY output format or as the result of ANY >database query. They are meaningless in all but two contexts; > >#1. My raw text editor, where they were created. >#2. An XML application that treats them as meaningful because it >doesn't have a stylesheet, doesn't know what "application >conventions" to apply and doesn't know what to do with them. Again, if you type in anything other than the "true data", then why would you expect an application to understand what you meant. DWIM interfaces have failed... >I think that any syntactic specification that subscribes behaviour for >a _particular application_ is in trouble. You've specified a >mechanism for a particular class of applications to "figure out" >what was meant, but not for the larger set of ALL applications. This mechanism is described as an application convention and has nothing at all to do with syntax. A syntax in which all any characters between tag boundaries are significant is pretty rigorous to my way of thinking. There is also another benefit to this approach: we would be able, just from a simple ESIS tree to create an instance that would create the exact same ESIS tree on parsing, *without* a DTD. Nice feature for testing purposes... >Making ALL newlines (outside of verbatim elements) NOT data is also >unambiguous, but preserves the SGML/HTML convention of using whitespace for >formatting without affecting the parse tree. Yes, but this is a *worse* hack, because now you *are* confluging syntax and semantics. You have to be able to tell the parser "this element is verbatim"... >> An SGML (or XML with DTD) parse will not be ESIS- >> identical to an XML parse without DTD, but after application >> conventions are applied, the result will be identical. Isn't that >> what matters? > >The problem is what you describe as "application conventions" are conventions >at the level ABOVE the XML parser (as opposed to implementing XML as a set >of SGML application conventions). So you are depending on "smart >applications". Actually, I cannot see why a DTD would even make a difference in a case where record boundaries don't occur, though I must confess to not having thought it through completely. Date: Thu, 3 Oct 1996 13:48:17 -0400 From: gtn@ebt.com (Gavin Nicol) To: paul@arbortext.com CC: w3c-sgml-wg@w3.org Subject: Re: RS/RE: basic questions >the application gets control. Therefore, Adept as written would not >cope with making whitespace significant that isn't significant per 8879. This is inaccurate unless you also specify the exact syntax in effect. Date: Thu, 03 Oct 96 13:02:00 CDT From: Michael Sperberg-McQueen Subject: DTD equivalence and expressive power To: W3C SGML Working Group In its meeting on 2 October, the ERB reached consensus on the following issues relating to the equivalence of document instances and of DTDs in XML and SGML. The brief statement of the points of consensus is followed by some discussion and examples. 1 For any XML DTD XD, it will be possible to generate, without human intervention, an SGML DTD SD, such that (a) SD will accept all the document instances accepted by XD, and (b) SD will produce the same ESIS for them (modulo any exceptions required by the XML handling of white space and record boundaries) 2 If possible, XML will be defined in such a way that for any XML DTD XD, a corresponding SGML DTD SD can be generated, without human intervention, such that in addition to 1(a) and 1(b), (a) SD will accept *only* documents which are ESIS-equivalent to some document instance accepted by XD, and (b) if SD is translated back into XML, producing a third DTD XD', then XD and XD' will accept an ESIS-equivalent set of documents (i.e. for each document accepted by XD, there is a document accepted by XD' which has the same ESIS, and for each document accepted by XD', there is a document accepted by XD which has the same ESIS). ------- Discussion As may be seen, point 2 puts a slightly heavier burden on XML than point 1, requiring in item (a) that if XML DTDs are translated into SGML, the resulting DTD enforces all the constraints of the original XML DTD, and in item (b) that XML DTDs preserve their expressive power and accept equivalent languages even after round-trip conversion into and from full SGML. It's not clear to everyone that this heavier burden can always be met, so point 2 is expressed as a goal, not a hard requirement. Another way of expressing point 2(a) is that XML will not have greater expressive power than Full SGML. This means, for example, that point 2(a) forbids XML to accept arbitrary regular expressions as content models, since some regular expressions cannot be translated into SGML content models. A hypothetical XML DTD with could in theory be translated into SGML as which would fulfil the requirements of point 1, since any document satisfying the first declaration also satisfies the second. A document containing , however, would satisfy the SGML DTD without satisfying the XML DTD. Rule 2(a) says XML can't allow that to happen. This has the effect that XML and SGML tools can both preserve the validity of XML documents, assuming that they validate the documents at all. The view of the ERB is, in short, that suggestions for increasing the expressive power of SGML DTDs -- of which we have several -- will need to go into the WG8 revision work, not into XML. It should probably be noted that the ERB did not discuss, and so has not achieved consensus, on whether XML DTDs may be *less* expressive than SGML DTD's, i.e. whether rule 2 should also work if one switched the names SGML and XML around in it. That is, the jury is still out on whether eliminating constructs like EMPTY, inclusion and exclusion exceptions, etc., is ruled out in principle or not. ------- Further discussion for the hard-core set theorists ... If we consider that each DTD defines (or 'generates') a language, then the set of DTDs possible using some notation generates a set of languages. Call it the LG-set (for 'languages generated set') of the notation. Formally, rule 2(a) requires that LG-set of XML be a subset of SGML's LG-set. Adding a rule 3 which replaces 'SGML' with 'XML' and vice versa would require that SGML's LG-set be a subset of XML's, i.e. that the two LG-sets be identical (using, as always, ESIS-equivalence module RE/RS differences as a test of identity of instances). -CMSMcQ From: Paul Prescod Subject: Re: RS/RE: Yet Another Proposal To: crm@ebt.com (Christopher R. Maden) Date: Thu, 3 Oct 1996 13:57:29 -0400 (EDT) Cc: w3c-sgml-wg@w3.org While I could live with the proposal to treat all whitespace as significant (which isn't unprecedented...think of Fortran), I really feel that we've got to be clear that you either lose the ability to format with whitespace or you lose control over your "true data". This is the last time, I promise. > The newlines are really there, as "true data". They are not > *displayed*, per application convention dictating whitespace > normalization after parsing (e.g., in the DSSSL grove). ... > The newlines go into the database. All of them. They are part of the > data. The spaces between my table cells cannot be considered part of the True Data. "True data" is what I have in my head, before I sit down at the computer. "True data is meaning." Spaces for formatting are NOT part of that true data. They are meaningless. I put them as part of the encoding process as an encoding convenience. They have no meaning in ANY output format or as the result of ANY database query. They are meaningless in all but two contexts; #1. My raw text editor, where they were created. #2. An XML application that treats them as meaningful because it doesn't have a stylesheet, doesn't know what "application conventions" to apply and doesn't know what to do with them. > o All new lines are data, except those known to be in element content > (by virtue of SEPCHAR). I don't understand. How can I know which ones are in element content? > o When formatting XML (for display, transformation into RTF, or > printing): > ... I think that any syntactic specification that subscribes behaviour for a _particular application_ is in trouble. You've specified a mechanism for a particular class of applications to "figure out" what was meant, but not for the larger set of ALL applications. This is acceptable if we do not expect people to ever encode their data in XML, but not otherwise. If XML is really, truly _delivery and display only_ then this is fair. Otherwise, not. > The problem is that true content, without one hack or another, is > different between an SGML parse and an XML parse. Quoting is going to > make XML unusable, IMO. By making *all* newlines data, handling is > unambiguous. Making ALL newlines (outside of verbatim elements) NOT data is also unambiguous, but preserves the SGML/HTML convention of using whitespace for formatting without affecting the parse tree. > An SGML (or XML with DTD) parse will not be ESIS- > identical to an XML parse without DTD, but after application > conventions are applied, the result will be identical. Isn't that > what matters? The problem is what you describe as "application conventions" are conventions at the level ABOVE the XML parser (as opposed to implementing XML as a set of SGML application conventions). So you are depending on "smart applications". A "stupid" application should be able to take the outputs of a rigorous parse and work on them without fear of extraneous, meaningless data. > Alternatives: > 1) Implement a shortref-based hack that won't work in most current > SGML systems and complicates the markup, for a reason that won't be > explainable to most users or implementers. > 2) Define a simple application convention that won't work in most > current SGML systems, simplifies markup, and is easy to explain. 3) Make all newlines and tabs meaningless except those that are lexically distinguished either through a section delimiter, an escape character, an entity reference or a simple GI. (we have many options of how to make them obvious) Make all spaces meaningful. * Most existant documents will parse immediately. * The output of SGML normalizers will parse immediately. * The output of almost all SGML tools will parse immediately. * The markup looks basically identical to that which the SGML community is accustomed to. cons: * incautious use of space characters as formatting or newlines "as" space characters will fail. BUT, where and when was it written that a newline and a space are equivalent anyhow? Isn't that just a bad habit, a subtle abuse of markup? I think that Lee made this point before. If you want a space, put in a space. If you want a newline, put in a newline. > > editors do this for you automatically. On both Windows and the Mac, > > the standard text editor widgets Do the Right Thing. > > They do? I've never had Notepad, Write, Wordpad, SimpleText, or > Claris Works insert spaces at the end of lines for me. i think I > would be upset if they did when I wasn't editing XML. And I don't see > them implementing an XML mode any time soon. Sorry I wasn't more explicit. If you insert a newline meaning to insert a space, none of these editors will put in another space for you. But why would you do that? If, on the other hand, you keep typing past the end of a line, all of these tools will word wrap for you. Some of them will insert newlines. Some will not. NONE will STRIP the space you just typed, before you knew (or cared) that the line was going to wrap. Therefore, unless you treat your text editor as a typewriter (which I admit, I do, in vi, because it does not do automatic word wrapping) you will be okay. Emacs seems to also do the right thing. Distressingly, (and surprisingly) fmt does not. vi users would have to be careful and put spaces before every newline, but anyone using a text editor written in the last two decades shouldn't have a problem. Lee pointed out that mail programs often strip trailing spaces (I don't know why they would, but...). This is a further restriction. One other interesting thing: this is VERY easy to warn the user about in a validating parser (with DTD) "Warning, there is no space between lines 20 and 21 in mixed content. Lines will be concatenated." If we don't trust editors, there are other alternatives. We could make the start tags for mixed content elements lexically distinct. We could make a rule that says that a newline becomes a space if the element has had data comment in it already. That fails in obscure cases like this:

This is a document. but the solution is simple: put a space in, instead of (or in addition to) a newline. Again, this can be easily checked. Paul Prescod Date: Thu, 03 Oct 96 17:01:52 CDT From: Michael Sperberg-McQueen Subject: Questions for decision 9 October To: W3C SGML Working Group The period of open unstructured discussion in the WG is coming to an end (reminder: the voting booth closes soon; if you haven't voted or wish to revise your votes, do so now!), and on 9 October, the ERB will begin voting on specific questions regarding the technical content of the XML specification. Our current plan is to vote on questions in batches: one batch each on 9, 16, 23, and 30 October. Each week, we will announce what questions we expect to vote on at the next week's meeting, so that members of the work group can make their views known on questions important to them. On behalf of the ERB, I am about to send out a series of email messages with the items to be decided next Wednesday, one item per message, to make it easier to follow individual threads. As will be clear, some of these are relatively non-controversial, at least as far as the ERB can tell, while others need (further) discussion to clarify the pros and cons of the issue. If the ERB is described as leaning a particular way on an item, and you agree, there is probably no need for comment; if you disagree, a note giving the arguments in favor of the opposing view is in order. When the ERB does not have a clear leaning, we need full discussion of all sides of the question. It should be noted in advance that not all of the questions are perfectly independent of each other; the ERB is acutely aware that some later decisions may require us to revisit earlier decisions, to ensure consistency of the final result. Thanks, and my apologies in advance for the avalanche of messages which will follow this one by a few minutes. -C. M. Sperberg-McQueen From: lee@sq.com Date: Thu, 3 Oct 96 19:10:35 EDT To: Charles@sgmlsource.com Subject: Re: RS/RE: basic questions Cc: w3c-sgml-wg@w3.org > >There's no point in practice in being compatible with 8879 -- instead, XML > >has to be compatible with actual tools. This has been quoted out of context. Say rather, there is no point in being compatible with SGML but not compatible with existing SGML tools. > Hey, it worked for HTML, didn't it? I am not sure what you mean by that -- are you referring to the IETF HTML 2.0 work, which in fact produced an SGML DTD? When HTML was invented, there *were* no existing HTML tools. It _is_ unfortunate that Tim didn't know enough about SGML -- ``just enough to be dangerous'', as Yuri put it -- and that Dan wasn't involved sooner... However, I would say that until the major tool manufacturers changed, and it became the case that the major implementors were not cooperating with the IETF working group, it was successful. But yes, it was a lot of work. Lee -- Liam Quin, SoftQuad Inc. | lq-text freely available Unix text retrieval lee@sq.com +1 416 544-9000 | FAQs: Metafont fonts, OPEN LOOK UI, OpenWindows http://www.softquad.com/ | xfonttool (Unix xfontsel in XView) SGML - HTML - Other Things | The barefoot programmer Date: Thu, 03 Oct 96 18:20:49 CDT From: Michael Sperberg-McQueen Subject: A3: SGML features? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.3 XML will have no OMITTAG, DATATAG, SHORTREF, LINK, CONCUR, RANK, or SUBDOC features (7.3.1, 7.3.1.1, 7.3.1.2, 7.3.2, 7.4, 7.5, 7.6, 7.7, 7.8, 9.4.6, 11.2, 11.5, 11.6, 13.5). Date: Thu, 03 Oct 96 18:21:21 CDT From: Michael Sperberg-McQueen Subject: A4: XML's use of SHORTTAG? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.4 XML will make only partial use of the SHORTTAG feature: - no minimized start-tags or, probably, end-tags (7.4, 7.5) - no omission of attribute name and value indicator (7.9.1.2) But - possible use of empty end-tag (7.5) (to be decided separately) - omission of attribute-value specification will be legal for attributes not declared REQUIRED (7.9) (applies only when DTD is supplied and used) The final point, on omitted attribute-value specifications, raises the general question of how XML systems will behave when no DTD, or a partial DTD, is provided -- if such omitted or partial DTDs are allowed. It also raises the question of providing a way for a document to signal that its DTD can be skipped without loss of information (e.g. because it has no default attribute values, or no empty elements, etc.). These questions are to be discussed and decided separately. Date: Thu, 03 Oct 96 18:23:07 CDT From: Michael Sperberg-McQueen Subject: A7: CDATA, RCDATA, TEMP marked sections? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the question needs further discussion in the work group. A.7 Should XML have CDATA, RCDATA, and TEMP marked sections or not? Date: Thu, 03 Oct 96 18:20:00 CDT From: Michael Sperberg-McQueen Subject: A2: SGML declaration? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.2 All or virtually all the information provided by a normal SGML declaration will be fixed for all documents; no SGML declaration will be necessary. (Possible exception: character-set information may vary document to document, but will be conveyed in other ways.) (6.2.3) Date: Thu, 03 Oct 96 18:21:59 CDT From: Michael Sperberg-McQueen Subject: A5: no quantities, no capacities? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.5 XML will have no quantities or capacities (7.3.3, 7.4.2, 7.9.2, 7.9.4.5, 9.4.1, 9.4.2, 9.8, 11.3.1, 13.2). Date: Thu, 03 Oct 96 18:26:07 CDT From: Michael Sperberg-McQueen Subject: A12: quote all attribute values? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.12 XML will require all attribute-value specifications to take the form of attribute-value literals (7.9.3, 7.9.3.1). Date: Thu, 03 Oct 96 18:29:59 CDT From: Michael Sperberg-McQueen Subject: A18: types of markup declarations? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.18 XML will have declarations for elements, attributes, and non-XML notations, but not for short-references or links (11.1). (Whether it will have declarations for entities depends on question A.17.) Date: Thu, 03 Oct 96 18:26:45 CDT From: Michael Sperberg-McQueen Subject: A13: require semicolon in entity refs? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.13 XML will not allow RE to end an entity or character reference; an explicit refc must provided, and it must be a semicolon (9.4.4). Date: Thu, 03 Oct 96 18:22:30 CDT From: Michael Sperberg-McQueen Subject: A6: no asynchronous marked sections? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.6 XML will not allow asynchronous marked sections -- marked sections must begin and end in the same element. Date: Thu, 03 Oct 96 18:29:33 CDT From: Michael Sperberg-McQueen Subject: A17: keep or drop entities? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the question needs further discussion in the work group. A.17 Should XML have entities, or not? Date: Thu, 03 Oct 96 18:23:42 CDT From: Michael Sperberg-McQueen Subject: A8: INCLUDE/IGNORE marked sections? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the question needs further discussion in the work group. A.8 Should XML have INCLUDE and IGNORE marked sections or not? (If this question is answered YES, it leads to a separate question, how to achieve conditional inclusion in XML markup declarations. This related question is to be decided separately.) Date: Thu, 03 Oct 96 18:30:43 CDT From: Michael Sperberg-McQueen Subject: A19: parsing rules? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the question needs further discussion in the work group. A.19 XML will retain fundamentally the same parsing rules as SGML, though they may be expressed differently. (N.B. there is some sentiment for making XML's rules more restrictive than SGML's.) Date: Thu, 03 Oct 96 18:33:26 CDT From: Michael Sperberg-McQueen Subject: A21: restrictions on attribute values? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.21 like SGML, XML will forbid empty strings as attribute values for non-CDATA attributes, require FIXED attributes to take their default values (7.9.4.1, 7.9.4.2), and distinguish implied values from null-string values (11.3.4). Date: Thu, 03 Oct 96 18:24:15 CDT From: Michael Sperberg-McQueen Subject: A9: CDATA and RCDATA elements? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.9 XML will have no CDATA or RCDATA elements (11.2.3). Date: Thu, 03 Oct 96 18:28:27 CDT From: Michael Sperberg-McQueen Subject: A16: resolve character refs in PIs? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the question needs further discussion in the work group. A.16 XML will stipulate that character references within processing instructions should be resolved by the XML parser (8). Date: Thu, 03 Oct 96 18:19:09 CDT From: Michael Sperberg-McQueen Subject: A1: only one concrete syntax? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.1 XML will have only one concrete syntax, fixed at XML specification time, not document-instance parse time (0.2, 13.3, 13.4). Date: Thu, 03 Oct 96 18:24:54 CDT From: Michael Sperberg-McQueen Subject: A10: how to escape markup characters in data? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the question needs further discussion in the work group. A.10 How should XML escape markup delimiter characters in content (especially if (R)CDATA elements and marked sections are not allowed)? Date: Thu, 03 Oct 96 18:25:37 CDT From: Michael Sperberg-McQueen Subject: A11: element content and mixed content? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.11 XML will retain the distinction between element content and mixed content (7.6, 11.2.4). (Applies only if DTD supplied and used.) Date: Thu, 03 Oct 96 18:27:45 CDT From: Michael Sperberg-McQueen Subject: A15: use notation names in 'formal' processing instructions? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.15 XML will use a sort of 'formal processing instruction': the first token of the PI's system data will be the name of a declared notation (e.g. or ) (7.6, 8) (Applies only if XML allows processing instructions, and if there is a DTD supplied and used.) Date: Thu, 03 Oct 96 18:27:21 CDT From: Michael Sperberg-McQueen Subject: A14: allow processing instructions? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the question needs further discussion in the work group. A.14 Should XML allow processing instructions or not? Date: Thu, 03 Oct 96 18:35:35 CDT From: Michael Sperberg-McQueen Subject: A25: recursive entity reference? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.25 Like SGML, XML will forbid recursive entity reference (9.4). Date: Thu, 03 Oct 96 18:36:30 CDT From: Michael Sperberg-McQueen Subject: A27: precedence and behavior of , | & ? * + ? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.27 XML will behave like SGML as regards behavior and precedence of occurrence indicators and connectors in content models (11.2.4.1, 11.2.4.2). (Whether to abolish the AND connector will be decided separately.) Date: Thu, 03 Oct 96 18:34:20 CDT From: Michael Sperberg-McQueen Subject: A23: CURRENT, FIXED, REQUIRED, IMPLIED attributes? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.23 XML will have no CURRENT attributes, but it will have FIXED, REQUIRED, and IMPLIED attributes, and attributes with explicit defaults. Date: Thu, 03 Oct 96 18:33:57 CDT From: Michael Sperberg-McQueen Subject: A22: CONREF? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.22 XML will have no CONREF attributes (11.3.3, 7.3, 7.9.4.4). Date: Thu, 03 Oct 96 18:35:57 CDT From: Michael Sperberg-McQueen Subject: A26: ANY keyword? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.26 Like SGML, XML will allow elements to be declared ANY (11.2.4). (Whether other similar shorthand declarations will be defined, e.g. for any subelements but not allowing PCDATA, will be decided separately.) Date: Thu, 03 Oct 96 18:34:58 CDT From: Michael Sperberg-McQueen Subject: A24: references to external data entities? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.24 Unlike SGML, XML will not allow direct references to external data entities from within parsed character data (9.4). Date: Thu, 03 Oct 96 18:37:34 CDT From: Michael Sperberg-McQueen Subject: A28: syntax of markup declarations? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the question needs further discussion in the work group. A.28 Should XML use the markup-declaration syntax described by ISO 8879 clauses 10-11, or should XML define a specialized document type and let its markup declarations use the document-instance syntax, as proposed by MGML? Date: Thu, 03 Oct 96 18:31:15 CDT From: Michael Sperberg-McQueen Subject: A20: comment syntax in XML? To: W3C SGML Working Group On 9 October 1996, the ERB will vote to decide the following question. A non-binding preliminary vote indicates the ERB is now leaning to Yes. A.20 XML will retain the notion and syntax of comments (= 8879's 'comment declarations') (7.6, 10.3), but comment declarations will contain at most one comment: comments will take the form '' or else will begin with '' (no space allowed), and may not contain '--'. (Editor's note: the ERB did not explicitly discuss empty comments; the consensus may extend only to the form, and not to keeping . -CMSMcQ) Date: Thu, 03 Oct 96 18:40:38 CDT From: Michael Sperberg-McQueen Subject: note on the meaning of 'Yes' To: W3C SGML Working Group Along about question A22, it occurred to me that my boiler-plate phrase "the ERB is now leaning to Yes" was not necessarily clear when the question is phrased as a negative proposition like "XML will have no bananas." Does 'Yes' mean 'yes it will too have bananas!' or the opposite? In the boilerplate phrase, 'Yes' means 'in favor of the position or state of affairs described in the proposition', as in 'yes, XML will have no bananas.' Sorry for any confusion. -CMSMcQ Date: Thu, 03 Oct 1996 17:22:36 -0700 To: W3C SGML Working Group From: Tim Bray Subject: Re: Questions for decision 9 October At 05:01 PM 10/3/96 CDT, Michael Sperberg-McQueen wrote: >The period of open unstructured discussion in the WG is coming to an end... >Thanks, and my apologies in advance for the avalanche of messages... The warders will be along shortly to chain you to your debating chairs. Only bread and water until the job's done. - Tim Date: Thu, 3 Oct 96 19:39:12 CDT From: paul@arbortext.com (Paul Grosso) To: w3c-sgml-wg@w3.org Subject: Re: A10: how to escape markup characters in data? > From: Michael Sperberg-McQueen > > A.10 How should XML escape markup delimiter characters in content > (especially if (R)CDATA elements and marked sections are not allowed)? The way that seems easiest and most intuitive to me is to represent them as entity references of some kind, e.g.:

<ELEMENT foo (a & b)>

At least some tools do this automatically for authors now. Date: Thu, 03 Oct 1996 17:52:56 -0700 To: W3C SGML Working Group From: Tim Bray Subject: Re: A28: syntax of markup declarations? (LONG) A.28 Should XML use the markup-declaration syntax described by ISO 8879 clauses 10-11, or should XML define a specialized document type and let its markup declarations use the document-instance syntax, as proposed by MGML? This is a long posting in favor of using instance syntax for XML markup >declarations. 1. Terminology and Examples 2. Motivation 3. SGML Interoperability Issues 4. HTML Interoperability Issues 5. Appropriateness, Timing, and Jurisdiction Issues 1. Terminology and Examples If we use instance syntax for XML DTD's, then to avoid misunderstanding we shouldn't call them DTD's. I have been using the term "Document Structure Declaration" (DSD) since XML syntax is hardwired, and what's being declared is really structure. If we are using DSDs, and DSDs are XML documents, then clearly there must be a DSD for DSD's - I have been calling this the XML Reference DSD. I have made several documents available to help out: A draft of the XML reference DSD: http://www.textuality.com/dsd/xml-ref.dsd An SGML DTD for xml-ref.dsd: http://www.textuality.com/dsd/xml-ref.dtd A draft of a simple DSD for papers: http://www.textuality.com/dsd/paper.dsd The initial draft of the paper on this subject that I submitted to SGML'96 - accepted, but superseded by our work in this group. Only here to provide a reference point for paper.dsd: http://www.textuality.com/dsd/paper.xml I should note that xml-ref.dsd includes some significant contributions of both thought and syntax from Michael Sperberg-McQueen. Whether you want to read this stuff before or after the rest of this posting is up to you. I would advise waiting; the issue is not (supposed to be) the quality of the DSD, but the advisability of having one. 2. Motivation 2.1 Quality of Specification This is close to my heart as co-editor of the XML spec. If we use DTD syntax, the spec will have to: (a) specify the DTD syntax, (b) specify how the declarations in the DTD syntax impose constraints on instances. If we use instance syntax, we lose almost all of (a); it turns out that if you explain how recognizers for elements and attributes work, then you can do most of the job just by listing fragments from the Reference DSD and explaining the effect of the various elements and attributes. Michael Sperberg-McQueen has provided a minimalist literate-programming kind of DTD so that the spec can actually include the master copy of the reference DSD. If we use instance syntax, we will have a shorter, more elegant, more airtight, and more comprehensible spec, and something upon which the prospective XML processor author can easily get bootstrapped. 2.2 Integrity I for one, once we get XML done, plan to start browbeating the document management and authoring practitioners of the world along the lines of "now that XML exists, you have NO EXCUSE for not using descriptive markup in your structured documents!" If I convince them, and they then start using XML, and the first structured document they encounter is a DTD, with its rather ad-hoc syntax, then I've lied to them. I find this completely unacceptable. 2.3 Ease of Implementation If there is only one syntax for declaration and instance, then the prospective XML processor author only needs one lexer and one parser. Granted, the DTD language is not the hardest in the world, but two lexer/parsers are always harder than one to build. Once you've built one little perl/VB/rexx/Java thingie that can pull apart an XML instance (which we're planning to make easy), you can then pull apart XML declarations too. 2.4 Familiarity If we can go to the HTML community and tell them "not only can you now add your own structures to documents and still deliver them on the web, but here's how you do it, and it's just another bunch of tags", this removes another significant barrier to resistence. I cannot in good conscience defend to these people the proposition that to do this something straightforward, obvious, and good, they have to learn another language. 2.5 Tools If DTDs are instances, then you can use your existing SGML editors and document management systems and searchers and other value-adds to manage them - thus bringing an important class of metadata into the domain of an important class of tools. Yes, I think that formatting directives and search-metadata and entity catalogs should also be in SGML! 2.6 Other Improvements The proposed reference DSD makes it impossible to declare pernicious mixed content. Perhaps there are other things that could be wired in as well. 3. SGML Interoperability Issues There are some problems here. First of all, (a) how do we deal with the fact that SGML documents need to have a . Similarly, I propose Date: Fri, 4 Oct 96 09:32:25 BST To: W3C SGML Working Group Subject: Re: A11: element content and mixed content? > On 9 October 1996, the ERB will vote to decide the following question. > A non-binding preliminary vote indicates the ERB is now leaning to Yes. > > A.11 XML will retain the distinction between element content and mixed > content (7.6, 11.2.4). (Applies only if DTD supplied and used.) Has the idea of an optional DTD digest been completely abandoned, i.e. an optional set of useful information from the DTD, such as content type for non-element-only content tags and attribute default (type)? To retain SGML compliance this could be in the form of PIs, e.g. ht Date: Fri, 4 Oct 1996 10:46:30 -0400 From: gtn@ebt.com (Gavin Nicol) To: U35395@UICVM.CC.UIC.EDU CC: w3c-sgml-wg@w3.org Subject: Re: A2: SGML declaration? >necessary. (Possible exception: character-set information may vary >document to document, but will be conveyed in other ways.) (6.2.3) Do you mean "character set" or "encoding"? Date: Fri, 04 Oct 1996 08:48:50 -0700 To: W3C SGML Working Group From: Tim Bray Subject: Re: A11: element content and mixed content? At 09:32 AM 10/4/96 BST, Henry S. Thompson wrote: >> A.11 XML will retain the distinction between element content and mixed >> content (7.6, 11.2.4). (Applies only if DTD supplied and used.) > >Has the idea of an optional DTD digest been completely abandoned, Nope. That'll come up later. And mixed content would certainly be one of the useful things to signal in such a thing. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Fri, 04 Oct 1996 13:36:11 -0400 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: A2: SGML declaration? At 06:20 PM 10/3/96 CDT, you wrote: >On 9 October 1996, the ERB will vote to decide the following question. >A non-binding preliminary vote indicates the ERB is now leaning to Yes. > >A.2 All or virtually all the information provided by a normal SGML >declaration will be fixed for all documents; no SGML declaration will be >necessary. (Possible exception: character-set information may vary >document to document, but will be conveyed in other ways.) (6.2.3) Chould we reserve a place in the syntax for referring to one in the future so that we can gracefully introduce some form of SGML declaration if we want to in the future? For instance, perhaps if we make some wrong decision here, we will need to make a future version and differentiate them in the SGML declaration. Paul Prescod Date: Fri, 04 Oct 1996 15:14:28 -0400 To: From: Paul Prescod Subject: Re: A8: INCLUDE/IGNORE marked sections? At 06:23 PM 10/3/96 CDT, Michael Sperberg-McQueen wrote: >On 9 October 1996, the ERB will vote to decide the following question. >A non-binding preliminary vote indicates the question needs further >discussion in the work group. > >A.8 Should XML have INCLUDE and IGNORE marked sections or not? >(If this question is answered YES, it leads to a separate question, how >to achieve conditional inclusion in XML markup declarations. This >related question is to be decided separately.) If marked sections are required to be element-synchronous, couldn't you make and elements or attributes and use the stylesheet to make them disappear at delivery time? I think that this sort of thing should fulfill the needs of most people without introducing this new feature for parsers to handle. Paul Prescod Date: Fri, 04 Oct 1996 15:14:21 -0400 To: From: Paul Prescod Subject: Re: A11: element content and mixed content? At 06:25 PM 10/3/96 CDT, Michael Sperberg-McQueen wrote: >On 9 October 1996, the ERB will vote to decide the following question. >A non-binding preliminary vote indicates the ERB is now leaning to Yes. > >A.11 XML will retain the distinction between element content and mixed >content (7.6, 11.2.4). (Applies only if DTD supplied and used.) Why would we want to keep this distinction? Paul Prescod Date: Fri, 04 Oct 1996 17:36:51 -0400 To: From: Paul Prescod Subject: Re: A7: CDATA, RCDATA, TEMP marked sections? At 06:23 PM 10/3/96 CDT, Michael Sperberg-McQueen wrote: >On 9 October 1996, the ERB will vote to decide the following question. >A non-binding preliminary vote indicates the question needs further >discussion in the work group. > >A.7 Should XML have CDATA, RCDATA, and TEMP marked sections or not? It would be really handy to have some mechanism, to allow arbitrary non-SGML data (in the same character encoding). The author should be allowed to specify the delimiter a la MIME or Perl. I think that this is pretty easy to parse, and to write, and would be a MAJOR advantage over HTML which does these things through . If we do not provide such a mechanism, there is a danger that the XML convention will end up being the same as HTML. This convention hurts SGML/XML-based tools by making significant data in the document invisible to them. It should be distressing to us that there could be some valid XML documents that look better if the browser did NOT use a proper XML parser (which should throw away comments). I think that a rigorous specification would be an obvious feature to roll into SGML-NG. On the other hand, documents that used this feature would usually not be SGML compatible, or compatible with SGML tools. You would have to run them through an SGMLizer that encoded the non SGML data in an SGML-compatible way. Paul Prescod Date: Fri, 04 Oct 96 15:38:37 CDT From: Michael Sperberg-McQueen Subject: Re: A8: INCLUDE/IGNORE marked sections? To: W3C SGML Working Group On Fri, 4 Oct 1996 15:20:11 -0400 Paul Prescod said: >At 06:23 PM 10/3/96 CDT, Michael Sperberg-McQueen wrote: >>A.8 Should XML have INCLUDE and IGNORE marked sections or not? >>(If this question is answered YES, it leads to a separate question, how >>to achieve conditional inclusion in XML markup declarations. This >>related question is to be decided separately.) > >If marked sections are required to be element-synchronous, couldn't >you make and elements or attributes >and use the stylesheet to make them disappear at delivery time? I >think that this sort of thing should fulfill the needs of most people >without introducing this new feature for parsers to handle. I think most of those who deal with variant text of this sort agree that conditional elements or text-variant elements are the best way to handle textual variation. Unlike marked sections, such elements place the display under style-sheet control, so that users can display the variants of their choice. In theory, one might do this with marked sections, as well, but it's a rare display engine that accepts display-time changes to parameter-entity values. But marked sections are used in prologs, too, not just instances. Any multi-flavored DTD (such as HTML 2 -- not just gargantuan DTDs like the TEI main DTD) needs user control over parse-time customization, to suppress some elements or select among multiple declarations of elements or attributes or entities or .... Currently the TEI and HTML both handle this using clever sequences of parameter entity declarations, marked sections, and the occasional eye of newt. I think it's a hard requirement for XML to allow this kind of parse-time customization. If it does, XML can fulfill the goal of being a language one can *work* in; as Dave Hollander put it early on, a language we can author in, and edit in, and process in, and publish in without needing a one-way transform at the publication step, into a seriously dumber language. Without parse-time customization of DTDs, XML will be much less useful, just a write-only publication language, filling the same niche that HTML currently fills for some of us -- more flexible than HTML, but not usable as the day-to-day maintenance format for one's documents. As far as I can tell, there are these main possibilities for conditional inclusions in markup declarations: - multiple declarations, with rules about which one wins, just like SGML entity declarations. I'm not sure this is quite enough, but it might be. - marked sections for INCLUDE/IGNORE, controlled (as now) by entities the user declares in the subset or in a system-dependent way (e.g. by command-line option) - if the markup declarations use instance syntax (which I think is an idea whose time is NOW), then we can use the same technique as Paul Prescod describes: elements which enclose the conditional material and the contents of which are processed or not, depending on the conditions. For example, something like (I omit the instance-syntax equivalent, but can supply it if anyone is really curious) So a TEI-ish DTD driver might read ... ... ... &base.dtd and an instance might read ?> [Disclaimer -- this is a sketch of one way of handling the local subset in instance syntax, not of THE way. If you can think of a better way, Tim and I want to talk to you.] I could live with any of these approaches (marked sections, conditional elements, or multiple declarations--assuming they suffice). But *some* method of specifying conditional inclusion has to be there, or XML will be only a publication medium, not a working medium. Slightly better than HTML, but not something we can *work* in; we'll have to work in some other notation and translate into XML. - If we're worried about the load on browsers and other software that wants to be lightweight, we can define a subset of the C preprocessor into the language and say servers should support #ifdef and #ifndef and maybe #if, but that clients need not. Then the TEI DTD reads #if &TEI.base; = 'prose' ... #elsif &TEI.base = 'verse' ... #elsif &TEI.base = 'drama' ... #endif &base.dtd and an instance might read as above, or (A server/client distinction in the language has been suggested before, but not extensively discussed. One of the design goals suggests we want to avoid such divisions, but accepting it may be better than having a language too heavy for lightweight processors but not strong enough for real work. I don't know the best answer.) -C. M. Sperberg-McQueen Date: Sun, 6 Oct 1996 12:03:15 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org cc: bosak@atlantic-83.eng.sun.com Subject: Test: w3c-sgml-wg This is a test of the list w3c-sgml-wg@w3.org posted 12:05 p.m. California time on 1996.10.06. Date: Sun, 6 Oct 1996 13:07:25 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org cc: bosak@atlantic-83.eng.sun.com Subject: SGML WG list up again The mailing list w3c-sgml-wg@w3.org is working again, but we may have lost a lot of mail. Someone removed /etc/mail/aliases from W3C's mailing list server. The support guy tells me that all the mail that bounced in the meantime should automatically get sent again. Problem is, none of the messages I sent while the list was down actually bounced, they just disappeared into the void. The last message in the thread index is Michael Sperberg-McQueen's posting "Re: A8: INCLUDE/IGNORE marked sections?" dated Friday, 1996.10.04 at 15:38:37 CDT. If you sent a message between then and approximately 1 p.m. California time on Sunday, 1996.10.06, and you don't see it appear on the list by 5 p.m. California time on Sunday, 1996.10.06, please repost it to the list. To help avoid confusion, it would probably be best to put "(Resend)" at the beginning of your subject line in case the original lost messages perversely come to life after you have reposted them. Please contact me directly if a message you post to the list doesn't show up within an hour or so. Jon Date: Sun, 06 Oct 1996 14:31:22 +0000 To: W3C SGML Working Group From: Tim Bray Subject: Re: A26: ANY keyword? At 06:35 PM 10/3/96 CDT, Michael Sperberg-McQueen wrote: >A.26 Like SGML, XML will allow elements to be declared ANY (11.2.4). Upon further consideration, I think I'll vote NO on this one. I can't think of a single time I've actually *used* ANY, and can't (in the abstract) think of situations where it is actually a worthwhile thing to do - particularly if we decide to lose exclusion exceptions. Thus, given that it makes the parser a bit more difficult, and the 80/20 rules we're supposed to be applying, I can't see why we should put it in XML. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Sun, 6 Oct 96 18:23:16 CDT From: paul@arbortext.com (Paul Grosso) To: w3c-sgml-wg@w3.org Subject: Re: A26: ANY keyword? > From: Tim Bray > > At 06:35 PM 10/3/96 CDT, Michael Sperberg-McQueen wrote: > > >A.26 Like SGML, XML will allow elements to be declared ANY (11.2.4). > > Upon further consideration, I think I'll vote NO on this one. I can't > think of a single time I've actually *used* ANY, and can't (in the abstract) > think of situations where it is actually a worthwhile thing to do - I've rarely used ANY too, but we've talked about using it several times when creating simple DTDs or, more likely, taking some sort of partial declaration and then needing to invent a declaration for an undeclared element. I think I can imagine various scenarios when ANY would be useful when trying to intuit a DTD--not that there might not be better ways of intuiting DTDs, but we may be making the problem of intuiting DTDs a lot harder if we remove ANY. From: lee@sq.com Date: Sun, 6 Oct 96 19:33:35 EDT To: tbray@textuality.com Subject: Re: A26: ANY keyword? Cc: w3c-sgml-wg@w3.org > >A.26 Like SGML, XML will allow elements to be declared ANY (11.2.4). > > Upon further consideration, I think I'll vote NO on this one. I can't > think of a single time I've actually *used* ANY, and can't (in the abstract) > think of situations where it is actually a worthwhile thing to do - > particularly if we decide to lose exclusion exceptions. There are some things that are hard to do without it... (1) versioning. SGML has no `transparent' element, that does not affect the content model in which it appears. You can get a similar effect to a MODREQ element by using an EMPTY element with a marked section, where you put attributes on the element (e.g. ACTION=(delete|insert|revise), WHO=CDATA, and so on), but then you need to manage keeping the marked section (used to show the span of the change) with the empty element, and you really want an inclusion to allow it everywhere. (2) When you can't work out the DTD; FRED helps a lot here, but I don't know if FRED could cope with the OED, for example. (3) When you really don't care... But ANY is always equivalent to (#PCDATA|every|element|in|the|DTD)* so it can be dropped. Actually I would favour dropping all connectors except |, so that XML contents would always be (a|b|c|d)* and also dropping rcdata and cdata. That way, you can get one of the two effects of an inclusion easily (allow anywhere at this level, not in children) by adding to the or-group, and the complexities of content models are considerably reduced from all points of view. Failing that, use normal regexp's and keep ANY... Lee Date: Sun, 6 Oct 96 18:19:24 CDT From: paul@arbortext.com (Paul Grosso) To: w3c-sgml-wg@w3.org Subject: (Resend) Re: A7: CDATA, RCDATA, TEMP marked sections? > From: Paul Prescod > > At 06:23 PM 10/3/96 CDT, Michael Sperberg-McQueen wrote: > > > >A.7 Should XML have CDATA, RCDATA, and TEMP marked sections or not? > > It would be really handy to have some mechanism, to allow arbitrary non-SGML > data (in the same character encoding). The author should be allowed to > specify the delimiter a la MIME or Perl. I think that this is pretty easy > to parse, and to write, and would be a MAJOR advantage over HTML which does > these things through . > > If we do not provide such a mechanism, there is a danger that the XML > convention will end up being the same as HTML. This convention hurts > SGML/XML-based tools by making significant data in the document invisible > to them. I see no reason for TEMP, and extremely little for RCDATA. If we decide to accept PaulP's suggestion that we should have some way to allow, for example, the inclusion of SGML "examples" in XML documents, then I suggest we use as an end delimiter. Otherwise, rather than invent new incompatible syntax, I suggest we just tell people to use an element with #PCDATA content, escape < and & (using < and & or some such), and indicate via the style sheet that the element is "verbatim" (and perhaps set it in a monospaced font). paul Date: Sun, 06 Oct 1996 19:34:52 -0400 To: W3C SGML Working Group From: Paul Prescod Subject: Re: A17: keep or drop entities? At 06:29 PM 10/3/96 CDT, Michael Sperberg-McQueen wrote: >On 9 October 1996, the ERB will vote to decide the following question. >A non-binding preliminary vote indicates the question needs further >discussion in the work group. > >A.17 Should XML have entities, or not? We can divide entities into parse-time entities and application-request entities. We should probably discuss those separately, because they have very different characteristics, especially in a networked environment. I wonder if we can do away with parse-time entities. I am going to argue against them, in this article, not because I am confident that we can do without them, but because I am not sure if we need them or not, and would like to get some feedback. I can imagine complications regarding document validition, entity retrival failures, communication of those problems from the parser to the application, etc. What do you do if an entity representing a paragraph cannot be retrieved across the web? Declare the document invalid and not show it? Report the failure to the application and allow them to put a "missing paragraph" marker in? Insert a placeholder in the output of the parser itself? It seems like the application and parser must have a fairly sophisticated language for communicating these failures to each other and to the user. It might be simpler if the application controlled all network transactions so that the only thing that could make a parse fail is a really invalid document or a network failure in the middle of a download. (this is the current model used by HTML browsers) So, if you wanted to include a paragraph, you would just do something along the lines of or or The existance or non-existance of PARA.XML would not be the parser's concern. It would merely report to the application that this element was referenced, and the application would fetch it or not fetch it depending on its needs, and the network availability of the object. In this way, communications between the entity manager, the parser and the application could be minimized, which would simplify parser. Since the application must already be able to handle entity downloading, I do not think that it complicates the application code at all. There are downsides: a) an XML "document" could not be made up of multiple entities in the same sense that an SGML document is. For instance, ID references across entities could not be checked by the parser (in fact, you might have to use some form of HyTime/application notation for these references). Same with content-model adherance and inclusions/exclusions. b) an XML "text entity" could not span elements. I think we've already decided against that feature anyway, so this isn't a major loss. c) It's not clear where the conventions I described above for including text >from another document would be standardized. In the XML standard itself? In each application? d) To what extent would these conventions tie application designers/writers hands? Paul Prescod Date: Sun, 06 Oct 1996 20:50:52 -0400 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: (Resend) Re: A7: CDATA, RCDATA, TEMP marked sections? At 06:19 PM 10/6/96 CDT, Paul Grosso wrote: >I see no reason for TEMP, and extremely little for RCDATA. If we decide >to accept PaulP's suggestion that we should have some way to allow, for >example, the inclusion of SGML "examples" in XML documents, then I suggest >we use > as a start delimiter and > ]]> >as an end delimiter. This isn't as flexible as the MIME and Perl equivalents, but I think it is a good tradeoff of compatibility vs. flexibility. Paul Prescod From: "Lauren Wood" To: "W3C SGML Working Group" Subject: Re: ANY keyword? Date: Sun, 6 Oct 1996 14:40:29 -0700 > From: Michael Sperberg-McQueen > > On 9 October 1996, the ERB will vote to decide the following question. > A non-binding preliminary vote indicates the ERB is now leaning to Yes. > > A.26 Like SGML, XML will allow elements to be declared ANY (11.2.4). > (Whether other similar shorthand declarations will be defined, > e.g. for any subelements but not allowing PCDATA, will be decided > separately.) Why do we need ANY? Lauren --- Lauren Wood, Softquad, Inc. Date: Sun, 06 Oct 1996 22:58:43 -0500 From: Len Bullard To: Paul Prescod CC: W3C SGML Working Group Subject: Re: A17: keep or drop entities? Paul Prescod wrote: > > > > >A.17 Should XML have entities, or not? > > We can divide entities into parse-time entities and application-request > entities. We should probably discuss those separately, because they have > very different characteristics, especially in a networked environment. Two features of SGML that have been cited in the past as hypermedia-like have been notations and general entities. In HyTime, these features have both been used and result in indirection. To many, the amount of indirection appears to be artificial. Where we are using entities for boilerplate text, we can use typed hyperlinks. For the network environment, it seems to me that link types serve the purpose as well as entities, and in effect, have the same results. Can we discuss this subject without getting into the issues reserved for the next phase? Jon? Len Bullard Lockheed Martin Date: Mon, 07 Oct 1996 00:06:29 -0400 To: Michael Sperberg-McQueen From: Harvey Bingham Subject: Resend: Re: A6: no asynchronous marked sections? Cc: w3c-sgml-wg@w3.org At 18:22 96/10/03 CDT, you wrote: >On 9 October 1996, the ERB will vote to decide the following question. >A non-binding preliminary vote indicates the ERB is now leaning to Yes. > >A.6 XML will not allow asynchronous marked sections -- marked sections >must begin and end in the same element. > > That is not a strong-enough condition. Presumable all such marked sections are contained in the base document element. The intent is that marked sections must begin and end in the same parent element. Proposed rewording (1) assuming no tag omission but EMPTY or #CONREF A.6 XML will not allow asynchronous marked sections -- any starttags in the marked section must have corresponding endtags in that marked section except that no endtags are present if the element declared content is EMPTY or a present attribute denoted #CONREF. Above rewording can be simplified if no EMPTY declared content or present attribute denoted #CONREF are permitted. Proposed rewording (2) A.6 XML will not allow asynchronous marked sections -- any starttags in the marked section must have corresponding endtags in that marked section. This allows mixed content marked sections. Regards/Harvey Bingham Date: Sun, 6 Oct 1996 22:19:45 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: cbullard@HiWAAY.net CC: papresco@calum.csclub.uwaterloo.ca, w3c-sgml-wg@w3.org Subject: Re: A17: keep or drop entities? | Where we are using entities for boilerplate text, we can | use typed hyperlinks. For the network environment, it seems to me that | link types serve the purpose as well as entities, and in effect, | have the same results. | | Can we discuss this subject without getting | into the issues reserved for the next phase? Jon? Insofar as you restrict the discussion to the question stated in A.17, yes. That question is: A.17 Should XML have entities, or not? As long as we keep the discussion of hyperlinking sufficiently abstract that we're speaking in hypotheticals for the purpose of answering this question and not engaging in the design process we have scheduled for the November-April time frame, no problem. The first paragraph quoted above is certainly well within acceptable bounds for this phase of the discussion. Jon Date: Mon, 7 Oct 1996 09:02:14 -0400 From: gtn@ebt.com (Gavin Nicol) To: tbray@textuality.com CC: w3c-sgml-wg@w3.org Subject: Re: A26: ANY keyword? >>A.26 Like SGML, XML will allow elements to be declared ANY (11.2.4). > >Upon further consideration, I think I'll vote NO on this one. Given that we're talking about DTD-less parsing, it seems that we *must* have this one... Date: Mon, 07 Oct 1996 09:15:37 -0400 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: A26: ANY keyword? At 09:02 AM 10/7/96 -0400, Gavin Nicol wrote: >>>A.26 Like SGML, XML will allow elements to be declared ANY (11.2.4). >> >>Upon further consideration, I think I'll vote NO on this one. > >Given that we're talking about DTD-less parsing, it seems that we >*must* have this one... I don't think that a DTD-less parser should make any presumptions about content model. It shouldn't care, or presume that a particular element has element content, mixed content or ANY content. That's why we have to have a parsing model that has no such distinctions. In other words, I think that if we keep the ANY keyword, it would be entirely for that subset of the parsers that DO USE DTDs. Paul Prescod Date: Mon, 07 Oct 1996 15:10:17 +0000 To: W3C SGML Working Group From: James Clark Subject: Re: A17: keep or drop entities? I am only going to address entities in the document instance in this note. Consideration of other kinds of entity will need to wait until the syntax for prologs as been decided upon. I think different kinds of entities need to be considered separately. 1. External entities 1a. External text entities These are the most problematic kind of entity. I believe these very substantially complicate implementation. (i) With these you can no longer do useful processing with simple 5-line Perl scripts. (ii) It radically increases the complexity of interfacing a parser to an application. Without external text entities, you can just give a parser a file or a string and get back a tree or a sequence of events. With external text entities, the parser has to be able to call back into the application to get the contents of declared entities. In particular I think external text entities would make it much harder to interface XML parsers to typical Web browsers. (iii) The parser no longer reads from a single input stream: it has to have a stack of input streams and be able to switch between them. (iv) I think it's hard to write SGML editors that allow good control over entity structure at the same time as element structure. (v) External text entities make it much harder to do transformations on XML documents. Without external text entities, transformations can work as a filter that take a single file as input and write a single file as output, and so transformations can be easily pipelined together. If transformations must work work with and preserve external text entities, this sort of approach is no longer possible. What is the functionality that external text entities are providing? - The most important is a mechanism for reuse. But I would suggest that it's better to handle reuse by making a self-contained, independently parseable XML document of whatever it is you want to reuse, and having an element whose semantics are to include that document in context. Of course, this requires some way to express that semantic and ways to handle hyperlinking between independent documents; but these are problems we are going to have to solve anyway. - External text entities also provide a mechanism for subdividing a document into separate storage objects for convenience in, for example, version control or text editing. I think there are much more powerful approaches to this sort of problem. For example, if I keep the document in some sort of OO database, I can dynamically make an entity out of whatever fragment of the element structure I want. I think including support for external text entities makes it much harder to support these more powerful approaches. 1b. External data entities These are much less troublesome than text entities: a parser just has to pass the external identifier and notation to the application. However, I think the Web community is so used to the simplicity and convenience of being able to stick a URL directly in an attribute value that they are not going to buy into the idea that they should instead - make up an entity name; - include a separate entity declaration; - include a separate notation declaration; - create an SGML Open catalog. Using external data entities also has the substantial disadvantage that each document typically ends up requiring its own DTD subset that declares the external data entities it needs. I think this will make DTD-less parsing much harder. The only extra information that using external data entities provides is the notation, but this is marginally useful in an HTTP world where that information is in the MIME header. Information in data attributes can just as well be put as extra attributes on the element that includes the URL attribute value. 2. Internal entities 2a. Internal CDATA entities I think these are useful especially for allowing the use of readable entity names instead of numeric character references. Since they can contain neither elements nor nested entity references, they complicate implementation very little. They also don't crop up in the ESIS, so they don't complicate the data model. On the whole I would be in favour of keeping these, maybe with the restriction that they must contain exactly one character. 2b. Internal SDATA entities Given that we can have Unicode character references, and given that Unicode has a private use zone, and given internal CDATA entities, I don't think these offer any really useful functionality. They also complicate the interface: with internal SDATA entities, you can't treat attribute simply as strings (unless you application maps them onto characters). 2c. Internal PI entities Probably insufficiently useful to justify the complexity. 2d. Internal text entities These have some of the implementation problems that external text entities have. I would be against including them (but not as strongly as I am against external text entities). In summary, the more I think about it, the more I am convinced that XML will fail if it includes external text entities. I would recommend having exactly one kind of entity, namely internal CDATA entities, which is, in fact, exactly what HTML does. James Date: Mon, 07 Oct 1996 10:32:03 -0500 From: Len Bullard To: James Clark CC: W3C SGML Working Group Subject: Re: A17: keep or drop entities? James Clark wrote: [Snipped: agree] > The only extra information that using external data entities provides is the > notation, but this is marginally useful in an HTTP world where that > information is in the MIME header. Information in data attributes can just > as well be put as extra attributes on the element that includes the URL > attribute value. Inquiry: when we lose the notation here, we are not restricting its use in a DTD where the notation has more value. MIME typing via a mailcap is essentially the notation declarations in a separate file. I believe an SGML notation can say more about the type. 1. It is both convenient and formal to have these as part of the Document Type to ensure that the formal requirements for the type are noted. 2. The HTTP world may be expanded to include other protocols (e.g. stateful). I inquire only to ensure we keep the valuable piece: a completely specified interface. In all other respects vis a vis entities, I agree with James. External entities look "weird" to current hypertext/hypermedia practicioners. However, we will have to come back and establish the means for indirect hyperlink locations as the notion of independent linking is too valuable to lose, IMO. Len Bullard Lockheed Martin Date: Mon, 7 Oct 1996 09:41:55 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org CC: bosak@atlantic-83.eng.sun.com Subject: Voting extended 48 hours Due to the fact that our WG mail list was down over the weekend, preventing the distribution of a final reminder to use the "voting booth", the XML co-editors and I have decided to leave it open an extra 48 hours to allow those of you who may have forgotten to vote one last chance to register your opinion on the inclusion of various 8879 features in XML. Votes will continue to be accepted through Tuesday evening, October 8. The URL is: http://www.textuality.com/sgml-erb/dd-1996-0002.html This survey is not binding on the ERB, and you will have the opportunity to discuss each point in much more detail during the next several weeks as specific questions come up for consideration, but this initial snapshot of WG sentiment will play an important role in focusing the discussion on the points that seem to need the most attention and will serve as an indicator of where the SGML community stands as it is represented in the WG. Jon Date: Mon, 07 Oct 96 11:30:09 CDT From: Michael Sperberg-McQueen Subject: Re: (Resend) Re: A7: CDATA, RCDATA, TEMP marked sections? To: W3C SGML Working Group On Sun, 6 Oct 1996 19:37:40 -0400 Paul Grosso said: >I see no reason for TEMP, and extremely little for RCDATA. If we decide Little, but not nil. I've needed RCDATA when my SGML examples are trying to illustrate the syntax of marked sections: since CDATA marked sections can't have nested marked sections within them, one needs something like ]]> The &nil; in the penultimate line prevents the marked section from ending prematurely. To be sure, this is a very rare requirement. But it is in fact reasonably easy to handle, with RCDATA; hence my sense that CDATA and RCDATA may as well be handled alike. >Otherwise, rather than invent new incompatible syntax, I suggest we >just tell people to use an element with #PCDATA content, escape >< and & (using < and & or some such), and indicate via the >style sheet that the element is "verbatim" (and perhaps set it in >a monospaced font). This works. I do hate to edit files like this using Ascii editors; marked sections make it much easier to read the examples. But any argument in favor of retaining (R)CDATA marked sections seesm to rely exclusively on aesthetics and convenience. It's clear we *can* get by without CDATA marked sections at all, and *should* do so, if they make parsers harder. They don't seem to me to complicate things, particularly if we require the keyword to be a literal, not a parameter entity reference, and so on the whole they seem to me to be worth retaining. But if we want, as Len Bullard puts it, to K.I.S.S. until your lips bleed then they should definitely go, unless our lips are already bloody >from some other decisions. Michael Date: Mon, 07 Oct 1996 12:59:05 -0400 To: From: Paul Prescod Subject: Re: (Resend) Re: A7: CDATA, RCDATA, TEMP marked sections? At 11:30 AM 10/7/96 CDT, Michael Sperberg-McQueen wrote: >This works. I do hate to edit files like this using Ascii editors; >marked sections make it much easier to read the examples. But any >argument in favor of retaining (R)CDATA marked sections seesm to rely >exclusively on aesthetics and convenience. Let me add to that list: document validity and maintenance. Not long ago, I ran into a set of documents on SGML Open's site that were invisible in my browser because the content had not been properly escaped. I can't remember the exact series of documents, but they were written by a prominant member of the community and encoded, probably by the SGML Open web maintenance team. If these knowledgable people can make this mistake, (by accident, certainly not through ignorance) then what of Joe Webmeister and Jane PerlHacker? I think that this is the kind of feature that browser vendors and other tool-makers would be glad to implement in order to make their lives easier. And as I mentioned before, if we don't include it, we risk having them come up with their own solutions, a la: Paul Prescod Date: Mon, 7 Oct 1996 10:08:23 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org cc: bosak@atlantic-83.eng.sun.com Subject: First month of discussions available For my own purposes, I have extracted from my mail archive everything I received from the W3C SGML WG list during the month of September (plus the very first message, which was sent at the end of August). This is not in any way an official archive, just a copy of all the messages I received from the W3C mail server during that period. I've tried to eliminate fragments of my private mail from the extract, but there may very well be a few duplicates, and, of course, messages are sometimes slightly out of order because of the order in which they arrived at my site. Some people may find this file easier to work with than the official W3C archive. It's available as a uuencoded zip or gzip file about 411 KB in length (just under 300 KB when uudecoded, just over 1 MB when unzipped). Send me a private request if you would like to be mailed a copy. Jon Date: Mon, 07 Oct 1996 10:47:03 +0000 To: James Clark , W3C SGML Working Group From: Tim Bray Subject: Re: A17: keep or drop entities? At 03:10 PM 07/10/96 +0000, James Clark wrote: >I am only going to address entities in the document instance in this note. I am in agreement with most of James' points. The one where (at the moment) I part company is best exemplified by a particular piece of text that Michael and I are working with heavily these days - namely the outline of the XML spec. In the middle, one fines the following: &Intro; &Docs; &Elements; &Entities; .... It's tough to imagine how we could do collaborative authoring in Vancouver and Chicago without this kind of facility, which certainly doesn't require the application of an OO database, etc.; and this is quite a non-trivial task. Or am I missing something? BTW, James is totally right that supporting external text entities does increase the plumbing complexity in the parser - the one time I did this it seemed not to break the grad-could-do-it-in-a-week barrier, but it is outside the scope of the 5-line perl hack job. I'm not sure I believe that "the parser has to call into the call back into the application to get the contents of declared entities" - when I did it, all that was done down in the lexer (yes, the lexer had to see the dsd) - which made available to the parser (and app) a picture of the entity tree - if they wanted it. His points about the difficulty of selling this to HTML-heads who are used to just jamming in a URL are good too; arguably though, the transclude & parse semantic is one that is not currently available in HTML - so they're going to have to learn something new anyhow... and we'll certainly support the kind of transclude & process a la the HTML To: W3C-SGML-WG@w3.org CC: lou@vax.ox.ac.uk Subject: re A28 I haven't been able to pay as much attention to this list as I should (pressure of real life intervening) and I must say that such of it as I have read has been often been rather remote from my interests (eg jesuitical distinctions between varieties of spaces between words)... *howsomever* I would like to go on record as enthusiastically endorsing Preposition 28 -- that the same syntax be used both for expressing and for defining XML document structure. I think Tim's arguments in favour are both cogent and expedient. This would be a decision that would make entirely clear what it is that XML has which distinguishes it from both SGML and from HTML, at the same time as showing that it has the strengths of both. Lou Date: Mon, 07 Oct 1996 14:04:43 -0400 To: Michael Sperberg-McQueen , W3C SGML Working Group From: "Eve L. Maler" Subject: Re: (Resend) Re: A7: CDATA, RCDATA, TEMP marked sections? At 11:30 AM 10/7/96 CDT, Michael Sperberg-McQueen wrote: >On Sun, 6 Oct 1996 19:37:40 -0400 Paul Grosso said: >>I see no reason for TEMP, and extremely little for RCDATA. If we decide > >Little, but not nil. I've needed RCDATA when my SGML examples are trying >to illustrate the syntax of marked sections: since CDATA marked >sections can't have nested marked sections within them, one needs >something like > > This paragraph has one sentence. > Or two? > ]&nil;]> > ]]> > >The &nil; in the penultimate line prevents the marked section from >ending prematurely. > >To be sure, this is a very rare requirement. But it is in fact >reasonably easy to handle, with RCDATA; hence my sense that CDATA and >RCDATA may as well be handled alike. Also, if you want the code being demonstrated to be "executable," isn't it better to declare an external CDATA entity (with a notation in this case, say, of "SGML fragment") so that you can change the &nil; back to "]" and use/test the entity contents normally? This also solves the problem of editing this material in ASCII editors (or, for that matter, CASE tools). Of course, this can get tiresome when you have lots of little fragments, but that's a convenience argument too, and environments can be constructed to ease their management and creation. Eve From: Charles@sgmlsource.com (Charles F. Goldfarb) To: Michael Sperberg-McQueen Cc: W3C SGML Working Group Subject: Re: A28: syntax of markup declarations? Date: Mon, 07 Oct 1996 18:20:55 GMT ----=_32594a07966058550dd11929.MFSBCHJLHS On Thu, 03 Oct 96 18:37:34 CDT, Michael Sperberg-McQueen wrote: >A.28 Should XML use the markup-declaration syntax described by ISO 8879 >clauses 10-11, or should XML define a specialized document type and let >its markup declarations use the document-instance syntax, as proposed >by MGML? XML should use a proper subset of the ISO 8879 declaration syntax, for = several reasons: 1. The necessary subset is small, clean, and easily explained. I have = attached the grammar to this note. It has fewer than 30 productions. (SGML has = almost 200.) 2. 20,000 or so people already know the DTD language. That is 20,000 more= than know MGML. 3. It is the semantics of markup declarations that presents learning difficulties, not the syntax. The semantics will be the same in any case. 4. The same is true for implementation. While a second syntax is a = burden, it is a relatively small and easily automated one. 5. SGML instance markup is a great language for representing structured information. It is a poor language for defining it. Tim's paper.dsd is = three times the size (in lines) of the attached paper.dtd. 6. All SGML tools can handle markup declarations.=20 7. There are no SGML interoperability issues because it *is* SGML. 8. There is no problem putting markup declarations in "XML masquerading = as HTML". Declarations just look like long unknown tags. (HTML users may = even find them familiar for that reason.) 9. XML needs to be a conforming subset of SGML; otherwise it will be seen= as a competitor to SGML whatever our good intentions to the contrary. That = perceived competition will confuse users at best; at worst, it will ripen into real competition, with users and vendors choosing sides. 10. Our objective for XML is to increase the SGML market by making it = easier to understand and implement. We only get this result if XML *is* SGML; = otherwise, SGML doesn't change at all. If XML is a conforming profile of SGML, it = can be the core of SGML97 -- the basic conformance level. The rest of SGML would= be defined as a delta on the core SGML; core XML/SGML users would never have= to read it. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- ----=_32594a07966058550dd11929.MFSBCHJLHS XML as Conforming SGML (aka Core SGML) Charles F. Goldfarb This document is a specification of a DTD subset notation that allows, to facilitate comparison, exactly the XML constraints proposed by Tim Bray in MSGML. (This author has somewhat different preferences for XML constraints.) In addition, it includes formal definitions for the distinct kinds of data (such as public identifiers) that MSGML just labels "CDATA". The notation used is not intended to be read by parser-generator tools. It is designed to show how little of SGML is needed for a useful core and how concisely it can be presented and explained. Time did not permit the creation of an equivalent grammar in a form suitable for parser-generator input, but it is obviously possible to create one. The 8879 metalanguage is used to allow easy comparison with full SGML, with the following enhancements: 1. Delimiters are shown as literals, since XML has a fixed concrete = syntax. 2. The sequence commas are omitted. 3. Spaces between tokens (rather than "s") indicate when whitespace is allowed. When tokens are concatenated, no whitespace is allowed. Note that "--" is not permitted in any declaration. elemtype-dcl =3D "" choice =3D "(" elem-cont ("|" elem-cont)* ")"("?"|"*"|"+") seq =3D "(" elem-cont ("," elem-cont)* ")"("?"|"*"|"+") mixed =3D "(" mixed-cont ("|" mixed-cont)* ")*" elem-cont =3D (Gi("?"|"*"|"+") | choice | seq) mixed-cont =3D (Gi("?"|"*"|"+") | "#PCDATA" | mixed) attlist-dcl =3D "" attdef =3D Attname dec-value default-spec dec-value =3D ("CDATA" | tokenized | enum-list | notname-list) tokenized =3D ("ID"|"IDREF"|"IDREFS"|"ENTITY"|"ENTITIES"|"NAME" = |"NAMES" = |"NMTOKEN"|"NMTOKENS"|"NUMBER"|"NUMBERS"|"NUTOKEN"|"NUTOKENS") enum-list =3D "(" NMTOKEN ("|" NMTOKEN)* ")" notname-list =3D "NOTATION" "(" Notname ("|" Notname)* ")" default-spec =3D ("#REQUIRED" | "#IMPLIED" | ("#FIXED"? a-literal)) entity-dcl =3D "" external-spec =3D external-id ("NDATA" Notname)? p-entity-dcl =3D "" notation-dcl =3D "" external-id =3D ("PUBLIC" m-literal) | ("SYSTEM" system-id) | ("PUBLIC" m-literal system-id) ">" system-id =3D (('"' fsi "') | ("'" fsi "'") | p-literal) fsi =3D "<" smname ("base" "=3D" literal)? = ">"(#CHAR|entref)*"" smname =3D ("url" | "osfile") p-literal =3D (('"'(#CHAR|p-entref)*'"') | ("'"(#CHAR|p-entref)*"'")) p-entref =3D "%"P-entname";" a-literal =3D (('"'(#CHAR|g-entref)*'"') | ("'"(#CHAR|g-entref)*"'")) g-entref =3D "&"Entname";" m-literal =3D (('"'#MINCHAR*'"') | ("'"#MINCHAR*"'")) WHERE: 1. Syntactic variables in initial caps are NAMES in the XML concrete syntax; that is: Gi Attname Entname Notname. 2. Syntactic variables in all caps are character strings whose spelling is defined by the XML concrete syntax; that is: NMTOKEN. 3. #CHAR is a character in the XML character set. 4. #MINCHAR is a minimum data character in the XML character set. ----=_32594a07966058550dd11929.MFSBCHJLHS ----=_32594a07966058550dd11929.MFSBCHJLHS-- Date: Mon, 07 Oct 1996 14:28:30 -0400 To: Tim Bray , James Clark , W3C SGML Working Group From: Paul Prescod Subject: Re: A17: keep or drop entities? At 10:47 AM 10/7/96 +0000, Tim Bray wrote: >At 03:10 PM 07/10/96 +0000, James Clark wrote: >>I am only going to address entities in the document instance in this note. > >I am in agreement with most of James' points. The one where (at the >moment) I part company is best exemplified by a particular piece of text >that Michael and I are working with heavily these days - namely the outline >of the XML spec. In the middle, one fines the following: > > >&Intro; >&Docs; >&Elements; >&Entities; >.... Can't this be done through: Paul Prescod From: lee@sq.com Date: Mon, 7 Oct 96 14:45:13 EDT To: w3c-sgml-wg@w3.org Subject: Re: A17: keep or drop entities? > BTW, James is totally right that supporting external text entities does > increase the plumbing complexity in the parser - the one time I did this it > seemed not to break the grad-could-do-it-in-a-week barrier, but it is > outside the scope of the 5-line perl hack job. This is true. It's one reason that having the C preprocessor available as a separate program is so useful -- macros (like SGML genera/character entities) get expanded before the parser does its job. This is good because it simplifies the parser & encourage reuse, and horribly bad because it means that error handling in the parser is in terms of an intermediate representation that is alien to the user. However, the ability to produce an "expanded" form of an XML document, with all external entities in place, would reduce the onus on parser writers, if only because it would demonstrate that it can be done in a separate pass. Let's get this right. It's hard to do in SGML because of marked sections and the way that entity definitions work (although I have not written a validating parser & am open to being corrected here). It was impossible in troff -- "soelim" does not work in the general case -- because of the way that conditional text interacted with file inclusion. If included files are visible to the parser both before and after inclusion, error messages can be more helpful -- e.g. if the file is in place, the contents are here, expanded=yes, and the optional "when" attribute gives the GMT date of expansion Lee Date: Mon, 07 Oct 1996 15:43:34 -0400 To: W3C SGML Working Group From: Paul Prescod Subject: Re: A28: syntax of markup declarations? Both Tim and Charles have good points, but I side with Charles, because I have never seen XML as an SGML-like-language, or an SGML-competitor, but as an SGML subset. I think that we should avoid fixing all of the problems with SGML here like the plague, except where the "problem" is complexity, or something else that makes SGML unfit for Internet use. But both participants confuse me with their assumption that XML needs a markup-declaration language at all. What kinds of tools do we expect are going to use these markup-declarations? Probably not browsers (after all, we've bent over backwards to exempt them from needing DTDs). Probably not search engines (for the same reason). Nor do hard-coded tools that use XML as a syntax for passing well-defined objects back in forth. So who? Probably structured editors and document validators. But these editors and validators ALREADY EXIST and they are validating SGML editors and parsers. So do we really expect a new set of XML validating editors/ to come into existance that support the XML declaration language, but not the SGML declaration language? >From my knowledge of the "internals" of validating editors, parsing the DTD and declarations is just a small part of the problem. Maintaining an interactive, intuitive user interface to a document WHILE you are editing is the hard part. In other words, solving the problem of "import" is easy. It's "maintainance" and "UI" that's a hassle. So I do not think we are going to see a flurry of new XML editors if we simplify the parsing and markup declaration syntax: that's not what's keeping people out of the market. As evidence I will site the fact that there are no tag-level validating HTML editors (a la HoTMetaL and Spider) that are not based on full SGML parsers. All other HTML editors are either non-validating, or are massively "WYSIWYG" so that users do not work with tags at all. AFAIK, nobody has implemented a validating HTML structural editor from scratch. I am similarly not convinced that there is a dirth of high-quality validating parsers. Is the SGML DTD syntax really so difficult to read that it is the "bottleneck" in implementing these things? I suspect that is the SGML instance syntax that is the problem. In other words, I don't think that there is ANY problem with the current situation w.r.t validating parsers and editors, and I don't think XML should try to fix it. If XML simply does not address the issue of structural validity, * those that care about structural validity will use XML as an SGML subset, and buy stripped-down XML as SGML tools, * those that don't care will have a smaller, simpler specification, and will use cheap tools that do not do structural validation, * those that want to use a completely different technology for validating, such as Java, Java Script, Scheme or DSSSL can use that. * Maybe the time has come to investigate "DTD alternatives" and maybe the Web community is just the diverse user base that should do that. My experience with SGML editors suggests that parsing the DTD is only the FIRST STEP in making a usable structural editing environment, so maybe we need more powerful editing-rules definition languages. Similarly, a DTD is only the first step in determining if a document conforms to a particular SGML application, so maybe we need more powerful document-application description languages. If we give the Web community some lattitude to experiment, they/we may come up with some neat things. If so (or if not) we can always standardize a markup declaration syntax in XML 2.0 (a new version), or in XML-validation-language (an add-on), or, best of all, in SGML97. If there is TRULY a market for a non-SGML markup declaration language, then the market will become apparent between the launch of XML and the first revision. We will have had lots of time to think it through, and can decide carefully where to exceed SGML's limitations, and where to use a subset. If the market interest does not materialize, then we will have saved ourselves some effort now (which could translate into a better spec in other areas), and simplified the spec by a few pages. Paul Prescod Date: Mon, 07 Oct 1996 21:24:05 +0000 To: Tim Bray From: James Clark Subject: Re: A17: keep or drop entities? Cc: W3C SGML Working Group At 10:47 07/10/96 +0000, Tim Bray wrote: >I am in agreement with most of James' points. The one where (at the >moment) I part company is best exemplified by a particular piece of text >that Michael and I are working with heavily these days - namely the outline >of the XML spec. In the middle, one fines the following: > > >&Intro; >&Docs; >&Elements; >&Entities; >.... > >It's tough to imagine how we could do collaborative authoring in Vancouver >and Chicago without this kind of facility, which certainly doesn't require >the application of an OO database, etc.; and this is quite a non-trivial >task. Or am I missing something? Some SGML parsers already allow the document entity to be spread out of multiple files; for example, with nsgmls, you could say nsgmls header.sgm Intro.sgm Docs.sgm Elements.sgm Entities.sgm and the effective document entity would be the concatenation of all files specified as arguments. If your XML parser doesn't do this, you can just use "cat". Whether it's something simple like file concatenation or something complex like a OO database, I think ways can be found to solve this sort of problem outside of XML. James Date: Mon, 7 Oct 1996 13:27:51 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org CC: bosak@atlantic-83.eng.sun.com Subject: Re: A28: syntax of markup declarations? (LONG) (Speaking only for myself) I have thought about Tim Bray's proposal to use the same syntax for instances and markup declarations and find that I am in complete agreement. 1. It makes no sense to claim that we have created a syntax suitable for the markup of structured data in general and then not use it for the markup of the structured document that defines a particular schema. From a marketing standpoint this is indefensible; from a logical standpoint it is absurd. I have heard no one deny this. 2. I am not qualified to certify the correctness of the proposed DSD syntax, but I can guarantee that it would be no harder to teach than the current DTD syntax, and I strongly suspect that it would be easier. 3. Unlike many other features -- marked sections, for example -- that we can defer for the moment if we wish and introduce in a later version of XML, this is not something that can wait. We have to take this approach from the beginning or it will never happen. If there are flaws in the DSD syntax included in the November draft, then early implementors will soon expose them, but if it's not in that draft, we're stuck with an illogical and indefensible failure to employ the very philosophy that we are espousing. I think we should do it. Jon Date: Mon, 07 Oct 1996 17:33:36 -0400 To: bosak@atlantic-83.eng.sun.com (Jon Bosak), w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: A28: syntax of markup declarations? (LONG) Cc: bosak@atlantic-83.eng.sun.com At 01:27 PM 10/7/96 -0700, Jon Bosak wrote: >(Speaking only for myself) > >I have thought about Tim Bray's proposal to use the same syntax for >instances and markup declarations and find that I am in complete >agreement. > >1. It makes no sense to claim that we have created a syntax suitable >for the markup of structured data in general and then not use it for >the markup of the structured document that defines a particular >schema. From a marketing standpoint this is indefensible; from a >logical standpoint it is absurd. I have heard no one deny this. I do! I do! :) You can stretch the definition of "structured document" to include whatever you want, but I think that users have common sense, and will recognize that there are limits to that which a particular document can do. The first few words of Annex A of the SGML standard are "Text processing and word processing". The definition of "markup" in ISO 8879 is "markup: text that is added to the data of a document in order to convey information about it." (if I'm reading the S.H. right), Without the markup in your DSD, there _is no data_. In practice, there is nothing at all wrong with encoding non-textual information in SGML for consistency, standardization, too reuse, etc. but I don't think that anyone should feel compelled to encode non-textual data in SGML. We don't need to standardize a DTD (or DSD) for GIFs or JPEGs to prove that we believe in SGML. Why doesn't DSSSL look like this: Helvetica ISO/IEC 10179:1996//Color-Space Family::Device RGB" Note that a DSSSL script _is_ "wrapped" in an SGML document. If we wanted to do that with traditional SGML DTDs, I would be much less critical, but many of the benefits would be lost. Furthermore, I don't think that anyone other than SGML purists would even consider using a markup language for encoding documents without text-data. They would certainly not think us odd for not doing so. After all, the people we are talking about don't see anything wrong with mixing JavaScript, VBScript and HTML into one document. Perhaps in the fullness of time, SGML2000 should go the opposite way: it should provide simple mechanisms for changing markup syntaxes that would allow us to specify a "DTD for DTDs" and a "DTD for scheme programs" without expanding DTD and scheme program instances into the verbose-existant SGML instance syntax. It isn't right to unify all languages under the SGML "banner" by making them harder to read and longer to download. In short, I think we can be consistent philosophically without using instance syntax for DTD syntax, by using the "the right tools for the right jobs" argument. >2. I am not qualified to certify the correctness of the proposed DSD >syntax, but I can guarantee that it would be no harder to teach than >the current DTD syntax, and I strongly suspect that it would be >easier. To teach, yes. To use? I'm not sure. >3. Unlike many other features -- marked sections, for example -- that >we can defer for the moment if we wish and introduce in a later >version of XML, this is not something that can wait. We have to take >this approach from the beginning or it will never happen. Why can't it happen later? Standards change. Sometimes new standards are built on top of them. >If there are flaws in the DSD syntax included in the November >draft, then early implementors will soon expose them, but if it's not >in that draft, we're stuck with an illogical and indefensible failure >to employ the very philosophy that we are espousing. I don't think that it is a departure from our philosophy for the reason I described above. A DTD isn't a text document, any more than an MPEG is, and I don't feel guilty in the least telling clients that SGML doesn't wash dishes. I don't want to start a chain of anecdotal contradictions, but I've never had a newbie ask me why DTDs aren't SGML documents (though they may have wondered why the syntax was so different...I don't know). Once again, I find the idea attractive, but I have doubts. DTDs are "on the line" of documents that should be SGML-encoded. There are standardization benefits if they are, but efficiency benefits if they are not. I like the IETF philosophy: "when in doubt, leave it out." I fear that if we veer from that, we'll make a mistake that will cost us in the long run. "The market" has had the opportunity to implement this idea, just as Tim described it, with DTD->DSD translators. Near and Far, Fred and DTDtoHTML could have an "output SGML Instance" option, but don't. If DSDs are useful, I think that we should prove so in the marketplace of ideas and tools. I am more than willing to "play around" with DSD, DSD converters and DSD-based manipulation tools in the next few months, to see if they solve actual problems that I have. But in the meantime, I think that we may be solving a non-problem (or minor problem). XML isn't the place for that. I would argue (again) that we should defer this decision by leaving structural validity testing out of the XML standard. That's what SGML is for. Paul Prescod Date: Mon, 07 Oct 1996 17:06:09 -0500 From: Len Bullard To: Jon Bosak CC: w3c-sgml-wg@w3.org Subject: Re: A28: syntax of markup declarations? (LONG) Jon Bosak wrote: > > (Speaking only for myself) > > I have thought about Tim Bray's proposal to use the same syntax for > instances and markup declarations and find that I am in complete > agreement. > > 1. It makes no sense to claim that we have created a syntax suitable > for the markup of structured data in general and then not use it for > the markup of the structured document that defines a particular > schema. From a marketing standpoint this is indefensible; from a > logical standpoint it is absurd. I have heard no one deny this. I deny it. This seems a very large move toward reinventing SGML and putting a separate market into place that can take advantage of neither SGML or HTML or any of the tools we currently have. I don't want to have to pay to have the neighborhood built. This doesn't sound like a recipe for success. We lost all of the investment the SGML community has made in both products and in knowledge and training. This would be a language with NO support outside this mailing list. I support a simplified SGML; not an out and out replacement. Were that the case, I would say do LISP and forget about it. Someone is going to have make a very convincing argument for this. No, don't just point to the papers. Convince us here. Len Bullard Lockheed Martin Date: Mon, 7 Oct 1996 18:20:22 -0400 From: "Steven R. Newcomb" To: w3c-sgml-wg@w3.org from Paul Prescod on Mon, 07 Oct 1996 15:43:34 -0400) Subject: Re: A28: syntax of markup declarations? > Date: Mon, 07 Oct 1996 15:43:34 -0400 > From: Paul Prescod > > Both Tim and Charles have good points, but I side with Charles, because I > have never seen XML as an SGML-like-language, or an SGML-competitor, but as > an SGML subset. Hear, hear. If XML competes with SGML, it's clearly a disaster for both of them. If SGML tools can be used to process XML, both are winners. I find Charles's list of arguments on this point unassailable; no need to repeat them here. I can only add that, personally, I look at this as a professional issue. As information management professionals, I can't see any way we can recommend XML to our clients unless their information assets -- and our clients themselves -- are already in the realm of SGML: (1) should they later discover needs which XML cannot meet and which SGML can, and (2) because it is so important to be in the mainstream of information representation as defined by international standards: (a) for the sake of our clients, for the ultimate protection of their information assets from the fallout of wars between commercial interests, (b) for the sake of ourselves, so that our professional asses are covered and so we are maximally protected against lawsuits from erstwhile clients, and (c) not least because it's the right thing to do for humanity and world peace and prosperity, other things being equal, to show respect for international accords, and to try to live with them. I realize that the last item, 2c, is the kind of statement that is unpersuasive and even annoyingly irrelevant to younger engineers. I direct their attention to the technical and economic arguments in favor of XML being a proper subset of SGML (as amended), and in favor of positioning XML in such a way as to influence the amendment of SGML; these arguments are certainly persuasive enough. Best regards, --Steve *************************************************************** * Steven R. Newcomb | President * * direct +1 716 389 0964 | TechnoTeacher, Inc. * * main +1 716 389 0961 | (courier: 3800 Monroe Avenue, * * fax +1 716 389 0960 | Pittsford, NY 14534-1330 USA) * * Internet: srn@techno.com | P.O. Box 23795 * * FTP: ftp.techno.com | Rochester, New York 14692-3795 * * WWW: http://www.techno.com | USA * *************************************************************** Date: Mon, 07 Oct 96 17:49:01 CDT From: Michael Sperberg-McQueen Subject: Re: A28: syntax of markup declarations? To: Paul Prescod , W3C SGML Working Group On Mon, 7 Oct 1996 15:56:27 -0400 you said: >But both participants confuse me with their assumption that XML needs >a markup-declaration language at all. What kinds of tools do we >expect are going to use these markup-declarations? Probably not >browsers (after all, we've bent over backwards to exempt them from >needing DTDs). Probably not search engines (for the same reason). Nor >do hard-coded tools that use XML as a syntax for passing well-defined >objects back in forth. The goals for the project include making it possible to recognize the element boundaries of the instance without having to consult the DTD; we may also choose to make it possible to handle the element/mixed content distinction without reading the DTD in full. Any application that needs no more than successful recognition of element boundaries can, if we meet this goal, work happily without a DTD, or skip one if one is present. Since such applications *cannot* skip the DTD in SGML, this would be a net gain. But the goals also include making XML usable for all types of application, not just publication on wide area networks. If we succeed in this, then publication can be as simple as copying files into a public_xml directory, instead of involving a down-translation into some other language. Copying is better than down-translation in part because its faster, simpler, and less error-prone, but more important because down-translating as a publication step means readers cannot get access to documents as useful as our working copies. There may be administrative or policy reasons to remove some information before publishing a document, but it should be a policy decision, not a technical necessity. XML does need markup declarations; without them, it's not usable for applications that need document grammars. -CMSMcQ Date: Tue, 08 Oct 1996 07:27:23 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: A28: syntax of markup declarations? At 06:20 PM 10/7/96 -0400, Steven R. Newcomb wrote: I think I've said my piece on the main issues here. But there is one Steve Newcomb's statements that I *must* challenge: > As information management professionals, I >can't see any way we can recommend XML to our clients unless their >information assets -- and our clients themselves -- are already in the >realm of SGML: If this is true, we're wasting our time. The main benefit of XML is that it might well expand - immensely - the number of people who are using descriptive markup and standardized formats, instead of presentational/mongrel markup and proprietary formats. I personally plan to recommend XML, vociferously, to the Web publishing population, when they hit the HTML brick wall, and the document management profession, when they get tired of dealing with documents that are opaque proprietary bit blobs. XML is the easy on-ramp to SGML. If we go on preaching to the existing SGML choir, why bother doing this? Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Mon, 07 Oct 1996 19:45:07 -0400 To: Paul Prescod , bosak@atlantic-83.eng.sun.com (Jon Bosak), w3c-sgml-wg@w3.org From: "Eve L. Maler" Subject: Re: A28: syntax of markup declarations? (LONG) Cc: bosak@atlantic-83.eng.sun.com I agree with Paul P. and Charles on A28. I actually do believe it's possible and worthwhile to make a DTD for markup model specifications because of the benefits of managing the specs and, to a smaller degree, writing them, but this isn't a self-evident characteristic of the language. (FOSIs are SGML documents, which may not have been the most natural syntax choice in the world, but they do automatically get the benefits of SGML information creation, storage, and management.) However, I don't think we're the right agency to do this. I had earlier felt that this should be a WG8 project, but Paul's analysis of the market solutions pointed out that the "invisible hand" agency is appropriate too. I have no problem issuing an "advisory" or "informational" DTD for DTDs (along with the appropriate transformers) as a side project, whenever we get around to it -- we've certainly got the right talent lying around! And I think there's nothing wrong with seemingly having two notations for DTDs (which would go against our principle of having one way to do things); this should be in the realm of a helper application. If it's popular enough, it can later be nailed down as a standard. If XML is popular, more people will start writing DTDs, and to the extent that the technology is brought down to the individual-contributor level, the proportion of DTDs to instances will probably go up. However, for the kinds of DTDs that will contribute most to the increase at first, syntax won't be the big barrier. This just isn't a five-alarm problem. Eve At 05:33 PM 10/7/96 -0400, Paul Prescod wrote: >At 01:27 PM 10/7/96 -0700, Jon Bosak wrote: >>(Speaking only for myself) >> >>I have thought about Tim Bray's proposal to use the same syntax for >>instances and markup declarations and find that I am in complete >>agreement. >> >>1. It makes no sense to claim that we have created a syntax suitable >>for the markup of structured data in general and then not use it for >>the markup of the structured document that defines a particular >>schema. From a marketing standpoint this is indefensible; from a >>logical standpoint it is absurd. I have heard no one deny this. > >I do! I do! :) > >You can stretch the definition of "structured document" to include whatever >you want, but I think that users have common sense, and will recognize that >there are limits to that which a particular document can do. The first few >words of Annex A of the SGML standard are "Text processing and word >processing". The definition of "markup" in ISO 8879 is > >"markup: text that is added to the data of a document in order to convey >information about it." (if I'm reading the S.H. right), > >Without the markup in your DSD, there _is no data_. > >In practice, there is nothing at all wrong with encoding non-textual >information in SGML for consistency, standardization, too reuse, etc. but I >don't think that anyone should feel compelled to encode non-textual data in >SGML. We don't need to standardize a DTD (or DSD) for GIFs or JPEGs to prove >that we believe in SGML. > >Why doesn't DSSSL look like this: > >Helvetica > > > ISO/IEC 10179:1996//Color-Space Family::Device RGB" > > >Note that a DSSSL script _is_ "wrapped" in an SGML document. If we wanted to >do that with traditional SGML DTDs, I would be much less critical, but many >of the benefits would be lost. > > >Furthermore, I don't think that anyone other than SGML purists would even >consider using a markup language for encoding documents without text-data. >They would certainly not think us odd for not doing so. After all, the >people we are talking about don't see anything wrong with mixing JavaScript, >VBScript and HTML into one document. > >Perhaps in the fullness of time, SGML2000 should go the opposite way: it >should provide simple mechanisms for changing markup syntaxes that would >allow us to specify a "DTD for DTDs" and a "DTD for scheme programs" without >expanding DTD and scheme program instances into the verbose-existant SGML >instance syntax. It isn't right to unify all languages under the SGML >"banner" by making them harder to read and longer to download. > >In short, I think we can be consistent philosophically without using >instance syntax for DTD syntax, by using the "the right tools for the right >jobs" argument. > >>2. I am not qualified to certify the correctness of the proposed DSD >>syntax, but I can guarantee that it would be no harder to teach than >>the current DTD syntax, and I strongly suspect that it would be >>easier. > >To teach, yes. To use? I'm not sure. > >>3. Unlike many other features -- marked sections, for example -- that >>we can defer for the moment if we wish and introduce in a later >>version of XML, this is not something that can wait. We have to take >>this approach from the beginning or it will never happen. > >Why can't it happen later? Standards change. Sometimes new standards are >built on top of them. > >>If there are flaws in the DSD syntax included in the November >>draft, then early implementors will soon expose them, but if it's not >>in that draft, we're stuck with an illogical and indefensible failure >>to employ the very philosophy that we are espousing. > >I don't think that it is a departure from our philosophy for the reason I >described above. A DTD isn't a text document, any more than an MPEG is, and >I don't feel guilty in the least telling clients that SGML doesn't wash >dishes. I don't want to start a chain of anecdotal contradictions, but I've >never had a newbie ask me why DTDs aren't SGML documents (though they may >have wondered why the syntax was so different...I don't know). > >Once again, I find the idea attractive, but I have doubts. DTDs are "on the >line" of documents that should be SGML-encoded. There are standardization >benefits if they are, but efficiency benefits if they are not. I like the >IETF philosophy: "when in doubt, leave it out." I fear that if we veer from >that, we'll make a mistake that will cost us in the long run. > >"The market" has had the opportunity to implement this idea, just as Tim >described it, with DTD->DSD translators. Near and Far, Fred and DTDtoHTML >could have an "output SGML Instance" option, but don't. If DSDs are useful, >I think that we should prove so in the marketplace of ideas and tools. I am >more than willing to "play around" with DSD, DSD converters and DSD-based >manipulation tools in the next few months, to see if they solve actual >problems that I have. > >But in the meantime, I think that we may be solving a non-problem (or minor >problem). XML isn't the place for that. I would argue (again) that we should >defer this decision by leaving structural validity testing out of the XML >standard. That's what SGML is for. > > Paul Prescod > > > Date: Mon, 07 Oct 1996 20:41:42 -0400 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: A28: syntax of markup declarations? At 07:27 AM 10/8/96 -0700, Tim Bray wrote: >XML is the easy on-ramp to SGML. In your opinion (which is to say, each reader's), #1. Will an XML document _have_ to have a DTD or DSD to be considered valid? And if so, is XML really much of an "easy on-ramp" for authors, or just for tool vendors. #2. And if not, how do you feel about the fact that some XML documents will not be valid SGML documents (i.e. they will not conform to any DTD). The answers to these questions will influence my opinions on question A28, and are, I think more fundamental than the question of the syntax of markup declarations. Paul Prescod From: Charles@sgmlsource.com (Charles F. Goldfarb) To: lee@sq.com Cc: papresco@calum.csclub.uwaterloo.ca, w3c-sgml-wg@w3.org Subject: Re: RS/RE: basic questions Date: Thu, 03 Oct 1996 09:55:34 GMT On Wed, 2 Oct 96 22:44:32 EDT, lee@sq.com wrote: >There's no point in practice in being compatible with 8879 -- instead, XML >has to be compatible with actual tools. Hey, it worked for HTML, didn't it? And think of all the time we could save not having to write and review standards -- XML and SGML. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- Date: Tue, 08 Oct 1996 09:45:07 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: A28: syntax of markup declarations? At 08:41 PM 10/7/96 -0400, Paul Prescod wrote: >#1. Will an XML document _have_ to have a DTD or DSD to be considered valid? >And if so, is XML really much of an "easy on-ramp" for authors, or just for >tool vendors. That is an excellent question, which has received relatively little discussion so far. My personal opinion is that there are actually two levels of validity: an XML document is well-formed if it has correct tag & attribute syntax and all the elements nest properly. An XML document is valid if parses according to some DTD/DSD. I think we can all agree that these two levels are easy to understand and are meaningful. The question of whether the XML spec should discuss both of these in a normative way, and if not, which one, is open and interesting. Your postings would lead me to suspect you want to discard validation in the SGML sense, and consider a well-formed XML document valid. Which is a reasonable point of view, but not mine. At the moment I think that both levels should in fact be defined normatively in the XML spec. >#2. And if not, how do you feel about the fact that some XML documents will >not be valid SGML documents (i.e. they will not conform to any DTD). A display or search application doesn't care all that much. A professional authoring/publishing tool cares a whole lot. We should support the needs of both. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Mon, 7 Oct 1996 22:45:59 -0400 From: "Steven R. Newcomb" To: tbray@textuality.com CC: w3c-sgml-wg@w3.org from Tim Bray on Tue, 08 Oct 1996 07:27:23 -0700) Subject: Re: A28: syntax of markup declarations? > At 06:20 PM 10/7/96 -0400, Steven R. Newcomb wrote: > I think I've said my piece on the main issues here. But there is > one Steve Newcomb's statements that I *must* challenge: > > As information management professionals, I > >can't see any way we can recommend XML to our clients unless their > >information assets -- and our clients themselves -- are already in the > >realm of SGML: > If this is true, we're wasting our time. The main benefit of XML is > that it might well expand - immensely - the number of people who are > using descriptive markup and standardized formats, instead of > presentational/mongrel markup and proprietary formats. I personally > plan to recommend XML, vociferously, to the Web publishing > population, when they hit the HTML brick wall, and the document > management profession, when they get tired of dealing with documents > that are opaque proprietary bit blobs. > XML is the easy on-ramp to SGML. > > If we go on preaching to the existing SGML choir, why bother doing this? Tim, I've read your remark 4 times, and I must admit I don't understand why you think it challenges or contradicts my concerns. I can only assume I did not make myself clear. I agree wholeheartedly with everything you say above, _provided_ that XML's syntax is definable as a proper subset of SGML's syntax. That's what I meant by "already in the realm of SGML." XML can't be "the easy on-ramp to SGML" if the ramp can't take you to SGML. I'm going to hazard a guess as to what you were thinking when you wrote the above. If you're saying that the use of 1. It makes no sense to claim that we have created a syntax suitable > for the markup of structured data in general and then not use it for > the markup of the structured document that defines a particular > schema. From a marketing standpoint this is indefensible; from a > logical standpoint it is absurd. I have heard no one deny this. Precisely to the contrary, I maintain for the reasons given above that it is good marketing to distinguish the steering wheel and other user interface doodads from the rest of the automobile and it is indefensible to do otherwise. I also fail to understand why, from a logical standpoint (whatever that means), it is absurd either. To me it makes plain good sense. Most people want to be able to tell the difference, at a single glance, between the spare tire and the steering wheel, even though there is no theoretical reason why cars can't run on four steering wheels, or why the driver can't mount the spare tire on the steering column and use it for a steering wheel. For the record, please let me add that this is familiar territory to me. The GCA CApH activity, which I chair, created a DTD for SGML DTDs. The work on that project was mainly done by Wayne Wohler. I learned during that time that there have been several projects to accomplish the same goal, which is to provide the ability to use the same information management tools and techniques on DTDs that can be used on SGML documents. It's a worthy goal, and I don't regret the exercise. I can imagine situations in which it would be appropriate, but it's no longer fundamentally interesting because there's now a better way to accomplish the same goal. The new way is to regard the DTD information (just as formally, if not more so) as the values of properties of components of document instances (see Clause 9 of DSSSL -- the SGML Property Set). This maintains the meta-ness of DTD constructs, keeping them in the context in which they have their intended effect, and it does not unnecessarily and confusingly elevate DTDs to the status of documents in themselves. Best regards, --Steve *************************************************************** * Steven R. Newcomb | President * * direct +1 716 389 0964 | TechnoTeacher, Inc. * * main +1 716 389 0961 | (courier: 3800 Monroe Avenue, * * fax +1 716 389 0960 | Pittsford, NY 14534-1330 USA) * * Internet: srn@techno.com | P.O. Box 23795 * * FTP: ftp.techno.com | Rochester, New York 14692-3795 * * WWW: http://www.techno.com | USA * *************************************************************** From: lee@sq.com Date: Mon, 7 Oct 96 23:34:10 EDT To: srn@techno.com, w3c-sgml-wg@w3.org Subject: Re: A28: syntax of markup declarations? > Hear, hear. If XML competes with SGML, it's clearly a disaster for > both of them. If SGML tools can be used to process XML, both are > winners. Or if there is easy conversion. > (1) should they later discover needs which XML cannot meet and which > SGML can, and then they may have to write a DTD and do a simple mechanical conversion. > (2) because it is so important to be in the mainstream of information > representation as defined by international standards: If XML is successful, it may well become an international standard. But note that HTML 2.0 is on the way to becoming an international standard; I am not sure that it has helped very much, although it was a great deal of work to get it tht far. If you can convert from XML to SGML automatically and 100% reliably, amenability to SGML tools will always be only a step away. In the mean-time, if we have a language that is significantly more widely implemented, and that has a much simpler and clearer specification, your clients will be happy they chose XML... I will comment on Tim & Michael's syntax separately -- I _do_ think it is a sensible approach. We are not far away from a single unified syntax, using only element-style markup. That is an admirable goal, and will be a great benefit. It will also reduce the change of confusing XML and SGML document instances :-) Lee Date: Tue, 08 Oct 1996 00:18:35 -0400 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: A28: syntax of markup declarations? At 09:45 AM 10/8/96 -0700, Tim Bray wrote: >The question of whether the XML spec should discuss both of these in a > normative way, and if not, which one, is open and interesting. Your postings > would lead me to suspect you want to discard validation in the SGML sense, > and consider a well-formed XML document valid. Which is a reasonable point > of view, but not mine. At the moment I think that both levels should in fact > be defined normatively in the XML spec. I think if we are serious about our design principles, especially the four "non-negotiable" ones, I should be able to answer my own question (for both DTD and RE) It wasn't a trick question, Tim, I just decided to check the principles again.: --- 3. XML shall be compatible with SGML. 1.Existing SGML tools will be able to read and write XML data. 2.XML instances are SGML documents as they are, without changes to the instance. 3.For any XML document, a DTD can be generated such that SGML will produce "the same parse" as would an XML processor. 4.XML should have essentially the same expressive power as SGML. Note: #1 and #2 describe our goal in its ideal form. If this goal is not achievable in its fullest form, then we may back out to a weaker form: it shall be simple to transform XML documents into equivalent SGML documents, and vice versa. Our intention, however, is to *bite the bullet* and ensure if we can that no transformation is needed to allow SGML tools to read and write XML document instances. #3 and #4 indicate our intentions accurately, but it is not yet clear how best to formalize and explain the phrase "the same parse", or the phrase "essentially the same expressive power". These remain open questions; see point 8 also. --- According to those principles, SGML-compatible DTDs cannot be optional. As much as I would like to make a language where DTDs are totally optional, I'll have to do it another day. If XML has its own markup declaration syntax, it must be in addition to an SGML-compatible DTD on every instance. I think that the first principle that could be construed to support a new syntax (or my preference, none at all) is #6, XML Documents should be human-legible and reasonably clear. But #6 isn't in the realm of "non-negotiable" and it is arguable how much more readable or clear DSD is. Can we agree that our principles say we should use SGML compatible DTDs? Can we agree that the principles should still prevail? Paul Prescod From: lee@sq.com Date: Tue, 8 Oct 96 02:07:02 EDT To: Charles@sgmlsource.com, U35395@UICVM.CC.UIC.EDU Subject: Re: A28: syntax of markup declarations? Cc: w3c-sgml-wg@w3.org [this message is much shorter than you think. The last 166 lines are a yacc grammar] Dr. Charles F. Goldfarb wrote: | XML should use a proper subset of the ISO 8879 declaration syntax, for several | reasons: He then gave a series of reasons... I have marked some of them as spurions, which is not a very good word. I do not mean that I do not think they are good points. I mean that I think they are not germane to XML. Charles' point 10 is the most important, though, and I have copied it at the start. if you read nothing else, read that! | 10. Our objective for XML is to increase the SGML market by making is | easier to understand and implement. We only get this result if XML *is* SGML; | otherwise, SGML doesn't change at all. | | If XML is a conforming profile of SGML, it can be | the core of SGML97 -- the basic conformance level. The rest of SGML would be | defined as a delta on the core SGML; core XML/SGML users would never have to | read it. (I have commented on this point in sequence below, but I have reproduced it here in case people missed it. I did not think that changing SGML itself was a possible goal of a W3C working group -- what did I miss? * | 1. The necessary subset is small, clean, and easily explained. This is spurious, since if you didn't use that grammar, you would not have to explain it at all. I xcertainly would not accuse the SGML DTD grammar of being clean -- you can't even put comments everywhere you can put white space. It is quirky, idiosyncratic, and hard to explain. Perhaps I am not a good teacher. It is true that with OMITTAG NO, you get rid of the "- -", and if there is no CDATA, we don't have the problem that CDATA means different things in different contexts. | I have attached the grammar to this note. It has fewer than 30 productions. | (SGML has almost 200.) Well, that's true. I have turned it into a YACC-style grammar that can be read by we mortal C-programmers :-), and appended that below. Actually I am rusty with yacc and have not tested it for reduce/reduce conflicts, but I think it's probably OK. I wouldn't bet my socks on it, though. I have done this so that people can judge more easily its value; the SGML-style form is harder to turn directly into a parser using normal compiler-writing tools. For my own part, though, I would far rather see something like Tim & Michael's syntax, and would far rather implement it, given a choice. | 2. 20,000 or so people already know the DTD language. | That is 20,000 more than know MGML. And considerably more than 980,000 fewer than know HTML. I.e. hardly anyone knows SGML by comparison. Since it is hoped that XML will be used by non-SGML users (no?), this is spurious. | 3. It is the semantics of markup declarations that presents learning | difficulties, not the syntax. The semantics will be the same in any case. No, the syntax hinders it -- but only slightly, I agree. There are a few gotchas -- e.g. you can't put comments in model groups, and it's impossible to remember whether it's #CDATA or #EMPTY or not, and when you need brackets, because it's so inconsistent. So it would be very nice to have something cleaner. | 4. The same is true for implementation. While a second syntax is a burden, | it is a relatively small and easily automated one. I am not sure what you mean by an easily automated implementation here. It can be done with lex and yacc in under a day, I think, unless there is something about it I have missed (not unlikely)! And it will take up maybe 2 or 3 of our 20 pages. I have simplified the grammar below, using "string" instead of system id, public id, entity value, etc., assuming that the parser will check the values at a higher level. | 5. SGML instance markup is a great language for representing structured | information. It is a poor language for defining it. Tim's paper.dsd is three | times the size (in lines) of the attached paper.dtd. If it is thirty times clearer and three times larger, is it ten times better? | 6. All SGML tools can handle markup declarations. All the ADA tools in the world can handle ADA declarations, but that is not a reason to use them. If we wanted to use only SGML tools, there would be no XML. The right question is whether XML files can be trivially, automatically and accurately converted to SGML files. | 7. There are no SGML interoperability issues because it *is* SGML. Modulo application conventions, CAPACITY, NAMELEN, RS/RE, etc. In fact, since HTML is SGML (see RFC1866) why don't we just use that? HTML 3.2 with the CLASS definition is very close to SGML, if you don't want to use "obscure" features like marked sections, entities, your own content models, processing instructions, notation, etc. There are people on this list (I think) who don't find HTML to be a very elegant language. But it is more widely deployed than SGML, and one reason is that SGML is too complex. | 8. There is no problem putting markup declarations in "XML masquerading as | HTML". Declarations just look like long unknown tags. | (HTML users may even find them familiar for that reason.) Er, have you tried this? You are in for some interesting surprises :-) | 9. XML needs to be a conforming subset of SGML; otherwise it will be seen | as a competitor to SGML whatever our good intentions to the contrary. I don't believe this. If it is a problem, call it SGML Lite. 0.5 :-) | 10. Our objective for XML is to increase the SGML market by making is | easier to understand and implement. We only get this result if XML *is* SGML; | otherwise, SGML doesn't change at all. | If XML is a conforming profile of SGML, it can be | the core of SGML97 -- the basic conformance level. The rest of SGML would be | defined as a delta on the core SGML; core XML/SGML users would never have to | read it. I think this is the first really strong point you've made here, but it is a good one. If these are your objectives, though, this task should be done by WG8 and *not* by the W3C. The W3C is not a standards body, and can only make recommendations to standards bodies such as the IETF or ISO. If you want to change SGML, this is not, and cannot be, the forum. I _do_ agree that your point number 10 is a good one. Lee DTD : declarations ; declarations : declaration declarations | /* empty (this allows an empty DTD) */ ; declaration : elemtype-decl | attlist-decl | entity-decl | notation-decl ; elemtype-decl : "" ; modeltype : "EMPTY" | "ANY | "(" expression ")" ; expression : "(" expression ")" numop_opt | term binop expression ; numop_opt : numopt | /* empty */ ; numop : "?" | "*" | "+" ; binop : "," | "|" | "&" ; term : "#PCDATA" | Gi ; attlist-decl : "" ; attdefs : attdef more_attdefs ; more_attdefs : attdef more_attdefs | empty ; attdef : gi attvalue attdefault ; /* note: gi is used for any name; the client may * to keep separate namespaces, uyt they are not * distniguished syntactically */ attvalue : "CDATA" | tokenised | enumlist | notations ; tokenised : "ID" | "IDREF" | "IDREFS" | "ENTITY" | "ENTITIES" | "NAME" | "NAMES" | "NMTOKEN" | "NMTOKENS" | "NUMBER" | "NUMBERS" | "NUTOKEN" | "NUTOKENS" ; /* do we really need all (any) of those for xml? * why not just have cdata plus a regular expression * to match an attribute value? Easier to implement, * simpler, more powerful and more general! */ enumlist : "(" token tokenlist ")" ; token : gi | whole_number ; /* is that right? */ tokenlist : token tokenlist | /* empty */ ; notations : "NOTATION" "(" gi morenots_opt ")" ; morenots_opt : "|" gi morenots_opt | /* empty */ ; attdefault : "#REQUIRED" | "#IMPLIED" | "#FIXED" string | string /* irregular: a default value has no keyword */ ; entity-decl : "" ; entity_spec : gi entity_value | "%" gi entity_value ; /* Note: this allows a parameter entity (%) to have NDATA; * what would that mean? */ entity_value : external_spec ndata_entity_opt | string ; external_spec : "PUBLIC" string string_opt | "SYSTEM" string ; string_opt : string | /* empty */ ; ndata_entity_opt: "NDATA" gi /* data attributes go here? */ | /* empty */ ; notation-decl : "" ; /* Notes: * I have assumed that the tokeniser eats unquotes whitespace, and that * comments are reduced to whitespace and then eaten. * Hence, both comments and whitespace separate tokens. * * A comment is only recognised between the of a declaration; * comments should really be allowed anywhere, but that would not be * compatible with SGML. * * The tokeniser should also recognise entity references and replace them. * The syntax is such that special care needs to be taken with * * because the whitespace is significant after the % sign -- if there * was no space, it would be an entity reference, This relies on the * SGML notion that % and & are only special when followed by a name * start character! * * I have not distinguished between a string, a formal public identifier, * a system identifier, a new-fangled-non-sgml(yet)-formal-system-id, * and an entity value. It would make the grammar easier to use if I did, * though. */ end. Lee Date: Tue, 08 Oct 1996 03:18:53 -0400 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: A28: syntax of markup declarations? At 02:07 AM 10/8/96 EDT, you wrote: >| 2. 20,000 or so people already know the DTD language. >| That is 20,000 more than know MGML. >And considerably more than 980,000 fewer than know HTML. >I.e. hardly anyone knows SGML by comparison. Since it is hoped that >XML will be used by non-SGML users (no?), this is spurious. I think that there is a major fallacy here. If XML *requires* markup declarations, then it is structural, and strict, and is for SGML-types. The HTML community will not really embrace it. There is ABSOLUTELY NO CHANCE that the "HTML crowd" is going to embrace markup declaration no matter how warm and fuzzy you make the syntax. If it *allows* markup declarations, then only "CS-types" (and a few odd-ball Artsy types =) ) will use that feature, and the SGML markup declaration language will be sufficient. There is no circumstance where XML becomes everybody's favourite language because it has a beautiful markup declaration syntax. Therefore, I think that we are wasting our time treating the issue as if it were on the scale of entities, style sheets, linking syntax and other make-it-or-break-it decisions. Someone's sure to respond sarcastically about RS/RE, but the issue of whether XML _looks_ and _feels_ like HTML or SGML is a major one, and RS/RE can make or break that. Paul Prescod From: Charles@sgmlsource.com (Charles F. Goldfarb) To: lee@sq.com Cc: U35395@UICVM.CC.UIC.EDU, w3c-sgml-wg@w3.org Subject: Re: A28: syntax of markup declarations? Date: Tue, 08 Oct 1996 08:09:17 GMT On Tue, 8 Oct 96 02:07:02 EDT, lee@sq.com wrote: >| If XML is a conforming profile of SGML, it can be >| the core of SGML97 -- the basic conformance level. The rest of SGML would be >| defined as a delta on the core SGML; core XML/SGML users would never have to >| read it. > >(I have commented on this point in sequence below, but I have reproduced >it here in case people missed it. I did not think that changing SGML itself >was a possible goal of a W3C working group -- what did I miss? It can't be an *official* goal of W3C, but if an idea is good for SGML, WG8 should consider it as a user requirement. I think a Core SGML conformance level is a good idea. I think a simplified presentation of 8879 in which the Core can stand by itself is a good idea. Therefore I am going to propose those things to WG8. Ideally, the Core SGML and XML would be identical. >| 1. The necessary subset is small, clean, and easily explained. >This is spurious, since if you didn't use that grammar, you would not have >to explain it at all. I xcertainly would not accuse the SGML DTD grammar >of being clean -- you can't even put comments everywhere you can put >white space. It is quirky, idiosyncratic, and hard to explain. > >Perhaps I am not a good teacher. It is true that with OMITTAG NO, >you get rid of the "- -", and if there is no CDATA, we don't have the >problem that CDATA means different things in different contexts. I was following Tim's MGML proposal. I assumed that *no* comments were allowed *within* declarations; only standalone comment declarations *between* markup declarations. That is easy to explain. >| I have attached the grammar to this note. It has fewer than 30 productions. >| (SGML has almost 200.) > >Well, that's true. I have turned it into a YACC-style grammar that can >be read by we mortal C-programmers :-), and appended that below. Actually >I am rusty with yacc and have not tested it for reduce/reduce conflicts, >but I think it's probably OK. I wouldn't bet my socks on it, though. > >I have done this so that people can judge more easily its value; the >SGML-style form is harder to turn directly into a parser using normal >compiler-writing tools. I would expect Core SGML to have a standard YACC definition that could be turned directly into a parser. >| 2. 20,000 or so people already know the DTD language. >| That is 20,000 more than know MGML. >And considerably more than 980,000 fewer than know HTML. >I.e. hardly anyone knows SGML by comparison. Since it is hoped that >XML will be used by non-SGML users (no?), this is spurious. We are talking about the DTD language. HTML users don't, by and large, know about DTDs. If they do, then they know the SGML DTD syntax because that's what HTML DTDs are written in. > >| 3. It is the semantics of markup declarations that presents learning >| difficulties, not the syntax. The semantics will be the same in any case. >No, the syntax hinders it -- but only slightly, I agree. >There are a few gotchas -- e.g. you can't put comments in model groups, >and it's impossible to remember whether it's #CDATA or #EMPTY or not, >and when you need brackets, because it's so inconsistent. So it would be >very nice to have something cleaner. My proposal removes these inconsistencies. > >| 8. There is no problem putting markup declarations in "XML masquerading as >| HTML". Declarations just look like long unknown tags. >| (HTML users may even find them familiar for that reason.) >Er, have you tried this? You are in for some interesting surprises :-) Yes, with Netscape 2. > >| 9. XML needs to be a conforming subset of SGML; otherwise it will be seen >| as a competitor to SGML whatever our good intentions to the contrary. > >I don't believe this. >If it is a problem, call it SGML Lite. 0.5 :-) To call it SGML Lite, it has to be an SGML subset. That is my point. >| 10. Our objective for XML is to increase the SGML market by making is >| easier to understand and implement. We only get this result if XML *is* SGML; >| otherwise, SGML doesn't change at all. > >| If XML is a conforming profile of SGML, it can be >| the core of SGML97 -- the basic conformance level. The rest of SGML would be >| defined as a delta on the core SGML; core XML/SGML users would never have to >| read it. > >I think this is the first really strong point you've made here, but it is >a good one. If these are your objectives, though, this task should be >done by WG8 and *not* by the W3C. The W3C is not a standards body, and >can only make recommendations to standards bodies such as the IETF or ISO. > >If you want to change SGML, this is not, and cannot be, the forum. >I _do_ agree that your point number 10 is a good one. If XML is a conforming subset of SGML, then *by definition* it does not require a change in the standard. If it is anything but a conforming subset, then *by definition* it competes with the standard. I have proposed an example of a conforming subset that is much simpler than the full DTD syntax. Your BNF has added back complex things that I took out (like comments within declarations). Your notes address design questions as to what to include in the subset, which is a separate issue. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- To: w3c-sgml-wg@w3.org From: Martin Bryan Subject: Re: A28: syntax of markup declarations? (LONG) Date: Tue, 8 Oct 1996 10:14:15 +0100 >Once again, I find the idea attractive, but I have doubts. DTDs are "on the >line" of documents that should be SGML-encoded. There are standardization >benefits if they are, but efficiency benefits if they are not. I like the >IETF philosophy: "when in doubt, leave it out." I fear that if we veer from >that, we'll make a mistake that will cost us in the long run. To my way of thinking DTDs are programs which tell other programs how they could be customized to suit a particular application. They are not data, though they may need to be accompanied by explanatory data in the form of user manuals. Such manuals have their own structure, which is entirely independent of the DTD. ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ Date: Tue, 08 Oct 1996 07:27:54 -0500 From: Len Bullard To: lee@sq.com CC: srn@techno.com, w3c-sgml-wg@w3.org Subject: Re: A28: syntax of markup declarations? lee@sq.com wrote: > > If XML is successful, it may well become an international standard. > But note that HTML 2.0 is on the way to becoming an international standard; > I am not sure that it has helped very much, although it was a great deal > of work to get it tht far. It helps. It takes time. > If you can convert from XML to SGML automatically and 100% reliably, > amenability to SGML tools will always be only a step away. All conversions are *a step away*. It creates a separate market. It creates separate and additional expense. It does not provide anything that S-expressions or even BNF do not provide. It is yet another markup system. Why? > In the mean-time, if we have a language that is significantly more widely > implemented, and that has a much simpler and clearer specification, your > clients will be happy they chose XML... No they won't. My clients have SGML. They will ask "what is the advantage?" I reply meekly "well, it can be implemented in a week by a CS undergrad." They will say, "The work habits of the average CS grad are insignificant. I already pay them well and I have not problem with it taking two or three weeks if I can get a system improved, but not too far from my present course which has been chosen at some risk and some expense. I already have SGML tools, SGML parsers, SGML expertise, and significant amounts of SGML data. All I asked for was to get that on the Web *without conversion*. You gave me a new system. Next." > I will comment on Tim & Michael's syntax separately -- I _do_ think it > is a sensible approach. We are not far away from a single unified syntax, > using only element-style markup. That is an admirable goal, and will be > a great benefit. It will also reduce the change of confusing XML and SGML > document instances :-) I understand the factored appeal of what is being suggested. I think, like Dr. Newcomb, it is misguided. Someone please list the advantages. I see none. Clearly distinguishing the schema from the data is a feature of SGML. I want SGML to be easier; not to replaced. Len Bullard Lockheed Martin Date: Tue, 08 Oct 1996 14:14:36 +0000 To: W3C SGML Working Group From: James Clark Subject: Re: A28: syntax of markup declarations? (LONG) I'm strongly against Tim's proposal. >2.1 Quality of Specification > >This is close to my heart as co-editor of the XML spec. If we use DTD >syntax, the spec will have to: (a) specify the DTD syntax, (b) specify how >the declarations in the DTD syntax impose constraints on instances. If >we use instance syntax, we lose almost all of (a) You don't lose it. You still have to have to include the DSD for DSDs. I don't think that's going to be any shorter that a BNF gramar for the SGML DTD syntax. All potential implementors will be familiar with BNF. Almost none will be familiar with DSDs. An implementor coming fresh to the spec is going to have a hard time understanding DSDs if the only specification of what is a legal DSD is itself in the form of a DSD. I think the only real gain in the spec is that the prolog doesn't have any additional lexical structure that has to be specifeid. Overall I would say that the best spec would be one that gave a clean BNF for a simplified subset of SGML DTDs. >2.2 Integrity > >I for one, once we get XML done, plan to start browbeating the document >management and authoring practitioners of the world along the lines of "now >that XML exists, you have NO EXCUSE for not using descriptive markup in your >structured documents!" If I convince them, and they then start using XML, >and the first structured document they encounter is a DTD, with its rather >ad-hoc syntax, then I've lied to them. I find this completely unacceptable. I think this is totally bogus. A DTD is structured information. That doesn't make it a document, unless every set of structured information constitutes a document. It is no part of the philosophy of SGML to insist that SGML notation be used to represent every kind of structured information. Do you feel the urge to convert C programmers over from writing: int main(int argc, char **argv) { printf("Hello world\n"); return 0; } to Hello world&newline; ? I hope not. Why do you feel the urge to impose such a change for DTDs? >2.3 Ease of Implementation > >If there is only one syntax for declaration and instance, then the >prospective XML processor author only needs one lexer and one parser. >Granted, the DTD language is not the hardest in the world, but two >lexer/parsers are always harder than one to build. I don't believe this is true. With the DSD syntax an implementation has to construct its internal representation of DTDs from its internal representation of elements and attributes. This would probably be more work than writing the YACC actions that construct the representation of DTDs using the existing SGML syntax. I don't think YACC is really a good tool for parsing instances, but I think that it could work quite well for DTDs. I would concede that the lexer gets a little more complicated using the existing SGML syntax, so overall I would say that there's not much difference in implementation difficulty. >2.4 Familiarity > >If we can go to the HTML community and tell them "not only can you now add >your own structures to documents and still deliver them on the web, but >here's how you do it, and it's just another bunch of tags", this removes >another significant barrier to resistence. I cannot in good conscience >defend to these people the proposition that to do this something >straightforward, obvious, and good, they have to learn another language. Why is it so much easier to learn a bunch of new tags/attributes than it is to learn a few more declarations? The most complex part of DTDs is content models. I think people will have a much easier time understanding the current SGML syntax because of its similarity to regular expressions. The DSD syntax for content-models is an order of magnitude less readable than the existing DTD syntax. >3. SGML Interoperability Issues >(b) I propose hiding the XML markup declarations from SGML processors inside > a processing instruction as follows: . > Similarly, I propose [note: either we change PIC, or we live with a lot of > in markup > declarations, or we figure out a better way to hide XML markup > declarations] The fact that the DSD proposal is going to require some sort of gigantic kluge like this makes it a non-starter for me. I just don't see how people can think that a proposal that involves this sort of thing is an improvement over the existing SGML syntax. >This is the RIGHT THING TO DO. The SGML syntax for DTDs is not going to go away. Anything that ends up forcing SGML users and implementors to deal with two totally incompatible syntaxes for exactly the same semantics is not my idea of the RIGHT THING TO DO. James Date: Tue, 8 Oct 1996 10:25:37 -0700 (PDT) From: Bill Smith Subject: Re: A4: XML's use of SHORTTAG? To: w3c-sgml-wg@w3.org This is a resend of aposting that went into the void. ------------- Begin Forwarded Message ------------- To: w3c-sgml-wg@w3.org Subject: Re: A4: XML's use of SHORTTAG? Cc: Message-Id: > > The final point, on omitted attribute-value specifications, raises the > general question of how XML systems will behave when no DTD, or a > partial DTD, is provided -- if such omitted or partial DTDs are allowed. > It also raises the question of providing a way for a document to signal > that its DTD can be skipped without loss of information (e.g. because it > has no default attribute values, or no empty elements, etc.). These > questions are to be discussed and decided separately. > I would argue that we not allow default attributes, etc. in the first version of XML. Allowing default attribute values demands that we require a DTD or some DTD subset for all documents that make use of this shorthand notation. Unfortunately, this places a burden on all XML client developers of parsing and interpreting DTDs - correctly. A server-side mechanism can be employed to transform SGML document instances that employ default attribute values to an XML equivalent. Subsequent versions of XML could allow default attribute values, etc. ------------- End Forwarded Message ------------- Date: Tue, 8 Oct 1996 09:43:25 -0700 (PDT) From: Bill Smith Subject: Re: A17: keep or drop entities? To: w3c-sgml-wg@w3.org James Clark wrote: > > Some SGML parsers already allow the document entity to be spread out of > multiple files; for example, with nsgmls, you could say > > nsgmls header.sgm Intro.sgm Docs.sgm Elements.sgm Entities.sgm > > and the effective document entity would be the concatenation of all files > specified as arguments. If your XML parser doesn't do this, you can just > use "cat". Whether it's something simple like file concatenation or > something complex like a OO database, I think ways can be found to solve > this sort of problem outside of XML. > Excactly. Below is the text of a message I posted while the list server was down. __________________ I think the first version of XML should not support entities. Successive versions should. For those using SGML and entities, entity resolution and inclusion can be performed on the server side. XML clients would expect "complete" documents just as today's HTML clients expect complete documents. This has proven an acceptable model for many documents and applications. Entity substitution on the client side is not in great demand but extensible markup is. Let's first include the features required to accomplish the leap from one-size-fits-all markup to extensible markup. Once that is accomplished, we can add other useful features that we all know, love, and use. One final comment on this. If we require general entity support initially, it will be necessary to specify, design, develop, implement, and distribute a Web-based entity resolution mechanism. Agreement on such a mechanism has proven difficult in the past and may again. Our energies are better spent on other issues. Date: Tue, 08 Oct 1996 10:40:56 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: A17: keep or drop entities? My feelings on this have been going back and forth like a ping-pong ball. If we take the actual constellation of *usage* of SDATA & CDATA internal entities, I don't see why we need them in XML. Charles says that the way they are used is not exactly congruent with the way they were designed, but nonetheless I don't see that the value they add makes up for the extra complexity in specification. External NDATA entities, it seems, cost very little. James points out correctly that what SGML does with NOTATIONS, the web does with mime-types - still, one of the grievous shortcomings of the web is that the browser can't tell what kind of an object it is until it follows the link. Having a typed external reference seems like a plus, and almost no extra work for the parser. Obviously, HTML is not going to abandon , but that's an application semantic, not an XML-level issue. Or, you could say that "the problem that NDATA is trying to solve is really one of linkage semantics, thus let's defer it". Internal text entities would be *awfully* hard to live without. I think these would be a big selling point for people who are hitting some of the HTML walls, in particular for big complicated forms, table markup, and the like. They complicate the parser - but not nearly as much as external text entities. And they're not really outside the scope of a perl script [do a 2-script pipeline, one of which stupidly switches in local entities, the second perls away in blissful ignorance]. External text entities are much more expensive to parser writers, but gosh, they're useful. It's really good to have a facility whereby you can tell management - "a compliant XML processor will never display this paragraph without this embedded cover-management's-ass warning that's replicated 389 times throughout the document". You can't do that with cat. But if we try to support external text entities in a general way, we can kiss the perl-script implementations goodbye. could we distinguish between an XML processor and an XML external entity manager? could we distinguish between client-side and server-side XML? could we think of some middle road along the lines of transclude-but-not-necessarily-validate? I'm going to have trouble voting on this. And it's *way* more important than the syntax of markup declarations. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Tue, 08 Oct 1996 14:15:27 -0400 To: Michael Sperberg-McQueen From: masonjd@ornl.gov (James David Mason) Subject: Re: A28: syntax of markup declarations? Cc: W3C SGML Working Group >A.28 Should XML use the markup-declaration syntax described by ISO 8879 >clauses 10-11, or should XML define a specialized document type and let >its markup declarations use the document-instance syntax, as proposed >by MGML? > Breaking my general inclination to keep quiet in this discussion, I weigh in on the side of those who believe the the XML declaration syntax, if there is to be such, should be derived from the SGML syntax. I've spent the past 15 years watching people try to improve on SGML syntax. I've occasionally joined the effort. I still hold that there are some things we could leave out and most users wouldn't miss them. But in spite of the fact that someone might be able to come up with a more "computer sciencely" way of doing what SGML does, we still have a sizeable base of (1) people who actually know how to use the beast and (2) tools that have been written in support of that effort. I just can't see dumping that to prove a point, whether that there's a more elegant way or a way to do it in instance syntax or whatever. I keep coming back to the nature of SGML: it's a metalanguage. The things we loosely call "SGML documents", our instances, aren't what I'd call roperly SGML documents. They're documents in languages we've created using SGML to define grammars. SGML isn't by any means perfect, easy, etc., but it is at least a workable language for defining the grammars of instances. To go over to instance grammars and then try to use them to define subinstance grammars is a case of using inappropriate tools. I just can't buy that. I second Steve Newcomb's assertion that trying to devise a new metalanguage -- because that's what it would be -- would hurt it, HTML, SGML, and all of us. There are enough SGML users out there for us to make a living supporting them. But there aren't enough of us to support a lot of "standards" projects, and we're already in danger of spreading ourselves too thin. Let's concentrate on making a reasonable, usable, selection of things to adopt >from SGML, then, if we find we really have to change something, let's do it in a way that can feed back into SGML 97. Jim Dr. James D. Mason (ISO/IEC JTC1/SC18/WG8 Convenor) Lockheed Martin Energy Systems Information Management Services SGML Systems Development 1060 Commerce Park, M.S. 6480 Oak Ridge, TN 37831-6480 U.S.A. Telephone: +1 423 574-6973 Facsimile: +1 423 574-0004 Network: masonjd@ornl.gov Date: Tue, 08 Oct 1996 14:29:59 -0400 To: Tim Bray From: masonjd@ornl.gov (James David Mason) Subject: Re: A28: syntax of markup declarations? Cc: w3c-sgml-wg@w3.org At 07:27 AM 10/8/96 -0700, Tim Bray wrote: >If we go on preaching to the existing SGML choir, why bother doing this? > Some of us in the choir (I sing Bass II) might like life made easier for us, too. Jim Mason Date: Tue, 8 Oct 1996 13:09:33 -0400 From: "Steven R. Newcomb" To: w3c-sgml-wg@w3.org James Clark on Tue, 08 Oct 1996 14:14:36 +0000) Subject: Re: A28: syntax of markup declarations? (LONG) [James Clark:] > Overall I would say that the best spec > would be one that gave a clean BNF for a simplified subset of SGML DTDs. Bravo. This is the shortest and clearest expression of the advice I've been trying to give, and that I would have given if I had had the necessary wit. Best regards, --Steve *************************************************************** * Steven R. Newcomb | President * * direct +1 716 389 0964 | TechnoTeacher, Inc. * * main +1 716 389 0961 | (courier: 3800 Monroe Avenue, * * fax +1 716 389 0960 | Pittsford, NY 14534-1330 USA) * * Internet: srn@techno.com | P.O. Box 23795 * * FTP: ftp.techno.com | Rochester, New York 14692-3795 * * WWW: http://www.techno.com | USA * *************************************************************** Date: Tue, 8 Oct 1996 10:32:46 -0700 (PDT) From: Bill Smith Subject: Re: A28: syntax of markup declarations? To: w3c-sgml-wg@w3.org Tim Bray wrote: > If this is true, we're wasting our time. The main benefit of XML is that it > might well expand - immensely - the number of people who are using descriptive > markup and standardized formats, instead of presentational/mongrel markup and > proprietary formats. I personally plan to recommend XML, vociferously, to the > Web publishing population, when they hit the HTML brick wall, and the document > management profession, when they get tired of dealing with documents that are > opaque proprietary bit blobs. > > XML is the easy on-ramp to SGML. > > If we go on preaching to the existing SGML choir, why bother doing this? > Tim has stated the real issue quite plainly here. I'll go a bit further on this topic though and suggest that XML doesn't need any markup declarations - in an intitial version. We have stated that a desirable goal for XML is that browsers be able to properly display documents without the need for markup declarations. We can assure achieving that goal by eliminating markup declarations from XML 1.0. If we allow markup declarations, there will be a great temptation to use them and we will not achieve the goal of DTD-less processing. Markup declarations should become a part of XML after initial accepatance in order to support validation, etc. Date: Tue, 8 Oct 1996 12:39:33 -0700 From: lauren@sqwest.bc.ca (Lauren Wood) To: w3c-sgml-wg@w3.org Subject: Re: A17: keep or drop entities? % From: Bill Smith % % I think the first version of XML should not support entities. Successive % versions should. % % For those using SGML and entities, entity resolution and inclusion can be % performed on the server side. XML clients would expect "complete" documents just % as today's HTML clients expect complete documents. This has proven an acceptable % model for many documents and applications. Entity substitution on the client % side is not in great demand but extensible markup is. % % Let's first include the features required to accomplish the leap from % one-size-fits-all markup to extensible markup. Once that is accomplished, we can % add other useful features that we all know, love, and use. I disagree. I think even the first version of XML should support entities, including external text entities. If we want people to support XML (other than the we-already-use-SGML crowd), we need to give them something that doesn't exist in HTML, and entities are an important part of this. Any given user agent, whether for HTML or XML, must know whether the object/file that is referenced in a document is to be shown inline, or not shown. In the case of HTML, that is accomplished by having defined names for elements. Thus an HTML UA knows that the SRC attribute on IMG points to a file whose contents are to be shown inline, but the HREF attribute on A points to a file whose contents are not to be shown. Instead, the contents of the A element (which typically don't contain a file name or object reference of any sort) are to be highlighted in some way. How is your typical XML UA to know what to do? The information will have to be passed in some way. The standard &chap1; or ← tells an SGML UA to show the contents of the entity. How would the URL method cope with only inserting one paragraph out of a list of paragraphs (or a database, for that matter)? We would have to come up with an extension to the URL syntax to do so. One of the beauties of using external entities is to have one document that contains all the boilerplate texts I need for all my documents, and just change the name of the entity in the document shown to get the appropriate boilerplate text showing up. This to me is a reason for people to use XML. Some of the other reasons are rather nebulous to the HTML users. And they are a large proportion of our potential XML users. If we don't give these users enough first time around, they're not going to bother waiting for the second attempt. Lauren -- Lauren Wood, SoftQuad, Inc Date: Tue, 8 Oct 1996 13:22:23 -0700 (PDT) From: Bill Smith Subject: Re: A17: keep or drop entities? To: w3c-sgml-wg@w3.org Lauren Wood wrote: > I disagree. I think even the first version of XML should support entities, > including external text entities. If we want people to support XML (other than > the we-already-use-SGML crowd), we need to give them something that doesn't > exist in HTML, and entities are an important part of this. XML gives structure and tag extensibility. From my vantage point, this is such a huge increment over HTML that everything else is noise. External text entities add complexity where it isn't required. > > Any given user agent, whether for HTML or XML, must know whether the > object/file that is referenced in a document is to be shown inline, or not > shown. In the case of HTML, that is accomplished by having defined names for > elements. Thus an HTML UA knows that the SRC attribute on IMG points to a file > whose contents are to be shown inline, but the HREF attribute on A points to a > file whose contents are not to be shown. Instead, the contents of the A element > (which typically don't contain a file name or object reference of any sort) are > to be highlighted in some way. How is your typical XML UA to know what to do? > The information will have to be passed in some way. The standard &chap1; or > ← tells an SGML UA to show the contents of the entity. > Stylesheets tell UAs how to interpret (display) content. They might also be used to construct complete UIs given structure and tag extensibility. > How would the URL method cope with only inserting one paragraph out of a list > of paragraphs (or a database, for that matter)? We would have to come up with > an extension to the URL syntax to do so. One of the beauties of using external > entities is to have one document that contains all the boilerplate texts I need > for all my documents, and just change the name of the entity in the document > shown to get the appropriate boilerplate text showing up. This to me is a > reason for people to use XML. Some of the other reasons are rather nebulous to > the HTML users. And they are a large proportion of our potential XML users. If > we don't give these users enough first time around, they're not going to bother > waiting for the second attempt. I don't disagree with this other than on complexity and therefor timing. Including external text entities in XML 1.0 could slow down development of the standard and will make XML UA development more difficult (translates into time). ime). We are dealing with Web time not ISO time. From: lee@sq.com Date: Tue, 8 Oct 96 11:11:48 EDT To: Charles@sgmlsource.com, lee@sq.com Subject: Re: A28: syntax of markup declarations? Cc: U35395@UICVM.CC.UIC.EDU, w3c-sgml-wg@w3.org Just a quick note: > I have proposed an example of a conforming subset that is much simpler > than the full DTD syntax. Your BNF has added back complex things that > I took out (like comments within declarations). Actually my BNF does not include comments at all. I noted that I would expect the tokeniser to deal with them (as in most programming languages), which is much simpler than they way they are defined in 8879, where the grammar actually includes the whitespace. I assumed that your grammar was treating them the same way. I do not believe that I added anything that wasn't in your grammar, but if I did, I'll gladly correct it, if it is of use to anyone. I made it by editing your grammar to make it regular in the sense of parser-generating tools -- e.g. no use of + ? or *, but pure BNF. Which is easier to read depends on what you're used to, of course -- your form (like that of ISO8879) is much terser, but is not as widespread in computer science. The lack of comments wouldn't matter too much to me -- although I would simply say it was a badly designed language and shiver and say ugh :-) -- because I could define entities for my comments, I have done that experimentally in the past for element descriptions, so the parser/application can access them. Lee From: lee@sq.com Date: Tue, 8 Oct 96 19:52:53 EDT To: w3c-sgml-wg@w3.org, bill.smith@Eng Subject: Re: A17: keep or drop entities? > XML gives structure and tag extensibility. HTML 3.2 has the CLASS attribute, so that instead of inventing a PERSON element type, you can do to accomplish the same thing. You can't add your own attributes and stay within the DTD (although HTML 2 did allow youto do that). There is some structure with the DIV element. > Stylesheets tell UAs how to interpret (display) content. The CSS1 draft isn't the world's best thought-out style sheet language, but it is a style sheet language. Microsoft IE 3.0 implements some of it. Netscape Navigator 4.0 will implement some or all of it. If XML is no more powerful than HTML, and does not offer better presentation, why use it? Because it is simpler than something else also not being widely used? I think XML has to be simple enough to be implemented easily, but no simpler. A 20 page spec in computer-science terminology and precision is a good goal for a way of representing structured objects in a networked environment. (are there any IDL & CORBA people on this list?) Note that the HTML tag allows for the WWW equivalent of NDATA entities -- you can include arbitrary objects. It's slightly more flexible, in that you don't have to specify the notation, but can if you want to, and of course keeps notation (content encoding) separate >from data format (content type). On the other hand, as far as I know, the people implementing OBJECT do not let allow transclusion/embedding of HTML, but only non-HTML data types. So in some areas, HTML is catching up fast. If the goal is to address only those people currently using SGML, it'd be better to specify ways of converting SGML to HTML with CSS1 style sheets -- e.g. using Jade or other DSSSL implementations when such things happen. So clearly the goal must be to address people who are _not_ currently using SGML, but who are or are about to be publishing on the Internet. At any rate, it is unise, I think, to assume that structure, extensibility and stylesheets are the altogether province of SGML and not at all of HTML. Lee -- Liam Quin, SoftQuad Inc. | lq-text freely available Unix text retrieval lee@sq.com +1 416 544-9000 | FAQs: Metafont fonts, OPEN LOOK UI, OpenWindows http://www.softquad.com/ | xfonttool (Unix xfontsel in XView) SGML - HTML - Other Things | The barefoot programmer From: "Peter Sharpe" Date: Tue, 8 Oct 1996 17:42:45 -0700 "Re: A7: CDATA, RCDATA, TEMP marked sections?" (Oct 4, 5:36pm) To: Paul Prescod Subject: Re: A7: CDATA, RCDATA, TEMP marked sections? Cc: w3c-sgml-wg@w3.org On Oct 4, 5:36pm, Paul Prescod wrote: > >A.7 Should XML have CDATA, RCDATA, and TEMP marked sections or not? > > It would be really handy to have some mechanism, to allow arbitrary non-SGML > data (in the same character encoding). > I agree. Javascript, VBScript, Denali, invariably contains what an SGML parser would believe to be markup. There are several requirements for the mechanism by which the markup is escaped: 1. It has to be simple and intuitive. I strongly believe that CDATA marked sections violate this requirement. HTML authors are used to having both syntax and symantics for their markup. If they use SCRIPT, I believe they would naturally expect the "parser" to understand that it should ignore everything until it sees "". To have to add additional markup would neither be intuitive nor welcomed. 2. It has to be easily parseable by a simple lexer on the server side. There is no reason to expect that the server-side application will be using an XML parser. While it is not rocket science to detect the marked section start and end, it certainly complicates things. I do not believe that there is an acceptable solution to these requirements using SGML. The choices are very few: CDATA elements, CDATA marked sections and "structured comments". CDATA elements fail to hide markup which looks like end-tags. CDATA marked sections are too much of a burden. And "structured comments"...well, that's the worst kind of hack, in my opinion. I do believe that there is a fairly simple solution that would cover almost all cases, and the cases it doesn't cover would be obvious to the author: Proposal: The only markup which terminates the content of a CDATA element is an end-tag that matches the element's start-tag. For example, the only markup that would end a SCRIPT element would be "". In the case where there is no DTD, there either would be no possibility of CDATA elements or else there would be some alternate way to indicate the content type. And, finally, I would propose that some term other than CDATA be used to describe these elements. Peter PS These comments relate to A10 as well. And there are implications for A9. -- Peter Sharpe, Chief Scientist, SoftQuad Inc. Tel: +1 604 585 1999 ext. 312 #108-10070 King George Highway, Surrey, B.C., CANADA V3T 2W4 Fax: 585 1926 Internet: peter@sq.com or peter@sqwest.bc.ca World Wide Web: www.sq.com From: Paul Prescod Subject: Re: A7: CDATA, RCDATA, TEMP marked sections? To: peter@sqwest.bc.ca (Peter Sharpe) Date: Tue, 8 Oct 1996 21:13:21 -0400 (EDT) Cc: w3c-sgml-wg@w3.org > On Oct 4, 5:36pm, Paul Prescod wrote: > > >A.7 Should XML have CDATA, RCDATA, and TEMP marked sections or not? > > > > It would be really handy to have some mechanism, to allow arbitrary non-SGML > > data (in the same character encoding). > > > There are several requirements for the mechanism by which the markup is > escaped: > 1. It has to be simple and intuitive. > I strongly believe that CDATA marked sections violate this requirement. I have to disagree. The hardwired CDATA MS syntax somebody (Michael?) proposed a few days ago is not that non-intuitive. I originally had an idea like yours, but have come to embrace that convention. > HTML authors are used to having both syntax and symantics for their > markup. If they use SCRIPT, I believe they would naturally expect the > "parser" to understand that it should ignore everything until it sees > "". To have to add additional markup would neither be intuitive > nor welcomed. No, but they are already going to have to learn to "shape up" to move into the XML world. If you look at it in a certain way, you might consider it _easier_ for both the author and the parser to have a SINGLE syntax for turning on and off CDATA content instead of a potentially infinite list (". > > In the case where there is no DTD, there either would be no possibility of > CDATA elements or else there would be some alternate way to indicate the > content type. I think that our design should presume the non-availability of a DTD as the "norm". So we should specify that "alternate way", and I suspect it will turn out to be hard-coded CDATA marked sections. I had the original non-SGML compatible proposal in this area (and it was similar to yours). I now feel that we should settle for a reasonable, workable SGML-compatible compromise: CDATA marked sections. Perhaps for SGML 97 we can get soething more flexible (i.e. something that would allow us to embed ]]). Paul Prescod From: Charles@sgmlsource.com (Charles F. Goldfarb) To: "Peter Sharpe" Cc: Paul Prescod , w3c-sgml-wg@w3.org Subject: Re: A7: CDATA, RCDATA, TEMP marked sections? Date: Wed, 09 Oct 1996 01:44:35 GMT On Tue, 8 Oct 1996 17:42:45 -0700, "Peter Sharpe" wrote: >There are several requirements for the mechanism by which the markup is >escaped: >1. It has to be simple and intuitive. >2. It has to be easily parseable by a simple lexer on the server side. >In the case where there is no DTD, there either would be no possibility of >CDATA elements or else there would be some alternate way to indicate the >content type. > >And, finally, I would propose that some term other than CDATA be used to >describe these elements. Here's how I would tell users to address these requirements using DTD-less XML: Chapter 5. The CLEARDATA Tag. When you need to put scripts or other data in your document that isn't SGML, you mark it with special tags called CLEARDATA tags (CDATA for short). These tags look different from normal tags because they are warning the browser that the data is clear -- that is, it doesn't have any SGML markup in it. The browser needs to be warned because if it sees, within CDATA, something that looks like markup, it has to know it isn't really markup, it's just data. The CLEARDATA start-tag is: -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- From: "Peter Sharpe" Date: Tue, 8 Oct 1996 19:15:00 -0700 "Re: A7: CDATA, RCDATA, TEMP marked sections?" (Oct 9, 1:44am) <9610081742.ZM2148@west.sq.com> <3263feef.22271083@mail.alink.net> To: Charles@sgmlsource.com Subject: Re: A7: CDATA, RCDATA, TEMP marked sections? Cc: Paul Prescod , w3c-sgml-wg@w3.org On Oct 9, 1:44am, Charles F. Goldfarb wrote: > Here's how I would tell users to address these requirements using DTD-less > XML: > > Chapter 5. The CLEARDATA Tag. > > When you need to put scripts or other data in your document that isn't SGML, > you mark it with special tags called CLEARDATA tags (CDATA for short). > > The CLEARDATA start-tag is: The CLEARDATA end-tag is: ]]> > An elegant solution. However, it doesn't solve the correct problem. I'm not so much talking about the case where you just want to escape some characters, but where you also want to label those characters. No semantic information other than "this is clear text" can be attached to the CLEARDATA start-tag (sic). So you need additional markup. This means that you are asking the HTML author, for example, to use markup like My point is that that kind of markup will be non-intuitive and considered completely unnecessary by the author. I might also note that Appendix B of the standard (B.13.1.1, the paragraph starting at 29) says that CDATA content is only ended by an end-tag which matches the start-tag (or ancestor!?). So the SGML committee must have considered that a good idea at one time. (Or the author of the appendix had an intuitive, but incorrect, understanding of CDATA declared content.) If there was a good reason why the committee changed its mind about that, I'd be very interested to hear it. Peter -- Peter Sharpe, Chief Scientist, SoftQuad Inc. Tel: +1 604 585 1999 ext. 312 #108-10070 King George Highway, Surrey, B.C., CANADA V3T 2W4 Fax: 585 1926 Internet: peter@sq.com or peter@sqwest.bc.ca World Wide Web: www.sq.com From: Paul Prescod Subject: Re: A7: CDATA, RCDATA, TEMP marked sections? To: peter@sqwest.bc.ca (Peter Sharpe) Date: Tue, 8 Oct 1996 22:44:15 -0400 (EDT) Cc: Charles@sgmlsource.com, w3c-sgml-wg@w3.org > However, it doesn't solve the correct problem. I'm not so much talking about > the case where you just want to escape some characters, but where you also > want to label those characters. No semantic information other than "this > is clear text" can be attached to the CLEARDATA start-tag (sic). So you need > additional markup. This means that you are asking the HTML author, for > example, to use markup like > > Compared to what they use now, it is a model of simplicity, elegance and clarity. I don't see why anyone should think it is redudant at all. You can put a script directly in > >My point is that that kind of markup will be non-intuitive and considered >completely unnecessary by the author. > I agree. A tool could hack the excess markup into place, if and when XML would move it back into SGML. >I might also note that Appendix B of the standard (B.13.1.1, the paragraph >starting at 29) says that CDATA content is only ended by an end-tag which >matches the start-tag (or ancestor!?). So the SGML committee must have >considered that a good idea at one time. (Or the author of the appendix had an >intuitive, but incorrect, understanding of CDATA declared content.) If there >was a good reason why the committee changed its mind about that, I'd be >very interested to hear it. > Annex B does not form an integral part of 8879. The relevant specification of that unfortunate behavior is found in 7.6, lines 6-11. "The content of an element declared as character data or replaceable character data is terminated only by an etago delimiter-in-context (which need not open a valid end-tag) or a valid net. Such termination is an error if it would have been an error had the content been mixed content. Seems to me that the termination on first occurring etago, then checking to see if it is a proper termination else error is less desirable than to require the proper matching endtag. Regards. Harvey Bingham Date: Wed, 09 Oct 1996 12:12:22 -0400 To: w3c-sgml-wg@w3.org From: "Steven J. DeRose" Subject: Re: RS/RE: basic questions At 10:44 PM 10/02/96 EDT, lee@sq.com wrote: >There's no point in practice in being compatible with 8879 -- instead, XML >has to be compatible with actual tools. Probably the most widespread >tools that read SGML are HoTMetaL, Panorama, Adept, and A/E, with NSGMLS >and SGMLS and Omnimark being the most widespread on the `next layer down' >(conceptually, I don't mean to deprecate them!). I don't list DynaText >because the viewer is the widespread part, and it reads a compiled form. Sorry to have to point this out, but this claim is no longer accurate. The shipping version of DynaText does read uncompiled SGML and view it just fine. Like most other tools, we don't do SHORTREF at this time, so that overall point still holds. Steve Date: Wed, 9 Oct 1996 09:20:47 -0700 (PDT) From: Bill Smith Subject: Re: A11: element content and mixed content? To: w3c-sgml-wg@w3.org David Durand wrote: > If we drop the distinction, we must treat spaces and tabs as data elements > in element content (like tables) because without a DTD we can't tell > whether or not it is element content, and therefore "insignificant". Of > course, I'd say that we should drop it, and accept the restrictions on > whitespace, but the current leaning reflects the "code-formatting" needs > you and others have advocated. I don't think David is suggesting requiring a DTD. I'm firmly opposed to requiring a DTD, even a short form, to "properly" handle XML. If we cross the DTD/DTDless line, I think we will significantly reduce the liklihood that XML will be adopted. We should drop the distinction. Date: Wed, 9 Oct 1996 13:00:06 -0400 To: w3c-sgml-wg@w3.org From: "David G. Durand" (David G. Durand) Subject: Re: A17: keep or drop entities? At 12:18 PM 10/9/96, James Clark wrote: >At 10:48 09/10/96 -0400, Gavin Nicol wrote: >>>There is the problem, that I discovered following the pointer that you >>>provided me, that not all of the character entities have an equivalent >>>in Unicode. I've only skimmed over this issue so far, so I don't have >>>complete information, but I think it is still an issue. >> >>We need SDATA internal entities for precisely the reason given >>here. > >Why can't we use the private use zone in 10646 for this? As a user (or developer) I would prefer that characters not in the reference character set be identified by mnemonic strings rather than by numbers. If the character is not on my local system, at least I have a name that someone thought was indicative of the character in question. With a private-use character code I know nothing other than that I don't know how to render it. I might even have a local convention for private-use that conflicts with the one in the document. Private use should be private, and XML must be useful for interchange as well. I think of SDATA as the "alt" attribute of character sets. -- David _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ http://www.dynamicdiagrams.com/services_map_main.html Date: Wed, 09 Oct 1996 09:46:32 -0900 To: W3C SGML Working Group From: "W. Eliot Kimber" Subject: Re: A22: CONREF? At 03:04 AM 10/9/96 -0500, Robert Streich wrote: >At 06:33 PM 10/3/96 CDT, Michael Sperberg-McQueen wrote: >>On 9 October 1996, the ERB will vote to decide the following question. >>A non-binding preliminary vote indicates the ERB is now leaning to Yes. >> >>A.22 XML will have no CONREF attributes (11.3.3, 7.3, 7.9.4.4). > >Too bad. I guess this means I won't get to argue for conloc as a more >reasonable method of transclusion. Losing CONREF doesn't preclude conloc, as such--it only means that you have to have a separate element type with a declared content of EMPTY to do the conloc reference: Not quite as nice as having #CONREF for this, but it does avoid the serious problem of conref attributes. Cheers, Eliot -- W. Eliot Kimber (kimber@passage.com) Senior SGML Consultant and HyTime Specialist Passage Systems, Inc., (512)339-1400 10596 N. Tantau Ave., Cupertino, CA 95014-3535 (408) 366-0300, (408) 366-0320 (fax) 2608 Pinewood Terrace, Austin, TX 78757 (512) 339-1400 (fone/fax) http://www.passage.com (work) http://www.drmacro.com (home) "If I never had existed, would you still remember me?..." --Austin Lounge Lizards, "1984 Blues" Date: Wed, 9 Oct 1996 13:11:29 -0400 To: w3c-sgml-wg@w3.org From: "David G. Durand" (David G. Durand) Subject: Re: A11: element content and mixed content? At 9:20 AM 10/9/96, Bill Smith wrote: >David Durand wrote: > >> If we drop the distinction, we must treat spaces and tabs as data elements >> in element content (like tables) because without a DTD we can't tell >> whether or not it is element content, and therefore "insignificant". Of >> course, I'd say that we should drop it, and accept the restrictions on >> whitespace, but the current leaning reflects the "code-formatting" needs >> you and others have advocated. > >I don't think David is suggesting requiring a DTD. I'm firmly opposed to >requiring a DTD, even a short form, to "properly" handle XML. If we cross the >DTD/DTDless line, I think we will significantly reduce the liklihood that XML >will be adopted. We should drop the distinction. Well, I am making the point that if we want nonsignificant whitespace in tables, we need the distinction _and_ a DTD. I agree that this is bad. If we keep the distinction, that means that we either need a DTD (bad) or we need to disallow "insignificant whitespace". This means that you can't put spaces, newlines, or tabs inside element content to pretty-print your code. I favor the prohibition. I was interpreting this level of SGML-compatibility as preserving the SGML distinction, but explicitly disallowing (in XML documents) something SGML allows. Of course, you are right that it really says that in XML there is no distinction, so my explanation (and thoughts) were muddy. Nuke it! -- David RE delenda est. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ http://www.dynamicdiagrams.com/services_map_main.html Date: Wed, 9 Oct 1996 15:07:23 -0400 From: gtn@ebt.com (Gavin Nicol) To: jjc@jclark.com CC: w3c-sgml-wg@w3.org Subject: Re: A17: keep or drop entities? >>We need SDATA internal entities for precisely the reason given >>here. > >Why can't we use the private use zone in 10646 for this? It's too small for some situations, and not quite as flexible. Date: Wed, 09 Oct 1996 20:26:51 +0000 To: gtn@ebt.com (Gavin Nicol) From: James Clark Subject: Re: A17: keep or drop entities? Cc: w3c-sgml-wg@w3.org At 15:07 09/10/96 -0400, Gavin Nicol wrote: >>>We need SDATA internal entities for precisely the reason given >>>here. >> >>Why can't we use the private use zone in 10646 for this? > >It's too small for some situations, ISO/IEC 10646 provides in excess of 2^29 private use characters. James Date: Wed, 9 Oct 1996 15:38:30 -0400 From: gtn@ebt.com (Gavin Nicol) To: jjc@jclark.com CC: w3c-sgml-wg@w3.org Subject: Re: A17: keep or drop entities? >>It's too small for some situations, > >ISO/IEC 10646 provides in excess of 2^29 private use characters. Sorry, I thought you meant the BMP. Date: Wed, 09 Oct 1996 12:56:39 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Report from the SGML ERB meeting of Oct. 9th The SGML ERB met Wed. Oct 9th and voted on quite a few items. In attendence: Bosak, Bray, Clark, DeRose, Hollander, Kimber, Maler, Paoli, Sperberg-McQueen, and Sharpe. Absent: Magliery and Connolly. By a recent resolution of the ERB, and at Dan Connolly's request, he is now a non-voting liaison member. Thus, "Unanimous" means 10 in favor. No votes were close enough that Tom's presence or absence would have made a difference. Several issues were left unresolved at the end of the meeting; the ERB will be meeting tomorrow and Saturday to get through this stuff. >A.1 XML will have only one concrete syntax, fixed at XML >specification time, not document-instance parse time (0.2, 13.3, >13.4). Passed, Unanimous >A.2 All or virtually all the information provided by a normal >SGML declaration will be fixed for all documents; no SGML declaration >will be necessary. (Possible exception: character-set information may >vary document to document, but will be conveyed in other ways.) >(6.2.3) Passed, Unanimous >A.3 XML will have no OMITTAG, DATATAG, SHORTREF, LINK, CONCUR, >RANK, or SUBDOC features (7.3.1, 7.3.1.1, 7.3.1.2, 7.3.2, 7.4, 7.5, 7.6, >7.7, 7.8, 9.4.6, 11.2, 11.5, 11.6, 13.5). Passed, Unanimous >A.4 XML will make only partial use of the SHORTTAG feature: > > * no minimized start-tags or, probably, end-tags (7.4, 7.5) > * no omission of attribute name and value indicator (7.9.1.2) >But > * omission of attribute-value specification will be legal for >attributes not declared REQUIRED (7.9) (applies only when >DTD is supplied and used) >The final point, on omitted attribute-value specifications, raises the >general question of how XML systems will behave when no DTD, or a >partial DTD, is provided -- if such omitted or partial DTDs are allowed. >It also raises the question of providing a way for a document to signal >that its DTD can be skipped without loss of information (e.g. because it >has no default attribute values, or no empty elements, etc.). These >questions are to be discussed and decided separately. Passed, Unanimous >A.5 XML will have no quantities or capacities (7.3.3, 7.4.2, >7.9.2, 7.9.4.5, 9.4.1, 9.4.2, 9.8, 11.3.1, 13.2). Passed, Unanimous >A.6 XML will not allow asynchronous marked sections -- marked >sections must begin and end in the same element. Passed; Unanimous. As Harvey Bingham pointed out, this needs careful phrasing to avoid ambiguity. >A.7 Should XML have CDATA, RCDATA, and TEMP marked sections or >not? XML will have CDATA marked sections, which must begin with the 9-character literal string "". This is essentially Charles Goldfarb's proposal, although we may not call them CLEARDATA. XML will not have RCDATA or TEMP marked sections. Both Unanimous. >A.8 Should XML have INCLUDE and IGNORE marked sections or not? >(If this question is answered YES, it leads to a separate question, how >to achieve conditional inclusion in XML markup declarations. This >related question is to be decided separately.) Split in two: XML will not have INCLUDE or IGNORE marked sections in document instances; Unanimous. The question of conditional markup in declarations is still open. >A.9 XML will have no CDATA or RCDATA elements (11.2.3). Passed, Unanimous. >A.10 How should XML escape markup delimiter characters in content >(especially if (R)CDATA elements and marked sections are not >allowed)? Unanimously agreed that CDATA marked sections are to be used for blocks of text. See A19 for more on this. >A.11 XML will retain the distinction between element content and >mixed content (7.6, 11.2.4). (Applies only if DTD supplied and >used.) Passed, DeRose dissenting. >A.12 XML will require all attribute-value specifications to take >the form of attribute-value literals (7.9.3, 7.9.3.1). Passed, Unanimous. >A.13 XML will not allow RE to end an entity or character >reference; an explicit refc must provided, and it must be a semicolon >(9.4.4). Passed, Unanimous. >A.16 XML will stipulate that character references within >processing instructions should be resolved by the XML parser (8). Defeated, Sperberg-McQueen dissenting. >A.18 XML will have declarations for elements, and attributes, but not for short-references or links (11.1). Passed, Unanimous, for elements and attributes. Notations and entities remain open. >A.19 XML will retain fundamentally the same parsing rules as >SGML, though they may be expressed differently. (N.B. there is some >sentiment for making XML's rules more restrictive than SGML's.) Agreed unanimously that the rules should be stricter than SGML in that the characters '&', '<', and '>' are deemed always to delimit markup, and must always be escaped, specifically as "&", "<", and ">", when appearing in parsed character data. The ERB recognizes that this impinges on the user's name space in an un-SGML-like way, but feels that this has already, de facto, happened. >A.21 like SGML, XML will forbid empty strings as attribute values >for non-CDATA attributes, require FIXED attributes to take their default >values (7.9.4.1, 7.9.4.2), and distinguish implied values from >null-string values (11.3.4). Passed, 7 in favor, DeRose, Hollander, and Sperberg-McQueen dissenting. >A.23 XML will have no CURRENT attributes, but it will have FIXED, >REQUIRED, and IMPLIED attributes, and attributes with explicit >defaults. Passed, Unanimous. >A.24 Unlike SGML, XML will not allow direct references to >external data entities from within parsed character data (9.4). Passed, Unanimous. >A.25 Like SGML, XML will forbid recursive entity reference >(9.4). Passed, Unanimous. >A.26 Like SGML, XML will allow elements to be declared ANY >(11.2.4). (Whether other similar shorthand declarations will be >defined, e.g. for any subelements but not allowing PCDATA, will be >decided separately.) Passed, Bray dissenting. >A.27 XML will behave like SGML as regards behavior and precedence >of occurrence indicators and connectors in content models (11.2.4.1, >11.2.4.2). (Whether to abolish the AND connector will be decided >separately.) Passed, Unanimous. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Wed, 09 Oct 96 16:34:31 CDT From: Michael Sperberg-McQueen Subject: A8 and A17: entities, conditional inclusion, what's XML for? To: W3C SGML Working Group The discussions of entities and conditional inclusion seem to me to suggest we may need clarification of some issues, raised by questions A8 and A17. We might disagree over these because of different views on whether entities, or conditional inclusion, are needed or desirable for XML as a mechanism for publication on wide-area networks. We might disagree because of different views on whether they are essential for document management in production work on any reasonably large body of documents. Or we might disagree because of different views on whether XML's task is solely to support network distribution and publication of documents, or to support, as far as possible, production work in managing those documents. There doesn't seem to be a lot of need for discussion on the first point. I think HTML demonstrates that neither external text entities, nor conditional inclusion in the DTD, is essential to wide deployment and acceptance of a system for network distribution of documents. Some may think HTML's lack of each of these is a weak point, and it would be better to have them, but we can't reasonably claim that 'No one will accept a document markup language that doesn't have external entities', any more than we could claim 'No one will accept a markup language that requires lots of angle brackets.' (I used to hear that a lot. I don't, so much, anymore. Hmm.) The second point may be more controversial. It seems so obvious to me that conditional inclusion and external text entities are essential for acceptable document management that I have a hard time making a serious argument; I tend to collapse into sputtering incoherence. But in case anyone really thinks they're not needed, I'll try. On conditional inclusion, I'll just note that there seem to be quite a few public DTDs which have found it necessary to have more than one flavor, and which use conditional inclusion of declarations to accomplish that feat. In some cases, there are just two flavors; in the case of the TEI, it's something like a few hundred thousand flavors (not counting variations caused by suppressing individual elements; if you count those, there are probably 2**400 or so flavors). For production work, it seems better if we can generate the required flavors from a single copy of the DTD, rather than keeping a copy of each flavor. This simplifies updates, too. The TEI may be unusual in its extreme variety, but even HTML has multiple flavors controlled by marked sections. On external entities: (1) External entities allow me to divide a document up into convenient chunks for editing, for exchange with others working on the same project, for check-in and check-out, version control, etc. Any division into files does this. (2) Keeping entity syntax allows me to express, in a standard way, how the different entities of which a document is composed fit together. The 'cat them all together then call the parser' technique does NOT allow me to do this: I lose the ability to document, in a standard way, how the entity structure of the document works. (It also forces me to split my root entity asynchronously in a way I normally avoid.) (3) In 8879, entities referred to are parsed in context, and the structure of the document as a whole is validated by the parser. The "just refer to them from an empty element' technique makes all my external entities opaque to validation: even if they are valid when viewed in isolation, this technique offers no guarantee that the elements in the external entities are valid at the point of reference. If I want to validate the external entities at all, this technique also limits me to single-element external entities, which is not necessarily exactly what I want. Validation using a document grammar is the jewel of SGML. I don't want to be forced to do without it. Reference to external entities via attribute values (which amounts to the reinvention of the GML Starter Set's INCLUDE tag, twenty-odd years later) also shifts responsibility for the include-and-handle-here behavior from the parser for the language to the application. In general, whenever there is some constraint which must be honored by every application, or some behavior which every application must perform, there's a good case for making the constraint expressible in the language itself, and making the basic processor responsible for the behavior. In databases, constraints on the data should be expressed in the schema and enforced by the DBMS. Leaving them to be enforced by every application programmer who touches the database for read or write is, in general, not a good idea. In SGML we have a well understood way of saying "That thing over there is part of this document; it goes HERE", using references to external entities. It's appropriate that it be in the markup language, not just in the applications' style sheet languages, since processing a document generally requires that we know what is and what is not part of the document. Moving that knowledge outside of the markup language is NOT a step forward in the history of document representation. (4) External entities can be (at least, I think they can) not just files but any data stream. External entities, that is, make it possible to build what Dave Sklar calls 'spontaneously combusting' documents: documents whose external entities are data streams created on demand, at parse time, and thus guaranteed up to date. Take away external entities, and how are we to do that? Even if everyone agrees that these constructs / features are (a) required for serious production work and (b) not required for net-based distribution, we may still disagree over whether they belong in XML. XML could be SGML-for-clients: nothing there that isn't essential to allow a client to parse it. Or it could be a more serious language, a flavor of SGML stripped down enough to allow easier implementation and make it feasible to implement in a client, but strong enough that a lot of serious work can be done in it, so that most of us could use it, most of the time, and publication on the net would not ALWAYS involve a serious down-translation and loss of information. On the whole, I'd rather have XML be a strong, useful language, not limited to use in network publishing. That's what I think Goal 2 is for. If most of us, most of the time, need more than XML will provide and will have to do our daily production work in Full SGML, then what will XML have bought us? A slightly better publication medium than HTML 2.0 or HTML 3.2? We don't need a group this high-powered to do that: the mountains give birth, and bring forth a mouse? In short: I think we both conditional inclusion of DTD fragments and normal SGML-style support for external text entities are essential, and belong in XML. If we want to specify that servers should expand references to external entities before serving to clients, that's OK by me, but we may want to look for other ways to specify whether inclusions should be done on the server side or the client side. Either way, the syntax for references to external text entities (and their declarations) needs to be in XML, unless we are content with a niche language when we could have a stronger one. -C. M. Sperberg-McQueen From: Charles@sgmlsource.com (Charles F. Goldfarb) To: "Christopher R. Maden" Cc: w3c-sgml-wg@w3.org Subject: Re: A17: keep or drop entities? Date: Thu, 10 Oct 1996 04:48:59 GMT On Wed, 9 Oct 1996 14:11:45 GMT, "Christopher R. Maden" wrote: > >I'm surprised Gavin hasn't posted about this yet. At lunch the other >day, he had the brilliant notion of defining the Unicode hex character >references not as CDATA entities but as FUNCHARs of the form >&#Uxxxxxxxx; where xxxxxxxx is the hex representation of the Unicode >code point. That is very close to the scheme that WG8 has proposed for SGML97. See N1855 at my web site. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- From: Charles@sgmlsource.com (Charles F. Goldfarb) To: Michael Sperberg-McQueen Cc: W3C SGML Working Group Subject: Re: A8 and A17: entities, conditional inclusion, what's XML for? Date: Thu, 10 Oct 1996 04:49:06 GMT On Wed, 09 Oct 96 16:34:31 CDT, Michael Sperberg-McQueen wrote: >The discussions of entities and conditional inclusion seem to me to >suggest we may need clarification of some issues, raised by questions A8 >and A17. >We might disagree over these because of different views on whether >entities, or conditional inclusion, are needed or desirable for XML as a >mechanism for publication on wide-area networks. >We might disagree because of different views on whether they are >essential for document management in production work on any reasonably >large body of documents. >Or we might disagree because of different views on whether XML's task is >solely to support network distribution and publication of documents, or >to support, as far as possible, production work in managing those >documents. Actually, these views can be reconciled once we accept the idea that XML is a conforming subset of SGML We have seen that this goal is possible if we: 1. Specity some unambiguous rule for RE handling that is a reasonable interpretation of the standard (i.e., one that would be a viable candidate for SGML97) 2. Define a simplified, but conforming, declaration syntax. If XML is a conformance level of SGML ("core SGML") then it doesn't need conditional DTDs because full SGML is available for that. Derived DTDs will be sufficient for casual editing of instances. No DTDs should be needed just to read the instances (but there must be a DTD available and identified in the DOCTYPE declaration). As for entities, James Clark's suggestion that internal entities be limited to a single character is a good one, as it provides a mnemonic capability for &#Unnnnn. I believe the question of external data entities -- the only other potentially necessary entity category -- should be deferred to the discussion of linking mechanisms, as that is what they are. (At present, I favor having them.) -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- From: Charles@sgmlsource.com (Charles F. Goldfarb) To: Robert Streich Cc: tbray@textuality.com, w3c-sgml-wg@w3.org Subject: Re: A26: ANY keyword? Date: Thu, 10 Oct 1996 04:49:03 GMT On Wed, 09 Oct 1996 01:58:45 -0500, Robert Streich wrote: >At 09:02 AM 10/7/96 -0400, Gavin Nicol wrote: >>>>A.26 Like SGML, XML will allow elements to be declared ANY (11.2.4). >>> >>>Upon further consideration, I think I'll vote NO on this one. > >I don't see any need to keep it, so why not let it disappear? I agree with >Tim. > Moreover: 1. It encourages unstructured DTDs. The objective of XML is to promote structure. 2. It is an inclusion in disguise. XML has banned inclusions. 3. It can be replaced by an equivalent model group. An XML design principle is to avoid redundancy. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- From: Charles@sgmlsource.com (Charles F. Goldfarb) To: Tim Bray Cc: w3c-sgml-wg@w3.org Subject: Re: A17: keep or drop entities? Date: Thu, 10 Oct 1996 04:49:01 GMT On Tue, 08 Oct 1996 10:40:56 -0700, Tim Bray wrote: >My feelings on this have been going back and forth like a ping-pong ball. > >If we take the actual constellation of *usage* of SDATA & CDATA internal entities, > I don't see why we need them in XML. Charles says that the way they are used is not exactly congruent with the way they were designed, but nonetheless I don't see that >the value they add makes up for the extra complexity in specification. Actually, Tim, I was supporting your position. The keyword SDATA in the ISO character entity set is unnecessary because the replacement text is a symbolic string. (My original intention was that a system would use an equivalent entity set in which the replacement text was real system data.) -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- From: Charles@sgmlsource.com (Charles F. Goldfarb) To: "David G. Durand" (David G. Durand) Cc: w3c-sgml-wg@w3.org Subject: Re: A11: element content and mixed content? Date: Thu, 10 Oct 1996 04:48:55 GMT On Wed, 9 Oct 1996 13:11:29 -0400, "David G. Durand" (David G. Durand) wrote: >Well, I am making the point that if we want nonsignificant whitespace in >tables, we need the distinction _and_ a DTD. I agree that this is bad. Not necessarily so. Just declare the left square bracket to be a NET-enabling start-tag. Then: -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- From: Charles@sgmlsource.com (Charles F. Goldfarb) To: Tim Bray Cc: w3c-sgml-wg@w3.org Subject: Re: Report from the SGML ERB meeting of Oct. 9th Date: Thu, 10 Oct 1996 04:49:05 GMT On Wed, 09 Oct 1996 12:56:39 -0700, Tim Bray wrote: >Agreed unanimously that the rules should be stricter than SGML in that the >characters '&', '<', and '>' are deemed always to delimit markup, and must >always be escaped, specifically as "&", "<", and ">", when appearing >in parsed character data. The ERB recognizes that this impinges on the user's >name space in an un-SGML-like way, but feels that this has already, de facto, >happened. You may be able to avoid the name space impingement by declaring them as named function characters. You could then say: &#amp;, et. al. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- From: Charles@sgmlsource.com (Charles F. Goldfarb) To: Tim Bray Cc: w3c-sgml-wg@w3.org Subject: Re: A17: keep or drop entities? Date: Thu, 10 Oct 1996 04:48:58 GMT On Tue, 08 Oct 1996 10:40:56 -0700, Tim Bray wrote: >I'm going to have trouble voting on this. And it's *way* more important than the syntax of markup declarations. >From a technical standpoint, Tim is correct. Compared to a simplified DTD syntax that captures only the XML information set, the perceived benefit of DSDs becomes vanishingly small. From a business and political standpoint, though, the divisiveness that a non-standard syntax would create overwhelms every other consideration. I urge Tim to withdraw the DSD proposal so that we can concentrate our energies on the important technical issues. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- From: Charles@sgmlsource.com (Charles F. Goldfarb) To: Michael Sperberg-McQueen Cc: W3C SGML Working Group Subject: Re: A7: CDATA, RCDATA, TEMP marked sections? Date: Thu, 10 Oct 1996 04:48:51 GMT On Thu, 03 Oct 96 18:23:07 CDT, Michael Sperberg-McQueen wrote: >On 9 October 1996, the ERB will vote to decide the following question. >A non-binding preliminary vote indicates the question needs further >discussion in the work group. > >A.7 Should XML have CDATA, RCDATA, and TEMP marked sections or not? > A number of thoughtful questions have been raised in this thread regarding the motivation for the CDATA design. I'll try to answer them. 1. The ugly (that is to say, distinctive) syntax. A design principle of SGML is that different kinds of information have different syntaxes. Processing instructions are different from comments are different from elements are different from specially-parsed data. The syntactic distinctions emphasize the semantic ones to remind the user that different rules are in effect and different behaviors will apply. The syntactic differences also allow SGML to avoid impinging on the user's name space, as it would have to do if, say, comments were element types. 2. Why a CDATA element is terminated by any end-tag. Because we goofed. An original objective of SGML, believe it or not, was to support DTD-less parsing of an instance. Obviously, we blew it in the final product by not making CDATA element types, etc., optional. (In fact, we had many more options at one time, but were pressured into "simplifying" things. SHORTTAG was a combination of what started as several independent features.) Multi-year projects in a highly political arena with changing personnel contributes to a loss of focus. WG8 has gotten much, much better at managing the standards process since then. The review process for SGML97 started out by agreeing on objectives and procedural rules and writing them down. XML has wisely done the same. The error was compounded by the fact that we couldn't agree (or occasionally forgot whether we agreed) whether we could expect a dumb parser to keep a stack. We'll fix this bug in SGML97. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- From: lee@sq.com Date: Thu, 10 Oct 96 01:28:07 EDT To: Charles@sgmlsource.com, U35395@UICVM.CC.UIC.EDU Subject: Re: A8 and A17: entities, conditional inclusion, what's XML for? Cc: w3c-sgml-wg@w3.org > As for entities, James Clark's suggestion that internal entities be > limited to a > single character is a good one, as it provides a mnemonic capability for > &#Unnnnn. Do you mean a single chracter or a single glyph? For example, is the sequence the latin letter small a combining accent acute one or two characters, or is is 8 characters (2 4-octet bit combinations)? Note that ISO 10646 relies on the use of combining characters to produce accented characters, except for a few special cases that are in ther for historical reasons. If you allow only a single codepoint, can I do and then use a´ to get an a-acute? This practice is quite common in SGML today, of course, as is the alternative where . indicates typographical superimposition, not functional composition :-) which leads to many more entity definitions. Note: I have used a-acute, which can be represented in two ways in Unicode and ISO 10646, as there is a pre-combined aacute character for compatibility with ISO 8859-1 (Latin 1). Pretend that there was no such combined character -- e.g. if I had to produce r-acute, and needed to use the combining characters -- as that is the more general case. If you have to handle a character sequence that maps to a single glyph, how much harder is it to allow an arbitrary sequence? It might be worth saying whether entity references can occur within entity values; they could be banned in XML to simplify implementation in environments that lack recursion, such as FORTRAN 66. (are there any FORTRAN 66 SGML-aware Unicode implementations???) Lee Date: Thu, 10 Oct 1996 09:20:07 +0000 To: Charles@sgmlsource.com From: James Clark Subject: Re: Report from the SGML ERB meeting of Oct. 9th Cc: w3c-sgml-wg@w3.org At 04:49 10/10/96 GMT, Charles F. Goldfarb wrote: >On Wed, 09 Oct 1996 12:56:39 -0700, Tim Bray wrote: > >>Agreed unanimously that the rules should be stricter than SGML in that the >>characters '&', '<', and '>' are deemed always to delimit markup, and must >>always be escaped, specifically as "&", "<", and ">", when appearing >>in parsed character data. The ERB recognizes that this impinges on the user's >>name space in an un-SGML-like way, but feels that this has already, de facto, >>happened. > >You may be able to avoid the name space impingement by declaring them as named >function characters. You could then say: &#amp;, et. al. That won't work. Entering the first character of a delimiter in context as a named character reference doesn't prevent its recognition as a delimiter. James From: Charles@sgmlsource.com (Charles F. Goldfarb) To: James Clark Cc: w3c-sgml-wg@w3.org Subject: Re: Report from the SGML ERB meeting of Oct. 9th Date: Thu, 10 Oct 1996 09:43:57 GMT On Thu, 10 Oct 1996 09:20:07 +0000, James Clark wrote: >>You may be able to avoid the name space impingement by declaring them as named >>function characters. You could then say: &#amp;, et. al. > >That won't work. Entering the first character of a delimiter in context as >a named character reference doesn't prevent its recognition as a delimiter. I was afraid of that ... which is why I said "you may". James is correct. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- Date: Thu, 10 Oct 1996 10:12:13 -0400 To: Charles@sgmlsource.com, Tim Bray From: "Eve L. Maler" Subject: Re: A17: keep or drop entities? Cc: w3c-sgml-wg@w3.org At 04:49 AM 10/10/96 GMT, Charles F. Goldfarb wrote: >Actually, Tim, I was supporting your position. The keyword SDATA in the ISO >character entity set is unnecessary because the replacement text is a symbolic >string. (My original intention was that a system would use an equivalent entity >set in which the replacement text was real system data.) The [xxxxxx] replacement text "templates" have been widely implemented to produce the desired glyphs. But this doesn't mean they're not system data, does it? It's still essentially a "processing instruction that returns data" (clause 8). Regular internal text entities aren't supposed to have this property. Eve From: Charles@sgmlsource.com (Charles F. Goldfarb) To: "Eve L. Maler" Cc: Tim Bray , w3c-sgml-wg@w3.org Subject: Re: A17: keep or drop entities? Date: Thu, 10 Oct 1996 20:53:59 GMT On Thu, 10 Oct 1996 10:12:13 -0400, "Eve L. Maler" wrote: >At 04:49 AM 10/10/96 GMT, Charles F. Goldfarb wrote: >>Actually, Tim, I was supporting your position. The keyword SDATA in the ISO >>character entity set is unnecessary because the replacement text is a symbolic >>string. (My original intention was that a system would use an equivalent entity >>set in which the replacement text was real system data.) > >The [xxxxxx] replacement text "templates" have been widely implemented >to produce the desired glyphs. But this doesn't mean they're not system >data, does it? It's still essentially a "processing instruction that >returns data" (clause 8). Regular internal text entities aren't >supposed to have this property. Eve has made a very sensible observation, so let me explain my reasoning. There are two principal purposes for labeling SDATA and PI: 1. To make it easy to locate and revise or remove system-specific information. This, of course, enhances document portability and reuse by containing system dependencies. 2. To prevent generated text from being parsed in context with the SGML document. This enhances portability and reuse by assuring that all applications will "see" the same data. The symbolic replacement text in the ISO 8879 character entity sets don't present a problem on either of those counts. They are not system-dependent and they parse identically in all environments. That is because the generation of system-specific data takes place in the *result* document; it is never seen by the parser. In pernicious SDATA, the entity text is system-specific and therefore needs to be labeled. In the context of creating a simplified subset of SGML that had to be usable in a wide range of system environments, it seemed reasonable to eliminate SDATA entities. In order to allow the ISO character entity sets to be used with the SGML subset, I proposed removing the SDATA keyword from them. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- Date: Thu, 10 Oct 1996 22:35:31 -0500 (CDT) From: DAVEP@acm.org Subject: Re: A17: keep or drop entities? To: Charles@sgmlsource.com Cc: W3C-SGML-WG@w3.org On Thu, 10 Oct 1996, Charles F. Goldfarb wrote: >On Thu, 10 Oct 1996 10:12:13 -0400, "Eve L. Maler" wrote: >>At 04:49 AM 10/10/96 GMT, Charles F. Goldfarb wrote: >>> The keyword SDATA in the ISO >>>character entity set is unnecessary because the replacement text is a symbolic >>>string. (My original intention was that a system would use an equivalent entity >>>set in which the replacement text was real system data.) >>The [xxxxxx] replacement text "templates" have been widely implemented >>to produce the desired glyphs. But this doesn't mean they're not system >>data, does it? It's still essentially a "processing instruction that >>returns data" (clause 8). Regular internal text entities aren't >>supposed to have this property. >Eve has made a very sensible observation, so let me explain my reasoning. >There are two principal purposes for labeling SDATA and PI: > >1. To make it easy to locate and revise or remove system-specific information. >This, of course, enhances document portability and reuse by containing system >dependencies. > >2. To prevent generated text from being parsed in context with the SGML >document. This enhances portability and reuse by assuring that all applications >will "see" the same data. >The symbolic replacement text in the ISO 8879 character entity sets don't >present a problem on either of those counts. They are not system-dependent and >they parse identically in all environments. That is because the generation of >system-specific data takes place in the *result* document; it is never seen by >the parser. In pernicious SDATA, the entity text is system-specific and >therefore needs to be labeled. There may have been just two principal purposes, but there is certainly a third in this day and age: When one is inserting an ordinary text entity, the replacement text is specifically intended to become characters in the document. When "&" is inserted in a document, one does not normally intend it to be the same as though "[amp ]" were directly in the document. Many processors rely on the SDATA designation to trigger the special handling. Leave it out and we're in trouble. Dave Peterson SGMLWorks! davep@acm.org From: lee@sq.com Date: Thu, 10 Oct 96 23:34:27 EDT To: Charles@sgmlsource.com Subject: Re: A7: CDATA, RCDATA, TEMP marked sections? Cc: w3c-sgml-wg@w3.org > A design principle of SGML is that different kinds of information have > different syntaxes. It's very valuable to hear comments like this -- that sort of principle wasn't obvious to me from the standard. Since it's counter to practice in both programming languages and natural language, perhaps it's not surprising. We have an opportunity to diverge from this route in XML. > Processing instructions are different from comments are different from > elements are different from specially-parsed data. Nouns are different from verbs, but they are not distinguished lexically in English. In Sanskrit and Ancient Greek and Latin they are distinguished by endings; morphological tagging has tended to weaken with time, however, so that languages are getting less complex in this regard. In programming languages, consider process(w) where w may be an integer or a string, but the same syntax is used. This uniform syntax is VERY highly prized in computer science. > The syntactic distinctions > emphasize the semantic ones to remind the user that different rules are in > effect and different behaviors will apply. If the rules were used everywhere, the language would be simpler, no different syntax would be needed, and the user would not get confused. > The syntactic differences also allow > SGML to avoid impinging on the user's name space, as it would have to do if, > say, comments were element types. It is true that the parser needs to have enough context to know what is a comment. E.g.: this is now a comment. Now there is no syntactic distinction, and yet the DTD-writer's namespace is unaffected. It would be more productive to introduce the idea of scope -- e.g. (title, P*) > in order to protect namespaces. (I am not proposing this syntax here, but only trying to point out that since SGML has only global names, it's in no position tobe proud about the user's namespace!) It would be far better to have a simpler syntax. There may have been some merit in having a separate comment syntax, if it had been a good one. Since the SGML comment syntax has the same open and close (at least by default) and suffers from non-rubust parity errors, as well as failing if a typewriter em-dash is included (which can't be escaped), I don't accept that the comment syntax is a good one. It's less robust than the usual element syntax (which is very good, by the way). If you want to have a syntactical distinction, you could use a similar syntax to elements, on the principle of least surprises, or you could use a coment that goes to the end of a line (or record, if you must) as in many programming languages. Element-style comments could nest, which removes some of the need for IGNORED marked sections. But I expect that XML will have SGML's weird multiple syntaxes... :-( Lee >Received: by chuckd.allette.com.au.allette.com.au id AA08193 (5.65/IDA-1.5 for Charles@sgmlsource.com); Fri, 11 Oct 1996 15:11:27 +1000 Date: Fri, 11 Oct 1996 15:11:27 +1000 (EST) From: Rick Jelliffe To: DAVEP@acm.org Cc: Charles@sgmlsource.com, W3C-SGML-WG@w3.org Subject: Re: A17: keep or drop entities? On Thu, 10 Oct 1996 DAVEP@acm.org wrote: > Many processors rely on the SDATA designation to trigger the special > handling. Leave it out and we're in trouble. OmniMark, for instance, uses the SDATA label. I'd suggest that usually SDATA entities are used as keys into a table of entities known by a public identifier (with ENTITIES in the identifier). Often these entities are single characters or glyphs. So, rather than getting rid of SDATA, perhaps it should be promoted as the way to signify this kind of indexing, with a view to allowing fast access to large internal-form (e.g. already in a grove or in a database) tables of data. Under this view, name qualifications could be added to XML and ISO8879 without much pain: e.g. &ISOlat1:eacute; meaning 'lookup in table keyed by "ISOlat1" the text keyed by "eacute"'. This mechanism would also allow convenient access to other simple databases apart from lists of characters/glyphs, which XML system creators might find useful. E.g. Today's price &DowJones2334:XmlCorp;Yesterday's price &DowJones2333:XmlCorp;
It would also be useful for getting glyphs, and any case where the number of SDATA entities might be in the tens or hundreds of thousands (i.e. where the SDATA mechanism is tedious and inefficient, but where a query mechanism is overkill):

The Chinese character for hamburger is &CMAC:C12344;.

Rick Jelliffe http://www.allette.com.au/allette/ricko email: ricko@allette.com.au ================================================================ Allette Systems http://www.allette.com.au email: info@allette.com.au 10/91 York St, 2000, phone: +61 2 9262 4777 Sydney, Australia fax: +61 2 9262 4774 ================================================================ >Received: by chuckd.allette.com.au.allette.com.au id AA08774 (5.65/IDA-1.5 for Charles@sgmlsource.com); Fri, 11 Oct 1996 18:24:56 +1000 Date: Fri, 11 Oct 1996 18:24:55 +1000 (EST) From: Rick Jelliffe To: lee@sq.com Cc: Charles@sgmlsource.com, w3c-sgml-wg@w3.org Subject: delimiters (Re: A7: CDATA, RCDATA, TEMP marked sections?) On Thu, 10 Oct 1996 lee@sq.com wrote: > But I expect that XML will have SGML's weird multiple syntaxes... :-( Is the programming language world really so simple? What about C++? It has three comment delimiters: // for C++ /* */ from C # (at beginnning of line) from cpp It has different delimiters for blocks ( {} ) and statements ( ; ) and function calls ( () ). People complain that LISP is difficult to read because it only has one delimiter for comments and one for function calls. The reasons for C++'s multiple comments delimiters might be artifacts of its implementation history, but C++ syntax is more popular than LISP syntax, as Java proves. One very good reason for tag delimiters and comment delimiters to be different is that if they are different non-SGML-aware text tools can play with them. E.g. off-the-shelf text editors can display markup in different colors, if they they can detect the markup easily. This is something we use a real lot here, and we even make sure our SGML documents are marked up so that the simple detection mechanism based on delimiters with no context will be reliable: e.g. < and & always must be done by entity references, and CDATA content models are deprecated. (I guess it means we already try to make our in-process documents more like XML already.) While it might be good to have a comment declaration like you suggested in SGML, I think it would be a terrible idea in XML: anything that requires context or previous declarations to parse should be viewed as suspect and complicating. But even in SGML, you could declare an architectural form or fixed attribute for an element that labels it as having the nature of a comment. Rick Jelliffe http://www.allette.com.au/allette/ricko email: ricko@allette.com.au ================================================================ Allette Systems http://www.allette.com.au email: info@allette.com.au 10/91 York St, 2000, phone: +61 2 9262 4777 Sydney, Australia fax: +61 2 9262 4774 ================================================================ Date: Fri, 11 Oct 1996 17:05:08 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: A7: CDATA, RCDATA, TEMP marked sections? At 12:37 PM 10/11/96 EDT, lee@sq.com wrote: >It's not clear to what extent XML has to be backwards-compatible with >SGML, and in what ways, in order to succeed, and I am trying to probe >that boundary. For example, it seems to have been agreeed that there >will be application conventions that an SGML system wouldn't know about, >and clearly if you have no DTD, most SGML systems are hosed. An area worth probing indeed; but in fact I know of no such application conventions that are apt to make it into XML. At the current time it seems that XML will be a very pure subset of 8879, with the possible exception of RS/RE rules. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 From: lee@sq.com Date: Fri, 11 Oct 96 12:37:51 EDT To: ricko@allette.com.au Subject: Re: A7: CDATA, RCDATA, TEMP marked sections? Cc: w3c-sgml-wg@w3.org > To be realistic, what about C++? > > It has 3 forms comment delimiters: > // from C++ > # (at start of line) for cpp > /* */ for # does not introduce a comment in C++ (nor in the cpp). It is used for directives that are handled separately -- e.g. #include There are few people who would say that C++ has a clean syntax, and even its designer, Bjarne Stroustrop, is not among them -- see his book The Design and Rationale of C++ (is that the right title?) for comments about why C++ is the way it is. Mostly it's backwards compatibility, which is so often why unpleasant designs get perpetuated. Vide MS-DOS... The people who worked on C++ were/ae some of the best in the programming language industry -- just as the people who worked on SGML were/are some of the best in the text processing industry. C++ has been successful because it builds on something insanely successful, and makes it more powerful -- at the cost of greatly increaased complexity. It has succeeded where none of the Pascal++ could. It is not in fact 100% compatible with C. But the incompatibilities are clearly documented and understood, and it looks close enough to C's popular syntax that C programmers are prepared to invest time and effort learning it. Sometimes when you give up compatibility, the world loses interest -- SmallTalk 80 was a fabulous language. NeWS was technically the best windowing system ever created. It's not clear to what extent XML has to be backwards-compatible with SGML, and in what ways, in order to succeed, and I am trying to probe that boundary. For example, it seems to have been agreeed that there will be application conventions that an SGML system wouldn't know about, and clearly if you have no DTD, most SGML systems are hosed. Lee Date: Fri, 11 Oct 1996 09:36:37 -0400 To: lee@sq.com From: masonjd@ornl.gov (James David Mason) Subject: Re: A7: CDATA, RCDATA, TEMP marked sections? Cc: Charles@sgmlsource.com, w3c-sgml-wg@w3.org At 11:34 PM 10/10/96 EDT, lee@sq.com wrote: > >Nouns are different from verbs, but they are not distinguished lexically in >English. In Sanskrit and Ancient Greek and Latin they are distinguished by >endings; morphological tagging has tended to weaken with time, however, so >that languages are getting less complex in this regard. While morphological distinctions in IE languages have tended to decrease with time, there have been compensating syantactic effects. Languages like English (only two morphological noun cases, down from four plus remnants of another in Anglo Saxon) or French (only one noun case, down from seven in Latin) tend to have developed very strongly positional syntax that their ancestors did not need. >In programming languages, consider > process(w) >where w may be an integer or a string, but the same syntax is used. > >This uniform syntax is VERY highly prized in computer science. > Another reason why those of us trained as natural-language find programming languages so unnatural. Jim Mason Dr. James D. Mason (ISO/IEC JTC1/SC18/WG8 Convenor) Lockheed Martin Energy Systems Information Management Services SGML Systems Development 1060 Commerce Park, M.S. 6480 Oak Ridge, TN 37831-6480 U.S.A. Telephone: +1 423 574-6973 Facsimile: +1 423 574-0004 Network: masonjd@ornl.gov Date: Fri, 11 Oct 1996 09:32:26 -0400 To: DAVEP@acm.org, Charles@sgmlsource.com From: "Eve L. Maler" Subject: Re: A17: keep or drop entities? Cc: W3C-SGML-WG@w3.org At 10:35 PM 10/10/96 -0500, DAVEP@acm.org wrote: >There may have been just two principal purposes, but there is certainly >a third in this day and age: When one is inserting an ordinary text entity, >the replacement text is specifically intended to become characters in the >document. When "&" is inserted in a document, one does not normally >intend it to be the same as though "[amp ]" were directly in the document. >Many processors rely on the SDATA designation to trigger the special >handling. Leave it out and we're in trouble. Yes, this is exactly what I meant. One way or another, you need the effect of saying "This is an instruction to the system to return something else." There are three main ways it gets used: o Trivial keyboardable cases (lt, amp, sometimes gt) to escape markup characters. These can be replaced with CDATA or text entities. (Of course, "keyboardable" depends on your keyboard!) o Nonkeyboardable cases. Maybe these can be replaced with character references of one kind or another. I get too lost in the UNICODE jungle to really understand this alternative, but I could be persuaded. o Non-special-symbol cases, where you're truly sending out to the system for some processing to happen and some value to be returned -- for example, "insert the current date and time here". Should this be done exclusively with non-SGML scripts instead? As Dave and Rick point out, many systems -- ADEPT is one of them, by the way -- key off the SDATA label to access an external table. If all documents potentially need the functionality of looking something up outside on the system, why not stick with SDATA and avoid making all applications reinvent this wheel? Eve Date: Sat, 12 Oct 96 14:51:22 CDT From: Michael Sperberg-McQueen Subject: more questions for 16 October To: W3C SGML Working Group The ERB has decided many, though not all, of the questions already posed to the WG as questions A.1 through A.28. We continue to discuss the remainder, and decisions will be announced as they are made. A second batch of questions is scheduled for consideration on 16 and 17 October; I will post the questions of this second batch to the list shortly. As before, the postings indicate when there appears to be a consensus in the ERB leaning toward a particular decision on the issue. -CMSMcQ Date: Sat, 12 Oct 96 15:04:59 CDT From: Michael Sperberg-McQueen Subject: B.1 Character-set rules? To: W3C SGML Working Group On 16 October 1996, the ERB will vote to decide the following question. A straw poll indicates the question needs further discussion in the work group. B.1 What should XML's character-set rules be? Should conforming XML documents be restricted to particular character sets? Should conforming XML processors be required to be able to parse all conforming XML documents (13.1)? Date: Sat, 12 Oct 96 15:06:39 CDT From: Michael Sperberg-McQueen Subject: B.5/B.6 Restrict PCDATA in content models? To: W3C SGML Working Group On 16 October 1996, the ERB will vote to decide the following question. A straw poll indicates the question needs further discussion in the work group. B.5 Should XML restrict the use of the PCDATA token in content models, to simplify RE handling or eliminate the Mixed Content Problem? (7.6.1, 11.2.4) * B.5 restrict PCDATA to models of the form (#PCDATA) * B.6 restrict PCDATA to models of the form (#PCDATA | x ... | z)* Date: Sat, 12 Oct 96 15:07:57 CDT From: Michael Sperberg-McQueen Subject: B.8 MSxCHAR markup-suppression? To: W3C SGML Working Group On 16 October 1996, the ERB will vote to decide the following question. A straw poll indicates the ERB is leaning to dropping these constructs. B.8 Should XML use MSOCHAR, MSSCHAR, and MSICHAR strings (9.7)? Date: Sat, 12 Oct 96 15:06:59 CDT From: Michael Sperberg-McQueen Subject: B.7 Conditional inclusion in DTDs? To: W3C SGML Working Group On 16 October 1996, the ERB will vote to decide the following question. A straw poll indicates the question needs further discussion in the work group. B.7 How should XML deal with the need for conditional inclusion of markup declarations, if XML has no marked sections (10.4.1)? Date: Sat, 12 Oct 96 15:08:24 CDT From: Michael Sperberg-McQueen Subject: B.9 Formal system, public identifiers? To: W3C SGML Working Group On 16 October 1996, the ERB will vote to decide the following question. A straw poll indicates the question needs further discussion in the work group. B.9 Should XML require system and public identifiers to be FORMAL (13.5)? Date: Sat, 12 Oct 96 15:05:56 CDT From: Michael Sperberg-McQueen Subject: B.3 Record-end handling? To: W3C SGML Working Group On 16 October 1996, the ERB will vote to decide the following question. A straw poll indicates the question needs further discussion in the work group. B.3 How should XML handle RE (7.6.1)? Date: Sat, 12 Oct 96 15:10:12 CDT From: Michael Sperberg-McQueen Subject: B.12 Multiple notation declarations? To: W3C SGML Working Group On 16 October 1996, the ERB will vote to decide the following question. A straw poll indicates the question needs further discussion in the work group. B.12 Should XML retain SGML's prohibition on multiple declarations for the same notation (11.4)? Date: Sat, 12 Oct 96 15:09:33 CDT From: Michael Sperberg-McQueen Subject: B.10 Empty elements? To: W3C SGML Working Group On 16 October 1996, the ERB will vote to decide the following question. A straw poll indicates the question needs further discussion in the work group. B.10 What form should EMPTY elements take, if there are EMPTY elements in XML: , (where '/' is assumed the NET string) (7.3, 11.2.3)? Date: Sat, 12 Oct 96 15:09:59 CDT From: Michael Sperberg-McQueen Subject: B.11 Empty end-tags? To: W3C SGML Working Group On 16 October 1996, the ERB will vote to decide the following question. A straw poll indicates the ERB is leaning to forbidding empty end-tags. B.11 Should XML forbid, allow, or require empty end-tags (7.5)? Date: Sat, 12 Oct 96 15:06:14 CDT From: Michael Sperberg-McQueen Subject: B.4 Drop comments, PIs in mixed content? To: W3C SGML Working Group On 16 October 1996, the ERB will vote to decide the following question. A straw poll indicates the question needs further discussion in the work group. B.4 Should XML forbid comments and processing instructions in mixed content, as a way of simplifying RE handling (7.6)? Date: Sat, 12 Oct 96 15:10:30 CDT From: Michael Sperberg-McQueen Subject: B.13 ENTITY attributes for notations? To: W3C SGML Working Group On 16 October 1996, the ERB will vote to decide the following question. A straw poll indicates the question needs further discussion in the work group. B.13 Should XML remove SGML's prohibition on ENTITY attributes for notations (11.4.1)? Date: Sat, 12 Oct 96 15:05:28 CDT From: Michael Sperberg-McQueen Subject: B.2 DTD required or optional? To: W3C SGML Working Group On 16 October 1996, the ERB will vote to decide the following question. A straw poll indicates the ERB is leaning to making DTDs optional. B.2 Should XML require each document instance to have a DTD or not (7.1)? From: lee@sq.com Date: Sat, 12 Oct 96 18:51:57 EDT To: U35395@UICVM.CC.UIC.EDU, w3c-sgml-wg@w3.org Subject: Re: B.11 Empty end-tags? > B.11 Should XML forbid, allow, or require empty end-tags (7.5)? I would very much like to see them allowed in SGML itself, as this can greatly simplify document migration. If end tags are not required, it will be very difficult to handle empty elements in XML if there is no DTD; you don't want to do a lot of look-ahead if you're doing progressive display as data arrives. Lee From: lee@sq.com Date: Sat, 12 Oct 96 18:54:17 EDT To: U35395@UICVM.CC.UIC.EDU, w3c-sgml-wg@w3.org Subject: Re: B.13 ENTITY attributes for notations? > B.13 Should XML remove SGML's prohibition on ENTITY attributes for > notations (11.4.1)? It's funny -- I came across this yesterday, as SoftQuad Panorama actually uses ENTITY notation attributes in its style sheet DTD (oops). They're used to specify a mask for an icon. I don't understand the restriction in SGML (maybe someone can explain it?) but I don't see how it can sensibly be lifted in XML unless it might also be lifted in SGML. Lee From: lee@sq.com Date: Sat, 12 Oct 96 18:49:20 EDT To: U35395@UICVM.CC.UIC.EDU, w3c-sgml-wg@w3.org Subject: Re: B.10 Empty elements? > B.10 What form should EMPTY elements take, if there are EMPTY elements > in XML: , (where '/' is assumed the NET string) > (7.3, 11.2.3)? Only one of these, , can be parsed reliably without a DTD, unles you are prepared to use lookahead to the end of the document or untiul the next close tag of a container. Two other possibilities: (1) use a naming convention, e.g. (1b) add something (e.g. @) as a name start character: <@BR> where all elements starting with @ are EMPTY, by convention. An amendment to SGML could allow <@ to be a delimiter to start a tag of an EMPTY element, and then the convention would become supported syntax. Both of these are using syntax, but are using a kind of syntax for which there is precedent. I don't have strong feelings on these, but wanted to raise them since they are both compatible with SGML as it stands today. Lee Date: Sat, 12 Oct 96 19:35:31 CDT From: Michael Sperberg-McQueen Subject: Re: B.11 Empty end-tags? To: Liam Quin , W3C SGML Working Group On Sat, 12 Oct 96 18:51:57 EDT Liam Quin said: >> B.11 Should XML forbid, allow, or require empty end-tags (7.5)? > >I would very much like to see them allowed in SGML itself, as this can >greatly simplify document migration. > >If end tags are not required, it will be very difficult to handle >empty elements in XML if there is no DTD; you don't want to do a >lot of look-ahead if you're doing progressive display as data arrives. For the record: in the sense of this question, empty end-tags are end-tags of the form '', not end-tags for EMPTY elements. The possibility of end-tags for EMPTY elements is *also* up for discussion, but it's question B.10, not B.11. -CMSMcQ From: lee@sq.com Date: Sat, 12 Oct 96 23:05:02 EDT To: U35395@UICVM.CC.UIC.EDU, lee@sq.com, w3c-sgml-wg@w3.org Subject: Re: B.11 Empty end-tags? > For the record: in the sense of this question, empty end-tags are > end-tags of the form '', not end-tags for EMPTY elements. oops, thanks for pointing that out. I have no objection to in XML as long as there is no OMITTAG, although it does slightly reduce robustness in the face of transmission error or user error. Lee Date: Sat, 12 Oct 1996 21:08:49 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org CC: bosak@atlantic-83.eng.sun.com Subject: Re: B.11 Empty end-tags? | On 16 October 1996, the ERB will vote to decide the following question. | A straw poll indicates the ERB is leaning to forbidding empty end-tags. | | B.11 Should XML forbid, allow, or require empty end-tags (7.5)? I said a while ago that I would speak up for the Desperate Perl Hacker (and others similarly situated). The DPH strongly desires that XML forbid empty end-tags -- or (as the DPH would put it), that all end-tags be required to contain the name of the element being ended. Yes, this does make the marked up document a bit longer. Yes, it does make it a little harder to type in manually (though it's very easy to imagine a version of emacs that automatically inserts the named end-tag using logic something like what it uses to track matching parens). But I worked for four years at Novell with SGML in which empty end-tags were forbidden, and it ain't that bad. In fact, the named end-tag can be very useful at times in understanding where you are in a document. In return for the little extra space, you reap enormous benefits for the person (and her name is legion) who has to make a quick change to, or extract information from, a large collection of documents. Guarantee that all end tags contain the GI, and you enable incredibly powerful operations to be performed on arbitrarily large collections of documents from a single Korn shell command line. Take away that guarantee, and suddenly you have to maintain a stack. This may not sound like much, but to a lot of people in the trenches, especially people facing deadlines, it's all the difference in the world. I suspect that the same is true for people who build cgi-bin scripts and similar one-off lightweight processors. Jon From: Charles@sgmlsource.com (Charles F. Goldfarb) To: Michael Sperberg-McQueen Cc: W3C SGML Working Group Subject: Re: B.12 Multiple notation declarations? Date: Sun, 13 Oct 1996 07:50:04 GMT On Sat, 12 Oct 96 15:10:12 CDT, Michael Sperberg-McQueen wrote: >On 16 October 1996, the ERB will vote to decide the following question. >A straw poll indicates the question needs further discussion in the work >group. > >B.12 Should XML retain SGML's prohibition on multiple declarations for >the same notation (11.4)? I thought the ERB had decided not to augment SGML. Why is this question even being asked? -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- Date: Sun, 13 Oct 1996 09:35:48 -0500 From: Len Bullard To: Jon Bosak CC: w3c-sgml-wg@w3.org Subject: Re: B.11 Empty end-tags? Jon Bosak wrote: > > | On 16 October 1996, the ERB will vote to decide the following question. > | A straw poll indicates the ERB is leaning to forbidding empty end-tags. > | > | B.11 Should XML forbid, allow, or require empty end-tags (7.5)? > > I said a while ago that I would speak up for the Desperate Perl Hacker > (and others similarly situated). [snip] > This may not > sound like much, but to a lot of people in the trenches, especially > people facing deadlines, it's all the difference in the world. I > suspect that the same is true for people who build cgi-bin scripts and > similar one-off lightweight processors. > > Jon I concur with Jon. Whatever space is saved by the empty end tag, it complicates the use of simple rewrite scripts and it makes it much harder to read the instance. Simpler scripts are important to making XML usable. len From: Charles@sgmlsource.com (Charles F. Goldfarb) To: bosak@atlantic-83.eng.sun.com (Jon Bosak) Cc: w3c-sgml-wg@w3.org, bosak@atlantic-83.eng.sun.com Subject: Re: B.11 Empty end-tags? Date: Sun, 13 Oct 1996 15:58:07 GMT On Sat, 12 Oct 1996 21:08:49 -0700, bosak@atlantic-83.Eng.Sun.COM (Jon Bosak) wrote: >Guarantee that all end tags contain the GI, and you enable incredibly >powerful operations to be performed on arbitrarily large collections >of documents from a single Korn shell command line. Take away that >guarantee, and suddenly you have to maintain a stack. I don't disagree with requiring all end-tags to contain the element type name. However, you'll still have to maintain a stack because, for example, OLs can nest in LIs, so you can't simply scan for the next to match a given
    . -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- Date: Sun, 13 Oct 1996 13:39:15 -0400 To: lee@sq.com, U35395@UICVM.CC.UIC.EDU, w3c-sgml-wg@w3.org From: "Eve L. Maler" Subject: Re: B.10 Empty elements? At 06:49 PM 10/12/96 EDT, lee@sq.com wrote: >> B.10 What form should EMPTY elements take, if there are EMPTY elements >> in XML: , (where '/' is assumed the NET string) >> (7.3, 11.2.3)? > >Only one of these, , can be parsed reliably without a DTD, >unles you are prepared to use lookahead to the end of the document or >untiul the next close tag of a container. > >Two other possibilities: >(1) use a naming convention, e.g. > > >(1b) add something (e.g. @) as a name start character: > <@BR> > where all elements starting with @ are EMPTY, by convention. > An amendment to SGML could allow <@ to be a delimiter to start > a tag of an EMPTY element, and then the convention would become > supported syntax. > >Both of these are using syntax, but are using a kind of syntax for >which there is precedent. > >I don't have strong feelings on these, but wanted to raise them since >they are both compatible with SGML as it stands today. > >Lee Another couple of possibilities: (2) provide the necessary markup declarations in a "partial DTD" so that empty elements can be understood: (2b) provide the necessary "instance parameters" rather than whole markup declarations (which generally provide more info than is needed for read-only processing functions): These relate to B.2, required vs. optional DTDs. I find the idea of partial DTDs really appealing, since you don't have to create redundant PIs (which could possibly get out of sync with the original DTD), but also find the PI idea appealing because certain kinds of markup declarations contain a lot of overkill info. I prefer both of these over the naming convention ideas because the latter mess up the user's available GI namespace. If supplying instance parameter in whatever form is a possibility, then there's no reason to ban EMPTY elements (which are quite popular in many DTDs today) from XML, and we can gracefully do away with the need to supply the extra info if SGML97 starts allowing the end-tag. Eve Date: Sun, 13 Oct 1996 10:53:23 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org CC: bosak@atlantic-83.eng.sun.com Subject: Re: B.11 Empty end-tags? [Charles Goldfarb:] | I don't disagree with requiring all end-tags to contain the element | type name. However, you'll still have to maintain a stack [...] For many things, yes. But not for a very large category of everyday, one-off reporting and document management tasks. I still remember the day several years ago when a Director of Internationalization came to me with a hopeless look in her eyes and said, "We need to go through all of our documentation, right now, and extract every string that appears in a user dialogue so that it can be translated separately from the documentation. Is that possible?" (She was thinking of what it would have taken to do this job working >from the FrameMaker files that had been our standard source until then.) I allowed that it might be, and with the aid of my first perl hack and the fact that in our newly converted SGML all text in user dialogues occurred only in strings of the form ... or ... I was able to deliver what she wanted, neatly alphabetized and sorted by type, in 30 minutes. (If it hadn't been my first attempt, it would have taken about five minutes.) From that day on she was one of the strongest SGML champions in the entire corporation. In that one instant she saw the advantages of markup ... and I saw the advantages of non-empty end tags! If you want to win ardent converts, this is one way to do it. Jon Date: Mon, 14 Oct 1996 09:53:51 +0100 To: U35395@UICVM.CC.UIC.EDU, w3c-sgml-wg@w3.org From: haitto@synex.se (Hasse Haitto) Subject: Re: B.13 ENTITY attributes for notations? Cc: lee@sq.com, mail@synex.se lee@sq.com wrote: >> B.13 Should XML remove SGML's prohibition on ENTITY attributes for >> notations (11.4.1)? > >It's funny -- I came across this yesterday, as SoftQuad Panorama actually >uses ENTITY notation attributes in its style sheet DTD (oops). > >They're used to specify a mask for an icon. > >I don't understand the restriction in SGML (maybe someone can explain it?) >but I don't see how it can sensibly be lifted in XML unless it might also >be lifted in SGML. > >Lee When we stumbled upon this silly restriction, I asked Charles about it, and he wrote me as follows: >The restrictions on data attribute declared values is on the list to be >examined during the revision of SGML. I introduced data attributes late >in the game so I did it conservatively (I was concerned about long chains >of entities that had data attributes among which were entities that had >data attributes, etc.) Soooo, we have kept the entity attribute in the style sheet DTD--which is proprietary anyway--and look forward to the day when, hopefully, entity attributes for notations are allowed in both SGML and XML. Cheers, Hasse -- Hasse Haitto Phone: +46 8 791 88 81 Synex Information AB Fax: +46 8 791 88 89 Stora Nygatan 20 Internet: haitto@synex.se S-111 27 Stockholm, Sweden WWW: http://www.synex.se To: Michael Sperberg-McQueen , W3C SGML Working Group From: Martin Bryan Subject: Re: B.10 Empty elements? Date: Mon, 14 Oct 1996 09:27:45 +0100 At 21:09 12/10/96 +0100, Michael Sperberg-McQueen wrote: >On 16 October 1996, the ERB will vote to decide the following question. >A straw poll indicates the question needs further discussion in the work >group. > >B.10 What form should EMPTY elements take, if there are EMPTY elements >in XML: , (where '/' is assumed the NET string) >(7.3, 11.2.3)? They should follow SGML's rule of a start-tag with no end-tag ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ To: Michael Sperberg-McQueen , W3C SGML Working Group From: Martin Bryan Subject: Re: B.9 Formal system, public identifiers? Date: Mon, 14 Oct 1996 09:27:43 +0100 At 21:08 12/10/96 +0100, Michael Sperberg-McQueen wrote: >On 16 October 1996, the ERB will vote to decide the following question. >A straw poll indicates the question needs further discussion in the work >group. > >B.9 Should XML require system and public identifiers to be FORMAL >(13.5)? This needs interpretation. The FORMAL keyword in SGML does not restrict system identifiers to any formal structure. The important question is: If system identifiers are retained in XML must they conform to the rules for formal system identifiers defines in ISO 10744? ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ To: Michael Sperberg-McQueen , W3C SGML Working Group From: Martin Bryan Subject: Re: B.7 Conditional inclusion in DTDs? Date: Mon, 14 Oct 1996 09:27:40 +0100 At 21:06 12/10/96 +0100, Michael Sperberg-McQueen wrote: >On 16 October 1996, the ERB will vote to decide the following question. >A straw poll indicates the question needs further discussion in the work >group. > >B.7 How should XML deal with the need for conditional inclusion of >markup declarations, if XML has no marked sections (10.4.1)? If XML does not retain marked sections it should not allow for conditional inclusion of markup declarations ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ To: Michael Sperberg-McQueen , W3C SGML Working Group From: Martin Bryan Subject: Re: B.13 ENTITY attributes for notations? Date: Mon, 14 Oct 1996 09:27:48 +0100 At 21:10 12/10/96 +0100, Michael Sperberg-McQueen wrote: >On 16 October 1996, the ERB will vote to decide the following question. >A straw poll indicates the question needs further discussion in the work >group. > >B.13 Should XML remove SGML's prohibition on ENTITY attributes for >notations (11.4.1)? This would make XML incompatible with SGML systems. We need a consistent view on whether or not we are seeking to achieve compatibility. ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ To: Michael Sperberg-McQueen , W3C SGML Working Group From: Martin Bryan Subject: Re: B.8 MSxCHAR markup-suppression? Date: Mon, 14 Oct 1996 09:27:37 +0100 At 21:07 12/10/96 +0100, Michael Sperberg-McQueen wrote: >On 16 October 1996, the ERB will vote to decide the following question. >A straw poll indicates the ERB is leaning to dropping these constructs. > >B.8 Should XML use MSOCHAR, MSSCHAR, and MSICHAR strings (9.7)? > > > MSOCHAR and MSICHAR do not make sense in XML, but should XML need an escaping mechanism for some characters, as has been suggested, it would seem silly to invent something new when MSSCHAR already provides a solution that will be acceptable to full SGML systems. ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ Date: Mon, 14 Oct 1996 07:55:48 -0400 From: gtn@ebt.com (Gavin Nicol) To: U35395@UICVM.CC.UIC.EDU CC: w3c-sgml-wg@w3.org Subject: Re: B.8 MSxCHAR markup-suppression? >B.8 Should XML use MSOCHAR, MSSCHAR, and MSICHAR strings (9.7)? Drop them. Date: Mon, 14 Oct 1996 07:56:59 -0400 From: gtn@ebt.com (Gavin Nicol) To: U35395@UICVM.CC.UIC.EDU CC: w3c-sgml-wg@w3.org Subject: Re: B.9 Formal system, public identifiers? >B.9 Should XML require system and public identifiers to be FORMAL >(13.5)? If this is equivalent to having a canonical grammar by which to parse them, that is extensible, yes. Date: Mon, 14 Oct 1996 07:57:22 -0400 From: gtn@ebt.com (Gavin Nicol) To: U35395@UICVM.CC.UIC.EDU CC: w3c-sgml-wg@w3.org Subject: Re: B.3 Record-end handling? >B.3 How should XML handle RE (7.6.1)? Get rid of it. Date: Mon, 14 Oct 1996 07:58:42 -0400 From: gtn@ebt.com (Gavin Nicol) To: U35395@UICVM.CC.UIC.EDU CC: w3c-sgml-wg@w3.org Subject: Re: B.4 Drop comments, PIs in mixed content? >B.4 Should XML forbid comments and processing instructions in mixed >content, as a way of simplifying RE handling (7.6)? Not necessary if you don't see it. Date: Mon, 14 Oct 1996 07:54:53 -0400 From: gtn@ebt.com (Gavin Nicol) To: U35395@UICVM.CC.UIC.EDU CC: w3c-sgml-wg@w3.org Subject: Re: B.1 Character-set rules? >B.1 What should XML's character-set rules be? Should conforming XML >documents be restricted to particular character sets? Should conforming >XML processors be required to be able to parse all conforming XML >documents (13.1)? In the interest of moving forward: 1) A single coded character set ISO 10646 should be used. 2) The primary encodings (required by all implementations) should be UTF-8 and UCS-2. The may be auto-detected. 3) Encodings other than these are allowed, in which case, an external mechanism must be used to indicate the encoding. Date: Mon, 14 Oct 1996 10:22:58 -0700 (PDT) From: Bill Smith Subject: Re: B.10 Empty elements? To: w3c-sgml-wg@w3.org Eve Maler wrote: > > (2) provide the necessary markup declarations in a "partial DTD" so that > empty elements can be understood: > Requiring partial DTDs is the first step on the slope to requiring DTDs and greatly complicating XML for everyone. How will validating XML parsers distinguish between partial DTDs and full DTDs? How will authors know the difference? Having said that, this approach has merit and I'm undecided on the best way to handle empt elements - other than to say it should be simple, easily understood, and accpetable to the masses. Date: Mon, 14 Oct 1996 09:42:43 -0700 (PDT) From: Bill Smith Subject: Re: B.3 Record-end handling? To: w3c-sgml-wg@w3.org > >B.3 How should XML handle RE (7.6.1)? > > Get rid of it. > Concur... make that agree Date: Mon, 14 Oct 1996 10:33:57 -0700 (PDT) From: Bill Smith Subject: Re: B.11 Empty end-tags? To: w3c-sgml-wg@w3.org Liam Quinn wrote: > I have no objection to in XML as long as there is no OMITTAG, > although it does slightly reduce robustness in the face of transmission > error or user error. This is an important point. The form permits graceful error recovery by an intelligent application. Also, Jon Bosak's comments on ease of processing are compelling. Date: Mon, 14 Oct 1996 10:25:12 -0700 (PDT) From: Bill Smith Subject: Re: B.2 DTD required or optional? To: w3c-sgml-wg@w3.org > > B.2 Should XML require each document instance to have a DTD or not > (7.1)? > I'd rather we forbid them than require them. Date: Mon, 14 Oct 1996 14:39:18 -0400 To: Bill Smith , w3c-sgml-wg@w3.org From: "Eve L. Maler" Subject: Re: B.10 Empty elements? At 10:22 AM 10/14/96 -0700, Bill Smith wrote: >Requiring partial DTDs is the first step on the slope to requiring DTDs and >greatly complicating XML for everyone. How will validating XML parsers >distinguish between partial DTDs and full DTDs? How will authors know the >difference? I don't think full DTDs ever *need* to be required; pretty much all the "read-only" applications (such as browsers) would do just fine if they had the tiny subset of information that includes, e.g., EMPTY elements. You make a really good point about distinguishing partial and full DTDs. Perhaps a PI such as is a good way to handle this; otherwise the DTD would be assumed to be full, and the instance could generate some errors accordingly if some declarations are missing. Eve Date: Mon, 14 Oct 1996 12:28:11 -0700 To: "Eve L. Maler" , lee@sq.com, U35395@UICVM.CC.UIC.EDU, w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: B.10 Empty elements? At 01:39 PM 10/13/96 -0400, Eve L. Maler wrote: >(2) provide the necessary markup declarations in a "partial DTD" so that > empty elements can be understood: > > > >(2b) provide the necessary "instance parameters" rather than whole markup > declarations (which generally provide more info than is needed for > read-only processing functions): > > Sign me up with Eve on (2) - this has the advantages that it's easy to explain, easy to parse, and is consistent with both SGML and HTML thinking. 2b - please, not another syntax for doing the same thing... whether we use DSD or DTD, let's use one thing throughout. The difficulty of parsing (2) as opposed to (2b) is nowhere near greater enough to justify getting into another syntax. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 From: lee@sq.com Date: Mon, 14 Oct 96 15:27:13 EDT To: w3c-sgml-wg@w3.org Subject: partial DTDs [was: Re: B.10 Empty elements?] I really don't like this "partial DTD" idea. (1) as Bill pointed out, you have to deal with distinguishing partial and full DTDs. That includes needing tools for merging changes made to a partial DTD back into a full one. Ugh. (2) if you can handle a partial DTD, you can handle a full one. Having less of something rarely makes the code much simpler! So we would be requiring all XML programs to handle DTDs, with their gruesome (sorry) syntax and arcane restrictions, both syntactic (no comments in model groups) and semantic (ambiguity, only one ID, etc). So we have to decide whether to (A) make every XML program a full SGML application, with a DTD and everything that can go into it (including a man-year of coding!) (B) make every XML program have a parser for a reduced DTD grammar, such as the one Charles proposed (and that for some insane reason I turned into a BNF grammar) (C) throw away the whole kit and bundle and have no DTD (D) if there is no SGML DTD, consider the possibility of an SGML representation of the structured information and documentation that are required to make a DTD usable, whether or not there is in fact a formal DTD. I would very much like to see a way of interchanging natural language element and attribute descriptions, for example. Style sheet editors are hard to use if you think TITLE refers to the book's title, and don't realise it is supposed to contain Dr., Earl, Lord, etc.... A prototype version of HoTMetaL had "Author" as the description for the element! (the person who did that has since been promoted, don't worry) It would also be useful to support something like HyLex, to give a regular expression that textual data must match -- but with a POSIX regular expression, *please*, so I can implement it in an hour instead of a week if I have to write an NDFA matcher instead of using Henry's libregexp!!! But since neither element descriptions nor regular expressions for text or attributes are part of SGML, there's no need to put them in a DTD. Another approach, then, might be SGML syntax with optional constraints where the constraints can be expressed simply, perhaps even in an XML subset, and documents can be parsed (in the computer science sense) without the constraints, and further checked (more like SGML validation) with them. Without a DTD, though, you have * no distinction between element context and mixed content * no endtagless EMPTY elements * no CONREF (already voted no on that, I think) * default attribute values must be a program convention (like #IMPLIED, although giving a default in a DTD may still be useful to humans, and in keeping software up to date) Note: I have avoided the term "XML application" since "SGML application" is used to mean both a DTD and a piece of software... Lee Date: Mon, 14 Oct 96 14:44:12 CDT From: paul@arbortext.com (Paul Grosso) To: w3c-sgml-wg@w3.org Subject: Re: B.10 Empty elements? > From w3c-sgml-wg-request@w3.org Mon Oct 14 14:21:16 1996 > Date: Mon, 14 Oct 1996 14:39:18 -0400 > To: Bill Smith , w3c-sgml-wg@w3.org > From: "Eve L. Maler" > Subject: Re: B.10 Empty elements? > Resent-From: w3c-sgml-wg@w3.org > Sender: w3c-sgml-wg-request@w3.org > > At 10:22 AM 10/14/96 -0700, Bill Smith wrote: > >Requiring partial DTDs is the first step on the slope to requiring DTDs and > >greatly complicating XML for everyone. How will validating XML parsers > >distinguish between partial DTDs and full DTDs? How will authors know the > >difference? > > I don't think full DTDs ever *need* to be required; pretty much all the > "read-only" applications (such as browsers) would do just fine if they > had the tiny subset of information that includes, e.g., EMPTY elements. > > You make a really good point about distinguishing partial and full DTDs. > Perhaps a PI such as is a good way to handle this; > otherwise the DTD would be assumed to be full, and the instance could > generate some errors accordingly if some declarations are missing. Another way would be to say that XML doesn't require full DTDs. Period. Without any way to say whether a DTD is full or partial. Basically, the application can opt to make whatever use it wants of whatever DTDs it's given. If the application comes across an element for which there is no declaration in the DTD, it does whatever it is able to do rather than necessarily issuing an error message. This way, someone could send what they think is a "full" DTD, an then if there is an extra element, the application could still work. Better yet, if the user of the application decides to extend the language (the one that is defined in the DTD that exists) and add a new element, they can do that if the application supports that. If we require that all DTDs be marked as full or partial, then what was a full DTD given one version of the instance can't automatically become a partial DTD for an extended version of the instance. That is, for XML, all DTDs become "possibly partial" where both an omitted DTD and a full DTD are just 'degenerate' cases. Tying this back to empty elements, if the instance uses no empty elements, a read-only application may well be able to get away with no DTD. If there are empty elements, perhaps only those declarations need to be included and sent in the partial DTD. Date: Mon, 14 Oct 1996 13:47:46 -0700 To: lee@sq.com, w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: partial DTDs [was: Re: B.10 Empty elements?] At 03:27 PM 10/14/96 EDT, lee@sq.com wrote: >I really don't like this "partial DTD" idea. I disagree with Lee comprehensively on this. We haven't considered carefully what exactly is allowed a DTD-less subset yet, but the following at least are obviously useful, and do not require any complex validation machinery:

    elements were within PGBRK. Of course, it could simply have been a user error... If empty elements were marked syntactically, e.g. <@PGBRK> then there would be no problem. This can be done by allowind @ as a name start character, and then saying that in XML, empty elements have names starting with @. If SGML could be augmented to allow a different open tag delimiter for EMPTY elements, it could use <@, and the entire problem would vanish. Lee Date: Mon, 14 Oct 1996 18:21:46 -0700 (PDT) From: Bill Smith Subject: Re: B.10 Empty elements? To: w3c-sgml-wg@w3.org Len Bullard wrote: > 3. Is the processing time severe for the case you state? > I realize this question has many hands to argue with. While the average case time may not be "severe", the worst case behavior may be and therefor cannot be ignored. If an empty element is inserted high in a document instance (say an within a high-level

    in HTML 3.2), the emptiness of cannot be inferred until the enclosing element is closed - or the parser performs lookahead. Either way, processing is delayed and application complexity increases. I might trade speed for complexity but I'd hate to lose speed while increasing complexity. Bad tradeoff. Date: Mon, 14 Oct 1996 20:39:29 -0500 From: Len Bullard To: Bill Smith CC: w3c-sgml-wg@w3.org Subject: Re: B.10 Empty elements? Bill Smith wrote: > > Len Bullard wrote: > > > 3. Is the processing time severe for the case you state? > > I realize this question has many hands to argue with. > > While the average case time may not be "severe", the worst case behavior may be > and therefor cannot be ignored. > > If an empty element is inserted high in a document instance (say an within a > high-level
    in HTML 3.2), the emptiness of cannot be inferred until the > enclosing element is closed - or the parser performs lookahead. Either way, > processing is delayed and application complexity increases. > > I might trade speed for complexity but I'd hate to lose speed while increasing > complexity. Bad tradeoff. Agreed that is a bad tradeoff. Forcing lookahead isn't good, and maintaining a stack seems to be undesirable for the PERL hacker. It appears though, that this is still a case where the absence of the DTD bites, and perhaps the is the best tradeoff. Thanks for making it clear, Bill and Lee. Get ready to answer this same question a few hundred times a year. No matter how we explain it, the looks redundant for an EMPTY element and a lot of SGML hackers are taught not to do it. It will be a tough habit to break because from the author's perspective, not the parser programmer, it looks like YetAnotherReasonSGMLIsUgly. Oh well, my drummer is ugly but he has good meter. len From: lee@sq.com Date: Mon, 14 Oct 96 22:38:26 EDT To: w3c-sgml-wg@w3.org Subject: Re: B.10 Empty elements? > Get ready to answer this same question a few hundred times > a year. No matter how we explain it, the looks redundant > for an EMPTY element and a lot of SGML hackers are taught not to > do it. It will be a tough habit to break because from the > author's perspective, not the parser programmer, it looks like > YetAnotherReasonSGMLIsUgly. Which is why I prefer something like or <@br> both of which are either legal with RCS or can be made legal with a small change to the SGML declaration. Sorry for beating a drum until it breaks... but I have lousy metre :-) Lee Date: Mon, 14 Oct 1996 23:44:33 -0400 To: w3c-sgml-wg@w3.org From: Arjun Ray Subject: Re: B.11 Empty end-tags? At 10:33 AM 10/14/96 -0700, Bill Smith wrote: >Liam Quinn wrote: > >> I have no objection to in XML as long as there is no OMITTAG, >> although it does slightly reduce robustness in the face of transmission >> error or user error. > >This is an important point. The form permits graceful error recovery by >an intelligent application. Unless the XML spec will have normative statements regarding error recovery, anything better than a coredump is graceful enough, IMHO. I'd like to repeat my earlier comments: - A popular language *will* have untutored/unsophisticated users. - It will take just one implementation modeled on Mosaic 2.4 to make tag salad an ugly reality, *especially* if "graceful recovery" becomes an *expected* feature. >Also, Jon Bosak's comments on ease of processing are compelling. The more general question is how much can the DPH expect to accomplish with regular expressions alone? 1. In general, stacks (or at a pinch, counters) will always be necessary: SGML/XML markup is essentially a Dyck language, which is homomorphic to the "hardest" CF language. Besides, one of the goals of XML is easy implementations of parsers. To the extent this becomes a reality, the DPH will have that much less work to do anyway -- somebody will have written a parser package:-) 2. Maximizing the reach of regular expressions involves a tradeoff between ease of use for the author (relatively freeform input) and ease of parsing. For example, parsing nroff files is basically trivial, but the format isn't very user-friendly. There's nothing to stop the DPH's talents being applied to an intermediate (or canonical) form with such properties instead of the "source". Regards, Arjun Date: Tue, 15 Oct 1996 01:45:32 -0500 To: cbullard@HiWAAY.net From: Robert Streich Subject: Re: B.10 Empty elements? Cc: W3C SGML Working Group At 06:27 PM 10/14/96 -0500, you wrote: >I disagree. To me, particularly when editing manually, this looks like >an insertion point for text or other efforts. I agree wholeheartedly with this. In one SGML editor that didn't have a reasonable representation for empty elements, I had to put something in the element via the stylesheet just to eliminate the confusion. bob Robert Streich streich@slb.com Schlumberger voice: 1 512 331 3318 Austin Research fax: 1 512 331 3760 Date: Tue, 15 Oct 1996 01:46:07 -0500 To: Tim Bray From: Robert Streich Subject: Re: B.2 DTD required or optional? Cc: W3C SGML Working Group At 01:47 PM 10/14/96 -0700, Tim Bray wrote: >At 03:05 PM 10/12/96 CDT, Michael Sperberg-McQueen wrote: > >>B.2 Should XML require each document instance to have a DTD or not >>(7.1)? > >One concept is that the XML spec should define two separate characteristics >that a document can have, well-formedness and validity. I agree with something along the lines of Tim's proposal. SGML requires a DTD, yet it hasn't stopped anyone from processing documents without a DTD. SGML addressed this by differentiating validating and non- validating parsers. I think we should use a similar approach. bob Robert Streich streich@slb.com Schlumberger voice: 1 512 331 3318 Austin Research fax: 1 512 331 3760 Date: Tue, 15 Oct 1996 01:46:12 -0500 To: Charles@sgmlsource.com From: Robert Streich Subject: Re: B.9 Formal system, public identifiers? Cc: Michael Sperberg-McQueen , W3C SGML Working Group At 10:42 PM 10/14/96 GMT, Charles F. Goldfarb wrote: >On Sat, 12 Oct 96 15:08:24 CDT, Michael Sperberg-McQueen > wrote: >>B.9 Should XML require system and public identifiers to be FORMAL >>(13.5)? > >Yes. XML should use formal public identifiers (per ISO 8879) and formal system >identifiers (per SGML Extended Facilities of ISO/IEC 10744). I agree with Charles. bob Robert Streich streich@slb.com Schlumberger voice: 1 512 331 3318 Austin Research fax: 1 512 331 3760 Date: Tue, 15 Oct 1996 01:44:45 -0500 To: cbullard@HiWAAY.net From: Robert Streich Subject: Re: B.11 Empty end-tags? Cc: Jon Bosak , w3c-sgml-wg@w3.org At 09:35 AM 10/13/96 -0500, Len Bullard wrote: >Jon Bosak wrote: > >[snip] > >> This may not >> sound like much, but to a lot of people in the trenches, especially >> people facing deadlines, it's all the difference in the world. I >> suspect that the same is true for people who build cgi-bin scripts and >> similar one-off lightweight processors. >> >> Jon > >I concur with Jon. Whatever space is saved by the empty end tag, it >complicates the use of simple rewrite scripts and it makes it much >harder >to read the instance. Simpler scripts are important to making XML >usable. If there are no technical reasons for excluding them, then I don't see why we should. Whether or not an author chooses to use them is the decision that the author will make dependent on the situation. I'd hate to see us get into the business of protecting people from themselves. bob Robert Streich streich@slb.com Schlumberger voice: 1 512 331 3318 Austin Research fax: 1 512 331 3760 To: w3c-sgml-wg@w3.org From: Martin Bryan Subject: Re: B.10 Empty elements? Date: Tue, 15 Oct 1996 07:43:22 +0100 .. No matter how we explain it, the looks redundant >for an EMPTY element and >If empty elements were marked syntactically, e.g. ><@PGBRK> >then there would be no problem. The first solution cannot be used as input to a standard SGML tool, so would require a specialised XML tool to edit the document. The second solution could be handled by any SGML tool that allowed extension to the name characacter definition. I would argue that this provides a strong case for selecting the second solution. ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ Date: Mon, 14 Oct 1996 23:59:54 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: streich@slb.com CC: cbullard@HiWAAY.net, w3c-sgml-wg@w3.org Subject: Re: B.11 Empty end-tags? | At 09:35 AM 10/13/96 -0500, Len Bullard wrote: | >Jon Bosak wrote: | > | >[snip] | > | >> This may not | >> sound like much, but to a lot of people in the trenches, especially | >> people facing deadlines, it's all the difference in the world. I | >> suspect that the same is true for people who build cgi-bin scripts and | >> similar one-off lightweight processors. | >> | >> Jon | > | >I concur with Jon. Whatever space is saved by the empty end tag, it | >complicates the use of simple rewrite scripts and it makes it much | >harder | >to read the instance. Simpler scripts are important to making XML | >usable. | | If there are no technical reasons for excluding them, then I don't see | why we should. Whether or not an author chooses to use them is the | decision that the author will make dependent on the situation. I'd hate | to see us get into the business of protecting people from themselves. Not from themselves; from each other. A world in which I am guaranteed that any XML data from any source will have no empty end tags is a different world from one in which I have no such guarantee. Jon From: "Henry S. Thompson" Date: Tue, 15 Oct 96 10:00:45 BST To: Tim Bray Subject: Re: B.2 DTD required or optional? Cc: W3C SGML Working Group Tim Bray writes > One concept is that the XML spec should define two separate characteristics > that a document can have, well-formedness and validity. > > A. A "Well-Formed XML Document" has > > 1. syntactically distinguishable elements, attributes, comments, and PI's > 2. one root element > 3. all elements properly nested within each other > 4. no text outside the root element > > Such a document may or may not contain markup declarations. If it does, > well-formedness also allows > > 5. empty elements, if they're declared > 6. entity references, if they're declared > > B. A "Valid XML document" means there's a full DTD and it's been validated > in the familiar SGML sense. > I think this is the right approach, with a moderate extension of the list of properties of well-formedness, to allow for defaulted and CONREF attributes, possibly mixed content model (note this is challenge to the 'just use the normal declaration' strategy, although I suppose would do.), possibly other things. To recapitulate something I said a while back, if the declarations are NOT there, a conforming application can do whatever it likes, e.g. with respect to empty elements (which can and should on this account remain as simply ) report an error after treating the element as normal, but if the declarations ARE there a conforming application must make use of them. What this amounts to for specification/documentation purposes is identifying a core and a set of (independent) extensions. The core is unequivocally processable without any declarations, but for each extension type there is a required accompanying declaration type. Oh yes, I agree with Martin that the single root requirement is too strong, all we need is 'all text in elements'. ht I like the idea of propaganda in sigs -- here's mine: Reference concrete syntax means just that: NO NEW SYNTAX Date: Tue, 15 Oct 1996 12:02:16 +0000 To: Bill Smith From: James Clark Subject: Re: B.10 Empty elements? Cc: w3c-sgml-wg@w3.org At 18:21 14/10/96 -0700, Bill Smith wrote: >Len Bullard wrote: > >> 3. Is the processing time severe for the case you state? >> I realize this question has many hands to argue with. > >While the average case time may not be "severe", the worst case behavior may be >and therefor cannot be ignored. > >If an empty element is inserted high in a document instance (say an within a >high-level
    in HTML 3.2), the emptiness of cannot be inferred until the >enclosing element is closed - or the parser performs lookahead. Either way, >processing is delayed and application complexity increases. Isn't the problem even worse than this? You don't just to figure out that empty elements are in fact empty, you also have to figure out that non-empty elements are not in fact empty. The first time I see a chapter tag, I can't tell that it is not in fact an empty tag until I see its close tag. So either I can't start displaying the chapter until I have got the whole chapter or I have to assume initially that every tag is non-empty and be able to go back and reformat when I discover one that's not. This just is not going to work. James Date: Tue, 15 Oct 1996 07:57:07 -0500 From: Len Bullard To: James Clark CC: Bill Smith , w3c-sgml-wg@w3.org Subject: Re: B.10 Empty elements? James Clark wrote: > > At 18:21 14/10/96 -0700, Bill Smith wrote: > >Len Bullard wrote: > > > >> 3. Is the processing time severe for the case you state? > >> I realize this question has many hands to argue with. > > > >While the average case time may not be "severe", the worst case behavior > may be > >and therefor cannot be ignored. > > > >If an empty element is inserted high in a document instance (say an > within a > >high-level
    in HTML 3.2), the emptiness of cannot be inferred > until the > >enclosing element is closed - or the parser performs lookahead. Either way, > >processing is delayed and application complexity increases. > > Isn't the problem even worse than this? You don't just to figure out that > empty elements are in fact empty, you also have to figure out that non-empty > elements are not in fact empty. The first time I see a chapter tag, I can't > tell that it is not in fact an empty tag until I see its close tag. So > either I can't start displaying the chapter until I have got the whole > chapter or I have to assume initially that every tag is non-empty and be > able to go back and reformat when I discover one that's not. This just is > not going to work. > > James But it is working. It requires the DTD and in cases for systems that already do SGML as hypermedia, it has required a stylesheet where a DTD was not used. I see the point of avoiding lookahead that it adds complexity, but in fact, empty elements are used successfully and they work. In the cases I'm thinking about, IDE/AS and IADS they don't format as they load, but many of these documents are considerably bigger than the average Web page and the performance is acceptable. I think you overstate. The looks worse than <@e>. It is consistent, but intuitively wrong. As I said, do as you will, but be prepared to explain it repetitively. len Date: Tue, 15 Oct 1996 14:07:44 +0000 To: cbullard@HiWAAY.net From: James Clark Subject: Re: B.10 Empty elements? Cc: Bill Smith , w3c-sgml-wg@w3.org At 07:57 15/10/96 -0500, Len Bullard wrote: >James Clark wrote: >> >> At 18:21 14/10/96 -0700, Bill Smith wrote: >> >Len Bullard wrote: >> > >> >> 3. Is the processing time severe for the case you state? >> >> I realize this question has many hands to argue with. >> > >> >While the average case time may not be "severe", the worst case behavior >> may be >> >and therefor cannot be ignored. >> > >> >If an empty element is inserted high in a document instance (say an >> within a >> >high-level
    in HTML 3.2), the emptiness of cannot be inferred >> until the >> >enclosing element is closed - or the parser performs lookahead. Either way, >> >processing is delayed and application complexity increases. >> >> Isn't the problem even worse than this? You don't just to figure out that >> empty elements are in fact empty, you also have to figure out that non-empty >> elements are not in fact empty. The first time I see a chapter tag, I can't >> tell that it is not in fact an empty tag until I see its close tag. So >> either I can't start displaying the chapter until I have got the whole >> chapter or I have to assume initially that every tag is non-empty and be >> able to go back and reformat when I discover one that's not. This just is >> not going to work. >> >> James > >But it is working. I meant that neither of the two possibilities in the penultimate sentence were viable: if you are guaranteed to have a DTD (or at least a partial one) there's no problem using for empty elements; if you are happy not to have the document display as it loads, there's also no problem. James Date: Tue, 15 Oct 1996 09:43:57 -0400 To: cbullard@HiWAAY.net, Charles@sgmlsource.com From: "Eve L. Maler" Subject: Re: B.7 Conditional inclusion in DTDs? Cc: Michael Sperberg-McQueen , W3C SGML Working Group At 07:14 PM 10/14/96 -0500, Len Bullard wrote: >Charles F. Goldfarb wrote: >> >> On Sat, 12 Oct 96 15:06:59 CDT, Michael Sperberg-McQueen >> wrote: >> >> >On 16 October 1996, the ERB will vote to decide the following question. >> >A straw poll indicates the question needs further discussion in the work >> >group. >> > >> >B.7 How should XML deal with the need for conditional inclusion of >> >markup declarations, if XML has no marked sections (10.4.1)? >> >> XML shouldn't have conditional DTDs. They defeat the objective of simplicity. >> Those who need complex DTD management will be using full SGML and can easily >> generate XML from it. >> -- >> Charles F. Goldfarb > >Also concur. As do I (I think). Presenting DTDs to new XML users will be easier if we constrain them (the DTDs :-) to single "state spaces" for now. >I would personally like to see reduced complexity in >DTD design practice. Right now, I am creating style sheets. >The complexity of the conditional DTDs I am working with is absurd. >Modularization by parameter entity, conditionals, inclusions, >exclusions, just to hit the obvious ones, make the state space >of these DTDs bigger than Pluto's Orbit. While they are legal, >they make any menu or dialog system generated from them almost >worthless as a way to guide an author. They would be much >better in production if broken up into multiple DTDs. This >is a design habit I think XML can promote. The next >generation of DTDs would not give one the uncomfortable >feeling of walking in the temples of ancient Mexico where >one can admire the intricacy, but despair at the blood shed >to stack rocks that high. Authors are paying a high >price for the convenience of DTD version management which >is really not that convenient and not done that often. The >rate of change isn't that high. While not a technical >concern, promoting simplification for the sake of >effective production is a worthy goal. As one of the perpetrators of outrageously parameterized DTDs, I have to disagree. Most DTDs don't need and shouldn't use parameterization on the level of a DocBook or TEI, but any DTD with a very diverse user base typically needs customization in order to be useful. (Many more people cannibalize DocBook than use it out of the box.) Since these audiences don't share any particular DTD management system, the ability to customize is needed in 8879 form. And maintenance truly is a nightmare if you're maintaining both your changed portions and the original portions, which you need to do whenever the base DTD gets an upgrade (as DTDs with huge audiences tend to do regularly). Plus, a DTD customization layer provides a machine-readable (and relatively human-readable) specification of precisely how your markup model differs from the base one. DTD analysis tools are popular and available enough to help people wend their way through forests of parameter entities, to the point where DTD readability is much less of a concern (though I still feel that readability should be an important *secondary* goal of a raw DTD). Finally, in any one "reading" of a parameterized DTD, the markup model is stable; in my experience, authors pay no extra price for DTDs that are parameterized. In fact, the ability to rip out the elements that are irrelevant to one's own situation has meant that DocBook (for one) has become quite a bit *more* usable. The management of SGML documents and associated tools (including DTDs) is still in its infancy. I believe in using the 8879 techniques at my disposal until a portable, functional way of doing the same thing arises. Eventually, the XML user base should be let in on these methods as well (whatever they are at the time). Eve Date: Tue, 15 Oct 1996 14:58:42 +0000 To: lee@sq.com From: James Clark Subject: Re: B.10 Empty elements? Cc: w3c-sgml-wg@w3.org At 22:38 14/10/96 EDT, lee@sq.com wrote: >> Get ready to answer this same question a few hundred times >> a year. No matter how we explain it, the looks redundant >> for an EMPTY element and a lot of SGML hackers are taught not to >> do it. It will be a tough habit to break because from the >> author's perspective, not the parser programmer, it looks like >> YetAnotherReasonSGMLIsUgly. > >Which is why I prefer something like > >or ><@br> >both of which are either legal with RCS or can be made legal with a >small change to the SGML declaration. Another variation on this theme is to require empty elements to be written as >From the SGML point of view we would be making the NET delimiter "/>" and requiring the start-tags of all empty elements to be net-enabling start-tags. In SGML '97 this could be a distinct delimiter. James Date: Tue, 15 Oct 1996 14:58:45 +0000 To: W3C SGML Working Group From: James Clark Subject: Re: B.9 Formal system, public identifiers? >B.9 Should XML require system and public identifiers to be FORMAL >(13.5)? For XML 1.0, we need to keep things as simple as possible. The simplest solution is: - No public identifiers, formal or otherwise. - System identifiers must be URLs. I don't think it's part of our job in this WG to try and solve the WWW's naming/addressing problems. The mechanism that the WWW currently uses for identifying entities is URLs. We should simply adopt that into XML. If we incorporate public identifiers, we would need to define a mechanism for resolving them to URLs. There is not time to do that before November. They can always be added in XML 1.1. I don't think formal system identifiers buy us any functionality that we have to have in XML 1.0. They can be added in a future version; this would be backwards compatible since any legal formal system identifier is not a legal URL. James Date: Tue, 15 Oct 1996 11:16:10 -0400 From: gtn@ebt.com (Gavin Nicol) To: jjc@jclark.com CC: w3c-sgml-wg@w3.org Subject: Re: B.9 Formal system, public identifiers? >For XML 1.0, we need to keep things as simple as possible. The simplest >solution is: > >- No public identifiers, formal or otherwise. > >- System identifiers must be URLs. I disagree. Using FPI's and FSI's for naming is simple enough. The really hard part is name *resolution* (ie. turning a name into a physical adress that can be accessed). I agree that we cannot solve the resolution side by November, but XML naming conventions certainly can be resolved by then. Again, if we have an extensible naming scheme with a robust grammer, then I am all for it. Date: Tue, 15 Oct 1996 10:58:01 -0500 From: Len Bullard To: "Eve L. Maler" CC: Charles@sgmlsource.com, Michael Sperberg-McQueen , W3C SGML Working Group Subject: Re: B.7 Conditional inclusion in DTDs? Eve L. Maler wrote: > > As one of the perpetrators of outrageously parameterized DTDs, > I have to disagree. Most DTDs don't need and shouldn't use > parameterization on the level of a DocBook or TEI, but any DTD > with a very diverse user base typically needs customization in > order to be useful. (Many more people cannibalize DocBook than > use it out of the box.) We have to agree to disagree and then agree. As someone sitting here at the moment with a requirement to write a stylesheet for Docbook, as beautiful a piece of SGML as it is, it is awfully hard to tell the user of for example, Author/Editor why some of these combinations are legal. But, yes, it can be subset and that is the typical solution. Two years ago we brought up the use of subsets and provable subsets on CTS for this reason. > Since these audiences don't share any particular DTD management > system, the ability to customize is needed in 8879 form. And > maintenance truly is a nightmare if you're maintaining both your > changed portions and the original portions, which you need to do > whenever the base DTD gets an upgrade (as DTDs with huge audiences > tend to do regularly). Plus, a DTD customization layer provides a > machine-readable (and relatively human-readable) specification of > precisely how your markup model differs from the base one. What I would trade is the nightmare of the DTD maintainer for the production speed and quality of the author. DTDs with huge audiences can usually break up the audience into smaller domains. > DTD analysis tools are popular and available enough to help people > wend their way through forests of parameter entities, to the point > where DTD readability is much less of a concern (though I still feel > that readability should be an important *secondary* goal of a raw DTD). I agree, but readability isn't my problem, after the first hour. It is using a relatively cheap seat and employee to produce a quality result by constraining and directing their effort via the DTD. > Finally, in any one "reading" of a parameterized DTD, the markup > model is stable; in my experience, authors pay no extra price for > DTDs that are parameterized. In fact, the ability to rip out the > elements that are irrelevant to one's own situation has meant that > DocBook (for one) has become quite a bit *more* usable. Parameterization is not at issue although it can contribute to the problem if one parameterizes prior to understanding the root-to-leaf branches, or simpler, the *engine* productions where actual content is inserted such that absurd productions don't occur such as footnotes in titles or some such. What tends to happen in the large DTDs whose domains are unconstrained is simply too many potential choices. Then the cost of the employee goes up as the requirement for deep content knowledge increases. This is tough to justify in conversion work, and often, in technical writing where the actual content is generated upstream, but there is no configuration management, or database. > The management of SGML documents and associated tools (including > DTDs) is still in its infancy. I believe in using the 8879 techniques > at my disposal until a portable, functional way of doing the same > thing arises. Eventually, the XML user base should be let in on > these methods as well (whatever they are at the time). As long as we agree that to focus the production effort, subsets or cannibalization might be required, I think we agree. We probably should move this to CTS. Len Bullard Date: Tue, 15 Oct 1996 10:44:53 -0500 From: Len Bullard To: James Clark CC: w3c-sgml-wg@w3.org Subject: Re: B.10 Empty elements? James Clark wrote: > > At 07:57 15/10/96 -0500, Len Bullard wrote: > >James Clark wrote: > >> > >> At 18:21 14/10/96 -0700, Bill Smith wrote: > >> >Len Bullard wrote: > >> > > >> >> 3. Is the processing time severe for the case you state? > >> >> I realize this question has many hands to argue with. > >> > > >> >While the average case time may not be "severe", the worst case behavior > >> may be > >> >and therefor cannot be ignored. > >> > > >> >If an empty element is inserted high in a document instance (say an > >> within a > >> >high-level
    in HTML 3.2), the emptiness of cannot be inferred > >> until the > >> >enclosing element is closed - or the parser performs lookahead. Either way, > >> >processing is delayed and application complexity increases. > >> > >> Isn't the problem even worse than this? You don't just to figure out that > >> empty elements are in fact empty, you also have to figure out that non-empty > >> elements are not in fact empty. The first time I see a chapter tag, I can't > >> tell that it is not in fact an empty tag until I see its close tag. So > >> either I can't start displaying the chapter until I have got the whole > >> chapter or I have to assume initially that every tag is non-empty and be > >> able to go back and reformat when I discover one that's not. This just is > >> not going to work. > >> > >> James > > > >But it is working. > > I meant that neither of the two possibilities in the penultimate sentence > were viable: if you are guaranteed to have a DTD (or at least a partial one) > there's no problem using for empty elements; if you are happy not to > have the document display as it loads, there's also no problem. > > James I buy that. Which do you prefer? or <@e> The <@e> solution is attractive to me because it is easy to explain, easy to use, and meets the rule of thumb Dr. Goldfarb mentions that different features need different syntax. I can live with either, it's just easier to explain <@e> because most SGML hackers have stumbled over empty elements and know there is a problem there. Like mixed content, just say, "oh that solves the mixed content fubar" and they will be happy. Otherwise, we say, "you use empty elements, you need a DTD", or, "please get used to this syntax convention that was proposed to make your production simpler, and the cost of your tools less?" len Date: Tue, 15 Oct 1996 09:25:26 -0700 (PDT) From: Bill Smith Subject: Re: B.11 Empty end-tags? To: w3c-sgml-wg@w3.org Arjun Ray wrote: > Unless the XML spec will have normative statements regarding error recovery, > anything better than a coredump is graceful enough, IMHO. I'd like to repeat > my earlier comments: > > - A popular language *will* have untutored/unsophisticated users. > > - It will take just one implementation modeled on Mosaic 2.4 to > make tag salad an ugly reality, *especially* if "graceful > recovery" becomes an *expected* feature. Let application developers decide how to handle error recovery. Some will opt for graceful recovery, whether as a result of "untutored/unsophisticated users" or some other "failure" like transmission errors. Others will opt for a coredump - like some 30% of Unix utilities. I don't think we should specify the expected behavior. We should take into consideration whether robust applications can be developed around XML and part of that is graceful error recovery. My point on is that it enables an application to recover from what is likely to frequently occur in the input stream. Date: Tue, 15 Oct 1996 09:08:03 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org CC: bosak@atlantic-83.eng.sun.com Subject: Re: B.9 Formal system, public identifiers? [Gavin, beginning with a quote from James:] | >For XML 1.0, we need to keep things as simple as possible. The simplest | >solution is: | > | >- No public identifiers, formal or otherwise. | > | >- System identifiers must be URLs. | | I disagree. Using FPI's and FSI's for naming is simple enough. | The really hard part is name *resolution* (ie. turning a name | into a physical adress that can be accessed). | | I agree that we cannot solve the resolution side by November, but | XML naming conventions certainly can be resolved by then. Again, | if we have an extensible naming scheme with a robust grammer, then | I am all for it. Putting on a third hat this time... (Note that I am now speaking as one with a vested interest in this question.) In the general scales-to-the-edges-of-the-galaxy scenario in the context of which this is usually discussed, the resolution problem is a well-known nightmare. In the more limited case where one is only interested in defining location-independent identifiers for a few thousand publications and only needs to resolve those identifiers within a corporate intranet space, however, the resolution problem is a pussycat. At Sun, we are putting into place an intranet publishing solution that relies on FPIs and the SGML Open catalog system (including Tauber's DELEGATE extension) to provide location-independent naming of our technical manuals and other publications. This system allows users to install individual publications in multiple locations on a network without worrying about what happens if one of the servers goes offline for a while, and we believe that it will provide very substantial document management benefits moving forward. We don't expect everyone to follow our lead in this, but it would be a good thing for us to have FPIs as they are currently defined be part of what you can legally put in an XML document. So this (not disinterested) user strongly supports Gavin's position. Jon Date: Tue, 15 Oct 1996 10:01:29 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: B.10 Empty elements? A survey of the options: - easy to understand. But of course then we can't retain the EMPTY 8879 declaration in XML, thus making XML<->SGML round-tripping difficult. Also I agree that it looks like a place where data ought to be put. Also, I think that XML ought to be able to deal with the following common formulation: ; call this HTML interoperability. <@e> and - requires annoying extra markup declarations. On the other hand, allows full SGML and HTML interoperability. Also, it is A Good Thing to make people start thinking about markup declarations. I think that SGML and HTML interoperability are worth a whole lot of elegance and cleverness. So it's for me. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Tue, 15 Oct 1996 18:24:55 +0000 To: w3c-sgml-wg@w3.org From: James Clark Subject: Re: B.9 Formal system, public identifiers? At 11:16 15/10/96 -0400, Gavin Nicol wrote: >>For XML 1.0, we need to keep things as simple as possible. The simplest >>solution is: >> >>- No public identifiers, formal or otherwise. >> >>- System identifiers must be URLs. > >I disagree. Using FPI's and FSI's for naming is simple enough. I just counted the number of lines of code in the part of SP that handles external entities; I included just the URL and OSFILE storage mangers, and just the UTF-8 and Unicode encodings. It was more than 5000 lines. I don't see how we can possibly meet our goals for ease of implementation if this part of XML has this level of complexity. Don't get me wrong: I love FPIs and FSIs. But I strongly believe this whole XML exercise is going to be a total waste of time unless we exercise extreme self-restraint in the features we put in XML, especially in version 1.0. James Date: Tue, 15 Oct 1996 10:57:18 -0700 (PDT) From: Bill Smith Subject: Re: B.10 Empty elements? To: w3c-sgml-wg@w3.org Following up on Tim's option survey: Can (should) we allow both <@e> or and with markup declarations? is my strong preference but only if we specify that the required markup declarations be part of the document instance. Date: Tue, 15 Oct 1996 12:17:22 -0400 From: Derek Denny-Brown To: w3c-sgml-wg@w3.org James Clark on Tue, 15 Oct 1996 14:58:45 +0000) Subject: Re: B.9 Formal system, public identifiers? > From: James Clark > >B.9 Should XML require system and public identifiers to be FORMAL > >(13.5)? > > For XML 1.0, we need to keep things as simple as possible. The simplest > solution is: > > - No public identifiers, formal or otherwise. > > - System identifiers must be URLs. I agree that this is a usefull immediate solution. One extension to URLs that I believe that XML would need is a means to refer to other parts of a MIME multipart message. To my knowledge, there exists no current URL formalism for this. Such a thing would be usefull beyond XML, so this might be become proposal for extending URLs (in general) to support MIME multipart messages. Or XML could define its own syntax of referencing parts (by their id)... -enjoy, derek --------------------------------------------------------------------------- Derek Denny-Brown | Technical Staff @ TechnoTeacher, Inc. http://www.techno.com/~derdb/ | work-phone: (716) 389-0963 SGML/HyTime/DSSSL/WWW | http://www.techno.com/ Date: Tue, 15 Oct 1996 11:14:51 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: B.9 Formal system, public identifiers? At 06:24 PM 10/15/96 +0000, James Clark wrote: >I just counted the number of lines of code in the part of SP that handles >external entities; I included just the URL and OSFILE storage mangers, and >just the UTF-8 and Unicode encodings. It was more than 5000 lines. So this was just fSi, not fPi? Interesting; and a powerful argument. Would you have any insights as to what the "tough bits" are that make this so hard? It seems on the face of it that FSI handlers for URLs and OSFiles ought to be pretty easy; would it be possible to contemplate cutting a lot of the options out, so that all you had left was something along the lines of [yeah, I know we don't have CDATA in XML] This at least has the virtue that (a) it slips into the base of what SGML offers, and (b) allows room for clean extensibility. Until I read James' posting, I was ready to lose FPI's (& PUBLIC for that matter) and keep FSI's. I'd still like to keep at least a subset of the nice clean FSI syntax. James? Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Tue, 15 Oct 1996 11:30:23 -0700 To: Bill Smith , w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: B.10 Empty elements? At 10:57 AM 10/15/96 -0700, Bill Smith wrote: > is my strong preference but only if we specify that the required markup >declarations be part of the document instance. This would normally be the case when there was no DTD... but you'd insist that even if there was, they be in the subset, not the DTD itself? -Tim Date: Tue, 15 Oct 1996 19:39:23 +0000 To: Tim Bray From: James Clark Subject: Re: B.9 Formal system, public identifiers? Cc: w3c-sgml-wg@w3.org At 11:14 15/10/96 -0700, Tim Bray wrote: >I'd still like to keep at least a subset >of the nice clean FSI syntax. What is the functionality you want from FSIs that you can't get from URLs? James Date: Tue, 15 Oct 96 13:43:25 CDT From: Michael Sperberg-McQueen Subject: Re: B.10 Empty elements? To: W3C SGML Working Group On Tue, 15 Oct 1996 03:10:23 -0400 Martin Bryan said: >.. No matter how we explain it, the looks redundant >>for an EMPTY element > >and > >>If empty elements were marked syntactically, e.g. >><@PGBRK> >>then there would be no problem. > >The first solution cannot be used as input to a standard SGML tool, >so would require a specialised XML tool to edit the document. The Not so: an element declared using an Elephant's Child type declaration can certainly take this form in SGML. The rule would simply be that elements declared empty in an XML DTD would be declared, in the equivalent SGML DTD, using declarations of the form One may or may not like this solution, but let's discuss it using reasons that actually apply, not erroneous claims about what can and cannot be handled by SGML processors. -C. M. Sperberg-McQueen From: streich@austin.sar.slb.com Date: Tue, 15 Oct 96 13:49:55 CDT To: tbray@textuality.com, jjc@jclark.com Subject: Re: B.9 Formal system, public identifiers? Cc: w3c-sgml-wg@w3.org > At 11:14 15/10/96 -0700, Tim Bray wrote: > > >I'd still like to keep at least a subset > >of the nice clean FSI syntax. > > What is the functionality you want from FSIs that you can't get from URLs? At least better opportunities for indirection. bob Date: Tue, 15 Oct 1996 19:39:21 +0000 To: Tim Bray From: James Clark Subject: Re: B.9 Formal system, public identifiers? Cc: w3c-sgml-wg@w3.org At 11:14 15/10/96 -0700, Tim Bray wrote: >At 06:24 PM 10/15/96 +0000, James Clark wrote: > >>I just counted the number of lines of code in the part of SP that handles >>external entities; I included just the URL and OSFILE storage mangers, and >>just the UTF-8 and Unicode encodings. It was more than 5000 lines. > >So this was just fSi, not fPi? No, FPIs as well: I was counting the entity manager (which in SP only deals with external entities) after removal of support for all encodings other than UTF-8 and Unicode and for all storage managers other than URL, OSFILE. I didn't count all the code in the parser proper for handling entity declarations and references. James Date: Tue, 15 Oct 1996 20:09:19 +0000 To: streich@austin.sar.slb.com From: James Clark Subject: Re: B.9 Formal system, public identifiers? Cc: tbray@textuality.com, w3c-sgml-wg@w3.org At 13:49 15/10/96 CDT, streich@austin.sar.slb.com wrote: >> At 11:14 15/10/96 -0700, Tim Bray wrote: >> >> >I'd still like to keep at least a subset >> >of the nice clean FSI syntax. >> >> What is the functionality you want from FSIs that you can't get from URLs? > >At least better opportunities for indirection. How do FSIs offer this? What makes you think URLs can't offer indirection? James Date: Tue, 15 Oct 1996 12:24:46 -0700 (PDT) From: Bill Smith Subject: Re: B.2 DTD required or optional? To: w3c-sgml-wg@w3.org I like Tim Bray's distinction of well-formed and valid. The concepts are easily understood and have practical (not just theoretical) utility. In my opinion, it must be possible to correctly process well-formed XML without the aid of a DTD. Ideally, well-formed documents would be processed without any form of "hinting". If we cross the DTD-less line for well-formed XML, I doubt that XML will be widely accepted. Date: Tue, 15 Oct 1996 15:36:10 -0400 From: gtn@ebt.com (Gavin Nicol) To: jjc@jclark.com CC: w3c-sgml-wg@w3.org Subject: Re: B.9 Formal system, public identifiers? >>I disagree. Using FPI's and FSI's for naming is simple enough. > >I just counted the number of lines of code in the part of SP that handles >external entities; I included just the URL and OSFILE storage mangers, and >just the UTF-8 and Unicode encodings. It was more than 5000 lines. I don't >see how we can possibly meet our goals for ease of implementation if this >part of XML has this level of complexity. I think the figures depend largely on how much code reuse is in effect. Most people dealing with XML probably already have code to use. For those that don't, I don't see 5000 lines of code as a major overhead, because the code is very straightforward. One other thing: any XML applications that try to do anything even slightly complex will require entity managers. by stating up front what the overall functionality is expected to be, people can design the entity managers with the future in mind. >Don't get me wrong: I love FPIs and FSIs. But I strongly believe this whole >XML exercise is going to be a total waste of time unless we exercise extreme >self-restraint in the features we put in XML, especially in version 1.0. I agree with you wholeheartedly. The real thrust I've been continually trying to make is that of building a good infrastructure that is open-ended. As such, I see nothing wrong with using FPI's and FSI's, possibly with some constraints (XML 1.0 only *requires* certain forms of FSI). The constraints can be removed over time, but the foundation cannot be relaid time and time again. Date: Tue, 15 Oct 1996 12:31:30 -0700 (PDT) From: Bill Smith Subject: Re: B.9 Formal system, public identifiers? To: w3c-sgml-wg@w3.org James Clark wrote: > I just counted the number of lines of code in the part of SP that handles > external entities; I included just the URL and OSFILE storage mangers, and > just the UTF-8 and Unicode encodings. It was more than 5000 lines. I don't > see how we can possibly meet our goals for ease of implementation if this > part of XML has this level of complexity. > > Don't get me wrong: I love FPIs and FSIs. But I strongly believe this whole > XML exercise is going to be a total waste of time unless we exercise extreme > self-restraint in the features we put in XML, especially in version 1.0. In my opinion, this is an especially strong argument against FPIs, FSIs, and external entities - in XML 1.0. They can be added in a subsequent version. From: Charles@sgmlsource.com (Charles F. Goldfarb) To: DAVEP@acm.org Cc: W3C-SGML-WG@w3.org Subject: Re: A17: keep or drop entities? Date: Tue, 15 Oct 1996 19:49:59 GMT On Thu, 10 Oct 1996 22:35:31 -0500 (CDT), DAVEP@acm.org wrote: >On Thu, 10 Oct 1996, Charles F. Goldfarb wrote: > >>On Thu, 10 Oct 1996 10:12:13 -0400, "Eve L. Maler" wrote: > >>>At 04:49 AM 10/10/96 GMT, Charles F. Goldfarb wrote: > >>>> The keyword SDATA in the ISO >>>>character entity set is unnecessary because the replacement text is a symbolic >>>>string. (My original intention was that a system would use an equivalent entity >>>>set in which the replacement text was real system data.) > >>>The [xxxxxx] replacement text "templates" have been widely implemented >>>to produce the desired glyphs. But this doesn't mean they're not system >>>data, does it? It's still essentially a "processing instruction that >>>returns data" (clause 8). Regular internal text entities aren't >>>supposed to have this property. > >>Eve has made a very sensible observation, so let me explain my reasoning. > >>There are two principal purposes for labeling SDATA and PI: >> >>1. To make it easy to locate and revise or remove system-specific information. >>This, of course, enhances document portability and reuse by containing system >>dependencies. >> >>2. To prevent generated text from being parsed in context with the SGML >>document. This enhances portability and reuse by assuring that all applications >>will "see" the same data. > >>The symbolic replacement text in the ISO 8879 character entity sets don't >>present a problem on either of those counts. They are not system-dependent and >>they parse identically in all environments. That is because the generation of >>system-specific data takes place in the *result* document; it is never seen by >>the parser. In pernicious SDATA, the entity text is system-specific and >>therefore needs to be labeled. > >There may have been just two principal purposes, but there is certainly >a third in this day and age: When one is inserting an ordinary text entity, >the replacement text is specifically intended to become characters in the >document. When "&" is inserted in a document, one does not normally >intend it to be the same as though "[amp ]" were directly in the document. >Many processors rely on the SDATA designation to trigger the special >handling. Leave it out and we're in trouble. We're only in trouble for non-portable SDATA. Let's approach this from another direction. All data has the potential of being application-specific, in the sense that the inherent semantics of the element type causes the data to be interpreted a particular way. For example, a "time" element in some document type may treat data content in the form "hh:mm.ss" as a time of day and always convert it to GMT in the formatted output. As the data recognized by the parser is always the same, this data is portable and non-pernicious, even though it is an "instruction that returns data" or, if you prefer, a data content notation. In the specific case of the ISO character entity sets and XML, the SGML system does not need to know that the symbolic character names are SDATA because they are portable (i.e., always produce the same ESIS). The XML application can rely on the convention that "[tttttttt]" is always treated as a symbolic character name for which a specific glyph is produced in the formatted output. We could formalize this convention by defining an "xmlpcdata" notation that is the fixed value of a notation attribute declared for every element type that has #pcdata in its content model. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- Date: Tue, 15 Oct 1996 12:47:52 -0700 (PDT) From: Bill Smith Subject: Re: B.10 Empty elements? To: w3c-sgml-wg@w3.org Tim Bray wrote: > At 10:57 AM 10/15/96 -0700, Bill Smith wrote: > > > is my strong preference but only if we specify that the required markup > >declarations be part of the document instance. > > This would normally be the case when there was no DTD... but you'd insist that > even if there was, they be in the subset, not the DTD itself? -Tim If there's another mechanism that doesn't require reference to some external object that's fine with me. If we require complex entity management in XML 1.0 we're lost. I place locating arbitrary (partial) DTDs over the Web in the too complex category. We can and should add this to subsequent versions. From: dgd@cs.bu.edu (David Durand) Date: Tue, 15 Oct 1996 16:27:13 -0400 To: w3c-sgml-wg@w3.org Subject: Re: B.9 Formal system, public identifiers? dated: Tue, 15 Oct 1996 20:09:19 +0000 To: streich@austin.sar.slb.com From: James Clark Subject: Re: B.9 Formal system, public identifiers? >> >> >I'd still like to keep at least a subset >> >of the nice clean FSI syntax. >> >> What is the functionality you want from FSIs that you can't get from URLs? > >At least better opportunities for indirection. How do FSIs offer this? What makes you think URLs can't offer indirection? James URLs can provide indirection (or at least a server can) but they bind the resolution protocol at the time the name is assigned. Using URLs as the only syntax for system identifiers doesn't allow other forms of name to be used later on. If we specify FSI syntax, and then restrict the FSIs to be URLs, there is a nice path to more-robust identifiers at no more cost than we incur by implementing URLs to begin with. Similarly, if we allow FPIs and restrict our catalogs to mapping FPI -> FSI, the implementation will be a lot simpler than current SGML implementations. We need not even use the SGML/Open catalog syntax if it makes parsing to hard. Having the FSI syntax to allow extension seems easy and foresighted to me. FPI's may add complexity (in that they need to be resolved, but if we have external entities at all, FPIs should be available -- We need the slot in the syntax if we ever want to have the FPI/FSI distinction -- and it is a critical distinction. So I guess I think we need to FPIs into the syntax even if we consciously decide not to specify how they should be resolved. (Assuming that we have external entities at all. On that question I can't make up my mind.) If we are to eliminate the distinction between FPI and FSI (or URN/URL or pointer/name, choose your terminology), then we would have to eliminate URLs and other pointers I think, and standardize on FPIs only. However, this would fly in the face of familiarity, and be hard to implement, so we should probably avoid the high road on this issue. But making it impossible to ever do the right thing and use location independent naming seems a high price to bear. -- David RE delenda est. Date: Tue, 15 Oct 96 17:16:18 CDT From: Michael Sperberg-McQueen Subject: Re: B.7 Conditional inclusion in DTDs? To: W3C SGML Working Group On Tue, 15 Oct 1996 09:52:51 -0400 Eve L. Maler said: >At 07:14 PM 10/14/96 -0500, Len Bullard wrote: >>Charles F. Goldfarb wrote: >>> XML shouldn't have conditional DTDs. They defeat the objective of >>> simplicity. >>> Those who need complex DTD management will be using full SGML and >>> can easily generate XML from it. >>Also concur. > >As do I (I think). Presenting DTDs to new XML users will be easier if >we constrain them (the DTDs :-) to single "state spaces" for now. I dissent strongly. XML should be as simple as possible, but not simpler. If multi-flavored DTDs are not allowed in XML, it ceases to be even potentially a read-edit-write language suitable for work with documents, and becomes exclusively a write-only language for publication. Not much reason to support the desperate Perl hacker in XML if XML is just for publication: the DPH will in general be working on the working copies of documents, not documents that have already been through an irreversible publication process. If publication were all that were needed, we could have been happy just all encouraging SoftQuad to release Panorama for other platforms. We need -- Dave Hollander has been particularly eloquent on this score -- a language suitable both for publication and for maintenance. Eliminating conditional inclusion in DTDs defeats the objective of XML as a strong, powerful language for work with texts, and abandons without necessity the goal of having XML possess expressive power approximately equal to that of Full SGML. When we are talking about a formalism not powerful enough to represent the DTD for HTML 2.0, I think we are talking about too weak a formalism. Production DTDs require parse-time customization. XML requires the power to handle production DTDs. Ergo, XML requires parse-time customization -- i.e. condition inclusion in DTDs. -C. M. Sperberg-McQueen Date: Tue, 15 Oct 1996 17:33:36 -0500 From: Len Bullard To: Michael Sperberg-McQueen CC: W3C SGML Working Group Subject: Re: B.7 Conditional inclusion in DTDs? Michael Sperberg-McQueen wrote: > When we are talking about a formalism not powerful enough to represent > the DTD for HTML 2.0, I think we are talking about too weak a formalism. I'd gladly dump the conditionals there in favor of more versions for what very very little good the HTML DTD does for the HTML user. > Production DTDs require parse-time customization. XML requires > the power to handle production DTDs. Ergo, XML requires parse-time > customization -- i.e. condition inclusion in DTDs. > > -C. M. Sperberg-McQueen Then why TEIlite? len Date: Tue, 15 Oct 1996 22:40:53 +0000 To: dgd@cs.bu.edu (David Durand) From: James Clark Subject: Re: B.9 Formal system, public identifiers? Cc: w3c-sgml-wg@w3.org At 16:27 15/10/96 -0400, David Durand wrote: > If we specify FSI syntax, and >then restrict the FSIs to be URLs, there is a nice path to more-robust >identifiers at no more cost than we incur by implementing URLs to >begin with. There is a cost to the user: instead of entering a kind of thing they are familiar with, namely a URL, they have to enter something that they are not familiar with, namely an FSI. If we stick to URLs now, we can still support FSIs in the future: any FSI must include < and > but these two characters are not allowed in URLs. It's ridiculous to force users to type at the beginning of all their URLs when it's completely unnecessary to do so. > But making it impossible to ever do the right thing and use >location independent naming seems a high price to bear. URL syntax is extensible: why can't it be used to do location-independent naming? Something like fpi://W3C/DTD/HTML_3.2 is a perfectly good URL. I can't resolve it, but then I can't resolve FPIs either. Location-independent naming is not a problem specific to XML. We should use whatever general solution gets adopted for the WWW, rather than trying to come up with a solution specifically for XML. James From: streich@austin.sar.slb.com Date: Tue, 15 Oct 96 18:04:25 CDT To: jjc@jclark.com Subject: Re: B.9 Formal system, public identifiers? Cc: w3c-sgml-wg@w3.org > >> >> What is the functionality you want from FSIs that you can't get from URLs? > >> > > >> >At least better opportunities for indirection. > >> > >> How do FSIs offer this? What makes you think URLs can't offer indirection? > > > >I didn't say that URLs don't offer indirection, only that there are > >better opportunities with FSIs. With an FSI, I can add my own storage > >object handler to pull the doc out of a DMS, for example. > > Why can't you add a URL scheme to do this? It's just as easy for an entity > manager to recognize a URL scheme name and pass the scheme part off to some > handler as it is for it to recognize a storage manager name and pass the SOI > off to some handler. Sure, but then the URL is just a dummy. Why force me to invent a scheme like that? It's too messy. I like to have things that do what you expect them to do. If it looks like a URL, it is a URL, and it should resolve a URL. Trying to make it into something else is too much like server-side includes, yeccchhh. I just can't see myself explaining to people: "Well, true. It looks like a URL, but it is really this other thing...." bob From: lee@sq.com Date: Tue, 15 Oct 96 19:17:41 EDT To: w3c-sgml-wg@w3.org Subject: Re: B.9 Formal system, public identifiers? gtn@ebt.com (Gavin Nicol) wrote: > I think the figures depend largely on how much code reuse is in effect. > Most people dealing with XML probably already have code to use. For those > that don't, I don't see 5000 lines of code as a major overhead, because > the code is very straightforward. No-one in the world has any XML code today. There are people with URL-handling code, but there are very few people in the world with FSI code. Most of it was written by James, in fact... For me, if something takes James 5,000 lines, it is probably complex enough that I would leave it out of the XML implementation I was thinking of writing in my spare time when it settles down. If FSIs end up being required by XML, I won't use XML for my purpose. Although I can write about 500 lines of working C in a day, I am very tired on the next day :-), and it takes several days of debugging. So I can get between 100 and 300 working, tested, debugged lines of code done in a day, averaged out over a week. So FSIs may take me more than ten weeks. Well, I don't think that counts as not a ``major overhead''. If it takes me ten weeks to implement a parser for all of XML, you can forget it. I would expect to be able to write a simple style-sheet aware browser in less time than that! (the last one took me about 2 or 3 weeks, but it wasn't very good :-) ) A possible compromise might be to allow a url: prefix to a URL, and say that. This is forwards compatible -- you can add URNs later when they are working, and any other mechanism you want. Note that one very very useful feature in Panorama is support for partial URLs. A system identifier of "docbook.dtd" is interpreted as relative to the file containing that identifier. Hence, if we used File->Open to open a file containing that system identifier, Panorama looks in the same directory as the document you just opened. But if ftp was used, Panorama will use ftp to fetch "docbook.dtd" from the remote computer, again in the same directory. And if http was used... well, you see the pattern :-) This means you can test your SGML locally on a networked or local disk, then yes exactly the same unchanged files to test with a web browser. No changes means no errors. If URLs are used in XML, let's also allow partial URLs, as in HTML, and say that _all_ system identifiers are treated in this way. It really does work, and there is a sample implementation you can try... Lee Date: Tue, 15 Oct 96 17:54:04 CDT From: Michael Sperberg-McQueen Subject: Re: B.7 Conditional inclusion in DTDs? To: W3C SGML Working Group On Tue, 15 Oct 1996 18:34:24 -0400 said: >Michael Sperberg-McQueen wrote: >> Production DTDs require parse-time customization. XML requires >> the power to handle production DTDs. Ergo, XML requires parse-time >> customization -- i.e. condition[al] inclusion in DTDs. >> >> -C. M. Sperberg-McQueen > >Then why TEIlite? Primarily for pedagogical reasons. (1) TEI Lite is smaller than the full TEI DTD and most people seem to find it reasonably manageable. (2) Any single-file DTD is easier to install and run on many existing SGML systems, since the user doesn't need to figure out how to tell the system where to find external entities. (This is getting better with the wider adoption of SGML Open entity catalogs, I think, but it's still a pain for some, and when we designed Lite it was worse.) So Lite is easier to get up and running with. I don't expect many serious projects to stick with it forever: the stuff in the full TEI DTD is there because people need it. (3) Any project that designs and uses a specific view of the TEI DTD is likely to want to generate a one-file version of the DTD, for speed and because some SGML systems (you know who you are!) are non-conforming in refusing to handle references to undeclared elements. TEI Lite, being available both in TEI.extensions.ent and TEI.extensions.dtd form and in single-file derived-DTD form, provides an illustration of how this can work. TEI Lite, in other words, is not a substitute for the full TEI DTD. The full TEI DTD allows the user to modify it in a rather large number of ways that cannot readily be supported without conditional inclusion in some form or other. Handling it outside of the language, with something like the C preprocessor or a specially designed DTD filter, is less reliable than handling conditional inclusion in ways that every conforming SGML system can and does support. -C. M. Sperberg-McQueen Date: Tue, 15 Oct 1996 20:49:05 -0400 From: gtn@ebt.com (Gavin Nicol) To: lee@sq.com CC: w3c-sgml-wg@w3.org Subject: Re: B.9 Formal system, public identifiers? >> I think the figures depend largely on how much code reuse is in effect. >> Most people dealing with XML probably already have code to use. For those >> that don't, I don't see 5000 lines of code as a major overhead, because >> the code is very straightforward. > >No-one in the world has any XML code today. True, but they do have URL and/or name resolution code. >For me, if something takes James 5,000 lines, it is probably complex enough >that I would leave it out of the XML implementation I was thinking of >writing in my spare time when it settles down. If FSIs end up being >required by XML, I won't use XML for my purpose. Well, that's you. I would use them. >Although I can write about 500 lines of working C in a day, I am very >tired on the next day :-), and it takes several days of debugging. So I >can get between 100 and 300 working, tested, debugged lines of code done >in a day, averaged out over a week. Well, one of my strengths is prototyping, and the best I've ever managed was 40,000+ lines of C++ in a month (working, reasonably well tested, but not production level code), and recently 27,000 lines of JAVA in 3 weeks, but that says little about the *type* of code. FSI's and whatnot are simple enough to make them a SMOP, especially if they are constrained by a "minimum" level of functionality. I would also expect free code to be made available over time. >A possible compromise might be to allow a url: prefix to a URL, and >say that. This is forwards compatible -- you can add URNs later when >they are working, and any other mechanism you want. Or constrain the possible set of FSI's... >Note that one very very useful feature in Panorama is support for partial >URLs. Relative URLs? They are very useful, and a standard feature in most browsers now. From: streich@austin.sar.slb.com Date: Tue, 15 Oct 96 16:56:07 CDT To: jjc@jclark.com Subject: Re: B.9 Formal system, public identifiers? Cc: tbray@textuality.com, w3c-sgml-wg@w3.org > At 13:49 15/10/96 CDT, streich@austin.sar.slb.com wrote: > >> At 11:14 15/10/96 -0700, Tim Bray wrote: > >> > >> >I'd still like to keep at least a subset > >> >of the nice clean FSI syntax. > >> > >> What is the functionality you want from FSIs that you can't get from URLs? > > > >At least better opportunities for indirection. > > How do FSIs offer this? What makes you think URLs can't offer indirection? I didn't say that URLs don't offer indirection, only that there are better opportunities with FSIs. With an FSI, I can add my own storage object handler to pull the doc out of a DMS, for example. With a URL, I would have to go through a gateway to accomplish the same purpose. If you've ever had a key gateway crash because the process table overflowed, you understand my reluctance to be cornered into this. bob Date: Tue, 15 Oct 1996 20:38:21 -0400 From: gtn@ebt.com (Gavin Nicol) To: jjc@jclark.com CC: dgd@cs.bu.edu, w3c-sgml-wg@w3.org Subject: Re: B.9 Formal system, public identifiers? >URL syntax is extensible: why can't it be used to do location-independent >naming? Something like fpi://W3C/DTD/HTML_3.2 is a perfectly good URL. I >can't resolve it, but then I can't resolve FPIs either. This was one of my original dislikes for FSI's: namely that they add a lot of syntax for no apparent benefit. In the long run though, it's all just syntax, and I think that one thing FSI's do bring to play is that you can add attributes to the address. This can be very useful for things like specifying encoding etc (yes, you could also use RFC 15XX mechanisms to encoding strings, but that is not a easy for users to type). >Location-independent naming is not a problem specific to XML. We >should use whatever general solution gets adopted for the WWW, rather >than trying to come up with a solution specifically for XML. True, but I haven't seen a good general purpose solution from the WWW yet either. BTW. Do you see URI's as addresses or names? I tend to think of them as pointers into a large shared address space. Date: Tue, 15 Oct 1996 22:59:34 +0000 To: streich@austin.sar.slb.com From: James Clark Subject: Re: B.9 Formal system, public identifiers? Cc: w3c-sgml-wg@w3.org At 16:56 15/10/96 CDT, streich@austin.sar.slb.com wrote: > >> At 13:49 15/10/96 CDT, streich@austin.sar.slb.com wrote: >> >> At 11:14 15/10/96 -0700, Tim Bray wrote: >> >> >> >> >I'd still like to keep at least a subset >> >> >of the nice clean FSI syntax. >> >> >> >> What is the functionality you want from FSIs that you can't get from URLs? >> > >> >At least better opportunities for indirection. >> >> How do FSIs offer this? What makes you think URLs can't offer indirection? > >I didn't say that URLs don't offer indirection, only that there are >better opportunities with FSIs. With an FSI, I can add my own storage >object handler to pull the doc out of a DMS, for example. Why can't you add a URL scheme to do this? It's just as easy for an entity manager to recognize a URL scheme name and pass the scheme part off to some handler as it is for it to recognize a storage manager name and pass the SOI off to some handler. James Date: Tue, 15 Oct 1996 22:51:51 -0500 From: Len Bullard To: James Clark CC: David Durand , w3c-sgml-wg@w3.org Subject: Re: B.9 Formal system, public identifiers? James Clark wrote: > > At 16:27 15/10/96 -0400, David Durand wrote: > > > If we specify FSI syntax, and > >then restrict the FSIs to be URLs, there is a nice path to more-robust > >identifiers at no more cost than we incur by implementing URLs to > >begin with. > > There is a cost to the user: instead of entering a kind of thing they are > familiar with, namely a URL, they have to enter something that they are not > familiar with, namely an FSI. > > If we stick to URLs now, we can still support FSIs in the future: any FSI > must include < and > but these two characters are not allowed in URLs. It's > ridiculous to force users to type at the beginning of all their URLs > when it's completely unnecessary to do so. > > > But making it impossible to ever do the right thing and use > >location independent naming seems a high price to bear. > > URL syntax is extensible: why can't it be used to do location-independent > naming? Something like fpi://W3C/DTD/HTML_3.2 is a perfectly good URL. I > can't resolve it, but then I can't resolve FPIs either. > > Location-independent naming is not a problem specific to XML. We should use > whatever general solution gets adopted for the WWW, rather than trying to > come up with a solution specifically for XML. > > James The question: are the WWW protocols the responsibility of any notation or any language for defining a notation? In a framework, protocols can always be added, but for system integration and stability, redoing existing ones is expensive. After so many years of work in HyTime, I do not wish to see the strong concepts of HyTime location and address types disappear. But at this time, the established WWW protocols are in place and they work. XML has the power to return to us the capability to design our own structures and use them as see fit in operations on the data. To me, there is the evolution of XML. Version 1.0 should be be a perfect fit to existing frameworks. Else, it will not be adopted. VRML used this strategy successfully. If we adopt this strategy, our interoperability problems and many of our portability problems are reduced significantly. I think James is right. 2.0 is a new day. len Date: Tue, 15 Oct 1996 21:16:50 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: [oh no, Mr. Bill! not] B.3 Record-end handling? I can't believe I'm addressing this again. But in a lengthy discussion of the Vancouver SGML ERB/WG caucus this afternoon, Peter Sharpe dreamed up the following and won't have time to post it, so I agreed to. So it's all his fault. It smells to me like it might work. 1. If you have a DTD and you know where element content is, you lose all white space in element content. 2. DTD or none, if a line contains some markup, and aside from that only white space, you lose said white space and the trailing record boundary character(s). "Markup" meaning tags, comments, and PIs. 3. Every byte in mixed or PCDATA content that is not lost in this fashion and is not markup is passed to the application. "Lose" means "don't pass to the application". This has the virtues that - it can be explained *very* briefly - the behavior is an awful lot like what ordinary people think that 8879 is trying to do - it's easy to build - it allows users to put all sorts of gratuitous white space in their data without getting in the way - it doesn't use the [inaccurate and counter-intuitive to most programmers] terms "RS" and "RE" On the downside, I suspect that this will eat a few white spaces between tags and RE's, and around comments and PIs, and maybe a few REs after comments and PIs, that a real SGML parser would pass on. But (a) few will ever notice, and (b) those that do will be surprised at the SGML behavior. It may not be perfect. But it does provide an example of the maximum level of complexity in this area that I for one am willing to tolerate in XML. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 From: Charles@sgmlsource.com (Charles F. Goldfarb) To: Michael Sperberg-McQueen Cc: W3C SGML Working Group Subject: Re: B.9 Formal system, public identifiers? Date: Wed, 16 Oct 1996 19:21:50 GMT On Sat, 12 Oct 96 15:08:24 CDT, Michael Sperberg-McQueen wrote: >On 16 October 1996, the ERB will vote to decide the following question. >A straw poll indicates the question needs further discussion in the work >group. > >B.9 Should XML require system and public identifiers to be FORMAL >(13.5)? Here is a possible solution to the impasse between FSI and URL-only. The FSI design effectively treats an informal system identifier (i.e., an ordinary string) as a default storage object specification. The usual default is OSFILE. For XML, we could let the default be URL. In other words, for a URL you could enter either: http://www.exasperated.org or http://www.exasperated.org For other storage managers that XML supports in future versions, the normal FSI syntax would be used. -- Charles F. Goldfarb * Information Management Consulting * +1(408)867-5553 13075 Paramount Drive * Saratoga CA 95070 * USA International Standards Editor * ISO 8879 SGML * ISO/IEC 10744 HyTime Prentice-Hall Series Editor * CFG Series on Open Information Management -- From: lee@sq.com Date: Wed, 16 Oct 96 15:30:22 EDT To: tbray@textuality.com, w3c-sgml-wg@w3.org Subject: Re: [oh no, Mr. Bill! not] B.3 Record-end handling? > 2. DTD or none, if a line contains some markup, and aside from that > only white space, you lose said white space and the trailing record > boundary character(s). "Markup" meaning tags, comments, and PIs. The only trouble with this is that it relies on non-portable line breaks. With the system-specific rules for what constitutes a line (or record, if you prefer that terminology), a file on a Macintosh with ASCII CR between lines copied to Unix, which recognises only LF as line breaks, causes an interoperability problem: the document is now all on one line, so you have to read it all before you can display past the first space. MIME uses CR LF for a line boundary, like SMTP, but most servers don't in fact translate properly as far as I can tell. We've heard a lot about how XML applications will have problems if there is whitespace in tables, but have not heard how an XML application is to recognise that something is a table. I have assumed so far the use of a style sheet to determine that. Perhaps the style sheet should tell the XML application how to display spaces and newlines. Then all spaces and newlines and carriage returns are passed retained by the lexical analyser and/or parser at the discretion of the application. I would expect multple blanks (of any kind) to be coalesced and compressed into a single space. A non-editing application would lose comments at the same time, of course. One thing seems to be generally (if not universally) agreed: the SGML rules for whitespace and RS/RE are too complex for XML, for a variety of reasons. There is an excellent paper produced by Software Exoterica on the handling of RS/RS in their parser. That someone could write a paper on whitespace (and still say that there are grey areas in which they had to make a decision, as I recall!) says that it is too complex. The problem is made complex by two things: (1) a desire to use whitespace to be part of the data in some places, and part of the markup in others, without any syntactic differentitation. In other words, in , the spaces might be important, but in another DTD, might be the same as . The same applies to newlines, with the additional twist that in some places, people might want a newline to be different froma space, and even the SGML parser can't tell where. (2) a fond imagining that the SGML parser can deal with people putting the parkup in the wrong place. We have been told that

    x x x

    is more natural to some people than

    x x x

    but that to others

    x x x

    is more natural. SGML special-cases the first of these. But since it is subjective, there is no clear reason to prefer any of these forms over any other. We have also been told that SGML can handle the case where a non-SGML- aware editor performs word wrap incorrectly, and attempts to divine the "True Data" that the user was imagining. But this is more than subjective, it's absurd. (please tell me privately if I am misrepresenting anyone, and I will gladly summarise to the list) All of these points seem very strange to anyone from a computer science background, and are in fact central to why SGML is not as widely implemented as it could be, I believe. These issues show how the `cultural assumptions' of SGML are utterly alien to the CS world. This is more important than complexity. If complexity were the only issue, we would see large numbers of partial implementations. We don't. There is no hope in trying to change the CS community in this regard; we have already tried that. There are things that SGML can teach CS, but there are also things SGML must learn from CS. Regularity is one of them. I do not believe there is any justification for using white-space rules other than those used in Pascal, LISP, C, C++, Java, Alogol, PostScript, TeX, and many other languages designed since the 1960s. I realise there are people with large amounts of legacy data in SGML, or other non-XML formats. But if the goal is to get wide deployment, you need to get software developers to _like_ your language. So you need to avoid irregularities and quirks. Yes, you can have multiple syntactic constructs. Most languages _do_ use a special syntax for comments, for example, so that the parser can ignore them easily. But try and make them fit into existing language constructs used elsewhere and you will gain many more converts, much more quickly. Maybe there are 10 or 20 thousand people who can write SGML DTDs today. But there are many more people than that who can write a C program. So I am afraid I do not think Peter's idea is the right approach. It is a good idea if you can't take the right approach. But we can. Lee