Date: Wed, 28 Aug 1996 03:13:56 -0400 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org Subject: Greetings from the chair To the members of the W3C SGML Working Group, welcome and thank you for your patience during construction of the mailing list. Our business will begin in earnest immediately following the Labor Day holiday in the U.S. (September 2). You will be provided at that time with an organized framework within which to conduct a discussion of an SGML application profile for Web use. In the meantime, you can start by familiarizing yourself with the W3C SGML ERB/WG charter [1] and the current statement of design principles [2]. You may also wish to review the current DSSSL Online Application Profile [3]. The SGML ERB/WG has three explicit deliverables: 1. The specification of an application profile defining a form of SGML designed for Internet transmission and processing by user agents. For purposes of discussion, the format thus defined has been given the temporary working name of Extensible Markup Language (XML). The ERB has set an ambitious goal of completing the initial draft for this specification by the SGML 96 conference (November 18-22). 2. The specification of basic hypertext link types for XML. These types have yet to be identified, but a basic goal is to go beyond the single link type in HTML to include the types that have been considered essential in hypertext systems for the last decade. It seems apparent that achieving this goal implies the implementation of the basic functionality of the HyTime clink and ilink architectural forms, but whether the mechanism used is architectural forms or extensions to DSSSL remains an open question to be dealt with after the initial XML specification draft has been completed. 3. The specification of extensions and public text needed to make DSSSL work in an Internet context. For example, a mechanism needs to be added to DSSSL to enable text to flow around objects. We will be taking these one at a time, beginning with #1. Be prepared for an extremely interesting discussion of this subject beginning next week. [1] http://www.textuality.com/sgml-erb/tr-1996-01.html [2] http://www.textuality.com/sgml-erb/dd-1996-0001.html [3] ftp://sunsite.unc.edu/sun-info/standards/dsssl/dssslo Jon --- Jon Bosak, Online Information Technology Architect Sun Microsystems, 2550 Garcia Ave., MPK17-101, Mountain View, CA 94043 Davenport Group::SGML Open::ANSI X3V1::ISO/IEC JTC1/SC18/WG8::W3C SGML ERB Date: Tue, 3 Sep 1996 09:17:31 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org Subject: DSSSL application available An alpha version of James Clark's Jade is now available for testing by qualified individuals. Jade is an application of DSSSL (ISO/IEC 10179:1996), the international standard language for specifying processing semantics to be applied to documents marked up in languages conforming to SGML (ISO 8879:1986). Jade takes as input an SGML DTD, a document marked up in the language specified by the DTD, and a DSSSL stylesheet. It produces as output an SGML representation of the resulting DSSSL flow object tree or, alternatively, an RTF file of the formatted document. For complete and up-to-date information on Jade, consult the Jade home page at http://www.jclark.com/jade. The distribution made available with this announcement contains an alpha version of Jade repackaged for easy installation on Win95 and Sparc Solaris 2.5 platforms. For demonstration purposes, it also contains a recent version of the HTML 3.2 DTD, a sample document marked up in HTML 3.2, and an HTML 3.2 DSSSL stylesheet. Complete source code is available from the same archive. To obtain the distribution, follow these steps: 1. Log in via anonymous ftp to sunsite.unc.edu (or one of its mirrors). 2. Set "bin" for binary transfer. 3. Change to the directory /pub/sun-info/standards/dsssl/jade/1996.08.31 4. Get the files README, DESCR, and HOWTO. 5. Get the appropriate set of binaries. This will be one of the files jade-win.zip, jade-sol.tar.gz, etc. 6. Read DESCR and HOWTO for further directions. Jon --- Jon Bosak, Online Information Technology Architect Sun Microsystems, 2550 Garcia Ave., MPK17-101, Mountain View, CA 94043 Davenport Group::SGML Open::ANSI X3V1::ISO/IEC JTC1/SC18/WG8::W3C SGML ERB Date: Thu, 5 Sep 1996 19:13:38 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org Subject: W3C SGML WG: The work begins This message marks the official start of our group deliberations regarding SGML simplification for Web use. The ultimate goal of this effort is the creation of a form of SGML that can be used to transmit documents (or document fragments) to a future generation of Web browsers and similar Internet client applications. This SGML subset -- or derivative, or conformance level, or application profile; we're not sure exactly what we're aiming for yet -- is known by the working name of Extensible Markup Language, or XML. To ensure wide acceptance, it is generally agreed that XML must be designed in such a way that the data stream delivered to a Web application can be handled by a parser that can be constructed by any competent computer science graduate. It is also generally agreed that it must retain the basic qualities of extensibility, validation, structure, and standardization that make SGML powerful enough to use for large-scale commercial publishing. And there is a strong desire to make XML compatible with existing SGML tools. These general goals are clear enough, but the details are not easily determined. Our uncertainty has two levels: we're not sure where the optimum balance is between SGML compatibility and ease of implementation as a general goal, and we're not sure which specific features of SGML should be retained in XML. A number of people have made specific proposals for SGML simplification, which will be placed on the ERB archive for your consideration. These proposals agree on many points, but on others they vary considerably. The orderly and rational process for designing XML would be to determine our goals in advance and then design to meet those goals. In preliminary discussions awaiting the formation of this W3C SGML Working Group list, however, the Editorial Review Board found it impossible to arrive at an exact statement of the goals independent of a technical discussion of the solution. We have decided, therefore, to put both levels of the problem before the whole Working Group for a general discussion with a specific time limit. The discussion will begin with this message and continue for approximately one month. More precisely, this phase will come to an end on Sunday evening, October 6. At that time, the ERB, under the guidance of an editor to be appointed from among its membership, will take all the input generated by the Working Group discussion and form an initial draft of the XML specification. It is our goal to have this draft available in time for review at a meeting of the Working Group scheduled for Sunday evening, November 17 -- the evening before the SGML 96 conference in Boston. The top level of the discussion, addressing larger goals and the question of SGML compatibility, can take as its starting point the preliminary discussion draft of design principles to be found at http://www.textuality.com/sgml-erb/dd-1996-0001.html This high-level discussion is free to range as far into the questions raised by the XML effort as necessary, but where possible it will be helpful to relate points to specific clauses of the document referred to above so that, in the end, we can arrive at a finished statement of design principles. To provide a framework for the discussion of specific features, the ERB (chiefly Michael Sperberg-McQueen and Tim Bray) have prepared a mechanism named "Features of ISO 8879 for Use in W3C Editorial Review Board Discussions" and informally known as "the voting booth". This is a way to record and display the current state of your judgement, and the judgement of the group as a whole, regarding the salient features of ISO 8879. You can find this document (collection of forms) at http://www.textuality.com/sgml-erb/dd-1996-0002.html It is our hope that the "voting booth" will help focus the discussion of XML details in several ways: * as a catalog of the salient features of 8879 * as a summary of the already existing proposals for SGML simplification (proposals will be added to the feature matrix and to the archive as we obtain copies) * as a way to arrive at a quick consensus on features that do not need further discussion * as an indicator of current mood as the discussion progresses (you can change your vote as often as you want; only the last one counts) * as a lasting record of the collective wisdom of the SGML WG to guide the ERB in forming an XML specification Please note in using the "voting booth" that you must enter the email name that we have on record for you. See the current list of WG members at the end of this message if you are not sure about your address-of-record. Features will be subdivided into separately votable pieces if it becomes clear in the course of discussion that finer granularity is warranted. Despite our informal name for it, the "vote" on features is not binding in any sense but rather serves to indicate the collective opinion of what we believe to be a representative and highly qualified cross-section of the SGML community. The final specification will not be mechanically driven by its results, but will be produced as a coherent design effort that uses your vote and the associated discussion as critical input. While we do ask that you begin with the larger goals addressed in dd-1996-0001 before turning to the details of 8879, the month-long debate that begins with this message is basically unmoderated. The only firm rule is that you must confine your comments to XML and not wander off into unrelated subjects. In particular, please resist the temptation to start discussions related to linking, addressing, and hypertext in general; these issues belong to a second phase of design that will begin after the XML specification is completed. The current roster of WG participants is given below. Let the work begin! Jon ======================================================================== Chair: Jon Bosak jon.bosak@sun.com W3C Contact: Dan Connolly connolly@w3.org Technical Lead: James Clark jjc@jclark.com SGML ERB (ex officio members of the WG): Tim Bray, Textuality tbray@textuality.com Steve DeRose, EBT sjd@ebt.com Dave Hollander, HP dmh@hpsgml.fc.hp.com Eliot Kimber, Passage Systems kimber@passage.com Tom Magliery, NCSA mag@ncsa.uiuc.edu Eve Maler, ArborText elm@arbortext.com Jean Paoli, Microsoft jeanpa@microsoft.com Peter Sharpe, SoftQuad peter@sqwest.bc.ca C. M. Sperberg-McQueen, U. of Illinois U35395@UICVM.UIC.EDU WG members who are not on the ERB: Sharon Adler, EBT sca@ebt.com Paula Angerstein, Texcel paula@texcel.no Todd Bauman, U. of Maryland bbauma1@cs.umbc.edu Anders Berglund, EBT alb@ebt.com Karl Best, Novell kbest@novell.com Michel Biezunski, High Text michel@hightext.com Harvey Bingham, SGML consultant hbingham@acm.org Steve Brown, InfoObjects sbrown@cortland.com Martin Bryan, SGML Centre mtbryan@sgml.u-net.com Mark Buckley, Microsoft mbuckley@microsoft.com Len Bullard, Lockheed-Martin cbullard@HiWAAY.net Lou Burnard, Oxford University lou@vax.ox.ac.uk Steve Byrne, JavaSoft steve.byrne@sun.com Kurt Conrad, Sagebrush Group conrad@cbvcp.com Keith Corbett, Harlequin kmc@harlequin.com Robin Cover robin@acadcomp.sil.org Dave Durand, Boston University dgd@cs.bu.edu Joe English, CR Laboratories jenglish@crl.com Ralph Ferris, Fujitsu OSSI ralph@ossi.com Lee Fogal, Digital Equipment fogal@zk3.dec.com Todd Freter, Novell tfreter@novell.com Matthew Fuchs, Disney Imagineering matt@wdi.disney.com Charles Goldfarb charles@sgmlsource.com Paul Grosso, ArborText paul@arbortext.com Eduardo Gutentag, SunSoft eduardo@eng.sun.com Hasse Haitto, Synex haitto@synex.se John Iobst, Newspaper Assoc. of America iobsj@naa.org Alan Karben, Wall Street Journal karben@interactive.wsj.com Debbie Lapeyre, Atlis dlapeyre@netcom.com Megan MacMillan, BDM Technologies megan@bdmtech.com Murray Maloney, SoftQuad murray@sq.com James Mason, ORNL masonjd@ornl.gov Alex Milowski, Copernican Solutions sgml@winternet.com Steve Newcombe, TechnoTeacher srn@techno.com Gavin Nicol, EBT gtn@ebt.com Nancy Paisner, Hitachi nancy@hi.com Dave Peterson, SGML Works davep@acm.org Paul Prescod, U. of Waterloo papresco@calum.csclub.uwaterloo.ca Lynne Price, SGML consultant lprice@ix.netcom.com Liam Quin, SoftQuad lee@sq.com Arjun Ray, Q2-II aray@nmds.com Bill Smith, SunSoft bill.smith@sun.com Bob Stayton, SCO bobs@sco.com Robert Streich, Schlumberger streich@slb.com Jeff Suttor, SunSoft jeff.suttor@sun.com James Tauber, U. of Western Australia jtauber@library.uwa.edu.au Wayne Taylor, Novell wtaylor@novell.com Henry Thompson, U. of Edinburgh ht@cogsci.ed.ac.uk Sam Wilmott, OmniMark s.wilmott@omnimark.com Chris Wilson, Microsoft cwilso@microsoft.com Wayne Wohler, IBM wohler@vnet.ibm.com Lauren Wood, SoftQuad lauren@sqwest.bc.ca To: w3c-sgml-wg@w3.org From: Martin Bryan Subject: Questions about The Questions Date: Mon, 9 Sep 1996 19:19:43 +0100 Before attempting to answer the questions asked in the forms associated with DD-1996-002.html it is necessary to raise some questions about the expectations of our user community. For example, before choosing whether or not to retain the distinction between abstract and reference concrete syntaxes we need to determine whether the reference concrete syntax as defined in 1986 is still applicable in 1996. For this we need to consider the following points: - HTML has extended the Quantities defined in the reference concrete syntax: should XML offer less flexibility than HTML? - the SGML community has already agreed on a new set of Quantity defaults for the next version of SGML: should XML offer less flexibitity than the next version of SGML - the reference concrete syntax only permits the use of Latin alphanumeric characters in names of elements, attributes and tokens: should XML be designed to allow users to define elements, attributes and their values in a form that is dependent on their local language, or must they restrict themselves to shared names that have meanings defined in English only? - only Arabic numerals are recognised in the 1986 version of 8879: should other forms of number be allowed for in XML attribute values? (This is asked somewhat tounge-in-cheek to show a corollary of the previous question.) - the default character set in 8879 matches that of the reference concrete syntax: should users be able to select which character set is most appropriate for their documents and specify an SGML declaration in which only a subset of ISO 10646 is recognized as valid while still retaining the reference concrete syntax for markup? There are many other questions that need to be asked with respect to the other questions we are asked to vote on, but there is no general provision for us to attach qualifications or comments to our answers. I would suggest that a Comments textarea could usefully be added to each question to allow us to qualify our responses where appropriate. ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ Date: Mon, 9 Sep 1996 12:36:28 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: mtbryan@sgml.u-net.com CC: w3c-sgml-wg@w3.org Subject: Re: Questions about The Questions Thanks to Martin for raising some interesting questions. I look forward to seeing replies from the rest of the group. Please respond to particular points under subject headings that accurately identify each point in the manner suggested below. | There are many other questions that need to be asked with respect to | the other questions we are asked to vote on, but there is no general | provision for us to attach qualifications or comments to our | answers. I would suggest that a Comments textarea could usefully be | added to each question to allow us to qualify our responses where | appropriate. That's the purpose of this mailing list. It's archived by thread on the W3C site, and mail is available to everyone in the group, whereas the Web is not. If you need to qualify your response to an item on the features list, then you have identified an issue that needs to be discussed here; please break it out under an appropriate subject line and start a thread to discuss it. There should eventually be a discussion underway for every point in the features list on which there is not a consensus or near-consensus. Jon --- Jon Bosak, Online Information Technology Architect Sun Microsystems, 2550 Garcia Ave., MPK17-101, Mountain View, CA 94043 Davenport Group::SGML Open::ANSI X3V1::ISO/IEC JTC1/SC18/WG8::W3C SGML ERB Date: Mon, 09 Sep 1996 12:52:45 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: Questions about The Questions Thanks, Martin, for getting this going. At 07:19 PM 9/9/96 +0100, Martin Bryan wrote: >Before attempting to answer the questions asked in the forms associated with >DD-1996-002.html it is necessary to raise some questions about the >expectations of our user community. Yes, and this is exactly the right forum to raise them in. Thus, to address your last question first: >There are many other questions that need to be asked...I would suggest >that a Comments textarea could usefully be added to each question to allow >us to qualify our responses where appropriate. I would suggest that we hash these things out here by email and at our meetings - I don't think it's really possible or even desirable to expand the 8879 voting booth to try to capture all the minutiae at this point; its purpose is, as much as anything, for us to discover where we have areas of broad agreement, so we can focus our thinking and time away from those. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Mon, 09 Sep 1996 12:52:47 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Concrete syntax, character sets Before addressing Martin's questions, a couple of meta-remarks: - I split the response into two - one on the meta/process issues, this on the substantive technical issues - and gave this one a different title. Since we're apt to be discussing different things in parallel here, we should make an effort to do this. - This is pretty well the first time these issues have been raised... the following represents my current opinion, highly susceptible to modification by opinions & experiences expressed in this group. At 07:19 PM 9/9/96 +0100, Martin Bryan wrote: > For example, before choosing whether or >not to retain the distinction between abstract and reference concrete >syntaxes If you want such a distinction, use real SGML. XML should use a hardwired concrete syntax. >- HTML has extended the Quantities defined in the reference concrete syntax: >should XML offer less flexibility than HTML? >- the SGML community has already agreed on a new set of Quantity defaults >for the next version of SGML: should XML offer less flexibitity than the >next version of SGML XML should have *no* concept of quantities. Names, nesting depths, whatever, can be as large as required to meet the requirements of the application. One straightforward way to do this and preserve compatibility with SGML is to require an XML processor to have the capability of writing an appropriate SGML declaration to set the quantities high enough to make a particular XML DTD valid. >- the reference concrete syntax only permits the use of Latin alphanumeric >characters in names of elements, attributes and tokens... >- only Arabic numerals are recognised in the 1986 version of 8879... If you want to use anything but 7-bit ASCII in markup, use real SGML. XML should have the reference concrete syntax hardwired in. >- the default character set in 8879 matches that of the reference concrete >syntax: should users be able to select which character set is most >appropriate for their documents and specify an SGML declaration in which >only a subset of ISO 10646 is recognized as valid while still retaining the >reference concrete syntax for markup? *Good* point... with modern parsing and encoding technology, it seems like it would be easy, and it would certainly be desirable, for XML data not to be limited to small old character sets. On the other hand, with XML, ultimate flexibility is of less importance than ease of implementation; would it be thinkable to say that "all XML data is always in UTF8"? It seems this would break almost nothing and allow almost anything you'd want to do. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Mon, 9 Sep 1996 13:44:59 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: tbray@textuality.com CC: w3c-sgml-wg@w3.org Subject: Re: Concrete syntax, character sets [Responding to Tim Bray:] | At 07:19 PM 9/9/96 +0100, Martin Bryan wrote: | | > For example, before choosing whether or | >not to retain the distinction between abstract and reference concrete | >syntaxes | | If you want such a distinction, use real SGML. XML should use a hardwired | concrete syntax. I totally agree. | >- HTML has extended the Quantities defined in the reference concrete syntax: | >should XML offer less flexibility than HTML? | >- the SGML community has already agreed on a new set of Quantity defaults | >for the next version of SGML: should XML offer less flexibitity than the | >next version of SGML | | XML should have *no* concept of quantities. Names, nesting depths, whatever, | can be as large as required to meet the requirements of the application. | One straightforward way to do this and preserve compatibility | with SGML is to require an XML processor to have the capability of writing | an appropriate SGML declaration to set the quantities high enough to make | a particular XML DTD valid. I agree with Tim's intent here, but aren't some quantities related to the particular document instance? (Correct me if I'm wrong, but I seem to recall cases where declarations worked fine for me until a specific instance blew them up.) | >- the reference concrete syntax only permits the use of Latin alphanumeric | >characters in names of elements, attributes and tokens... | >- only Arabic numerals are recognised in the 1986 version of 8879... | | If you want to use anything but 7-bit ASCII in markup, use real SGML. | XML should have the reference concrete syntax hardwired in. Having just gone through a big struggle in WG8 and X3V1 over the ERCS proposal, I would feel pretty strange about limiting markup to something that not even Western Europeans could use the way they want to. I would like to see some serious discussion of this point. | >- the default character set in 8879 matches that of the reference concrete | >syntax: should users be able to select which character set is most | >appropriate for their documents and specify an SGML declaration in which | >only a subset of ISO 10646 is recognized as valid while still retaining the | >reference concrete syntax for markup? | | *Good* point... with modern parsing and encoding technology, it seems like | it would be easy, and it would certainly be desirable, for XML | data not to be limited to small old character sets. On the other hand, with | XML, ultimate flexibility is of less importance than ease of implementation; | would it be thinkable to say that "all XML data is always in UTF8"? It | seems this would break almost nothing and allow almost anything you'd want | to do. It's certainly thinkable to me. Is it thinkable to say that "all markup is in UTF8" as well? Jon Date: Mon, 09 Sep 96 16:01:24 CDT From: Michael Sperberg-McQueen Subject: Re: Questions about the Questions To: W3C SGML Working Group On Mon, 9 Sep 1996 14:34:50 -0400 Martin Bryan said: >Before attempting to answer the questions asked in the forms >associated with DD-1996-002.html it is necessary to raise some >questions about the expectations of our user community. ... >There are many other questions that need to be asked with respect to >the other questions we are asked to vote on, but there is no general >provision for us to attach qualifications or comments to our answers. >I would suggest that a Comments textarea could usefully be added to >each question to allow us to qualify our responses where appropriate. As one of the culprits responsible for the form taken by the voting booth, let me concur in the view that qualifications and comments on the questions before us are clearly necessary and important. Our idea -- my idea, at least, and I think Tim Bray and Jon Bosak agree -- was and remains that such comments are most usefully made on this list, in order that they can contribute to the general discussion. On the specific questions you raise, I will hazard some views, in separate postings (as requested by Jon Bosak), as a way of getting some discussion rolling. I'm not sure whether I am disagreeing with you or not; although your questions are phrased as if neutral, they seem to reflect some strong, though unspoken, biases on your part for or against particular answers. The questions you raise seem eminently relevant to those raised in the voting booth; I respectfully dissent, however, from your suggestion that they are logically prior questions which must be answered *before* those in DD-1996-0002. -C. M. Sperberg-McQueen ACH / ACL / ALLC Text Encoding Initiative University of Illinois at Chicago tei@uic.edu All opinions expressed in this note (except those I have quoted from other authors) are mine. They are not necessarily those of the Text Encoding Initiative, its executive committee or other participants, its sponsors, or its funders. Anyone who says otherwise is wrong. Date: Mon, 09 Sep 96 16:03:54 CDT From: Michael Sperberg-McQueen Subject: character sets To: W3C SGML Working Group Martin Bryan writes: >- the reference concrete syntax only permits the use of Latin >alphanumeric characters in names of elements, attributes and tokens: >should XML be designed to allow users to define elements, attributes >and their values in a form that is dependent on their local language, >or must they restrict themselves to shared names that have meanings >defined in English only? Here's my main concern with proposals to restrict XML to specific coded character sets (other than Unicode or UCS-4): we have a good chance here to provide a strong basis for internationalization, and requiring ISO 646, or any particular flavor of ISO 8859, or even *all* of the flavors of 8859, is not as good as defining XML >from the ground up as language-neutral. We should take good care that XML is compatible with the proposals for internationalization (or, as they say in the trade, i18n) already formulated by the relevant W3C working group. Those are good proposals, and XML should harmonize with them. (It would be nice to have a list of what that might entail, though; any volunteers?) >- the default character set in 8879 matches that of the reference >concrete syntax: should users be able to select which character set >is most appropriate for their documents and specify an SGML >declaration in which only a subset of ISO 10646 is recognized as >valid while still retaining the reference concrete syntax for markup? This does not seem, offhand, to help much in keeping XML simple to understand and implement. Or am I too pessimistic? -C. M. Sperberg-McQueen ACH / ACL / ALLC Text Encoding Initiative University of Illinois at Chicago tei@uic.edu All opinions expressed in this note (except those I have quoted from other authors) are mine. They are not necessarily those of the Text Encoding Initiative, its executive committee or other participants, its sponsors, or its funders. Anyone who says otherwise is wrong. Date: Mon, 09 Sep 96 16:08:31 CDT From: Michael Sperberg-McQueen Subject: quantity sets and capacity sets To: W3C SGML Working Group Martin Bryan writes: >- HTML has extended the Quantities defined in the reference concrete >syntax: should XML offer less flexibility than HTML? I don't know in what way HTML has extended the quantities; can you expand a bit? Nor am I clear what you mean by offering flexibility here. Is eliminating quantity limits entirely, or fixing them all at very high values, flexible? Or to be flexible must we allow users to prescribe name lengths of 6 and litlen of 20? As far as I can tell from reading them, virtually all of the proposals for simplifications or subsets of SGML propose doing one of two things with capacities and quantities: ignoring them, or doing away with them entirely. I have yet to encounter an implementer who said the limits of their implementation were well described -- or described at all! -- by the capacity set; some have said one might use the quantity set to do things like set buffer sizes for memory allocation, but I haven't yet met anyone who said they actually did so. I take the basic idea of the capacity set to be a simple and good one: allow an application to recognize at once, if it cannot handle a document because it is too large, so that the application may fail quickly and gracefully rather than slowly and painfully. I like the idea in theory. It does not seem to me to have proven itself in practice. At least one implementor has told me that all of the code in his parser for checking quantities and capacities was added on after the fact, purely for conformance reasons, and served no purpose in allowing early recognition of memory shortages or other resource problems. Some people might say that a standard which claims it makes no assumptions about implementation strategies ought perhaps to avoid trying to specify things like buffer sizes. >- the SGML community has already agreed on a new set of Quantity >defaults for the next version of SGML: should XML offer less >flexibitity than the next version of SGML This sounds interesting; can you point us to documentation for this new quantity set? This is the first I remember hearing about it. If restricting flexibility allows XML to be more easily implemented, and thus more widely implemented and cheaply supported, than all of 8879, then yes, I'm in favor of XML being inflexible, simple, and cheap, rather than flexible, baroque, and exorbitantly expensive. Where restricting flexibility doesn't help keep the implementation simple, I don't see much point in it. -C. M. Sperberg-McQueen ACH / ACL / ALLC Text Encoding Initiative University of Illinois at Chicago tei@uic.edu &disclaimer; Date: Mon, 09 Sep 1996 16:09:47 -0400 To: bosak@atlantic-83.eng.sun.com (Jon Bosak), tbray@textuality.com From: "Eve L. Maler" Subject: Re: Concrete syntax, character sets Cc: w3c-sgml-wg@w3.org At 01:44 PM 9/9/96 -0700, Jon Bosak wrote: >[Responding to Tim Bray:] >| XML should have *no* concept of quantities. Names, nesting depths, whatever, >| can be as large as required to meet the requirements of the application. >| One straightforward way to do this and preserve compatibility >| with SGML is to require an XML processor to have the capability of writing >| an appropriate SGML declaration to set the quantities high enough to make >| a particular XML DTD valid. > >I agree with Tim's intent here, but aren't some quantities related to >the particular document instance? (Correct me if I'm wrong, but I >seem to recall cases where declarations worked fine for me until a >specific instance blew them up.) This is correct. For example, IDCAP reflects the "bulk" of assigned ID attribute values, which is instance-specific. I suppose the same solution could work, though -- an SGML declaration could be constructed per SGML document, not just per DTD. Eve Date: Mon, 09 Sep 1996 15:12:51 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: Concrete syntax, character sets At 01:44 PM 9/9/96 -0700, Jon Bosak wrote: >| If you want to use anything but 7-bit ASCII in markup, use real SGML. >| XML should have the reference concrete syntax hardwired in. > >Having just gone through a big struggle in WG8 and X3V1 over the ERCS >proposal, I would feel pretty strange about limiting markup to >something that not even Western Europeans could use the way they want >to. I would like to see some serious discussion of this point. OK, for the moment I'll stick to my guns. Here are 2 arguments: 1. Document *data* is (mostly) for people to read, and thus of course has to support the languages they write in. Document *markup* is (mostly) for computer programs to read, plus the occasional unfortunate document designer. Given that these things are already monocased, and by industry habit that I doubt XML will break, short, it's not clear that expressing GI's & attribute names in Cyrillic or Chinese is all that important to the market. 2. Supporting bigger & more complex encodings in markup brings the benefit of making life easier & friendlier for document designers who want to use them. Restricting the markup character set down to 7 bits brings the benefit of making it quicker & easier to generate software that processes such markup. If I didn't already think that the second of these two incompatible benefits was more important, I wouldn't be working on XML. Jon, maybe you could bring some of the ERCS-folks' arguments alive for us? Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Mon, 09 Sep 1996 17:28:26 -0400 To: w3c-sgml-wg@w3.org From: "Eve L. Maler" Subject: Re: Concrete syntax, character sets You ask about quantities, and I reply about capacities. Sorry about that! In fact, some quantities, like LITLEN and ENTLVL, are also related to characteristics of instances. >>I agree with Tim's intent here, but aren't some quantities related to >>the particular document instance? (Correct me if I'm wrong, but I >>seem to recall cases where declarations worked fine for me until a >>specific instance blew them up.) > >This is correct. For example, IDCAP reflects the "bulk" of assigned >ID attribute values, which is instance-specific. I suppose the same >solution could work, though -- an SGML declaration could be >constructed per SGML document, not just per DTD. > > Eve Date: Mon, 9 Sep 1996 16:05:00 -0700 From: bsmith@atlantic-82.eng.sun.com (Bill Smith) To: w3c-sgml-wg@w3.org Subject: Re: Concrete syntax, character sets Time Bray wrote: > 1. Document *data* is (mostly) for people to read, and thus of course > has to support the languages they write in. Document *markup* is > (mostly) for computer programs to read, plus the occasional unfortunate > document designer. Given that these things are already monocased, > and by industry habit that I doubt XML will break, short, it's not > clear that expressing GI's & attribute names in Cyrillic or Chinese is all > that important to the market. Ask the question to the Cyrillic, Chinese, and other markets that can't live with 7-bit ASCII and I suspect you will get a very different answer. XML should *embrace* I18N not merely make it possible. > 2. Supporting bigger & more complex encodings in markup brings the benefit > of making life easier & friendlier for document designers who want to > use them. Restricting the markup character set down to 7 bits brings > the benefit of making it quicker & easier to generate software that > processes such markup. If I didn't already think that the second > of these two incompatible benefits was more important, I wouldn't > be working on XML. I don't see these as incompatible but rather as complementary requirements. XML should be both easy to use (document designers authors) and easy to implement (software developers). I don't see needing to trade them off, at least in this instance. We are agreed that 7-bit ASCII isn't sufficient for XML data. This will require significant effort to (properly) support. Suuprting non-7-bit ASCII in markup is trvial by comparison. What about the UTF8 suggestion for both markup and data? Date: Mon, 9 Sep 1996 14:02:38 -0700 From: bsmith@atlantic-82.eng.sun.com (Bill Smith) To: w3c-sgml-wg@w3.org Subject: Re: Concrete syntax, character sets Glad to see the discussion starting. Martin raised some interesting points and Tim's responses prompted me to reply. Tim Bray wrote: > XML should have *no* concept of quantities. Names, nesting depths, whatever, > can be as large as required to meet the requirements of the application. This I like. > One straightforward way to do this and preserve compatibility > with SGML is to require an XML processor to have the capability of writing > an appropriate SGML declaration to set the quantities high enough to make > a particular XML DTD valid. This I don't like. Requiring that XML processors have this capability (feature?) seems overly restrictive. Noting that it can be done would be sufficient for me. A reference application would be nice - perhaps something available from W3C. > If you want to use anything but 7-bit ASCII in markup, use real SGML. > XML should have the reference concrete syntax hardwired in. I think we should recognize that 7-bit ASCII isn't sufficient for something that professes to be "World Wide". I'm not aware of large technical problems with other encodings for markup but do know 7-bit ASCII restrictions are an issue with many people. I'd like to see XML support other encodings in markup. > *Good* point... with modern parsing and encoding technology, it seems like > it would be easy, and it would certainly be desirable, for XML > data not to be limited to small old character sets. On the other hand, with > XML, ultimate flexibility is of less importance than ease of implementation; > would it be thinkable to say that "all XML data is always in UTF8"? It > seems this would break almost nothing and allow almost anything you'd want > to do. I don't have a problem with UTF8 for data. Why not for markup as well? Date: Mon, 09 Sep 96 19:21:17 CDT From: Michael Sperberg-McQueen Subject: viewing the voting so far To: W3C SGML Working Group A word to the wise. If you look at the tally of voting so far, and it appears not to be including the votes you just cast, and you are using Netscape, you may be a victim of the Netscape's efforts to avoid reloading files already present in the cache. During Daylight Savings time, Netscape apparently often mistrusts the system clock (yours or the server's, I can't remember which) and may reload documents from the cache if they were recently fetched, even if they were more recently updated on the server. Workaround: shift-click on the Reload button. -CMSMcQ Date: Mon, 9 Sep 1996 17:36:27 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org Subject: Linux DSSSL application available A Linux compilation of the Jade alpha build of 1996.08.31 has been put together by Bill Lindsey (thanks, Bill!) and is now available at ftp://sunsite.unc.edu/pub/sun-info/standards/dsssl/jade/1996.08.31 The README file for the distribution appears below. I've taken advantage of this addition to revise the descriptive materials and rename the Win32 and Solaris binary files for greater uniformity, but their contents have not changed. Jon --- Jon Bosak, Online Information Technology Architect Sun Microsystems, 2550 Garcia Ave., MPK17-101, Mountain View, CA 94043 Davenport Group::SGML Open::ANSI X3V1::ISO/IEC JTC1/SC18/WG8::W3C SGML ERB ======================================================================== Jade is an application of DSSSL (ISO/IEC 10179:1996), the international standard language for specifying processing semantics to be applied to documents marked up in languages conforming to SGML (ISO 8879:1986). Jade takes as input an SGML DTD, a document marked up in the language specified by the DTD, and a DSSSL stylesheet. It produces as output an SGML representation of the resulting DSSSL flow object tree or, alternatively, an RTF file of the formatted document. This distribution contains an alpha version of Jade repackaged for easy installation on some common platforms (see HOWTO for a list). For demonstration purposes, it also contains a recent version of the HTML 3.2 DTD, a sample document marked up in HTML 3.2, and an HTML 3.2 DSSSL stylesheet. To obtain the distribution, follow these steps: 1. Log in via anonymous ftp to sunsite.unc.edu (or one of its mirrors). 2. Set "bin" for binary transfer. 3. Change to the directory /pub/sun-info/standards/dsssl/jade/1996.08.31 4. Get the files README, DESCR, and HOWTO. 5. Get the appropriate set of binaries. This will be one of the files jade-win32.zip, jade-sparc-sol2.5.tar.gz, etc. 6. Read DESCR and HOWTO for further directions. Last modified 1996.09.09 jon.bosak@sun.com Date: Mon, 09 Sep 1996 18:32:16 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: An item of interest to this group, I'd think If you go to www.packet.com - HotWired's new repackaged weekly news, which is pretty good - and go to last Friday, you will find a 28 minute RealAudio interview with Marc Andreesen. The guys give him a pretty hard time, and challenge him *specifically*, much to my surprise, on the issue of SGML compliance. Andreesen says (more or less, but the sense is crystal clear) "if it's broken in SGML terms, that's a bug and we'll fix it." Hmmmm Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Mon, 09 Sep 1996 21:35:12 -0500 From: Len Bullard To: Tim Bray CC: w3c-sgml-wg@w3.org Subject: Re: An item of interest to this group, I'd think Tim Bray wrote: > > If you go to www.packet.com - HotWired's new repackaged weekly news, which > is pretty good - and go to last Friday, you will find a 28 minute > RealAudio interview with Marc Andreesen. The guys give him a pretty hard > time, and challenge him *specifically*, much to my surprise, on the issue > of SGML compliance. Andreesen says (more or less, but the sense is > crystal clear) "if it's broken in SGML terms, that's a bug and we'll fix it." > > Hmmmm Very interesting. That sounds like some sort of progress. Let's wait and see. len bullard Date: Mon, 9 Sep 1996 23:17:37 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: tbray@textuality.com CC: w3c-sgml-wg@w3.org Subject: Re: Concrete syntax, character sets [Tim Bray:] | Jon, maybe you could bring some of the ERCS-folks' arguments alive for us? Imagine designing and working with a DTD in which all GIs and attribute names are restricted to the characters 0 through 9. I think that gives a rough idea of what it's like for an Asian restricted to 7-bit ASCII. Jon Date: Tue, 10 Sep 1996 00:56:28 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org Subject: Mail-list stuff Apologies to Steve Newcomb for misspelling his name. Note that Alex Milowski's email address is alex@copsol.com. We have two additions to the list: Derek Denny-Brown, TechnoTeacher (derdb@techno.com) Bill Lindsey, BDM (blindsey@BDMTech.com) Jon To: Michael Sperberg-McQueen , W3C SGML Working Group From: Martin Bryan Subject: Re: character sets Date: Tue, 10 Sep 1996 08:54:16 +0100 Michael Sperberg-McQueen/Tim Bray/Jon Bosak/Bill Smith et al have responded, as I hoped, to my feeler. Rather than write multiple responses I'm going to cut and paste bits of different messages into a single response: >>- the reference concrete syntax only permits the use of Latin >>alphanumeric characters in names of elements, attributes and tokens: >>should XML be designed to allow users to define elements, attributes >>and their values in a form that is dependent on their local language, >>or must they restrict themselves to shared names that have meanings >>defined in English only? > >Here's my main concern with proposals to restrict XML to specific >coded character sets (other than Unicode or UCS-4): we have a good >chance here to provide a strong basis for internationalization, and >requiring ISO 646, or any particular flavor of ISO 8859, or even >*all* of the flavors of 8859, is not as good as defining XML >from the ground up as language-neutral. > >We should take good care that XML is compatible with the proposals for >internationalization (or, as they say in the trade, i18n) already >formulated by the relevant W3C working group. Those are good >proposals, and XML should harmonize with them. (It would be nice to >have a list of what that might entail, though; any volunteers?) In fact i18n was the reason for asking the questions in this format. I will be flying from the Boston meeting of the XML team to Seville for a meeting of the i18n team, and expect to be quizzed on what you guys have decided about supporting multiple languages. The problem that internationalization really presents as far as XML is concernd centres around the use of name token groups. If we restirct these to ISO 646 (the version of ASCII used in 8879) users will not be able to declare tokens that include accented characters (or ideographs) without using entity/character references. Tim Bray and Jon Bosak wrote: >>| If you want to use anything but 7-bit ASCII in markup, use real SGML. >>| XML should have the reference concrete syntax hardwired in. >> >>Having just gone through a big struggle in WG8 and X3V1 over the ERCS >>proposal, I would feel pretty strange about limiting markup to >>something that not even Western Europeans could use the way they want >>to. I would like to see some serious discussion of this point. Jon has heard me blasting off in WG8 and knows that I am a long time supporter of removal of SGML's outdated restrictions. We need a standard for the 21st century multinational environment, not for a 19th century colonialist solution! (And this from a Brit!) >OK, for the moment I'll stick to my guns. Here are 2 arguments: > >1. Document *data* is (mostly) for people to read, and thus of course > has to support the languages they write in. Document *markup* is > (mostly) for computer programs to read, plus the occasional unfortunate > document designer. Given that these things are already monocased, > and by industry habit that I doubt XML will break, short, it's not > clear that expressing GI's & attribute names in Cyrillic or Chinese is all > that important to the market. GIs and attribute names I can agree to force to ASCII (providing the quanitiy issues are resolved properly!). The problem is that the same rules apply in 8879 to attribute values specified in terms of lists of options. This is where the restrictions become totally unacceptable. The problem is that SGML does not distinguish between naming of stored objects (which for program portability should be in a shared character subset) and values of stored objects, which must be as flexible as possible. >2. Supporting bigger & more complex encodings in markup brings the benefit > of making life easier & friendlier for document designers who want to > use them. Restricting the markup character set down to 7 bits brings > the benefit of making it quicker & easier to generate software that > processes such markup. If I didn't already think that the second > of these two incompatible benefits was more important, I wouldn't > be working on XML. User friendliness is far more important than programmer friendliness. If users won't accept the restrictions there is absolutely no point in bothering to write the program in the first place! Personally I strongly support the use of ISO 10646 BMP for both data and markup. I seem to have Bill Smith's support here. >>- the default character set in 8879 matches that of the reference >>concrete syntax: should users be able to select which character set >>is most appropriate for their documents and specify an SGML >>declaration in which only a subset of ISO 10646 is recognized as >>valid while still retaining the reference concrete syntax for markup? > >This does not seem, offhand, to help much in keeping XML simple to >understand and implement. Or am I too pessimistic? No - I was deliberately phrasing the question to invoke pessimism! The problem is that if we agree that data must conform to 10646 then we need to ask ourselves whether all XML tools must support all subsets of 10646. For example, I can envisage a Euro-XML which supports ISO 646, the Latin-1 and Latin-2 subsets of 10646, Greek, Hebrew and Cyrillic, but not the Arabic, Indic or the ideographic subsets. (There are already SGML tools that take this approach!) Should we force an "all of 10646" approach? Is our market really global or do we want to restrict ourselves to that part of the globe that uses the Latin alphabet? There are many marketing issues that need to be raised before make any technical decisions. ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ To: Michael Sperberg-McQueen , W3C SGML Working Group From: Martin Bryan Subject: Re: quantity sets and capacity sets Date: Tue, 10 Sep 1996 08:54:22 +0100 Again I am going to respond to two messages at once: At 10:08 PM 9/9/96 +0100, Michael Sperberg-McQueen wrote: >Martin Bryan writes: >>- HTML has extended the Quantities defined in the reference concrete >>syntax: should XML offer less flexibility than HTML? > >I don't know in what way HTML has extended the quantities; can you >expand a bit? RFC 1866 gives the following definition for Quantities: QUANTITY SGMLREF ATTSPLEN 2100 LITLEN 1024 NAMELEN 72 PILEN 1024 TAGLVL 100 GRPGTCNT 150 GRPCNT 64 (It also extends the default Capacity values.) > Nor am I clear what you mean by offering flexibility >here. I meant values lower than those specified in RFC 1866. > Is eliminating quantity limits entirely, or fixing them all at >very high values, flexible? Or to be flexible must we allow users to >prescribe name lengths of 6 and litlen of 20? Eliminating the values would be much more flexible and very desirable, but you must be aware of one very important thing - by eliminating the current limitations we would AUTOMATICALLY be abandoning the use of SGML's reference concrete syntax. Questionnaire 1, on 0.2, points out that all but one of the proposals we are considering are based on the SGML reference concrete syntax. The reference concrete syntax points, in SGMLREF, to the values in Figure 6 of 8879. If you change any of these values you have defined a variant concrete syntax, not SGML's reference concrete syntax. (There is nothing to stop us defining our own "XML reference concrete syntax" with ultra-high limits.) >As far as I can tell from reading them, virtually all of the proposals >for simplifications or subsets of SGML propose doing one of two things >with capacities and quantities: ignoring them, or doing away with them >entirely. I have yet to encounter an implementer who said the limits of >their implementation were well described -- or described at all! -- by >the capacity set; some have said one might use the quantity set to do >things like set buffer sizes for memory allocation, but I haven't yet >met anyone who said they actually did so. This was the theory of why they were included in the 1980's. With today's dynamic memory allocation solutions they are no longer relevant and can be safely ignored in XML. >I take the basic idea of the capacity set to be a simple and good one: >allow an application to recognize at once, if it cannot handle a >document because it is too large, so that the application may fail >quickly and gracefully rather than slowly and painfully. I like the >idea in theory. It does not seem to me to have proven itself in >practice. At least one implementor has told me that all of the code >in his parser for checking quantities and capacities was added on after >the fact, purely for conformance reasons, and served no purpose in >allowing early recognition of memory shortages or other resource >problems. > >Some people might say that a standard which claims it makes no >assumptions about implementation strategies ought perhaps to avoid >trying to specify things like buffer sizes. They will probably only have an advisory role in the update of SGML. >>- the SGML community has already agreed on a new set of Quantity >>defaults for the next version of SGML: should XML offer less >>flexibitity than the next version of SGML > >This sounds interesting; can you point us to documentation for this new >quantity set? This is the first I remember hearing about it. It has not been publicly documented yet, though I am told that SGML Open have been given the figures and have circulated them to their members. I will try to look up the proposed values sometime. (I can't offhand remember whether WG8 fixed NAMELEN at 32 or 64.) >If restricting flexibility allows XML to be more easily implemented, >and thus more widely implemented and cheaply supported, than all of >8879, then yes, I'm in favor of XML being inflexible, simple, and cheap, >rather than flexible, baroque, and exorbitantly expensive. Where >restricting flexibility doesn't help keep the implementation simple, >I don't see much point in it. Tim Bray wrote: > XML should have *no* concept of quantities. Names, nesting depths, whatever, > can be as large as required to meet the requirements of the application. to which Bill Smith responded: This I like. I concur with Bill Tim wrote > One straightforward way to do this and preserve compatibility > with SGML is to require an XML processor to have the capability of writing > an appropriate SGML declaration to set the quantities high enough to make > a particular XML DTD valid. and Bill responded: >This I don't like. Requiring that XML processors have this capability (feature?) >seems overly restrictive. Noting that it can be done would be sufficient for me. >A reference application would be nice - perhaps something available from >W3C. Again I concur with Bill. This proposal will conflict with Michael's plea about "XML being inflexible, simple, and cheap", to which I think we all concur. However I would add one vital extension: "and capable of handling the majority of documentary data that needs to be transmitted over the Internet"! If it cannot do this it is a waste of effort. We need to make sure XML does not impose unnecssary requirements while ensuring that it has the flexibility users so desperately need. ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ From: "Henry S. Thompson" Date: Tue, 10 Sep 96 09:05:13 BST To: bosak@atlantic-83.eng.sun.com Subject: Re: Concrete syntax, character sets Cc: tbray@textuality.com, w3c-sgml-wg@w3.org I'm actually typing this from Japan, where I've spent the afternoon being shown some SGML work done here in the context of language research, and it is noticeable that all the LOCALLY-produced DTDs use the full 16-bit Japanese character set for markup, otherwise the students couldn't always understand the markup. ht From: Gavin Nicol Date: Tue, 10 Sep 1996 14:31:27 GMT To: bosak@atlantic-83.eng.sun.com CC: tbray@textuality.com, w3c-sgml-wg@w3.org Subject: Re: Concrete syntax, character sets >Having just gone through a big struggle in WG8 and X3V1 over the ERCS >proposal, I would feel pretty strange about limiting markup to >something that not even Western Europeans could use the way they want >to. I would like to see some serious discussion of this point. I would feel somewhat strange about not supporting native language markup, particularly as we're going to have to use a variant concrete syntax to support native language content. It seems to me that the most reasonable thing to do would be to decide upon a syntax that used ISO 10646 for both data and markup... We had this same discussion in HTML-WG, and I pushed for a syntax that used ISO 10646 as the document character set. This, and other discussion led to the HTML I18N draft, which is moving towards proposed standard (and it'll probably be adopted by W3C in some HTML revision). It seems that in the interest of compatibility, we should have a similar concrete syntax, though with an extended markup character repertoire. The ERCS work that Rick did is very important, and I do not think it is a great burden for XML browsers to support it, at least to a minimal degree. In fact, given that we also have content negotiation in the WWW, and that HTTP 1.1 is becoming somewhat stricter on content labelling requirements, XML browsers would not need to support any encodings other than those deemed important by the companies producing them. >It's certainly thinkable to me. Is it thinkable to say that "all >markup is in UTF8" as well? No, it's not, because then you'd also require all content to be in UTF8, and many users have no way of creating such data. In the best cases, producing such data usually involves a conversion somwhere. Again, we had the same discussion in HTML-WG. There are many good reasons for selecting a single document character set, and then just looking upon SJIS and whatnot as encodings. From: Gavin Nicol Date: Tue, 10 Sep 1996 14:45:41 GMT To: tbray@textuality.com CC: w3c-sgml-wg@w3.org Subject: Re: Concrete syntax, character sets >1. Document *data* is (mostly) for people to read, and thus of course > has to support the languages they write in. Document *markup* is > (mostly) for computer programs to read, plus the occasional unfortunate > document designer. Given that these things are already monocased, > and by industry habit that I doubt XML will break, short, it's not > clear that expressing GI's & attribute names in Cyrillic or Chinese is all > that important to the market. For document designers, my experience has been that about 50% or the Japanese people I talk to wish for Japanese markup. The people who are happy with ASCII markup, usually feel that it is better for interoperability. However, the people who want native language markup usually cite usability as the prime reason: it is much more understandable to have "bunsho" in a Japanese document, and in stylesheets, it becomes even more desireable, they say. For Japanese, it is not an overly large problem, because they have a phonetic spelling of Japanese that uses ASCII (romanji), but for other languages, ASCII phonetics as markup don't win. >2. Supporting bigger & more complex encodings in markup brings the benefit > of making life easier & friendlier for document designers who want to < use them. Restricting the markup character set down to 7 bits brings > the benefit of making it quicker & easier to generate software that > processes such markup. If I didn't already think that the second > of these two incompatible benefits was more important, I wouldn't > be working on XML. This is a fallacy. If you are going to support native language content, you will have to have some way of decoding the octet stream in order to correvtly parse the document (otherwise you run into problems with bits of character codes that could be mistaken for markup). If you have a decoding module on the stream (or a bit combination transformation filter), you will also be able to support native language markup. From: Gavin Nicol Date: Tue, 10 Sep 1996 14:59:38 GMT To: ht@cogsci.ed.ac.uk CC: bosak@atlantic-83.eng.sun.com, tbray@textuality.com, w3c-sgml-wg@w3.org Subject: Re: Concrete syntax, character sets >I'm actually typing this from Japan, where I've spent the afternoon >being shown some SGML work done here in the context of language >research, and it is noticeable that all the LOCALLY-produced DTDs use >the full 16-bit Japanese character set for markup, otherwise the >students couldn't always understand the markup. One more reason is zenkaku/hankaku spaces etc.... Date: Tue, 10 Sep 1996 09:27:01 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: Concrete syntax, character sets I feel I should apologize in advance here; I am keenly aware of the fact that the 7-bit-centric-ness of contemporary computer systems is redolent of a lot of really bad Western attitudes, so arguing against complete i18n generality makes me really uncomfortable. And in fact some of the things that Henry & Gavin & Martin have been saying are very important. But, let's review the design principles. XML has to be *easy to write programs for* - I draw your attention to http://www.textuality.com/sgml-erb/dd-1996-0001.html, clause 4, which says that anyone competent ought to be able to cook up an XML processor in a few days. For this reason, we are probably going to discard a bunch of pieces of SGML that are really potentially very valuable, such as abstract syntax and SHORTREF and LINK - simply because we are happy with the trade-off of increased ease of implementation for loss of functional richness. So even though I may end up agreeing with Gavin, for the moment I'm going to argue the minimalist position, because someone needs to. My informal interpretation of our clause 4 ("It shall be easy to write programs which process XML documents") has been more or less, "I can do it with flex and yacc". Thus, what we need to do is avoid the extremes of (a) retreating into a 7-bit grass hut, and (b) trying to build something handles all the arcana of bit-patterns/glyphs/charsets/encodings. Rather, we want something that hits the right trade-off along the spectrum. Having said all that: At 02:31 PM 9/10/96 GMT, Gavin Nicol wrote: >I would feel somewhat strange about not supporting native language >markup, particularly as we're going to have to use a variant concrete >syntax to support native language content. It seems to me that the >most reasonable thing to do would be to decide upon a syntax that >used ISO 10646 for both data and markup... Well, data for sure. The problem is, if we keep markup in 7-bit land, I'm pretty darn sure I can build an efficient parsing/validating system out of flex & yacc in a week or so (because I have), and utf8 data, for example, won't break anything. Now, I'm not sure I *can't* do this if we let markup out of the ASCII box, but it's a question that we have to think about carefully, because we're in danger of compromising a central design goal. > you'd also require all content to be in >UTF8, and many users have no way of creating such data. In the best >cases, producing such data usually involves a conversion somwhere. Hmm, for us Westerners there's no problem, because the standard SGML repertoire is utf8 as it sits (right?). You're right that your average Japanese editing system doesn't emit UTF8... but I still think that it might be reasonable to say that "in XML, it must be utf8" - so the conversion gets applied on the way in. Question: conversions such as {S,N,EUC}JIS<->UTF8 and Big5<->UTF8 look very easy in principle - are they, in practice? And does software exist? >Again, we had the same discussion in HTML-WG. There are many good >reasons for selecting a single document character set, and then just >looking upon SJIS and whatnot as encodings. I think there are many good reasons - overwhelmingly good - for the SGML case. For XML, we should think about sweeping all the charset/encoding subtleties under the rug: it's UTF8 and that's all there is to it. Once again, if it's not crystal clear - we want anyone who wants to write a program to process XML to be able to go to one place and look at one simple set of rules, and we want the program, once written, to have a very high degree of probability of processing any XML doc in the world *as it sits*. If we get this, and as a consequence there is some proportion of documents that XML cannot address, that's a fair price to pay. E.g., a trade-off such as "If you *REALLY* *MUST* have the documents in the repository in native Shift-JIS, and can't do conversions in & out, then you have to use real SGML, because it has the machinery for that." Once again, I'm *really* not a monoglot cave-dwelling bigot. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Tue, 10 Sep 96 11:35:26 CDT From: Michael Sperberg-McQueen Subject: Re: quantity sets and capacity sets To: Martin Bryan , W3C SGML Working Group On Tue, 10 Sep 1996 04:05:43 -0400 Martin Bryan said: >Eliminating the values would be much more flexible and very desirable, >but you must be aware of one very important thing - by eliminating the >current limitations we would AUTOMATICALLY be abandoning the use of >SGML's reference concrete syntax. Questionnaire 1, on 0.2, points out Is this a problem? If so, what? If not, why are you raising it? >that all but one of the proposals we are considering are based on the >SGML reference concrete syntax. The reference concrete syntax points, Sorry, my slip. The note to item 0.2 may be misleading, since while all the schemes mentioned are based on the reference concrete syntax, four of the eight propose to ignore quantities and capacities. They do not propose an alternate set (unless memory fails me); they simply propose to ignore their values. Like the other four, therefore, they need no concept of a difference between concrete and abstract syntax. We are in any case not limited *at all* to the proposals summarized in document DD-1996-0002; they may be regarded as straw proposals, with the subtle difference that unlike a typical 'straw proposal' they already exist independently of this group. -C. M. Sperberg-McQueen ACH / ACL / ALLC Text Encoding Initiative University of Illinois at Chicago tei@uic.edu To: w3c-sgml-wg@w3.org From: Martin Bryan Subject: Common Constructs in XML (Clause 9) Date: Tue, 10 Sep 1996 18:48:29 +0100 Despite Michael's request that messages be kept to a single subject within the list I have decided not to bombard you all with one message per point I want to make, but have tried to cover questions raised by one of the set of forms in one posting. This set of points relates to dd-1996-0002-f5.html, which deals with Common Constructs. 9.4 (a) Why are Bas and Min in the Does not apply column? Surely data entities can be referenced in any SGML documents, even if subdocuments cannot. The real question is whether this distinction should be made for XML. 9.4 (b) SGML does not "forbid" references to undeclared entities - it simply makes them a reportable error. In what way could XML differ from SGML here? 9.4.5 I would like XML to only allow a REFC to be used to end an entity reference. 9.6 A simpler set of delimiter roles will probably be needed if we are to make parsing easy enough for a new CS batchelor to handle in a week ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ To: w3c-sgml-wg@w3.org From: Martin Bryan Subject: Element Structure for XML (Clause 7) Date: Tue, 10 Sep 1996 18:48:27 +0100 Despite Michael's request that messages be kept to a single subject within the list I have decided not to bombard you all with one message per point I want to make, but have tried to cover questions raised by one of the set of forms in one posting. This set of points relates to dd-1996-0002-f3.html, which deals with Element Structure, and also to the points relating to Processing Instructions in dd-1996-002-f4.html 7.1 One way to overcome the differences of how systems handle DocTypes might be for XML to only allow DTDs to be referenced using a public identifier (that would be resolved through a catalog), with no local subset, e.g. Such a line must be the first declaration in the document instance, but could be preceded by a comment (or PI) that indicates the public identifier of the SGML declaration to be used when parsing the document and DTD. 7.4 (a) Empty and unclosed start-tags are obviously not needed by XML, but could there be a case for NET-enabled start-tags to delimit numeric data? 7.6 (4th bullet) PIs don't make sense on a network. Their use should be restricted to the prolog, where they could be useful in setting up the parser (e.g. to identify a relevant SGML declaration). The thought of allowing a processing instruction within a document instance that is going to be displayable by many different plug-ins scares the hell out of me. 7.9.4.3 A useful restriction might be that the entities referred to can only be data entities defined using SGML (NDATA "SGML"), or valid SGML subdocuments. 8 (b) I feel that entities that map to Processing Instructions are not conducive to portability either. You can only refer to such entities within a document instance, and I have already made a point about the inamissibility of PIs in portable document instances. 8(c) If PIs are retained then PIC should be changed so that it is not > as this is required for many processing instructions. (An alternative may be to allow a character reference to be entered within a PI, but this would make XML incompatible with SGML.) ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ Date: Tue, 10 Sep 96 12:38:48 CDT From: Michael Sperberg-McQueen Subject: Re: Concrete syntax, character sets To: W3C SGML Working Group Help! We need answers to some simple factual questions. I don't know the answers. I sure hope someone else does. For example: - How easy is it to find libraries to deal with ISO 10646 in general, or Unicode in general, or UTF-8 in particular? - Do these libraries coexist well with current versions of yacc, lex, bison, and flex? - Are there relatively simple ways of either converting from the system character sets of prominent platforms into Unicode / UTF-8, or ways of persuading standard tools to emit Unicode/utf-8 data? I would like, on principle, to commit ourselves to proper support for i18n -- but I would equally like to keep to our goal of twenty pages of documentation. I think we can have both if: - there are good libraries, freely available, to handle wide characters -- at least utf-8 encoding of Unicode ... - they work with yacc and lex (or, probably more important, flex and bison) and reasonably widely available C compilers (notably gcc) - we can include a clear set of dos and donts for programmers to follow, so that those used to thinking of characters as seven-bit numbers can have a prayer of writing code that actually works with wide character sets. - we can point people to sources of information and instruction. - we can specify a reasonably straightforward way to work with XML on systems that don't have system support for Unicode. Current Java implementations may be worth emulating here; they seem to work very well with non-Unicode data despite the unbending fundamental principle that Java data and program source are all, always, Unicode, period. It isn't enough for internationalization to be *possible*; we need to say, crisply and clearly and *briefly*, what the requirements are and how to meet them. About the absolute necessity of non-Latin-1 characters and the relative importance of ease of implementation and support for culturally apt markup -- well, SGML has *always* made it possible to use non-Latin-1 characters, and HTML has not; at the same time, HTML has been relatively easy to implement, and SGML has not. Which is supported by more software? Which is used by more people? As noted: I'm in favor of i18n. The best way to advance that cause, though, is to provide a simple spec that shows implementors how to support i18n. Complexity of treatment, or even worse complexity of implementation, will not help the cause. If we want XML to support i18n, we have to find ways to help implementors find their way through the attendant problems. -C. M. Sperberg-McQueen ACH / ACL / ALLC Text Encoding Initiative University of Illinois at Chicago tei@uic.edu To: w3c-sgml-wg@w3.org From: Martin Bryan Subject: Entity structure for XML (Clause 6) Date: Tue, 10 Sep 1996 18:48:23 +0100 Despite Michael's request that messages be kept to a single subject within the list I have decided not to bombard you all with one message per point I want to make, but have tried to cover questions raised by one of the set of forms in one posting. This set of points relates to dd-1996-0002-f2.html, which deals with Entity Structure. 6.1 a) Should we allow SDATA entities as these require a system independent translation? This might be acceptable if XML were to fix on one particular notation, but would not generally be conducive to the receipt and display of XML files by any browser with a suitable plug-in. b) Should we predefine a set of NDATA notations that will be acceptable to all browsers? What mechanism would there be for extending the list of notations to take account of new developments in general support for a new notation (e.g how will we allow for MPEG-4 in 2000)? c) Should the use of subdocument entities be an option? 6.2 If you do not constrain parsing in a left-to-right manner how else would you deal with the detection of nested marked sections? Do you just ban these? 6.2.2 Should we require that entity references end with a REFC code (the semicolon)? 6.2.3 On the question of SGML declaration transmission I feel that the current text is in conflict with current practice. Most systems have an implied SGML declaration, which is normally an extension of the one defined in 8879 that is supposed to be used whenever you do not specifically receive one. WG8 have agreed in principle that it should be possible to identify a relevant SGML declaration using a public (or system) identifier in the same way that you can with a DocType declaration. This is the best long term solution. If XML is to have a fixed SGML declaration then we have no problem. If, to suit regional character set requirements, there is more than one option, there should be a set of predefined SGML declarations that can be referenced via a public identifier (perhaps through a PI or comment declaration) in front of the DocType declaration. ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ To: w3c-sgml-wg@w3.org From: Martin Bryan Subject: SGML Declaration in XML (Clause 13) Date: Tue, 10 Sep 1996 18:48:37 +0100 Despite Michael's request that messages be kept to a single subject within the list I have decided not to bombard you all with one message per point I want to make, but have tried to cover questions raised by one of the set of forms in one posting. This set of points relates to dd-1996-0002-f8.html, which deals with the SGML Declaration. 13.3 We should force XML to SCOPE DOCUMENT 13.4 There needs to be a separate vote on the various subclauses of the SYNTAX clause as there are lots of things associated with naming that need to be discussed. In particular I would point you to the ERCS Technical Corrigendum which has recently been sent to your national standards body for approval by ISO. (Its formal number is JTC1 N4178. Get your national body to vote YES by October 30th! You can get the source of this document online from http://www.ornl.gov/sgml/wg8/document/1861.htm.) This TC covers how to specify ISO 10646 characters as valid SGML name characters in an SGML declaration. 13.5 - SHORTTAG SHORTTAG YES is required to allow attribute values to be entered without literals (and without attribute name and value indicators) but XML should forbid the other options SHORTTAG currently permits. (In the next version of SGML these two roles of SHORTTAG will be clearly distinguished.) ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ Date: Tue, 10 Sep 1996 14:46:42 -0400 From: "Steven R. Newcomb" To: w3c-sgml-wg@w3.org Subject: Re: Concrete syntax, character sets > [Responding to Tim Bray:] > | At 07:19 PM 9/9/96 +0100, Martin Bryan wrote: > | If you want such a distinction, use real SGML. XML should use a hardwired > | concrete syntax. > > I totally agree. I agree to the extent that XML as defined right now should use a hardwired concrete syntax, but to force only one such syntax is asking for obsolescence. There needs to be a way to specify 'versions' of the concrete syntax used, where you might have a 7-bit ascii version and a Unicode version, etc. By putting the emphasis on a small number of various 'versions' of XML's concrete syntax, the XML documents themselves can be kept simple and concise. From the point of view of a parser, it is easiest if all the variables are known beforehand. The less a parser must adapt to the document itself, the easier it is to build the parser. There should be a declaration which specifies the syntax, using a public identifier to specify which XML syntax is to be used. As it stands now, there are very few tools which support portable text beyond 7-bit ascii in any reliable way. Given this framework, I think XML should start with ascii, as a base. Part of the whole concept here, as I saw it, was that I could fire up vi or notepad and view a document. (Though I might not enjoy doing it.) That paradigm breaks if XML tries to leap-frog currently used technology too much. --------------------------------------------------------------------------- Derek Denny-Brown | Technical Staff @ TechnoTeacher, Inc. http://www.techno.com/~derdb/ | work-phone: (716) 389-0963 SGML/HyTime/DSSSL/WWW | http://www.techno.com/ *************************************************************** * Steven R. Newcomb | President * * direct +1 716 389 0964 | TechnoTeacher, Inc. * * main +1 716 389 0961 | (courier: 3800 Monroe Avenue, * * fax +1 716 389 0960 | Pittsford, NY 14534-1330 USA) * * Internet: srn@techno.com | P.O. Box 23795 * * FTP: ftp.techno.com | Rochester, New York 14692-3795 * * WWW: http://www.techno.com | USA * *************************************************************** From: Gavin Nicol Date: Tue, 10 Sep 1996 19:35:19 GMT To: srn@techno.com CC: w3c-sgml-wg@w3.org Subject: Re: Concrete syntax, character sets >I agree to the extent that XML as defined right now should use a >hardwired concrete syntax, but to force only one such syntax is asking >for obsolescence. It depends. If you go for a 32 bit character repertoire (10646) for the document character set, then you'll be fine for the forseeable future. >There needs to be a way to specify 'versions' of the concrete syntax >used, where you might have a 7-bit ascii version and a Unicode >version, etc. I disagree. This can be handled quite adequately by the content negotiation mechanisms of the WWW. Also, different syntaxes means that numeric character references and other such things become dependent upon a given syntax, which could be a pain in email, and in translation servers. >As it stands now, there are very few tools which support portable >text beyond 7-bit ascii in any reliable way. Given this framework, I >think XML should start with ascii, as a base. Part of the whole >concept here, as I saw it, was that I could fire up vi or notepad and >view a document. (Though I might not enjoy doing it.) That paradigm >breaks if XML tries to leap-frog currently used technology too much. The fact that you use 32 bits for the document character set does not mean that you must use 32 bits internally. You can fake it by either using UTF-8 internally, or by restricting the acceptable input to 7 bit data via content negotiation. So long as the perser behaves in a conformant manner with the data that it takes in, all will be well. There are issues with numeric charcater references, SDATA, and other such constructs, ,but a) for portability, one needs to be careful here anyway, and b) SGML doesn't define exactly what a parser/application should do with data that it cannot dispaly, so any recovery scheme is acceptable. From: Gavin Nicol Date: Tue, 10 Sep 1996 19:55:42 GMT To: tbray@textuality.com CC: w3c-sgml-wg@w3.org Subject: Re: Concrete syntax, character sets >>syntax to support native language content. It seems to me that the >>most reasonable thing to do would be to decide upon a syntax that >>used ISO 10646 for both data and markup... > >Well, data for sure. The problem is, if we keep markup in 7-bit land, >I'm pretty darn sure I can build an efficient parsing/validating system >out of flex & yacc in a week or so (because I have), and utf8 data, for >example, won't break anything. Now, I'm not sure I *can't* do this if >we let markup out of the ASCII box, but it's a question that we have to >think about carefully, because we're in danger of compromising a central >design goal. If you don't restrict yourself to pure ASCII, you can still write a minimal parser in a week, or less, using flex and yacc. Yes, I have done this, and no, it is not difficult at all (in fact, I used a variant of the TEI grammars). >> you'd also require all content to be in >>UTF8, and many users have no way of creating such data. In the best >>cases, producing such data usually involves a conversion somwhere. > >Hmm, for us Westerners there's no problem, because the standard SGML >repertoire is utf8 as it sits (right?). You're right that your average >Japanese editing system doesn't emit UTF8... but I still think that it >might be reasonable to say that "in XML, it must be utf8" - so the >conversion gets applied on the way in. Question: conversions such as >{S,N,EUC}JIS<->UTF8 and Big5<->UTF8 look very easy in principle - are they, >in practice? And does software exist? You are getting closer to what I am thinking, and indeed, what you propose would be one scenario in what I have in mind. There are 2 problems here: 1) that some people confuse coded character sets and encodings, and 2) it depends on what you see an SGML parser manipulating. I imagine a system that has a parser which deals in a single normalised form (internal encoding). Whatever the data is actually encoded in get's converted into this normalised form, and if we have a single document character set, this conversion can be done blindly. I recommend that the syntax use ISO 10646 because it covers a great number of languages. Note that I do not believe it wise to prescribe what the *internal* representation is, nor the number of possible encodings and XML parser should handle. Note that the tables for writing converters from SJIS, EUC, etc. to UNICODE are publically available, and it is easy to use them to automatically generate converters for what you wish to handle. >I think there are many good reasons - overwhelmingly good - for the >SGML case. For XML, we should think about sweeping all the >charset/encoding subtleties under the rug: it's UTF8 and that's all >there is to it. UTF8 doesn't solve the worlds problems. I think we can fix the character repertoire, but fixing the encoding is arbitrary, and prescribes certain implementation details. It also complicates usage. From: Gavin Nicol Date: Tue, 10 Sep 1996 20:17:03 GMT To: U35395@UICVM.CC.UIC.EDU CC: w3c-sgml-wg@w3.org Subject: Re: Concrete syntax, character sets > - How easy is it to find libraries to deal with ISO 10646 in general, >or Unicode in general, or UTF-8 in particular? There are a few. I remember a group in France (I just moved so I've lost my URL list for the moment) produced such a library recently. Also, FreeBSD has a fairly good runes package. MSVC, and most modern OS's also come with wide version of all the str* faimily. > - Do these libraries coexist well with current versions of yacc, lex, >bison, and flex? Yes, thought not as easily as one would wish, it is still easy. I'm also surprised that everyone is so hooked on YACC. There are better tools available, and for the trivial grammars we are talking about, hand written parsers would take very little time to write. > - Are there relatively simple ways of either converting from the >system character sets of prominent platforms into Unicode / UTF-8, or >ways of persuading standard tools to emit Unicode/utf-8 data? Yes. The tables are available at www.unicode.org, and writing a tool to take the tables, and turn them into a conversion engine is fairly trivial. > - there are good libraries, freely available, to handle wide >characters -- at least utf-8 encoding of Unicode ... Most OS's/compilers have them... > - they work with yacc and lex (or, probably more important, flex and >bison) and reasonably widely available C compilers (notably gcc) This requires a little work, but not much. GCC can support wide characters, though the input side of GCC (last time I looked) still required more work to be truly I18N. The libraries from the GNU folk do support wide characters. > - we can include a clear set of dos and donts for programmers to >follow, so that those used to thinking of characters as seven-bit >numbers can have a prayer of writing code that actually works with wide >character sets. If client and servers do content negotiation correctly, a 7 bit parser will never see wide data (except via numeric character references etc. which can he handled in any way they wish). >- we can point people to sources of information and instruction. The I18N pages at W3C are a reasonable starting point. > - we can specify a reasonably straightforward way to work with >XML on systems that don't have system support for Unicode. Current >Java implementations may be worth emulating here; they seem to work >very well with non-Unicode data despite the unbending fundamental >principle that Java data and program source are all, always, Unicode, >period. Well. JAVA doesn't yet handle Japanese properly... nor do it's localised input streams work properly... >It isn't enough for internationalization to be *possible*; we need to >say, crisply and clearly and *briefly*, what the requirements are and >how to meet them. Easy to do. If we stick to a single document character set, the all we need to do is require (as the HTML I18N draft does) that all numeric character references be resolved in the document character set, whuich in practise *may* mean that an XML application just tosses data outside the acceptable range of it's internal representation. >markup -- well, SGML has *always* made it possible to use non-Latin-1 >characters, and HTML has not; at the same time, HTML has been relatively >easy to implement, and SGML has not. Which is supported by more >software? Which is used by more people? This is not due to I18N problems, though they certainly play a role. I would argue that other "features" are more to blame. >As noted: I'm in favor of i18n. The best way to advance that cause, >though, is to provide a simple spec that shows implementors how to >support i18n. Complexity of treatment, or even worse complexity of >implementation, will not help the cause. If we want XML to support >i18n, we have to find ways to help implementors find their way through >the attendant problems. Once the core syntax is decided, I would be more than happy to define the I18N features required. Actually, we could probably get Rick to join in that effort as well, to our benefit. Date: Tue, 10 Sep 1996 13:51:16 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: Concrete syntax, character sets Based on Gavin's experience that standard parsing tools can do the right thing with 10646 encodings, it seems like a very strong candidate for the best balance between flexibility, generality, and ease of implementation is: All XML documents will be encoded entirely in UTF8, data and markup. An XML processor will not perform any conversions on the data or markup, but will pass the data and markup to applications as they appear in the document. It seems like this tells implementors and users of tools *exactly* what they have to do, leaves no wriggle room, makes us language-independent to the extent that 10646 does (hard to beat), and supports implementation with standard tools. Obviously there are ways in which this could usefully be generalized; do any of these generalizations confer sufficient benefits to users that they are worth the extra implementation complexity? Gavin writes: >UTF8 doesn't solve the worlds problems. I think we can fix the >character repertoire, but fixing the encoding is arbitrary, and >prescribes certain implementation details. It also complicates usage. This is true; but I don't think the UTF-8 solution complicates usage, it just offloads the content conversion/interpretation problem from the parser. And the benefit - that anyone, anywhere, can write a simple program that will read *any XML document in the world* and, without recourse to any metadata, know what the bits mean - seems pretty large to me. Obviously, it would be of substantial public benefit to distribute, along with XML, a library of routines that convert stuff between UTF and {UCS*, ISO-8859-*, *JIS, etc...}. In fact, since the XML spec should include the API to the parser, we might even consider making at least some of these compulsory. But that's orthogonal to what the parser does. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 To: w3c-sgml-wg@w3.org From: Martin Bryan Subject: DTDs in XML (Clause 11) Date: Tue, 10 Sep 1996 18:48:32 +0100 Despite Michael's request that messages be kept to a single subject within the list I have decided not to bombard you all with one message per point I want to make, but have tried to cover questions raised by one of the set of forms in one posting. This set of points relates to dd-1996-0002-f7.html, which deals with DTDs. 11.3.2 This clause also contains (through a revision) a recommendation that all attributes whose declared value is ID should have the same name. Does XML want to enforce this horrible recommendation? WG8 have agreed in principle that there can be more than one attlist declaration for an element in future versions of SGML. It is possible that there will be new rules about the interpretation of attlists in which the same attribute name occurs in two declarations for a particular element. 11.3.4 (a) As well as deciding whether or not #CONREF should be retained the XML review process should also determine what is meant by #IMPLIED. Should it mean inheritance (a la TEI) or a system defined default, or a random value applied by the system? If inheritance is not accepted then we will need to define #IMPLIED clearly enough to ensure that different plug-ins do not assign different implied values. In particular the meaning of #IMPLIED with respect to elements defined using ID as the default value needs to be considered carefully. 11.3.4 (b) Remember that empty literals only apply when the attribute is of type CDATA. It then has a very distinct meaning - an empty string. This is definitely not the same as an implied value. ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ To: w3c-sgml-wg@w3.org From: Martin Bryan Subject: Markup Declarations in XML (Clause 10) Date: Tue, 10 Sep 1996 18:48:35 +0100 Despite Michael's request that messages be kept to a single subject within the list I have decided not to bombard you all with one message per point I want to make, but have tried to cover questions raised by one of the set of forms in one posting. This set of points relates to dd-1996-0002-f6.html, which deals with Markup Declarations. 10.1.6 I would personally like to ban system identifiers, making the use of public identifiers resolvable via a catalog compulsory within XML. 10.1.7 The current restriction of public identifiers to minimal literals that can only contain alphanumeric characters and a small set of special (punctuation) characters is becoming archane. It is a definite barrier to the international acceptability of SGML. XML should allow any ISO 10646 character to be used as an owner identifier or for the public text description, but not for the other parts of a formal public identifier. 10.4 Marked sections are useful in DTDs (a la HTML and TEI) but it might be valid to ban them within transmitted document instances conforming to XML. (XML should really be for the transmission of version independent documents. RCDATA should have its entities resolved before tranmission and with CDATA should have any embedded markup delimiters replaced by character references.) 10.4.1 (a) WG8 has recently decided that it will allow boolean operations to be performed on marked section status keywords! (Don't ask me what this really implies, I haven't got that far yet!!) We could place a restriction on the use of TEMP as this is just a mickey mouse version control mechanism. 10.5.3 For reasons stated elsewhere I would like to see SDATA and PI entities banned from XML. ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ From: bbauma1@cs.umbc.edu To: w3c-sgml-wg@w3.org Date: Wed, 11 Sep 1996 07:15:43 +0000 Subject: Re: Concrete syntax, character sets Priority: normal > * Notwithstanding the last point, we can (if we want) go farther than > saying that 10646 is the XML character set (in the SGML sense) and > also say that UTF-8 is the only (or recommended, or expected, or > default) encoding that we shall/should/expect to find when we open a > file containing an XML document or receive a byte stream during an > HTTP session. > > Right? > > Jon This sounds good as long as UTF-8 is recommended or defaulted. The other issue is what to present at the parser's API. Here to I do not want to be restricted to a UTF-8 encoding because if I were to write a parser in JAVA, a UTF-16 encoding would be more appropriate. Instead I view the parser proper as excepting UCS-4 on input and output. Encoding should be handled outside the parser in the storage manager. B. Todd Bauman Graduate Student University of Maryland, Baltimore County To: Gavin Nicol From: Martin Bryan Subject: Re: character sets Cc: U35395@UICVM.CC.UIC.EDU, w3c-sgml-wg@w3.org Date: Wed, 11 Sep 1996 06:47:07 +0100 Gavin >>In fact i18n was the reason for asking the questions in this format. I will >>be flying from the Boston meeting of the XML team to Seville for a meeting >>of the i18n team, and expect to be quizzed on what you guys have decided >>about supporting multiple languages. > >Is this the one organised by Mr. Carrasco? I may be attebnding it also. Hey great - at last we will get time for a chin-wag! ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ Date: Tue, 10 Sep 1996 22:32:04 -0700 From: bosak@atlantic-83.Eng.Sun.COM (Jon Bosak) To: gtn@ebt.com CC: tbray@textuality.com, w3c-sgml-wg@w3.org Subject: Re: Concrete syntax, character sets [Gavin:] > some people confuse coded character sets and encodings Yeah. Me, for instance. I apologize for getting this mixed up (once again) in an earlier comment. Let's review the basics for us slower members of the class. * The character set allowed in markup or data (e.g., 10646 BMP) and the character encoding put on the wire or in a disk file (e.g., UTF-8) are completely different issues. * We can specify 10646 as the character set (in the SGML sense) for all XML documents while still allowing a much more limited encoding (such as 7-bit ASCII) for the transfer of a particular document instance, assuming that information about which encoding is being used is conveyed when the transmission is established. * Notwithstanding the last point, we can (if we want) go farther than saying that 10646 is the XML character set (in the SGML sense) and also say that UTF-8 is the only (or recommended, or expected, or default) encoding that we shall/should/expect to find when we open a file containing an XML document or receive a byte stream during an HTTP session. Right? Jon From: bbauma1@cs.umbc.edu To: w3c-sgml-wg@w3.org Date: Tue, 10 Sep 1996 22:34:45 +0000 Subject: Re: Concrete syntax, character sets Priority: normal Although, I have a fairly limited grasp on ISO 10646 / UNICODE Ver. 2 I do have experience encoding TEI documents in a variety of unusual languages, and as such I strongly support Gavin's ideas. I believe that unless a strong case can be made to the difficulty of implementation, standardizing on the ISO 10646 character repertoire and not a particular encoding of that repertoire, to be the best choice. I agree with Tim Bray in that I do not want the XML parser to be responsible for encoding / decoding a particular character representation, and as such do not want UTF-8 to be specified as its internal format. As a matter of practicality, I realize that UCS-4 is not something that one would want to send across the "wire" and in that sense specifying that UTF-8 be the the initial encoding for transmission is fine. (Although the Reuters encoding scheme that I just read about in the UNICODE conference proceedings is very intriguing.) I also agree with Tim that a library of encoding conversion routines that fuction outside of the parser would be very useful in a reference implementation. Let me also weigh in on the side of having all markup and content in the same character set including GI's and attributes. Finally, I am a little concerned with the partitioning of functionality between XML and SGML. In particular, if XML becomes as successful as I hope, there may be strong reasons (i.e. product availability / price) for not using SGML and making due with XML. XML therefore cannot rely too heavily on the availability of its more functional parent to mitigate its own limitations. B. Todd Bauman Graduate Student University of Maryland, Baltimore County From: Gavin Nicol Date: Tue, 10 Sep 1996 14:58:47 GMT To: mtbryan@sgml.u-net.com CC: U35395@UICVM.CC.UIC.EDU, w3c-sgml-wg@w3.org Subject: Re: character sets >In fact i18n was the reason for asking the questions in this format. I will >be flying from the Boston meeting of the XML team to Seville for a meeting >of the i18n team, and expect to be quizzed on what you guys have decided >about supporting multiple languages. Is this the one organised by Mr. Carrasco? I may be attebnding it also. Date: Tue, 10 Sep 1996 20:11:22 -0700 From: bosak@atlantic-83.Eng.Sun.COM (Jon Bosak) To: w3c-sgml-wg@w3.org Subject: Additions to the WG New participants: Catherine Hamon, Hightext (hamon@hightext.com) Carol Ellerbeck, INSO (Carol_Ellerbeck@inso.com) Jon Date: Tue, 10 Sep 1996 22:45:26 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: The Hypermail archive I note that the hypermail archive at http://www.w3.org/pub/WWW/Archives/Public/w3c-sgml-wg/ is kind of confusing and a bit hard to read. For people just joining the group, or who want to retrace the discussion on a particular issue, I find the "thread index" to be a lot more useful - it's signposted from the page above, or available directly at http://lists.w3.org/Archives/Public/w3c-sgml-wg/threads.html This is best read from the bottom up, using the 'Prev' links, and going back to the list when they (occasionally) peter out. I've seen hypermail archives work better than this - are we doing something wrong here? Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Wed, 11 Sep 1996 12:16:14 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: tbray@textuality.com cc: w3c-sgml-wg@w3.org Subject: Re: The Hypermail archive | I've seen hypermail archives work better than this - are we doing | something wrong here? The person who set up W3C's list machinery has left and the remaining staff don't have the resources to make changes to the software. For the moment, we're stuck with it the way it is. Jon Date: Wed, 11 Sep 1996 13:13:36 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: SHORTTAG At 06:48 PM 9/10/96 +0100, Martin Bryan wrote: >13.5 - SHORTTAG >SHORTTAG YES is required to allow attribute values to be entered without >literals (and without attribute name and value indicators) but XML should >forbid the other options SHORTTAG currently permits. (In the next version of >SGML these two roles of SHORTTAG will be clearly distinguished.) The big problem here is the TEI, which makes heavy use of the minimized attribute formulation. But supporting SHORTTAG flies in the face of our design goal of being able to do useful processing of XML docs without having the DTD. The things that are TEI-encoded are already in SGML and thus probably aren't crying out for XML - I suspect that following the advent of XML, there will be a pretty neat & tidy XML-compliant TEI subset (right, Michael and Lou?). I think all forms of SHORTTAG complicate parsing, don't add enough, and should be dropped in XML. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Wed, 11 Sep 1996 13:13:30 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: REFC At 06:48 PM 9/10/96 +0100, Martin Bryan wrote: >6.2.2 >Should we require that entity references end with a REFC code (the semicolon)? Yes! Anyone disagree? Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Wed, 11 Sep 1996 13:13:33 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: PIs >7.6 (4th bullet) >PIs don't make sense on a network. Their use should be restricted to the >prolog, where they could be useful in setting up the parser (e.g. to >identify a relevant SGML declaration). The thought of allowing a processing >instruction within a document instance that is going to be displayable by >many different plug-ins scares the hell out of me. Hmm, I think I'm missing something. In SGML, a PI can be freely ignored by any application... what harm is done in supporting PI's as a place to put, well, Processing Instructions? In the full knowledge, of course, that nothing downstream can be guaranteed to understand or act on them, without prior arrangement. But such prior arrangements are common. Could you elaborate, Martin? >8(c) >If PIs are retained then PIC should be changed so that it is not > as this >is required for many processing instructions. (An alternative may be to >allow a character reference to be entered within a PI, but this would make >XML incompatible with SGML.) Good idea. I've heard this one bubble up in half a dozen different contexts of late, so it's not just us. Shouldn't be hard to come up with something, e.g. ?> or %>. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Wed, 11 Sep 1996 13:13:34 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Compliance with 8879, a moving target At 06:48 PM 9/10/96 +0100, Martin Bryan wrote: >WG8 have agreed in principle that there can be more than one attlist >declaration for an element in future versions of SGML. It is possible that >there will be new rules about the interpretation of attlists in which the >same attribute name occurs in two declarations for a particular element. Which raises an interesting question, when XML is going in a direction that it seems like WG8 is as well, should we (a) leap out ahead, or (b) wait for them to do it. For example, I noticed Charles Goldfarb on comp.text.sgml raising the idea of a syntactically distinguishable start tag for an EMPTY element. This would solve a big problem for XML (which we haven't kicked around here yet). Personally, I lean to (b) - we should figure out our own way to solve the problems while remaining maximally compliant with 8879 - while cheering them on, of course. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Wed, 11 Sep 1996 13:13:35 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Marked Sections At 06:48 PM 9/10/96 +0100, Martin Bryan wrote: >10.4 >Marked sections are useful in DTDs (a la HTML and TEI) but it might be valid >to ban them within transmitted document instances conforming to XML. (XML >should really be for the transmission of version independent documents. >RCDATA should have its entities resolved before tranmission and with CDATA >should have any embedded markup delimiters replaced by character references.) There's been no discussion of this, and early voting results do not show a trend. Marked sections clearly complicate parser construction, and I don't think they are useful enough to make up for this and get into XML. I think if you need this kind of sophistication, you should use Real SGML. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Wed, 11 Sep 1996 13:13:32 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: NET At 06:48 PM 9/10/96 +0100, Martin Bryan wrote: >7.4 (a) Empty and unclosed start-tags are obviously not needed by XML, but >could there be a case for NET-enabled start-tags to delimit numeric data? No. They complicate parser construction, are kind of hard to explain to people, and don't really add that much benefit. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Wed, 11 Sep 1996 21:30:57 +0100 From: Lou Burnard To: W3C-SGML-WG@w3.org Subject: short-tag considered unhealthy From: OXVAXD::LOU "Lou Burnard" 11-SEP-1996 21:28:04.58 To: MX%"tbray@textuality.com" CC: LOU Subj: RE: SHORTTAG > >The big problem here is the TEI, which makes heavy use of the minimized No sentence beginning like this can possibly be true. In point of fact, the TEI recommendations do not officially recommend shorttag. Au contraire. The BNC relies on it because it has 100 million words to tag with POS, but does so fully knowing that this is *not* kosher TEI. >I think all forms of SHORTTAG complicate parsing, don't add enough, and >should be dropped in XML. I agree. Lou Date: Wed, 11 Sep 1996 15:51:07 -0500 To: w3c-sgml-wg@w3.org From: mag@ncsa.uiuc.edu (Tom Magliery) Subject: Re: PIs At 1:13 PM 9/11/96, Tim Bray wrote: >>7.6 (4th bullet) >>PIs don't make sense on a network. Their use should be restricted to the >>prolog, where they could be useful in setting up the parser (e.g. to >>identify a relevant SGML declaration). The thought of allowing a processing >>instruction within a document instance that is going to be displayable by >>many different plug-ins scares the hell out of me. > >Hmm, I think I'm missing something. In SGML, a PI can be freely >ignored by any application... what harm is done in supporting PI's as a place >to put, well, Processing Instructions? In the full knowledge, of course, that >nothing downstream can be guaranteed to understand or act on them, without >prior arrangement. But such prior arrangements are common. Could you >elaborate, Martin? This started me thinking about something I've pondered before: There are (at least) two *sorta-different* kinds of comment-like metadata that we want to be able to embed in a document. The question here is whether these different things should be blessed with a different syntax. What can you do with PIs that you can't do with comments starting with a special well-known codeword that means "PI"? Syntactically, I think the answer is "well...nothing" (but welcome corrections). The standard answer: comments get ditched by SGML parsers, so some processing "engines" (that need to see the PIs) might never see them. PIs are good because they are not tossed by parsers. But we're not talking about SGML parsers here, we're talking about XML parsers, which (since they don't really exist yet) could be taught not to throw away comments out of hand, but keep them around so that "engines" could glance at them before throwing them away (or not). If the functionality of PIs is done in XML with the above-described special form of comment, this makes the syntax of XML slightly ximpler (hehe, that was a typo, but i decided to leave it) at the expense of moving the decision to ditch them a little further down the line. mag -- .---o Tom Magliery, Research Programmer .---o `-O-. NCSA, 605 E. Springfield (217) 333-3198 `-O-. o---' Champaign, IL 61820 O- mag@ncsa.uiuc.edu o---' Date: Wed, 11 Sep 96 16:04:36 CDT From: paul@arbortext.com (Paul Grosso) To: w3c-sgml-wg@w3.org Subject: Re: Element Structure for XML (Clause 7) > To: w3c-sgml-wg@w3.org > From: Martin Bryan > Subject: Element Structure for XML (Clause 7) > Date: Tue, 10 Sep 1996 18:48:27 +0100 > > . . . > > 7.6 (4th bullet) > PIs don't make sense on a network. Their use should be restricted to the > prolog, where they could be useful in setting up the parser (e.g. to > identify a relevant SGML declaration). The thought of allowing a processing > instruction within a document instance that is going to be displayable by > many different plug-ins scares the hell out of me. I don't understand and I disagree with the anti-PI sentiment. What is so scary about PIs? They're easy to recognize as PIs and easy to parse. And they're as simple as comments to handle: ignore them (unless you recognize them as being something you can and want to handle). PIs are the only good way to encode information in the SGML document that hasn't been pre-conceived of when the DTD was written, and there are lots of good reasons why this might be important. For example, Adept uses PIs to: 1. remember the position of the cursor when you last saved the file; 2. remember the "detail status" (whether this "division type" element was set to view as "collapsed" or "expanded" last time the file was saved) of elements; 3. insert certain instance-specific formatting information such as one-off inserted page breaks. Note that there is nothing scary about ignoring any of these PIs, but if such PIs are forbidden in XML, then there is no good way to encode this information in the document. Let's remember that 95% of the time, a given document instance is going to be read by the same tool that last wrote it. Sure, interchange is important, but allowing for user-friendly tools is too, and if we throw out the only safe, unscary way to support tool-specific amenities, we are really dooming ourselves to lowest common denominator standards of useability. Let's agree that it must be the case that parsing must be unchanged if all PIs within the document are ignored, but don't forbid the existence of PIs in document instances. > > . . . > > 8(c) > If PIs are retained then PIC should be changed so that it is not > as this > is required for many processing instructions. (An alternative may be to > allow a character reference to be entered within a PI, but this would make > XML incompatible with SGML.) I don't really understand what Martin is suggesting here. I realize that SGML's lack of reasonable escaping mechanisms (e.g., there is no way to put a PIC character in a PI) is problematic, but I don't see how changing the PIC from that in the RCS would really help anything. Most tools that make use of PIs have figured out some why to handle the PIC problem, so why should we address the issue? paul Date: Wed, 11 Sep 96 16:10:39 CDT From: paul@arbortext.com (Paul Grosso) To: w3c-sgml-wg@w3.org Subject: Re: Entity structure for XML (Clause 6) > To: w3c-sgml-wg@w3.org > From: Martin Bryan > Subject: Entity structure for XML (Clause 6) > Date: Tue, 10 Sep 1996 18:48:23 +0100 > > . . . > > b) Should we predefine a set of NDATA notations that will be acceptable to > all browsers? What mechanism would there be for extending the list of > notations to take account of new developments in general support for a new > notation (e.g how will we allow for MPEG-4 in 2000)? I think predefinition of the set of NDATA notations is unacceptable. SGML Open has started an effort to codify a method for notation interchange. While this has only just started, and we're not sure where it will go, I think some sort of "notation catalog" concept (not too logically dissimilar >from some sort of mailcap idea) might work. New notations and new tools to handle them will be invented every day, and we need to be able to handle this. > > . . . > 6.2.3 > On the question of SGML declaration transmission I feel that the current > text is in conflict with current practice. Most systems have an implied SGML > declaration, which is normally an extension of the one defined in 8879 that > is supposed to be used whenever you do not specifically receive one. WG8 > have agreed in principle that it should be possible to identify a relevant > SGML declaration using a public (or system) identifier in the same way that > you can with a DocType declaration. This is the best long term solution. If > XML is to have a fixed SGML declaration then we have no problem. If, to suit > regional character set requirements, there is more than one option, there > should be a set of predefined SGML declarations that can be referenced via a > public identifier (perhaps through a PI or comment declaration) in front of > the DocType declaration. I'm not sure what think about sgml decls, but I want to point out that the SGML Open catalog allows for sgml decls to be associated with DTDs. That is, the DTDDECL entry type of a catalog maps a DTD's FPI into the sgml decl that should be used. paul Date: Wed, 11 Sep 96 16:03:15 CDT From: Michael Sperberg-McQueen Subject: Re: short-tag considered unhealthy To: Lou Burnard , W3C SGML Working Group On Wed, 11 Sep 1996 16:31:52 -0400 Lou Burnard said: >The BNC relies on it [omission of attribute name when declared value is a name group -MSM] because it has 100 million words to tag with POS, >but does so fully knowing that this is *not* kosher TEI. > >>I think all forms of SHORTTAG complicate parsing, don't add enough, and >>should be dropped in XML. > >I agree. Well, I may be wrong, but the TEI editors can only very seldom resist the temptation to contradict each other in public, and now is not one of those times. TEI conformance does *not* forbid any use of SGML a user chooses to make; the TEI Interchange Format does restrict some SGML usages (including turning SHORTTAG off), but no one is required to use the TEI Interchange Format for interchange. (And in TEI P1, we even explicitly allowed omission of attribute names in the interchange format -- it continues to astonish me, but there it is.) So I don't care *what* Lou says, the BNC is perfectly kosher in this respect. However, Lou is right to say the TEI does *not* require, recommend, or otherwise abet the use of SHORTTAG in this way, and that we needn't keep SHORTTAG for the TEI's sake. So I agree with Tim, too, at least as regards omission of attribute names. Watching HTML users struggle to come to grips with this construct has persuaded me we really ought to get rid of it. On the other hand -- there are one or two uses of SHORTTAG that I don't think complicate parsing all that much, and might be retained: - empty end-tags - attribute values without quotes around them - the omission of attributes which have default values. Without SHORTTAG or OMITTAG turned on, 8879 requires that a value be specified for every attribute which isn't defaulted to #IMPLIED -- I think it would be better to allow omission of attribute value specifications for all attributes not #REQUIRED. -C. M. Sperberg-McQueen Date: Wed, 11 Sep 1996 16:37:39 -0500 From: Len Bullard To: Tim Bray CC: w3c-sgml-wg@w3.org Subject: Re: REFC Tim Bray wrote: > > At 06:48 PM 9/10/96 +0100, Martin Bryan wrote: > >6.2.2 > >Should we require that entity references end with a REFC code (the semicolon)? > > Yes! Anyone disagree? Not here. I'm tired of putting them back in where they are left out. Len Bullard Lockheed Martin Date: Wed, 11 Sep 96 16:23:30 CDT From: Michael Sperberg-McQueen Subject: Re: Element Structure for XML (Clause 7) To: W3C SGML Working Group Like Paul, I guess I think PIs aren't all that hard to deal with, though it might be nice to specify a canonical method of escaping PIC within them. The one thing I have most missed in PIs as currently constituted is a reasonably simple, mostly reliable method for different applications to avoid tripping over each other's PIs. I have the impression that some applications try to make all of their PIs begin with some keyword (say, the name of the application), so that they can readily tell, by looking just at the first token of the PI, whether the PI could possibly be theirs or not. So a document with could be processed by applications which recognize the keywords TeX, WScript, and teihtml, and each application can know from the keywords which PIs it should pay attention to, and which it can (and should!) ignore. Such a self-assigned keyword system is not, of course, 100% reliable, but it's somewhat more reliable than having no such pattern at all. Is this a convention for using PIs that XML ought to require or recommend? -C. M. Sperberg-McQueen Date: Wed, 11 Sep 1996 16:53:10 -0500 From: Len Bullard To: Tim Bray CC: w3c-sgml-wg@w3.org Subject: SGML or SGML-like : Was Re: SHORTTAG Tim Bray wrote: > > The things that are TEI-encoded are already in SGML and > thus probably aren't crying out for XML - I suspect that following the advent > of XML, there will be a pretty neat & tidy XML-compliant TEI subset (right, > Michael and Lou?). > > I think all forms of SHORTTAG complicate parsing, don't add enough, and > should be dropped in XML. I don't disagree as yet with shedding these features. I am uneasy with the "use real SGML" defense. This project has the public face of being SGML On The Web, not "SGML-like On The Web". Is it the intent of the participants to invent a different but similar language, or to define a strict subset of SGML that is efficient for distributed networks? This issue should be clarified before going further or it will be a subtext to all discussions that follow and an issue that will greatly complicate selling the solution. Len Bullard Lockheed Martin Date: Wed, 11 Sep 96 17:32:16 CDT From: paul@arbortext.com (Paul Grosso) To: w3c-sgml-wg@w3.org Subject: Re: Element Structure for XML (Clause 7) > From bsmith@atlantic-82.Eng.Sun.COM Wed Sep 11 17:23:02 1996 > Date: Wed, 11 Sep 1996 15:01:54 -0700 > From: bsmith@atlantic-82.Eng.Sun.COM (Bill Smith) > To: paul@arbortext.com > Subject: Re: Element Structure for XML (Clause 7) > > Paul Grosso wrote: > > > Let's remember that 95% of the time, a given document instance is going > > to be read by the same tool that last wrote it. Sure, interchange is > > important, but allowing for user-friendly tools is too, and if we throw > > out the only safe, unscary way to support tool-specific amenities, we > > are really dooming ourselves to lowest common denominator standards of > > useability. > > On the web, it's more likely that the tool used to view a document is not the > tool used to write it. But how many times did it go into and out of the authoring tool during its creation, drafting, modifying, editing, proofing, etc. before it got put up on the internet for viewing. > > PIs are not required to produce user-friendly tools. Tom Magliery's suggestion > seems like a reasonable alternative. I think PIs are unnecessary. > First, comments are comments, and as Tom says, they are ignored by many parsers. Why be non-compliant with SGML for (what I perceive as) no good reason? Second, there are existing tools that use PIs as they were intended to be used. Why break them and ask them to use comments for what they weren't intended? Third, comments have at least as annoying "escaping" problems as PIs. Do you know how often I try to type "--" in a long comment only to have a parse error (unless I'm using Adept which tries to avoid this)? The business about parsing -------- [that's 8 of them] in a comment is certainly more of a pain than scanning text following . Fourth, I still just don't get it. What's so hard about PIs? paul p.s. My message header seems to indicate that Bill sent this only to me. I hope I'm correct in assuming this was an oversight and that I'm not breaching e-etiquette by resending this to the entire mailing list. Date: Wed, 11 Sep 96 17:36:19 CDT From: paul@arbortext.com (Paul Grosso) To: w3c-sgml-wg@w3.org Subject: Re: Element Structure for XML (Clause 7) Various such schemes have been suggested in the past (Lynne Price once worked on such a suggestion in ANSI X3V1 TG8, but I don't believe it went anywhere.) ArborText has always had an ArborText-specific initial keyword for all its PIs (usually "Pub" from our earlier "Publisher" product). Codifying something like this in XML would be a good idea. If SGML Open can help such codification with some Technical Resolutions or Technical Memos, I'd be glad to offer to bring that up there. paul > Date: Wed, 11 Sep 96 16:23:30 CDT > From: Michael Sperberg-McQueen > Subject: Re: Element Structure for XML (Clause 7) > To: W3C SGML Working Group > > Like Paul, I guess I think PIs aren't all that hard to deal with, > though it might be nice to specify a canonical method of escaping > PIC within them. > > The one thing I have most missed in PIs as currently constituted is a > reasonably simple, mostly reliable method for different applications > to avoid tripping over each other's PIs. I have the impression > that some applications try to make all of their PIs begin with > some keyword (say, the name of the application), so that they can > readily tell, by looking just at the first token of the PI, whether > the PI could possibly be theirs or not. So a document with > > > > > > could be processed by applications which recognize the keywords > TeX, WScript, and teihtml, and each application can know from the > keywords which PIs it should pay attention to, and which it can > (and should!) ignore. > > Such a self-assigned keyword system is not, of course, 100% reliable, > but it's somewhat more reliable than having no such pattern at all. > > Is this a convention for using PIs that XML ought to require or > recommend? > > -C. M. Sperberg-McQueen > > Date: Wed, 11 Sep 1996 18:26:12 -0500 From: Robin Cover To: w3c-sgml-wg@w3.org Subject: Role of a DTD in XML I'm sure I should know know the answer to the following question, but I don't. Since knowledge is the only cure for ignorance, I'll ask. I'm having difficulty understanding the precise meaning and rationale for the principle articulated as list item #4 in "Draft DD-1996-0001 - Design Principles for XML," where the first explanatory sentence reads: In particular, it shall be straightforwardly possible to construct useful XML applications which do not read, or need to read, the DTD of the XML document. Question #1: Is "useful" the operative term here? Or is the meaning "useful *browser* (XML) applications"? It seems to me that authoring tools and search interfaces ought to be responsive to a grammar (DTD) [and/or a stylesheet] to be "useful." For example, with reference to "validating authoring tools" as candidate XML applications (see item #2 in "Design Principles for XML"): a system receiving an XML instance without a DTD could transduce some kind of XML grammar for the document, but (e.g., in terms of SGML's current 'declared content' data types) would not necessarily be able to enforce the "proper" constraints on attribute values. If it saw "

...

" and "..." in the instance, it (the validating editor) would not know whether to require that the value of 'n' for a new 'p' element should be unique (an ID), or a NAME, or CDATA, or what. Similarly, a user interface for a generic XML searching tool isn't going to be able to offer much assistance in the creation of meaningful queries (and in interpreting null results as meaningful or fortuitious) without a DTD. Question #2: If the first sentence in item #4 (quoted above) were to read, contrary to its current wording, something like "In all cases it should be possible for an XML application to process an XML instance without having to read a DTD..." -- then which clauses (sub-clauses) in the SGML Standard would need to dropped or revised in order to support the design and development of *great* (not merely "useful") XML applications? Does someone have a list? Thanks in advance for your forbearance and clarifications. Robin ------------------------------------------------------------------------- Robin Cover Email: robin@acadcomp.sil.org (or) 6634 Sarah Drive Email: robin@utafll.uta.edu ("uta-ef-el-el") Dallas, TX 75236 USA Tel: (1 214) 296-1783 (h) >>>>>>>>> The SGML Web Page <<<<<<<<< Tel: (1 214) 709-3346 (w) http://www.sil.org/sgml/sgml.html FAX: (1 214) 709-3380 ========================================================================= Date: Wed, 11 Sep 1996 16:26:44 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: short-tag considered unhealthy At 04:03 PM 9/11/96 CDT, Michael Sperberg-McQueen wrote: >On the other hand -- there are one or two uses of SHORTTAG that I don't >think complicate parsing all that much, and might be retained: > - empty end-tags Please, no; these save a tiny number of bytes, make it harder for both humans and computers to understand, and for people who don't already know SGML, have to be explained. Also, on purely CS-theory grounds, they push an XML parser over the edge from a pure automaton to something that has to keep a stack. OK, the cost of keeping a stack is not high, but neither is the benefit of using . The rest seem like good things for XML: > - attribute values without quotes around them > - the omission of attributes which have default values. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Wed, 11 Sep 96 18:20:16 CDT From: Michael Sperberg-McQueen Subject: Empty elements (and processing without a DTD) To: W3C SGML Working Group In the days before the SGML Work Group discussion list started up in earnest, the Editorial Review Board spent some time discussing the project's statement of goals -- and, like the Work Group as a whole, found itself drawn irresistibly into the discussion of particular technical points which came up in the course of larger discussions. Among other things, we discussed at some length what to do with EMPTY elements in XML. The other members of the ERB have encouraged me to post to the WG this summary of the ERB's discussion of EMPTY elements, originally posted to the ERB itself, so as to bring the results of that discussion into this one. It seems particularly apposite in the light of Robin Cover's recent query about why or when one might want to process a document without reading the DTD, even though it assumes rather than gives an answer to Robin's question. I have edited slightly for clarity and accuracy. -CMSMcQ ---- In private corresondence, Steve DeRose has identified (I think correctly) the following problems caused by EMPTY in its current form: A. it makes it hard, working only from the ESIS, to write out a document instance that is sure to be ESIS-equivalent to the original (i.e. to perform an ESIS-based identity-transform), since it's hard to know whether to emit "" or "" B. it makes it hard, if you have "

foo bar

" in input, to know whether when fully normalized this means / should mean:

foo bar

foo bar

unless you have read and understood the entire DTD -- which otherwise you might, under certain circumstances, be able to bypass entirely. Both of these things (correct understanding of the input, even sans DTD, and correct production of ESIS-equivalent structures, on the basis of the ESIS alone) seem plausible to want to do in XML. (They are plausible to want to do in SGML, too, it's just that without some knowledge of DTD or instance they are either very hard, or impossible.) One of the best ideas proposed by Claus Huitfeldt at the Wittgenstein Archives of the University of Bergen, and implemented in their markup scheme MECS (the Multi-Element Coding System), was to distinguish elements with content from those without content:

bar /p> and

/p> are completely distinct and can never look alike even with minimization:

bar />

/> I think we could get the function we need (empty elements with some enforcement/recognition that they are empty by definition, but also allowing simpler tools) in several ways, with different tradeoffs. 1 Don't Worry, Be Smart (Naggum). As I understand it, this is Erik Naggum's proposal for handling empty elements in stupid parsers. Even if this description isn't what he meant, however, it is still Erik from whom I acquired the ideas below. Keep the current syntax, and know that if you don't see an end-tag, the start-tag was for an empty element. I.e. every element is a potential empty element until you see an end-tag for it. If end-tag omission is prohibited, this means you only have to look for the next corresponding end-tag to be sure of what you have to do. If you're building a tree, you'll need to adjust it from time to time, or keep it in suspension until the ambiguity is resolved. When you see an end-tag, do this: if gi(currentTag) = gi(peekAtStack()) then call PopStack else do /* first update our internal tables: we now know that the GI at the top of the stack is for an EMPTY element, and if we like we can adjust our scanner so in the future we can do the right thing immediately ... */ giTemp = gi(peekAtStack()) empty.giTemp = 1 /* now update our tree information: delete everything we used to think was a child of the empty element, and make it a right-hand sibling instead */ call PromoteChildrentoRightSiblings(currentElement) end /* else */ Pro: simple, clean, obvious to SGML users. Con: possibly long lookahead for the second, third, ... start-tags encountered in the document; need big buffers or possibly two-pass parsing. 2 The Elephant's Child, or, the Insatiable Content Model (Clark). James has already mentioned [in the ERB discussions] this way of ensuring that an element is always empty, even though it is not necessarily declared using the EMPTY keyword. E.g. where 'nonesuch', of course, is a name for an element which may never be defined (because if it were, there would be one such!). Pro: provides the necessary functionality (ensures that blorts are always empty). Con: if Nonesuch is made a reserved name in XML, then DTD designers must remember to avoid it -- to make the prohibition easy to remember and obey, it needs to be a name like Nonesuch... so the rules can say "Never end a name in three dots: such names are reserved for use by XML systems which need magic names." If on the other hand Nonesuch is not a reserved name, then designers can use any name they choose, but (a) they have to understand the mechanism, to use it right, and (b) XML tools are liable to issue warnings that the DTD has a possible typo, or to offer the user Nonesuch... as a choice on a menu, unless they do some preliminary DTD analysis. From the reluctance of SoftQuad to handle undeclared elements properly, I infer that not all developers think such analysis simple or easy or worth while. 3 The Trapeze Act -- working *with* a net (Huitfeldt, DeRose). The problem is not in EMPTY, or in the absence of an end-tag, but in the inability to know without reading the DTD whether a given start-tag is for an empty element or not. We can remove the ambiguity by requiring -- as a matter of XML conformance, not of SGML -- that elements declared EMPTY should always use a NET start-tag:

foo is legal, while

foo bar

is either illegal, or means

foo bar

(This is slightly different from Steve's proposal, which involved two slashes, not one.) So if a yacc/lex parser defines terminals STAGO, NAME, TAGC, NET, we can define element : starttag content endtag | emptytag ; starttag : STAGO NAME attspecs TAGC ; emptytag : STAGO NAME attspecs NET ; Pro: solves the problem, for XML parsers which parse the source themselves. Con: many of us have trained ourselves to hate and despise NET, and recoil in horror at the prospect of finding useful work for it to do. More seriously, it's hard to imagine just how to tell existing SGML software how to emit a legal XML document using this mechanism, which means every time I touch my XML documents with an SGML tool, I am punished by being required to run a little cleanup filter. The distinction between and Pro: easy to handle using ESIS only (e.g. sgmls output) Con: does not do the job for software which reads the source itself; such software would have to read the DTD after all. (Combine with the Trapeze Act in order to avoid this problem?) 5 Borrowing from Peter to pay Paul (Maler, Goldstein, Bos): if you can't get the information in the instance (because FIXED attributes are defined in the DTD, which you don't want to read), or in the DTD (because you don't want to read it), put it somewhere else. Bob Goldstein and I suggested some time ago that you could put it (redundantly) in a style sheet (this in the old HTML to the Max paper); Eve has suggested PIs or other auxiliary material, as has Bert Bos in his proposal for SGML Lite (for which there is a pointer from the 'voting booth'). 6 Raw Bits, or What's So Hard about Reading DTDs? This resolves the problem by declaring it a non-problem: if one wants EMPTY elements, one has to declare them (as in PSGML -- Poor Folks' SGML, see pointer in the voting booth), and if the parser wants to know about them, it has to read the declarations. Since a non-validating parser needn't do much with other declarations except read past them, the DTD parser need only: - read and understand declarations of external parameter entities - recognize and obey reference to external parameter entities - recognize and handle declarations of EMPTY elements - skip over other types of declarations; the only place an MDC is legal is within a literal, a comment, or a marked section, so all you need is a lexical scanner that takes care of comments and marked sections, and something like this in the grammar: miscdeclaration : MDO misccontent MDC misccontent : /* nil */ | misccontent QUOTEDLITERAL | misccontent STRING /* string without MDC */ ; If there's no DTD to read, then there are no EMPTY elements. Period. Then an empty blort will have to read

foo bar

while

foo bar

is either illegal, or means

foo bar

as in the Trapeze Act. 7 Cooked Bits. A variant of the Raw Bits (parse it and like it!) approach would be to provide a different syntax for the declarations, to make them really, really easy to parse. Since we already agree that we'll need a filter to take an XML document and prepare an SGML declaration and prolog for it, we do have this as an option. It has been suggested, for example, that DTDs are structured information, and might usefully be represented using the same techniques we suggest for other structured information, namely SGML / XML document instances. This has the beneficial side effect of reducing the size and complexity of the context-free grammar by about half. ---- I don't know (a) whether there are other ways to get around this problem, or (b) which of these I think should be adopted in XML. I suspect everyone will agree that if XML document instances can be strictly conforming SGML, and XML can be elegant and easy to use, and XML can be simple to understand and implement, it would be good. It's just not clear to me how to optimize the values of all three aspects here, or in a number of other areas. -C. M. Sperberg-McQueen ACH / ACL / ALLC Text Encoding Initiative University of Illinois at Chicago tei@uic.edu / Tel. +1 (312) 413-0317 / Fax +1 (312) 996-6834 Date: Wed, 11 Sep 1996 16:43:27 -0700 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: tbray@textuality.com CC: w3c-sgml-wg@w3.org Subject: Re: short-tag considered unhealthy | >On the other hand -- there are one or two uses of SHORTTAG that I don't | >think complicate parsing all that much, and might be retained: | > - empty end-tags | | Please, no; these save a tiny number of bytes, make it harder for | both humans and computers to understand, and for people who don't | already know SGML, have to be explained. Also, on purely CS-theory | grounds, they push an XML parser over the edge from a pure automaton | to something that has to keep a stack. And, to bring it down to the level that I'm used to working at, they push a desperate human from a fix that can be accomplished in time to meet a deadline with a five-line perl hack to one that can't. Jon Date: Wed, 11 Sep 1996 17:35:15 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: Role of a DTD in XML At 06:26 PM 9/11/96 -0500, Robin Cover wrote: >I'm having difficulty understanding the precise meaning and rationale >for the principle articulated as list item #4 in "Draft DD-1996-0001 - > > In particular, it shall be straightforwardly possible to > construct useful XML applications which do not read, > or need to read, the DTD of the XML document. > >Question #1: Is "useful" the operative term here? Or is the meaning >"useful *browser* (XML) applications"? As part-author of this, I should offer some exegesis. Obviously, without some Markup Declaration, you can't validate against a grammar, or really do any meaningful "validation" aside from ensuring tags aren't mangled and are properly nested and so on. On the other hand, ensuring that tags aren't mangled, and constructing the de facto parse tree, are often useful things to do. You mention browsing applications, correctly; but search/retrieval is another case. Note that for both, you probably need some more information (stylesheets for browsing, some metadata about the elements for searching), but not of a type that's found in a DTD. However, with SGML, you just can't do this basic stuff without a DTD. And in particular you can't do it for a document *fragment* even if you have a DTD, unless you ship around some ancillary info like the SGML/Open fragment wrappers, and you have a parser that can interpret them and leap into the middle of a complex parse state. One of the motivations for XML is the desire to solve exactly this problem; the goal being that any XML doc or fragment, by itself, can be "parsed" in the sense of building an unambiguous tree. I think this addresses your question #2 as well: the desire is not to do things that need a grammar without a grammar; just to do things that shouldn't need a complete DTD without a complete DTD. A lot of applications will continue to require Markup Declaration. Which is one of the areas where XML is going to get really interesting. XML needs Markup Declaration, but: o Does it need *complete* markup declaration in every case? Partial markup declaration (these elements are empty, these attributes are #FIXED, etc.) could be very useful (and provides my favorite solution to the EMPTY element conundrum, what Michael calls "Cooked Bits".) o Should we use 8879 Markup Declaration (aka SGML Declaration & DTD) syntax for XML Markup Declaration? Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 From: "Henry S. Thompson" Date: Thu, 12 Sep 96 01:46:17 BST To: tbray@textuality.com Subject: Re: Marked Sections Cc: w3c-sgml-wg@w3.org Tim Bray writes > At 06:48 PM 9/10/96 +0100, Martin Bryan wrote: > > >10.4 > >Marked sections are useful in DTDs (a la HTML and TEI) but it might be valid > >to ban them within transmitted document instances conforming to XML. (XML > >should really be for the transmission of version independent documents. > >RCDATA should have its entities resolved before tranmission and with CDATA > >should have any embedded markup delimiters replaced by character references.) > > There's been no discussion of this, and early voting results do not show a > trend. Marked sections clearly complicate parser construction, and I don't > think they are useful enough to make up for this and get into XML. > Seconded. ht From: bbauma1@cs.umbc.edu To: w3c-sgml-wg@w3.org Date: Wed, 11 Sep 1996 22:09:22 +0000 Subject: Re: short-tag considered unhealthy Priority: normal > Please, no; these save a tiny number of bytes, make it harder for > both humans and computers to understand, and for people who don't > already know SGML, have to be explained. Actually a tiny number of bytes can add up quickly in large files, but I to am willing to do without . What I would like is to be able to leave off the end tag entirely and have XML assume that I wanted to close the immediately proceeding element so that Item 1 Item 2 Item 3 would be legal. Doing this should be easy assuming that the EMPTY element problem is dealt with. B. Todd Bauman Graduate Student University of Maryland, Baltimore County Date: Thu, 12 Sep 1996 08:34:27 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: PIs At 09:18 AM 9/12/96 GMT, Charles F. Goldfarb wrote: >>If PIs are retained then PIC should be changed so that it is not > as this >>is required for many processing instructions. (An alternative may be to >>allow a character reference to be entered within a PI, but this would make >>XML incompatible with SGML.) >This is backwards. Changing PIC breaks SGML. A character reference would not be >recognized by the SGML parser, but could be recognized by the XML application. Nice to have you aboard, Charles. I think on this one you're crediting us with too much subtlety. References aren't the issue - it's just that there are a lot of things that you'd like to put in PI's where it's irritating that you can't just use ">" to mean ">" - and since XML is almost certainly going to have a hardwired concrete syntax, why don't we change PIC to "%>" or "?>" or something, so as to make this irritation go away? Obviously the problem has no solution in the general case without getting into some of the other mechanisms Charles mentions, but the difficulties with the single character ">" are one of the many little things that add up to constitute the barrier to entry to SGML. Clearly this makes it harder to treat existing SGML documents with PI's as XML... which raises an important point: we've been focusing on building XML to encourage a flood of *new* documents out of proprietary formats into descriptive markup; should we be worried about supporting the existing SGML legacy base? I think not, but it's a valid question. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Thu, 12 Sep 1996 08:34:28 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: short-tag considered unhealthy At 09:26 AM 9/12/96 +0000, James Clark wrote: >> - the omission of attributes which have default values. ... > >I think there's an argument that this is an essential feature. Some >DTDs especially those using architectural forms have a lot of >defaulted attributes on almost every element. > >On the other hand it complicates DTD-less processing: for example, it >would make it hard for a browser to display a document correctly >without the DTD. This is another case where partial markup declaration would turn out to be useful; I might not want to supply a whole DTD, but on the other hand it would be real nice to send a few little morsels like the following along with a document. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Thu, 12 Sep 1996 08:19:26 -0700 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: RS/RE At 01:43 PM 9/12/96 GMT, Gavin Nicol wrote: >>One thing that's hard is the rules about how REs adjacent to PIs are >>handled. >David Durand and I independently came up with a good way of dealing >with such things I've heard about this but never seen it. Could you or David or someone please post it to the group? In our informal discussions before the advent of the WG, figuring out what to do about RS/RE, without busting our 8879 compliance, was one of the most worrying things. My "MGML" parsing system does the following: At any point in the document, the parser knows from the content model whether data can appear. If so, all bytes that are not markup are data. If not, all white space is ignored, and non-whitespace non-markup is an error. This sacrifices some flexibility, but has the immense advantage that you can explain it completely in 40 words to both users and programmers. Which is not the case for the SGML rules. Unfortunately, I don't think this can ever be made 8879-compliant. Our goal for XML is something that is crisp and clear as the above, but is also 8879-legal. If someone on the group could give us a flash of creativity on this, that would be wonderful. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 From: streich@austin.sar.slb.com Date: Thu, 12 Sep 96 11:15:50 CDT To: w3c-sgml-wg@w3.org Subject: Re: Marked Sections > At 06:48 PM 9/10/96 +0100, Martin Bryan wrote: > > >10.4 > >Marked sections are useful in DTDs (a la HTML and TEI) but it might be valid > >to ban them within transmitted document instances conforming to XML. (XML > >should really be for the transmission of version independent documents. > >RCDATA should have its entities resolved before tranmission and with CDATA > >should have any embedded markup delimiters replaced by character references.) > > There's been no discussion of this, and early voting results do not show a > trend. Marked sections clearly complicate parser construction, and I don't > think they are useful enough to make up for this and get into XML. On the contrary, I think they are very necessary in DTDs. I try to avoid them, but the inconsistencies in the current crop of tools force me to use them. XML is not going to be able improve this situation in the least. Tools will still have their quirks and many of those quirks will require some subtle change to the DTD in order to compensate for them. Variations on table models and equation markup are two such examples of things that XML will not address and will continue to be a problem in the future. Robert Streich streich@slb.com Schlumberger 512-331-3318 (voice) Austin Research 512-331-3760 (fax) Date: Thu, 12 Sep 1996 11:48:07 -0500 From: Len Bullard To: Tim Bray CC: w3c-sgml-wg@w3.org Subject: Re: PIs Tim Bray wrote: > > Clearly this makes it harder to treat existing SGML documents with PI's as > XML... which raises an important point: we've been focusing on building > XML to encourage a flood of *new* documents out of proprietary formats > into descriptive markup; should we be worried about supporting the existing > SGML legacy base? I think not, but it's a valid question. I don't think we have to directly. As long as we are 8879-legal, I have no problems with filtering mechanisms here. It is already the common practice. We can safely say, I believe, that XML was designed to enable efficient implementation and use of SGML online. Mileage depends on terrain. In this statement, all SGML applications are treated equally. That is, XML doesn't favor or disfavor HTML, 28001, TEI, on and on. We cannot go down the path of ensuring backward compatibiity where application-specific features have been introduced. len bullard From: Gavin Nicol Date: Thu, 12 Sep 1996 17:41:59 GMT To: tbray@textuality.com CC: w3c-sgml-wg@w3.org Subject: Re: RS/RE >>David Durand and I independently came up with a good way of dealing >>with such things > >I've heard about this but never seen it. Could you or David or >someone please post it to the group? In our informal discussions >before the advent of the WG, figuring out what to do about RS/RE, without >busting our 8879 compliance, was one of the most worrying things. Well, from my reading of the SGML Handbook, it seems that RE and RS are not *required* at all. If they occur, they are put there by the entity manager. In fact, RE and RS are not really even characters per se, they are kind of psuedo-characters (they have a code, and a name, but they aren't real characters). Anyway, if we assigned some character codes to them that are guaranteed to never occur in input, then the parser will never even see them (another syntax trick). This will, of course, mean that \n and \r will be seen in content, but they could be mapped such that they get converted to a space on input. I don't claim to be intimately familiar with all the effects that this will have in terms of markup regognition etc. but it seems to me that this would simplify parsers a great deal, and also get around problems with MIME text type requirements (canonical form) etc. To: W3C SGML Working Group From: Martin Bryan Subject: Re: Empty elements (and processing without a DTD) Date: Thu, 12 Sep 1996 18:55:36 +0100 >Pro: solves the problem, for XML parsers which parse the source >themselves. > >Con: many of us have trained ourselves to hate and despise NET, and >recoil in horror at the prospect of finding useful work for it to do. >More seriously, it's hard to imagine just how to tell existing SGML >software how to emit a legal XML document using this mechanism, which >means every time I touch my XML documents with an SGML tool, I am >punished by being required to run a little cleanup filter. The >distinction between and an sgmls-style front end. Such a filter is no worse than the one you would have to write to remove the end-tags from XML-EMPTY elements before sending them to a validating SGML tool! You can't win with either approach, but I like the use of NET here much better than the use of a (to me) unnecessary end-tag. Is keeping a list of empty elements that you skip over the end-tag processing rules for such a big deal for parser developers? As for telling an exisitingSGML parser how to emit XML, simply set the -x switch on Jade-lite and it will generate XML rather than RTF (if James accepts this as a useful transformation!) or put in a processing instruction *before* the base document element. >If there's no DTD to read, then there are no EMPTY elements. Period. This I like! >7 Cooked Bits. A variant of the Raw Bits (parse it and like it!) >approach would be to provide a different syntax for the declarations, to >make them really, really easy to parse. Bang goes any chance of being able to use SGML tools to generate XML! ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ To: Michael Sperberg-McQueen , W3C SGML Working Group From: Martin Bryan Subject: Re: Element Structure for XML (Clause 7) Date: Thu, 12 Sep 1996 18:55:44 +0100 At , Michael Sperberg-McQueen , W3C SGML Working Group, wrote >>>The one thing I have most missed in PIs as currently constituted is a >>>reasonably simple, mostly reliable method for different applications >>>to avoid tripping over each other's PIs. I have the impression >>>that some applications try to make all of their PIs begin with >>>some keyword (say, the name of the application), so that they can >>>readily tell, by looking just at the first token of the PI, whether >>>the PI could possibly be theirs or not. So a document with >>> >>> >>> >>> >>> >>>could be processed by applications which recognize the keywords >>>TeX, WScript, and teihtml, and each application can know from the >>>keywords which PIs it should pay attention to, and which it can >>>(and should!) ignore. But what happens when there is a new non-backward compatible variant for the language - do you have to have a new name to distinguish the new variant, or do you have to have some form of version control statement? What worries me is how to processes XML documents placed in the BNC 20 years after their creation if they contain processing instructions. ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ To: Tim Bray , w3c-sgml-wg@w3.org From: Martin Bryan Subject: Re: Marked Sections Date: Thu, 12 Sep 1996 18:55:30 +0100 Tim >>10.4 >>Marked sections are useful in DTDs (a la HTML and TEI) but it might be valid >>to ban them within transmitted document instances conforming to XML. (XML >>should really be for the transmission of version independent documents. >>RCDATA should have its entities resolved before tranmission and with CDATA >>should have any embedded markup delimiters replaced by character references.) > >There's been no discussion of this, and early voting results do not show a >trend. Marked sections clearly complicate parser construction, and I don't >think they are useful enough to make up for this and get into XML. > >I think if you need this kind of sophistication, you should use Real SGML. Are you calling HTML "Real SGML"? Even such simple applications need marked sections in their DTDs, as does any DTD that changes over time. ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ To: Tim Bray , w3c-sgml-wg@w3.org From: Martin Bryan Subject: Re: SHORTTAG Date: Thu, 12 Sep 1996 18:55:33 +0100 >>13.5 - SHORTTAG >>SHORTTAG YES is required to allow attribute values to be entered without >>literals (and without attribute name and value indicators) but XML should >>forbid the other options SHORTTAG currently permits. (In the next version of >>SGML these two roles of SHORTTAG will be clearly distinguished.) > >The big problem here is the TEI, which makes heavy use of the minimized >attribute formulation. But supporting SHORTTAG flies in the face of our >design goal of being able to do useful processing of XML docs without having >the DTD. The things that are TEI-encoded are already in SGML and >thus probably aren't crying out for XML - I suspect that following the advent >of XML, there will be a pretty neat & tidy XML-compliant TEI subset (right, >Michael and Lou?). > >I think all forms of SHORTTAG complicate parsing, don't add enough, and >should be dropped in XML. HTML extends SHORTTAG attribute processing to beyond that of SGML (by saying that unknown attribute names and associated values are ignored) for good reasons. Attribute entry is one of the areas where users are most prone to make typing errors (e.g. mistyped attribute names) or try to take short cuts (such as leaving out the literal quotes or just entering a token value). ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ To: paul@arbortext.com (Paul Grosso), w3c-sgml-wg@w3.org From: Martin Bryan Subject: Re: Element Structure for XML (Clause 7) Date: Thu, 12 Sep 1996 18:55:41 +0100 Paul Grosso wrote >I don't understand and I disagree with the anti-PI sentiment. > >What is so scary about PIs? They're easy to recognize as PIs and easy to >parse. And they're as simple as comments to handle: ignore them (unless >you recognize them as being something you can and want to handle). As far as parsing them is concerned PIs are a piece of cake: that is not the problem. I have no problem at all with PI's that precede the base document element. What I object to is PIs beyond that point. My problem is to do with what is the role of XML. You seem to be under the impression that XML is about creating interchangeable documents. I was under the impression that XML was about providing a simple mechanisms of being able to receive and understand an arbitrary document created by someone else using equipment I probably could never afford! To me PIs introduce the same level of complexity as company-specific tags in HTML. You just have to look at the Frames war in the HTML arena to see that if you get a document that has markup in that you cannot process the result is likely to be a document that does not completely define what the document generator intended. (Just try using Netscape to view a Microsoft site to see what the problem really is!) Whilst I have no problems with PI's being used as "hidden" info during document generation I have strong reservations about allowing them to be used to contain information that affects presentation or processing. Personally I much prefer to have processing information within clearly defined elements, such as HTML's