Date: Sun, 1 Dec 1996 09:15:10 -0800 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: gtn@ebt.com CC: w3c-sgml-wg@w3.org Subject: Re: Revised XML registration | I would recommend having charset be required, and also having an | optional language field. OK. Jon Date: Sun, 1 Dec 1996 09:21:59 -0800 (PST) From: Terry Allen To: w3c-sgml-wg@w3.org Subject: Characters in XML Further to the task of figuring out what characters are allowed where in XML, I've put together an abstract version of part of an SGML decl, using the EBNF productions in the XML spec instead of writing out character ranges. Two issues that arise are: 1. is XML using Unicode or 10646 2. is the document character set limited to assigned code points Comment invited. Subject: Re: SGML and XML To: Liam Quin , W3C SGML Working Group On Thu, 28 Nov 1996 16:17:22 -0500 Liam Quin said: >If SGML can be changed to disable RS/RE processing, or if Charles' >kludge with SHORTREF works, perhaps we're OK. We do have to get the >same whitespace behaviour with and without a DTD, so element content >and data content is not a distinction XML can make, I think. I think I agree that simplifying RE/RS handling would be good, but in the interests of clarity let me ask: Do you mean we have to make the DTD irrelevant to white-space handling * because that was decided at some point, or * as a direct consequence of some other decision, or * because that would be a cleaner design? I agree that it would be desirable to make rules which allow processors to skip the DTD and still get a wholly correct parse tree, er, grove, but I don't think that's possible, since a processor cannot resolve entity references or provide correct default attribute values without reading the DTD. I also believe that if XML makes no distinction between element content and mixed content, it will be impossible to build correct XML processors on top of SGML parsers. It might be worthwhile to distinguish clearly among (a) full validation, which checks the instance against the DTD and provides the correct grove, (b) full parsing, which doesn't validate but does provide character-for-character the correct grove, and which requires at least some attention to the DTD, and (c) minimal or partial parsing, which doesn't require a DTD and which is guaranteed (in XML) to vary from the correct grove only in specific ways: (i) it will have extra white space in element content, (ii) it won't provide correct values for defaulted attributes, (iii) it won't have information about attribute types like ID/IDREF. (iv) it won't resolve references to entities not hardcoded into the processor. I'm not sure (iii) is part of the standard grove output on which we want to model the XML processor's output; if it isn't, then that's one difference fewer. -C. M. Sperberg-McQueen Date: Sun, 1 Dec 1996 09:30:37 -0800 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org CC: bosak@atlantic-83.eng.sun.com Subject: Re: MIME type for XML [Keith Corbett:] | At 09:36 AM 11/29/96 -0500, Murray Maloney wrote: | >At 03:33 PM 27-11-96 -0800, Terry Allen wrote: | >>I'm glad you're thinking of this, Jon, but isn't it premature | >>to register the MIME type before the spec is finished? | >> | >>If a MIME type is needed for demos, let's use text/x-xml | > | >So, how about text/x-ml | | Be sure to check that the ML folks are not using it. I think that the conventional choice would be text/x-xml. That's what people will be expecting and that's what we should use until text/xml is registered. Jon Date: Sun, 1 Dec 1996 09:51:41 -0800 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org CC: bosak@atlantic-83.eng.sun.com Subject: Re: FPIs [Lee Quin:] | > My point was that you can make your own publisher prefix without | > getting permission from, or paying an annual fee to, a naming | > authority. | | Of course, when you discover that the European division of | Sun Life's software division has been using SUN:SUNSOFT as an owner | identifier for several years, you'll wish there had been a registry, and | all the trademark lawyers in Washington couldn't win you that battle :-) This is a question of internal coordination. We deal with much more difficult coordination problems than this one. | This is why the ISBN is used. [...] I don't see anything magic about ISBNs that will prevent people from misusing them. I only see recourse after the fact. Jon Date: Sun, 1 Dec 1996 10:16:42 -0800 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org CC: bosak@atlantic-83.eng.sun.com Subject: Re: FPIs [Charles Goldfarb:] | >> My point was that you can make your own publisher prefix without | >> getting permission from, or paying an annual fee to, a naming | >> authority. | | At the time 9070 was published, I checked with the U.S. ISBN agency | and was told (in writing) that they would support our use of publisher | prefixes for registered owner identifiers, and that there was no fee | for the assignment of a prefix. You do not have to be a publisher to | obtain one. That is interesting information. Is this a matter of public record, or do we just refer all complaints to you? | [Lee Quin:] | >Of course, when you discover that the European division of | >Sun Life's software division has been using SUN:SUNSOFT as an owner | >identifier for several years, you'll wish there had been a registry, and | >all the trademark lawyers in Washington couldn't win you that battle :-) | | Exactly! Using "-//SUN::SUNSOFT" as an unregistered, not guaranteed | unique prefix is perfectly o.k. Where Sun departs from the standard is | in trying to claim that it owns the prefix. Only registered owner | prefixes can be owned. Are you saying that we can't claim ownership of the string "SUN::SUNSOFT" in a context where someone other than Sun was using it in a way that is specifically designed to convey the authorship of documents? Excuse me, but I think that our legal department would disagree with you. | >At any rate I agree with Charles here that you should not try and invent | >your own standard. | | And there is no need to. Even if GCA-assigned ISO 9070 prefixes are | not yet available, ISBN prefixes are. I do not accept that I or anyone else needs to go to a naming authority for the right to name a document or to state my authorship of it. There is nothing new about this. I am suggesting that unregistered formal public identifiers provide a method by which publishers can (in practice) unambiguously identify texts in a way that allows widespread electronic distribution without typically resorting to a naming authority. Everyone agrees that in theory you have to be able to fall back to something like NAPTR if you want to guarantee resolution (though even NAPTR does not guarantee resolution in every case; I don't see how any system could do that). But there is a big difference between a scenario in which *all* location-independent resolution happens through a naming authority and one in which 99.99% of the resolutions can occur without reference to a naming authority and only a small residue need to fall back to NAPTR -- so small, in fact, that a publisher might choose to allow those references simply to fail. It is my claim that unregistered FPIs can enable the second scenario and thus allow publishing from the ground up rather than making all references go through a central registry to resolve every location-independent identifier. Jon Date: Sun, 1 Dec 1996 19:00:02 -0800 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org cc: bosak@atlantic-83.eng.sun.com Subject: New members; archives updated Having been duly certified to me to be SGML experts, the following persons have been added to the W3C SGML Working Group: David Abrahamson, Trinity College Dublin david.abrahamson@cs.tcd.ie Terry Allen, Fujitsu tallen@fsc.fujitsu.com David Birnbaum, U. Pittsburgh djbpitt+@pitt.edu Elaine Brennan, SGML Consultant elaine@netcom.com Franois Chahuneau, AIS fcha@ais.berger-levrault.fr Martin Cottreau, Andyne martinc@andyne.com Tony Graham, Mulberry tgraham@mulberrytech.com Mary McRae, Xyvision mary.mcrae@xyvision.com Norbert Mikula, U. Klagenfurt e_nmiku@utila.ifi.uni-klu.ac.at Gregory Murphy, CETH Princeton/Rutgers gjmurphy@cethsun.Princeton.EDU Steve Pepper, Falch pepper@falch.no Roger Price, U. Mass. Lowell rprice@cs.uml.edu Thomas Stadler, STEP ths@step.de Norm Walsh, O'Reilly norm@ora.com I have put the WG mail for the month of November out on sunsite.unc.edu in files named 9611a and 9611b: ftp://sunsite.unc.edu/pub/sun-info/standards/xml/wg/9611a.zip ftp://sunsite.unc.edu/pub/sun-info/standards/xml/wg/9611b.zip ftp://sunsite.unc.edu/pub/sun-info/standards/xml/wg/9611a.gz ftp://sunsite.unc.edu/pub/sun-info/standards/xml/wg/9611b.gz You can also use HTTP to retrieve these. As usual, I cannot guarantee that I received everything sent to the list, but these files contain everything that I did receive. The W3C archive for this group is publicly visible, so you are free to share these with anyone who might be interested in our transactions. New participants are advised to take a quick pass through 9611b to catch up with the present discussion. Jon Date: Mon, 2 Dec 1996 11:25:23 +0100 From: etxperl@stdoca.uab.ericsson.se (Per-Ake Ling UAB/K/U) To: w3c-sgml-wg@w3.org Subject: Stylesheets Hello, I have with interset followed the debate "CSS vs. DSSSL-O" and realized that although I have a fair knowledge of DSSSL-o, the same is not true of CSS. I will hunt around on the W3C site and see what I can find. However, it struck me that there may be others in a similar (or opposite) position, so I believe that it would be helpful if somebody could summarize the pros and cons of the respective specifications in an objective? or at least factual manner. I think the comparison would be helpful in focusing the attention on the issues that are of primary concern, eg design philosphy instead of syntax. Finally, I have a feeling that the stylesheet issue should be broadened to other alternatives, at least initially. In the end "CSS vs. DSSSL-O" may be appropriate, but it seems a little early to assume so. I have seen some emails that do discuss other alternatives (Java etc), but I think more of this is necessary before settling on the stylesheet flavor issue. Per-ke (Per-Åke for those that have 7-bit mailfeed) -- Per-ke Ling Ericsson Utcklings AB AXE Research and Development phone: +46 8 727 5674 mobile: +46 70 734 6472 email: Per-Ake.Ling@uab.ericsson.se From: Ken Holman To: "'w3c-sgml'" Subject: Client-side-resolved Indirection Date: Mon, 2 Dec 1996 09:42:30 -0500 [Len Bullard] >What I hear Debbie, Jon, Ken and others saying is that the >FPI provides functionality they need, some of which, depends >on formal registry practices and SGML Open standards for >catalogs. Does this undo the URL? Does this preclude >the use of the URL? Do the URN and the FPI/catalog conflict? Speaking for myself, I need "client-side-resolved indirection", which SGML's established PUBLIC mechanism, with "some kind of resolution", provides. The benefits I described earlier of identifer uniqueness, persistence, management and portability are all aspects of successfully implemented and utilized client-side-resolved indirection. I'm concerned that the (I hope useful) discussion of the use of ISBNs and Jon's observation of trademark lawyers' habits has distracted the readers in this group from the essence of the service provided by PUBLIC identifiers. BTW, I too was not charged anything to obtain an ISBN Publisher's Prefix, and I perceive the two-level hierarchy to be the Publisher's Prefix followed by the publisher's own management of their number space (the two-component value of the prefix is a mixture of both regional and language issues). We have since established internal publishing (manuals, diskettes, etc.) on the ISBN system. In short, is see no benefit difference _to the end user_ between our "+//ISBN 1-55160" and Jon's "-//SUN::SUNSOFT" because in both cases the end user has the opportunity to implement their client-side indirection however they need to without impacting on the (assumed) sacrosanct SGML files. If this client-side-resolved indirection dereferences PUBLIC values to URLs or internet-resolved URN/URIs or absolute filename specifications or relative filename specifications or SQL queries or DMS copy-out procedures or JCL tape mount directives or even some future yet-to-be-determined-or-standardized query mechanism, the SGML files remain inviolable. Isn't this inviolability critically important to people who want to invest in their data for the long term? Certainly the SGML Open scheme has its benefits and features, but I agreed with Debbie that the minimum is a simple string comparison. If I choose to go beyond that with an SGML Open catalog, I can choose to do so without impacting on my marked up source. Then again, need XML establish the "environment" in which XML PUBLIC identifers are dereferenced? Is it in the scope of XML to go outside the syntax of the markup to specify any obligatory mechanism by which information in the markup is handled? Users of XML who wish to hardwire SYSTEM identifiers can still choose to do so; hopefully the XML exercize will teach people more about the benefits they can build into their markup, one of which could be the implementation of client-side-resolution of indirect PUBLIC identifiers. >James says the FPI is hard to implement. So, it is something >not everyone should be required to implement. But this seems >to be a system implementation restriction and I don't think >we are here to restrict implementors. Agreed. ......... Ken -- G. Ken Holman Tel: +1 613 596-CADE(2233) /\ /\ Chief Technology Officer Fax: +1 613 596-5934 \/ \/ Computer Microstar Software Ltd. WATS: 1 800 267-9975 /\ /\ Aided 3775 Richmond Road Mail: gkholman@microstar.com \/ \/ Document Nepean Ontario Info: cade@microstar.com /\ /\ Engineering CANADA K2H-5B7 Web: http://www.microstar.com \/ \/ -- 1605 Mardick Court, Box 266 Phone: +1 613 489-2987 Kars, Ontario CANADA K0A-2E0 E-mail: gkholman@CanadaMail.com -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6 mQCNAjHOimAAAAEEAK47HbDtZZB8hJBk+r9Zh7m7QxdFHTaz/m200QQ7J9XX4QYs h6hsuP6ZqBJXyLdmIVMEwR6Ry6oxjKMd6BRJ8OGScD89eIghgbrpMX+900NxM0x2 v3yWO9ki2gKRPrn4vlCXznnmcmsyke0G02T2LefXbgZHyVSqDSOLy8nwuN7dAAUR tClHLiBLZW4gSG9sbWFuIDxhYjk0NUBmcmVlbmV0LmNhcmxldG9uLmNhPg== =IN3T -----END PGP PUBLIC KEY BLOCK----- From: "Christopher R. Maden" Date: Mon, 2 Dec 1996 15:04:38 GMT To: w3c-sgml-wg@w3.org Subject: Re: MIME type for XML I think that application/xml is necessary, too. All text/* types, I believe, are required to have normalized line endings (0x0d0a). This works for 8-bit encodings, but for 16-bit encodings, 0x000d000a becomes 0x000d0a000d0a, which isn't quite the same thing... -Chris -- http://www.ebt.com +1.401.421.9550 +1.401.521.2030 One Richmond Square, Providence, RI 02906 USA" NDATA SGML.Geek> Date: Mon, 02 Dec 1996 10:35:01 -0500 To: Ken Holman , "'w3c-sgml'" From: Paul Prescod Subject: Re: Client-side-resolved Indirection At 09:42 AM 12/2/96 -0500, Ken Holman wrote: >If this client-side-resolved indirection dereferences PUBLIC values to >URLs or internet-resolved URN/URIs or absolute filename specifications >or relative filename specifications or SQL queries or DMS copy-out >procedures or JCL tape mount directives or even some future >yet-to-be-determined-or-standardized query mechanism, the SGML files >remain inviolable. > >Isn't this inviolability critically important to people who want to >invest in their data for the long term? That is a very clear statement of the potential benefit of public identifiers. our documents can predict the eventual URN facilities of the Web and take advantage of them as soon as they become available. In the meantime we can use proprietary mechanisms, like SGML Open catalogs, PURLs, handles, or whatever. >Certainly the SGML Open scheme has its benefits and features, but I >agreed with Debbie that the minimum is a simple string comparison. If I >choose to go beyond that with an SGML Open catalog, I can choose to do >so without impacting on my marked up source. Could you clarify here? Do you and Debbie wans a simple established mechanism for name resolution, or just the option to put in the public identifier (which is what I have been pushing for). >Then again, need XML establish the "environment" in which XML PUBLIC >identifers are dereferenced? Is it in the scope of XML to go outside >the syntax of the markup to specify any obligatory mechanism by which >information in the markup is handled? Well, URLs must be handled according to URL semantics so we *could* specify a resolution semantic for public identifers. But I think that would put an extra burden on implementors, risk XML's acceptance, and inappropriately disqualify the URN effort before it got off of the ground. Paul Prescod Date: Mon, 02 Dec 96 09:46:33 CST From: Michael Sperberg-McQueen Subject: Re: Client-side-resolved Indirection To: W3C SGML Working Group On Mon, 2 Dec 1996 10:33:02 -0500 Paul Prescod said: >>Certainly the SGML Open scheme has its benefits and features, but I >>agreed with Debbie that the minimum is a simple string comparison. >>If I choose to go beyond that with an SGML Open catalog, I can >>choose to do so without impacting on my marked up source. > >Could you clarify here? Do you and Debbie wans a simple established >mechanism for name resolution, or just the option to put in the public >identifier (which is what I have been pushing for). I'm not sure I have understood you both correctly -- if I have, you are arguing in favor of adding PUBLIC identifiers without specifying how they are to be resolved. I can't believe my eyes. True, adding the syntax without describing the semantics would only add a few lines to the specification. It seems to me, however, that it would blow XML out of the water and more or less completely eliminate the chances that independent implementations would inter-operate except by accident. >>Then again, need XML establish the "environment" in which XML PUBLIC >>identifers are dereferenced? Is it in the scope of XML to go outside >>the syntax of the markup to specify any obligatory mechanism by which >>information in the markup is handled? > >Well, URLs must be handled according to URL semantics so we *could* >specify a resolution semantic for public identifers. But I think >that would put an extra burden on implementors, risk XML's >acceptance, and inappropriately disqualify the URN effort before it >got off of the ground. I think this boils down to having a paragraph in the spec which says (possibly in different words) "PUBLIC identifiers are magic cookies, which are resolved using a Magic Cookie Resolution Algorithm which lies outside the scope of this specification. As a consequence, no two XML processors are likely to resolve them in the same way; implementors are encouraged to distinguish their processors by the ingenuity of their resolution methods, using SGML Open catalogs, clever directory path tricks, ad hoc configuration files, magic flashing lights and sirens, or other mechanisms." Does that make anyone really want to use this system? Not me. We tried that approach once before, in 8879. It left all the details open to the implementors. The implementors exercised this freedom for a while, but ultimately chose (driven by their users? or by the odd desire to make 8879 live up to its billing?) to abandon it and specify what 8879 left unspecified. Now we have the SGML Open catalog, and PUBLIC identifiers work. To answer the question someone posed: No, I, for one, did not use SGML public identifiers before the advent of the SGML Open catalogue, because no two systems I had resolved them in the same way; with system identifiers, at least, my troubles were limited to the times when I moved documents from one system to another. With public identifiers and no common resolution mechanism, I saw no payback at all for the effort of learning a different ad hoc odd hack solution for each parser, and strewing copies of my DTDs all over my hard disk. Not all operating systems have symbolic links ... I think FPIs are a Good Thing, and I'd like to see them in XML. Like Tim and some others, I went into this process assuming FPIs and SGML Open catalogs would clearly be part of the spec. But FPIs are, I think, not Absolutely Essential, since I know from experience that one can work a long time with only SYSTEM identifiers. Maybe they don't belong in XML 1.0; get something out quickly that's easy to implement, and which can easily be seen to be easy to implement. Maybe they do; I'm agnostic on that point. But one thing seems clear to me: if we have them, we need to specify how to handle them. If we don't, we are giving up without reason on the goal of interoperability and complete explicit definition of the language. Or am I wrong? Michael Sperberg-McQueen Date: Mon, 2 Dec 1996 08:29:11 -0800 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: etxperl@stdoca.uab.ericsson.se CC: w3c-sgml-wg@w3.org Subject: Re: Stylesheets [Per-ke Ling:] | I have with interset followed the debate "CSS vs. DSSSL-O" and | realized that although I have a fair knowledge of DSSSL-o, the same is | not true of CSS. I will hunt around on the W3C site and see what I can | find. | | However, it struck me that there may be others in a similar (or | opposite) position, so I believe that it would be helpful if somebody | could summarize the pros and cons of the respective specifications in | an objective? or at least factual manner. | | I think the comparison would be helpful in focusing the attention on | the issues that are of primary concern, eg design philosphy instead of | syntax. I agree that this would be a very interesting and useful discussion, but this is not the right place for it. In this phase of our activity, we are scheduled to talk about hypertext mechanisms, with a specification due in April. This group picks up stylesheets as an activity as part of Phase III, which takes place between April and November of 1997. The list for discussions of which stylesheet language to use for the Web is www-style@w3.org. You should be able to subscribe to that list via the W3C home page at http://www.w3.org. A number of people on the w3c-sgml-wg list are also on the www-style list and would be happy to join such a discussion, which I think is long overdue. You would also find many more people on that list who could answer your questions about CSS. Jon Date: Mon, 2 Dec 1996 08:45:59 -0800 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org CC: bosak@atlantic-83.eng.sun.com Subject: Re: Client-side-resolved Indirection [Ken Holman:] | In short, is see no benefit difference _to the end user_ between our | "+//ISBN 1-55160" and Jon's "-//SUN::SUNSOFT" because in both cases | the end user has the opportunity to implement their client-side | indirection however they need to without impacting on the (assumed) | sacrosanct SGML files. | | If this client-side-resolved indirection dereferences PUBLIC values to | URLs or internet-resolved URN/URIs or absolute filename specifications | or relative filename specifications or SQL queries or DMS copy-out | procedures or JCL tape mount directives or even some future | yet-to-be-determined-or-standardized query mechanism, the SGML files | remain inviolable. | | Isn't this inviolability critically important to people who want to | invest in their data for the long term? My thanks to Ken for making my point better than I was making it. This is what I was trying to say. Jon Date: Mon, 02 Dec 1996 12:36:52 -0500 To: Michael Sperberg-McQueen , W3C SGML Working Group From: Paul Prescod Subject: Re: Client-side-resolved Indirection At 09:46 AM 12/2/96 CST, Michael Sperberg-McQueen wrote: >I'm not sure I have understood you both correctly -- if I have, you >are arguing in favor of adding PUBLIC identifiers without specifying >how they are to be resolved. I can't believe my eyes. Well, I'm glad someone came out in favour of the other side, because I thought there might be people who felt that way, but were not saying something, or that perhaps I was not being clear in my proposal. Believe your eyes, Michael. >True, adding the syntax without describing the semantics would only add >a few lines to the specification. It seems to me, however, that it >would blow XML out of the water and more or less completely eliminate >the chances that independent implementations would inter-operate except >by accident. Or by design. People other than this WG *can* design systems. >I think this boils down to having a paragraph in the spec which says >(possibly in different words) "PUBLIC identifiers are magic cookies, >which are resolved using a Magic Cookie Resolution Algorithm which lies >outside the scope of this specification. As a consequence, no two XML >processors are likely to resolve them in the same way; implementors are >encouraged to distinguish their processors by the ingenuity of their >resolution methods, using SGML Open catalogs, clever directory path >tricks, ad hoc configuration files, magic flashing lights and sirens, or >other mechanisms." Right! You've hit the nail on the head in a couple of respects. First, document *authors* should not care about the resolution mechanism of document *consumers*. They should be totally independant. When you download an SGML document from the web, and download a catalog with it, you have, in my opinion, basically abused the catalog feature. It is a *useful* abuse, and one that is perhaps unavoidable in SGML, but not necessarily in XML. Think of it this way: if I have a public identifier -//PAUL in the same downloadable package as a catalog that says that -//PAUL = "http://www.paul.com", I have basically *hard coded* "http://www.paul.com" into the package (though not the document). In other words, I have required my SGML vendor to implement the whole SGML catalog so that I could get one lousy level of indirection. As Lee has pointed out, "big deal!" If XML is designed right I should be able to get that same level of indirection through entities, through hytime, or through URL redirection. Plus there are all the system level redirection capabilities (where available). If you know C++, think of downloadable catalogs as a pointer to a pointer...useful, but not very exciting. But PublicIDs are different. They are *names* for things. The C++ equivalent might be a "resource name", let's say "lpt1:". Now when you code your C++ program you don't *care* how lpt1: is mapped to a physical device, as long as it is done correctly. The mechanisms for doing this on DOS, Windows, OS/2 and Windows NT are going to be *very* different. But all that is important to the programmer is the name. (yes, Unix uses a different name for this, but different names for the same thing is a sociological problem outiside of the realm of any namespace mechanism) >We tried that approach once before, in 8879. It left all the details >open to the implementors. The implementors exercised this freedom for a >while, but ultimately chose (driven by their users? or by the odd desire >to make 8879 live up to its billing?) to abandon it and specify what >8879 left unspecified. Now we have the SGML Open catalog, and PUBLIC >identifiers work. First, 8879 demonstrably succeeded. People used the open facility, decided what features they wanted and standardized. If it had forced a facility from the beginning, SGML's implementation might have been slowed and the facility chosen might have been suboptimal. Second, SGML Open standardization was primarily important because different applications on the same *system* were using different catalog files. That meant that every tool had to be configured individually, which was a pain. But what if I want to make my organization's FPI to SYSId lookup mechanism a relational database query? I don't see how SGML Open catalogs can be easily used to implement that. SGML was flexible enough to allow SGML Open catalogs to exist without restricting us to them. So 8879 was correct to leave the resolving mechanism open. When URNs are finished, they will obviously be implemented as a distributed database lookup, not as a flat SGML Open text file, so it would be even more deadly to standardize on a text file format now than it would have been in 1986. Finally, Public Identifers were useful before SGML Open Catalog as I will demonstrate in a minute. >To answer the question someone posed: No, I, for one, did not use SGML >public identifiers before the advent of the SGML Open catalogue, Didn't you ever download documents that used these FPIs in it? or "ISO 8879-1986//ENTITIES Added Latin 1//EN" or "ISO 8879-1986//ENTITIES Greek Symbols//EN" or "ISO 8879-1986//ENTITIES Publishing//EN" I must admit that I had really expected TEILite (at least!) to have a public identifier, but you got me there, it doesn't. I guess everybody had better have their TEILite file named "teilite.dtd". I encourage you to create one. >because >no two systems I had resolved them in the same way; >with system identifiers, at least, my troubles were limited to the times when I >moved documents from one system to another. With public identifiers and >no common resolution mechanism, I saw no payback at all for the effort >of learning a different ad hoc odd hack solution for each parser, and >strewing copies of my DTDs all over my hard disk. Not all operating >systems have symbolic links ... So the problem is with the multiple systems on your machine, not your machine vs. someone else's machine. Since SGML Open catalogs are available, I would expect XML processors to use it. But I am *not* prepared to limit them to it, just as SGML did not. Thanks to that flexibility, I *can* create a conforming SGML FPI-lookup system based on a relational database, or file system hack, or grep. >But one thing seems clear to me: if we have them, we >need to specify how to handle them. If we don't, we are giving up >without reason on the goal of interoperability and complete explicit >definition of the language. Public identifier lookup is a semantic. We have the option of specifying it or not. I hope I have shown that tying the semantic to a particular resolution mechanism at this stage of the game would be wrong. But the good side is that you don't *really* have to give up anything. If you require all of the vendors of the products that you use to use SGML Open catalogs, or a subset, then you will have all of the benefits you currently accrue, without interfering with my ability to require *my* vendors to talk to my relational database, or my "Handle" URN server or ... If URNs weren't around the corner, I would suggest that SGML Open should standardize an API for communication between entity managers and FPI managers (such as SGML Open catalogs or SQL databases). But URNs will solve that problem... Paul Prescod Date: Mon, 2 Dec 1996 09:32:56 -0800 (PST) From: "Deborah A. Lapeyre" Subject: RE: Simple solution? Pub. Idents. vs URN. To: Paul Prescod cc: Martin Cottreau , "w3c-sgml-wg@w3.org" Forgive me, I took the weekend off and am late catching up. What Paul is describing is exactly what I am asking for by way of an FPI mechanism. Simple, optional, no behavoir implied but one identity mechanism in place. Behavoir determined outside, but "names", that is identifiers (as well as URL addresses) allowed. --Debbie ======================================================================= Deborah A. Lapeyre Phone: 301-231-6933 Mulberry Technologies, Inc. Fax: 301-231-6935 6010 Executive Blvd. Suite 608 E-mail: dlapeyre@mulberrytech.com Rockville, MD USA 20852 ======================================================================= Date: Mon, 2 Dec 1996 10:04:54 -0800 (PST) From: "Deborah A. Lapeyre" Subject: Re: Client-side-resolved Indirection To: Michael Sperberg-McQueen cc: W3C SGML Working Group > identifiers work. > > To answer the question someone posed: No, I, for one, did not use SGML > public identifiers before the advent of the SGML Open catalogue, because > no two systems I had resolved them in the same way. Very true, and they were a pain. But I used them extensively. If they got resolved, I tried not to worry (other than once, per setup, per sofdtware package, per system) how. My point is even that was better than nothing! > > I think FPIs are a Good Thing, and I'd like to see them in XML. Like > Tim and some others, I went into this process assuming FPIs and SGML > Open catalogs would clearly be part of the spec. But FPIs are, I think, > not Absolutely Essentia. But one thing seems clear to me: if we have >them, we > need to specify how to handle them. If we don't, we are giving up > without reason on the goal of interoperability and complete explicit > definition of the language. 1) Yes, FPIs are a Good Thing, for me and for many of my clients. Yes, I used them even before SGML Open. 2) Yes, I would like a full spec for handling them as part of XML, but the consensus seems to be against that. 3) Yes, the Web requires URLs just now. 4) So, yes I will accept the "Magic Cookie" approach. It isn't good. But it gives me a hook. Yes, this is a compromise, and not one I like. But is this better than nothing? Yes, it is. --Debbie Lapeyre Date: Mon, 2 Dec 1996 09:54:47 -0800 (PST) From: "Deborah A. Lapeyre" Subject: Re: Client-side-resolved Indirection To: Paul Prescod cc: Ken Holman , "'w3c-sgml'" > > Could you clarify here? Do you and Debbie want a simple established > mechanism for name resolution, or just the option to put in the public > identifier (which is what I have been pushing for). > XML should not (as SGML did not) establish a resolution mechanism. I just want the ability to assign an unchanging identifier (not a location) to a document. (Also a DTD , DTD fragment, etc. -- but that can wait.) Does the document also need a URL? OF course, if it is going over the Web. But I agree with Paul. Can't the folks who can't handle a PUBLIC identifier just ignore it? As many others have mentioned, my document may have many URLs, but It will have only one PUBLIC name. What use I make of that is my application's business. Yes? NO? --Debbie Lapeyre Date: Mon, 02 Dec 96 12:04:38 CST From: Michael Sperberg-McQueen Subject: Re: Client-side-resolved Indirection To: W3C SGML Working Group cc: Paul Prescod On Mon, 2 Dec 1996 12:35:15 -0500 Paul Prescod said: > ... You've hit the nail on the head in a couple of respects. First, >document *authors* should not care about the resolution mechanism of >document *consumers*. They should be totally independant. As an author or publisher, I don't care about the mechanics; I do care, rather a lot, about ensuring (a) that the document consumer can actually resolve entity references and (b) that when they do, they get the right thing. Specifying a method of resolving FPIs seems to make those guarantees; specifying FPIs without a defined method of resolution seems to make neither guarantee. That's a level of independence between publisher and reader which I am not prepared to endorse. >>To answer the question someone posed: No, I, for one, did not use SGML >>public identifiers before the advent of the SGML Open catalogue, > >Didn't you ever download documents that used these FPIs in it? > > >or >"ISO 8879-1986//ENTITIES Added Latin 1//EN" >or >"ISO 8879-1986//ENTITIES Greek Symbols//EN" >or >"ISO 8879-1986//ENTITIES Publishing//EN" Not that I know of. Version 3.2 of HTML did not exist before the advent of SGML Open catalogues; the entity sets did, but I don't believe I ever downloaded SGML documents from the Web at that time; when on occasion I got SGML documents from other people, working with the DTD subset to make my SGML systems find the entity sets was just part of the job of getting the files to parse. >I must admit that I had really expected TEILite (at least!) to have >a public identifier, but you got me there, it doesn't. I guess >everybody had better have their TEILite file named "teilite.dtd". I >encourage you to create one. Don't believe your eyes; TEI Lite has a public identifier and has, as far as I can remember, always had one. I could be wrong, of course. Maybe it's only had one since Panorama became available and used the SGML Open catalogue. -C. M. Sperberg-McQueen Date: Mon, 02 Dec 1996 13:42:15 -0500 To: Michael Sperberg-McQueen , W3C SGML Working Group From: Paul Prescod Subject: Re: Client-side-resolved Indirection Cc: Paul Prescod At 12:04 PM 12/2/96 CST, Michael Sperberg-McQueen wrote: >On Mon, 2 Dec 1996 12:35:15 -0500 Paul Prescod said: >> ... You've hit the nail on the head in a couple of respects. First, >>document *authors* should not care about the resolution mechanism of >>document *consumers*. They should be totally independant. > >As an author or publisher, I don't care about the mechanics; I do >care, rather a lot, about ensuring (a) that the document consumer can >actually resolve entity references and (b) that when they do, they >get the right thing. Specifying a method of resolving FPIs seems to >make those guarantees; specifying FPIs without a defined method of >resolution seems to make neither guarantee. How so? How is this: harder to resolve correctly than this? and then, in an *external* file, PUBLIC "-//Paul//FOOBAR" http://www.paul.com/foobar The latter seems more likely to break, to me. And besides, what's the point? As I've pointed out, you have only shifted the resolution by one indirection level. You might as well have used HyTime mechanisms to point into an XML-syntax "catalog" file. >Not that I know of. Version 3.2 of HTML did not exist before the advent >of SGML Open catalogues; the entity sets did, but I don't believe I ever >downloaded SGML documents from the Web at that time; when on occasion I >got SGML documents from other people, working with the DTD subset to >make my SGML systems find the entity sets was just part of the job of >getting the files to parse. Ouch. Your experience was worse than mine. I suspect the big difference may be that TEI is a set of DTD fragments and not a monolithic DTD. As you may know I helped to run a technical writing course on the Web where all of the students used our public identifiers. We had hundreds of students, were using a mix of software with and without SGML Open catalog support and never had any problems with FPI lookup. In retrospect, I don't really know what *could* have gone wrong, or how SGML Open catalogs would have helped at all. As long as you have agreed upon a set of public identifier names, the system is quite robust, with or without catalogs. When people send you documents, do they usually send a catalog file? If not, then why do you care if the format is standardized. If so, then how could they have predicted the layout of your hard drive? And if they could have done so, they could have hard-coded the entity system ids, right? >Don't believe your eyes; TEI Lite has a public identifier and has, as >far as I can remember, always had one. I could be wrong, of course. >Maybe it's only had one since Panorama became available and used the >SGML Open catalogue. Okay, so TEILite is a good example. Although I still have software around that doesn't support public identifiers, if I were going to download more than three teilite document this week, I would modify the two or three catalog files on my hard disk to support it. Better to make those three changes today than one per document that I download. Paul Prescod Date: Mon, 2 Dec 1996 10:51:31 -0800 (PST) From: Terry Allen To: U35395@UICVM.UIC.EDU, w3c-sgml-wg@w3.org Subject: Re: Client-side-resolved Indirection Cc: papresco@calum.csclub.uwaterloo.ca Michael writes: >Specifying a method of resolving FPIs seems to make those guarantees; specifying FPIs without a defined method of resolution seems to make neither guarantee. That's a level of independence between publisher and reader which I am not prepared to endorse. If the defined method of resolution is "resolve as URNs", which various systems may do variously but with the same results (unless some succeed and some fail), does that suffice for you? There will be many (possibly competing) methods of resolving URNs; your system (NOT your XML system, your dealing-with-the-Internet system) will have local preferences about how to use them; you solve the problem there for all applications that need to handle URNs. As a reader, I want that kind of independence; as a writer, I want to avoid having to touch my docs after I'm finished with them; as a publisher, I'm more or less where I am with ISBNs, except that the resolution mechanisms are still being constructed. Regards, Terry Allen Fujitsu Software Corp. tallen@fsc.fujitsu.com "In going on with these experiments, how many pretty systems do we build, which we soon find outselves obliged to destroy?" - Benjamin Franklin A Davenport Group Sponsor: http://www.ora.com/davenport/index.html From: lee@sq.com Date: Mon, 2 Dec 96 15:53:56 EST To: w3c-sgml-wg@w3.org Subject: Fried Parrot Intestines in my XML Soup > If the defined method of resolution is "resolve as URNs", which > various systems may do variously but with the same results (unless > some succeed and some fail), does that suffice for you? For me it doesn't, because (1) an FPI isn't (last time a checked) a full valid URN (e.g. lacks the urn: prefix) (2) you can use URNs in the SYSTEM identifiers if you like. There is obviously a lot of emotional attachment to formal public identifiers. In some cases perhaps it's because people think that resolving an FPI always gets you the same data, It doesn't. There isn't even a resolution mechanism defined apart from Write a letter to ISO or the GCA. The CATALOG mechanism only helps you tie together an FPI and some data once you have already used some other mechanism to fetch the data -- e.g. typed it in probably mostly OK >from the SGML Handbook or the Annex or the JoanSmithPinkThing. It is true that a way to refer to commonly available files (DTDs, entity sets, the SunSoft logo :-), notation definitions perhaps) in a way that would allow a lookaside cache is useful. We include a number of common entity sets and DTDs with SoftQuad Panorama, so that if you put something on the web with one of those DTDs, the user doesn't need to download it... and the software knows that. That's useful for SGML and Panorama. Would it be useful for XML? If so, we need to consider a mechanism to do that in XML. It could be with FPIs. It could be with URNs. It could be something entirely different. First we have to understand what we are trying to achieve,and only then consider how to do it. Other people use FPIs as a kind of indirection, so that filenames are not wired in, e.g. in case the installation in EUROPE>SGML>SW.DIR;3 (Is that a valid PR1MOS networth path?) is moved. This is a different purpose, and although the same mechanism might solve both needs, it might also be appropriate to use two mechanisms. It's done in C, for example, with a file inclusion path: #include looks in a per-installation standard place (or places) for the file "stdio.h"; you can add more directories to the list, e.g. with -I on most C compilers, or in a dialogue box for preferences on some systems. #include "stdio.h" looks (by default) in the same place as the file containing that line, like a relative/partial URL. I am not here proposing C's semantics (or syntax!) for XML, but only pointing out that there are other mechanisms. We have focussed in very closely on FPIs, but I don't think we've talked enough about what problems we are trying to solve. For those of you who think it's a fought-and-won battle, I can only say that there mere fact that everyone didn't immediately say yes indicates that this isn't so. Hence, I suggest that we need the following: * a list of required functionality * discussion of possible approaches to providing that functionality * final choice of mechanism(s) * discussion on syntax, if needed * final shoice of syntax Lee Date: Mon, 2 Dec 1996 16:07:33 -0500 To: W3C SGML Working Group From: dgd@cs.bu.edu (David G. Durand) Subject: Re: Client-side-resolved Indirection Cc: Paul Prescod At 12:04 PM 12/2/96, Michael Sperberg-McQueen wrote: >On Mon, 2 Dec 1996 12:35:15 -0500 Paul Prescod said: >> ... You've hit the nail on the head in a couple of respects. First, >>document *authors* should not care about the resolution mechanism of >>document *consumers*. They should be totally independant. > >As an author or publisher, I don't care about the mechanics; I do >care, rather a lot, about ensuring (a) that the document consumer can >actually resolve entity references and (b) that when they do, they >get the right thing. Specifying a method of resolving FPIs seems to >make those guarantees; specifying FPIs without a defined method of >resolution seems to make neither guarantee. That's a level of >independence between publisher and reader which I am not prepared to >endorse. But it is one that is essential. There can be no _single_ resolution protocol for names, otherwise, they are simply locators, in the sense that the protocol is hardwired -- and it deterministically yields a location given a name. Names are useful for the cases where you want to be able to look things up. But let's be concrete for second: SGML allows PUBLIC without a system identifier: I am not proposing that; It is an adequate compromise that it even be _possible_ to have a public identifier in addition to the system ID. In this case, a public identifier is a way that an individual (or a motivated client) can safely change system IDs with confidence that the same resource is being addressed. No resolution mechanism required. We can accomodate the desire for a resolution mechanism by specifying (in advance of deployment) that FPIs should be resolvable via the FPI grandfathering of URNS. Alternatively, we can specify that PUBLIC will not be allowed, but that a URN can be included as (or in addition to) a URL. If we propose an experimental syntax for FPI-urns, that would be enough to enable interested XML applications to detect their use, and do our best to make sure that it becomes a standard (this would be pretty easy, I think). In this case, we must have a way to specify multiple URIs so that a "plain old URL" can also be provided, for the naming-impaired. >Not that I know of. Version 3.2 of HTML did not exist before the advent >of SGML Open catalogues; the entity sets did, but I don't believe I ever >downloaded SGML documents from the Web at that time; when on occasion I >got SGML documents from other people, working with the DTD subset to >make my SGML systems find the entity sets was just part of the job of >getting the files to parse. And this has always been a serious problem. We could enable off-line browsing of cached XML documents simply by specifying that PUBLIC IDs are guaranteed-good cache-keys. This is at the least more difficult with HTTP, as you need to do at least a HEAD to be sure that you have the correct version of anything. It's also easier and friendlier to prompt a user for the location of a public identifier than for a random URL. This is also more likely to be implemented, since implementors will be aware that there might be a possibility of manual resolution of FPIs -- since this is almost never the case for URLs, the option is an unlikely one to offer. -- David I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ Date: Mon, 2 Dec 1996 16:07:46 -0500 To: W3C SGML Working Group From: dgd@cs.bu.edu (David G. Durand) Subject: Re: SGML and XML First, we have a compriomise on this issue that is far from perfect, but took us at least a month of intense debate to reach: why are we re-opening it? Let's leave not-so-well-enough alone! I'm wistful for a better (different) solution, but given that everyone (including me) seems to find the compromise livable, shouldn't we leave it in place? At 11:04 AM 12/1/96, Michael Sperberg-McQueen wrote: >I think I agree that simplifying RE/RS handling would be good, but >in the interests of clarity let me ask: Do you mean we have to make >the DTD irrelevant to white-space handling > > * because that was decided at some point, or Yes. > * as a direct consequence of some other decision, or Yes, as well: if this is not the case, then applications will produce different results depending on whether a DTD is present or not. > * because that would be a cleaner design? I dunno, none of the proposals we've considered leads to a "clean design" other than making all whitespace significant (a policy that has been vetoed for compatibility reasons). >I agree that it would be desirable to make rules which allow >processors to skip the DTD and still get a wholly correct parse >tree, er, grove, but I don't think that's possible, since a >processor cannot resolve entity references or provide correct >default attribute values without reading the DTD. We already determined that resolving entity references is not required: the XML data structure cannot completely hide entity references from applications in that way that SGML parsers do. Default attributes at least have fixed values, so they must offer relatively little to an application that doesn't care about DTDs. >I also believe that if XML makes no distinction between element >content and mixed content, it will be impossible to build correct >XML processors on top of SGML parsers. We did determine, I think, that some differences would exist between the SGML and XML "data streams" in the case of RE near markup. This was deemed acceptable compared to the horrendously complex SGML status quo. >It might be worthwhile to distinguish clearly among > (a) full validation, which checks the instance against the DTD and >provides the correct grove, > (b) full parsing, which doesn't validate but does provide >character-for-character the correct grove, and which requires at >least some attention to the DTD, and > (c) minimal or partial parsing, which doesn't require a DTD and >which is guaranteed (in XML) to vary from the correct grove only in >specific ways: > (i) it will have extra white space in element content, > (ii) it won't provide correct values for defaulted attributes, > (iii) it won't have information about attribute types like >ID/IDREF. > (iv) it won't resolve references to entities not hardcoded into >the processor. This is far to complex: two kinds of parsing are enough. >I'm not sure (iii) is part of the standard grove output on which we >want to model the XML processor's output; if it isn't, then that's >one difference fewer. I don't know from groves, but if we repeat the mistakes of the ESIS on ID/IDREF (making them into crypto-types that the application can't see), we are truly foolish. for the same reason we need to make sure that empty elements are treated as a distinct type from elements with no content: another ESIS mistake that I thought we had forsworn. Or are we going to give up trivial XML->XML filtering? >-C. M. Sperberg-McQueen I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ Date: Mon, 2 Dec 1996 13:11:32 -0800 (PST) From: Terry Allen To: w3c-sgml-wg@w3.org Subject: FPIs in XML in response to Lee: what's wanted is persistent naming. | > If the defined method of resolution is "resolve as URNs", which | > various systems may do variously but with the same results (unless | > some succeed and some fail), does that suffice for you? | | For me it doesn't, because | (1) an FPI isn't (last time a checked) a full valid URN (e.g. lacks the | urn: prefix) "urn:" is now considered a matter of religious choice. If you like, when you send it to your URN resolver (outside your XML system) you can prepend "urn:fpi:" | (2) you can use URNs in the SYSTEM identifiers if you like. Up to the specification to say, isn't it? | There is obviously a lot of emotional attachment to formal public identifiers. "Emotional" is out of order. There is clearly attachment to something that people have found to work. They are also disturbed at the thought that they will have only URLs to work with. | In some cases perhaps it's because people think that resolving an FPI always | gets you the same data, It doesn't. There isn't even a resolution mechanism Resolving a URN always gives you the same data by the namer's definition of "same." That's the way they're defined. Regards, Terry Allen Fujitsu Software Corp. tallen@fsc.fujitsu.com "In going on with these experiments, how many pretty systems do we build, which we soon find outselves obliged to destroy?" - Benjamin Franklin A Davenport Group Sponsor: http://www.ora.com/davenport/index.html Date: Mon, 2 Dec 1996 16:34:54 -0500 To: w3c-sgml-wg@w3.org From: dgd@cs.bu.edu (David G. Durand) Subject: Re: Fried Parrot Intestines in my XML Soup At 3:53 PM 12/2/96, lee@sq.com wrote: >> If the defined method of resolution is "resolve as URNs", which >> various systems may do variously but with the same results (unless >> some succeed and some fail), does that suffice for you? > >For me it doesn't, because >(1) an FPI isn't (last time a checked) a full valid URN (e.g. lacks the > urn: prefix) Correction: use URN resolution in conjunction with the URN FPI grandfathering rules. Now objection 1 is moot. >(2) you can use URNs in the SYSTEM identifiers if you like I don't think that this is true -- if it is true, we've got at least a start on something resonable: we just need to get the URN people moving on a namespace ID for FPIs, and we can use FPIs. However, and this is pretty important: we lose tha bility to have an FPI _and_ a URL. So by insisting on inappropriate resolution stategies, we make FPIs harder to use for those already using them, and more likely to break the software of those not using them. Seems like a lose-lose proposition, unless we allow multiple URIs in the SYSTEM id. >There is obviously a lot of emotional attachment to formal public identifiers. There's a lot more than emotional attachment, but unfortunately some people just have a glitch about names: they don't understand resolution-protocol independent resolution. Looking something up in a book or writing a letter are _valid_, _useful_, perhaps even _necessary_ resolution methods, just as is the use of some TCP/IP protocol. >In some cases perhaps it's because people think that resolving an FPI always >gets you the same data, It doesn't. There isn't even a resolution mechanism >defined apart from Write a letter to ISO or the GCA. The CATALOG mechanism >only helps you tie together an FPI and some data once you have already used >some other mechanism to fetch the data -- e.g. typed it in probably mostly OK >from the SGML Handbook or the Annex or the JoanSmithPinkThing. FPIs do always get you the same data, if they are used correctly. That's the definition of them, and the contract, that you commit to when you assign an FPI. >It is true that a way to refer to commonly available files (DTDs, entity sets, >the SunSoft logo :-), notation definitions perhaps) in a way that would >allow a lookaside cache is useful. We include a number of common entity sets >and DTDs with SoftQuad Panorama, so that if you put something on the web >with one of those DTDs, the user doesn't need to download it... and the >software knows that. As you say. This is the point -- the resolution method you use in unique, but that's OK, because a name is a name is a name is a name. >If so, we need to consider a mechanism to do that in XML. >It could be with FPIs. It could be with URNs. It could be something >entirely different. First we have to understand what we are trying to >achieve,and only then consider how to do it. No, we can consider whether FPIs, which are good on their own merits, support this mechanism. And, surprise! -- they do. >Other people use FPIs as a kind of indirection, so that filenames are >not wired in, e.g. in case the installation in EUROPE>SGML>SW.DIR;3 >(Is that a valid PR1MOS networth path?) is moved. This is one perfectly valid way to use FPIs. >This is a different purpose, and although the same mechanism might solve >both needs, it might also be appropriate to use two mechanisms. >It's done in C, for example, with a file inclusion path: >#include >looks in a per-installation standard place (or places) for the file >"stdio.h"; you can add more directories to the list, e.g. with -I on >most C compilers, or in a dialogue box for preferences on some systems. >#include "stdio.h" >looks (by default) in the same place as the file containing that line, >like a relative/partial URL. >I am not here proposing C's semantics (or syntax!) for XML, but only >pointing out that there are other mechanisms. FPIs are a social discipline for naming, independent of mechanisms. Your "c" syntax is also an acceptable mechanism for FPIs. FPIS are _NOT_ about mechanism. >We have focussed in very closely on FPIs, but I don't think we've >talked enough about what problems we are trying to solve. Persistent, globally-unique, resolution mechanism-independent identifiers for internet resources, for use in XML documents. >For those of you who think it's a fought-and-won battle, I can only say >that there mere fact that everyone didn't immediately say yes indicates >that this isn't so. It shows that some people don't understand the difference between names and locations, nothing more. Read the URN group logs. There is no solution to this argument, either you agree to let the "namers" have their "harmless games" or they lose capabilities that they depend on and you don't believe in. I'm sorry, but there's not much more to it, once we have agreed to forswear philosophy. > >Hence, I suggest that we need the following: >* a list of required functionality Persistent, globally-unique, resolution mechanism-independent identifiers for internet resources, for use in XML documents. >* discussion of possible approaches to providing that functionality A way to assign strings to resources that will not be reused to identify other resources, in the next 500 years (assuming the continuance of people who care about such things). >* final choice of mechanism(s) Why not FPIs? they meet the reqwuirements, and are already part of SGML (and soon to be part of the URN facilities) >* discussion on syntax, if needed hierarchical authority-based name assignment (like FPIs) is the easiest name-assignment discipline to implement. >* final shoice of syntax Why not FPIs, they already exist, and are compatible, etc.?? -- David, who cannot believe he is having this same discussion again. I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ Date: Mon, 2 Dec 1996 16:34:46 -0500 To: w3c-sgml-wg@w3.org From: dgd@cs.bu.edu (David G. Durand) Subject: Re: Client-side-resolved Indirection At 10:51 AM 12/2/96, Terry Allen wrote: >If the defined method of resolution is "resolve as URNs", which >various systems may do variously but with the same results (unless >some succeed and some fail), does that suffice for you? There >will be many (possibly competing) methods of resolving URNs; >your system (NOT your XML system, your dealing-with-the-Internet >system) will have local preferences about how to use them; you >solve the problem there for all applications that need to handle >URNs. As a reader, I want that kind of independence; as a writer, >I want to avoid having to touch my docs after I'm finished with >them; as a publisher, I'm more or less where I am with ISBNs, >except that the resolution mechanisms are still being constructed. Hear hear. This is the correct way to proceed: we can point to URNs now, and we are all ready for experimental deployment today, and standardization tomorrow. And we don't have to revise XML based on the availability of URN resolution, crippling, in the meantime, those people who need and are willing to implement custom FPI resolution now. --- David _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ Date: Mon, 2 Dec 1996 16:56:40 -0500 From: gtn@ebt.com (Gavin Nicol) To: lee@sq.com CC: w3c-sgml-wg@w3.org Subject: Re: Fried Parrot Intestines in my XML Soup >Hence, I suggest that we need the following: >* a list of required functionality >* discussion of possible approaches to providing that functionality >* final choice of mechanism(s) >* discussion on syntax, if needed >* final shoice of syntax Hear hear! From: Ken Holman To: "'w3c-sgml'" Subject: RE: Client-side-resolved Indirection Date: Mon, 2 Dec 1996 16:57:33 -0500 [Paul Prescod] >[Ken Holman] >>Certainly the SGML Open scheme has its benefits and features, but I >>agreed with Debbie that the minimum is a simple string comparison. If I >>choose to go beyond that with an SGML Open catalog, I can choose to do >>so without impacting on my marked up source. > >Could you clarify here? Do you and Debbie wans a simple established >mechanism for name resolution, or just the option to put in the public >identifier (which is what I have been pushing for). [Michael Sperberg-McQueen] >But one thing seems clear to me: if we have them, we >need to specify how to handle them. If we don't, we are giving up >without reason on the goal of interoperability and complete explicit >definition of the language. I did not think that an established mechanism for name resolution was required, but I now think an XML catalog is necessary to communicate to a recipient the mapping of PUBLIC identifiers in a set of entities (if there is only one entity, there are no PUBLIC or SYSTEM identifiers). BTW, I think PUBLIC identifiers are far more "interoperable" than SYSTEM identifiers. If I build an (URL-only) XML Application for my colleagues, then I may have to hardwire the working URLs that are behind my firewall. When I choose to ship that application outside the firewall, I am obliged to modify the raw (working) sources to change the SYSTEM identifiers to URLs outside my firewall and then make the URLs publicly available so that the SGML files supplied can be used by the recipient without modification. If I can't make the files publicly available on my server (or the recipient doesn't have HTTP access) the recipient must determine from (perhaps cryptic) URLs which entities map to which SYSTEM identifiers. If, however, I shipped the package of SGML sources with only PUBLIC identifiers, then the recipient can choose where in their own environment to place those files, and only change a catalog file. Catalogs are not meant to be system independent (though, as I mentioned before, there are relative path conventions in the SGML Open catalog that may mean I don't have to change your catalog to use it on my system), they are meant to be system specific. Personally, I would suggest a subset of the SGML Open catalog format. We released one of our products before there was an SGML Open and much of our own implemented catalog format turned out to be compatible with what came from SGML Open. The base of what is needed is the keyword PUBLIC, followed by a public identifier, followed by the information required to identify the entity (if a system identifier with a relative path, the path is relative to the catalog file). By requiring both identifiers to be quoted, future versions of the XML catalog can implement other keywords. I also like having comments in catalogs, but that can wait (I've included it below for completeness). Since the use of PUBLIC is not obligatory in XML, the use of a catalog is not obligatory. Here's a thought ... could an XML catalog could be either a "system file" or "system documentation" based on how the author wishes to create it? For example, the following could be a catalog in the first file in a MIME envelope of three files: PUBLIC "+//ISBN 1-55160::MSL//DTD Presentation Slide Show//EN" "Second file in MIME envelope" PUBLIC "ISO 8879:1986//ENTITIES Added Latin 1//EN" "Third file in MIME envelope" (this is probably a poor example as I think I recall that the MIME parts can be uniquely identified, which would be the most appropriate SYSTEM values, but I hope that my point is getting across) >>Then again, need XML establish the "environment" in which XML PUBLIC >>identifers are dereferenced? Is it in the scope of XML to go outside >>the syntax of the markup to specify any obligatory mechanism by which >>information in the markup is handled? > >Well, URLs must be handled according to URL semantics so we *could* >specify >a resolution semantic for public identifers. But I think that would put >an >extra burden on implementors, risk XML's acceptance, and >inappropriately >disqualify the URN effort before it got off of the ground. Then perhaps the above use of a catalog as "system documentation" takes the "burden" off of implementors: if I choose an XML system implemented without the XML catalog, then the catalog file is documentation to me, the consumer; if I choose an XML system implemented with the XML catalog, then the catalog file is a system support file to my software. In another message I'll prepare a straw-man specification of changes to XML for PUBLIC identifiers. Responses to this thread can continue on the merits and drawbacks of the concept. ......... Ken -- G. Ken Holman Tel: +1 613 596-CADE(2233) /\ /\ Chief Technology Officer Fax: +1 613 596-5934 \/ \/ Computer Microstar Software Ltd. WATS: 1 800 267-9975 /\ /\ Aided 3775 Richmond Road Mail: gkholman@microstar.com \/ \/ Document Nepean Ontario Info: cade@microstar.com /\ /\ Engineering CANADA K2H-5B7 Web: http://www.microstar.com \/ \/ -- 1605 Mardick Court, Box 266 Phone: +1 613 489-2987 Kars, Ontario CANADA K0A-2E0 E-mail: gkholman@CanadaMail.com -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6 mQCNAjHOimAAAAEEAK47HbDtZZB8hJBk+r9Zh7m7QxdFHTaz/m200QQ7J9XX4QYs h6hsuP6ZqBJXyLdmIVMEwR6Ry6oxjKMd6BRJ8OGScD89eIghgbrpMX+900NxM0x2 v3yWO9ki2gKRPrn4vlCXznnmcmsyke0G02T2LefXbgZHyVSqDSOLy8nwuN7dAAUR tClHLiBLZW4gSG9sbWFuIDxhYjk0NUBmcmVlbmV0LmNhcmxldG9uLmNhPg== =IN3T -----END PGP PUBLIC KEY BLOCK----- Date: Mon, 02 Dec 1996 17:45:24 -0500 To: lee@sq.com, w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: Fried Parrot Intestines in my XML Soup At 03:53 PM 12/2/96 EST, lee@sq.com wrote: >> If the defined method of resolution is "resolve as URNs", which >> various systems may do variously but with the same results (unless >> some succeed and some fail), does that suffice for you? > >For me it doesn't, because >(1) an FPI isn't (last time a checked) a full valid URN (e.g. lacks the > urn: prefix) This isn't a problem. Just prepend urn:fpi: to the front. We may want to consider how we are going to address *full* urn:'s in the future, though. We should reserve an urn: prefix in the PublicID syntax now. >(2) you can use URNs in the SYSTEM identifiers if you like. We've discussed the downside of this already. * First, the spec. doesn't allow it. * Second, if we change "URL" to "URI" then we have no transition mechanism. We must wait until our readers all have name-enabled browsers to use them *at all*. * Third, all of our name-enabled SGML tools break, for no good reason. >There is obviously a lot of emotional attachment to formal public identifiers. No. There is a lot of technical attachment to names, and FPIs are a naming mechanism that work, that we can continue to use with only a small change to the standard. We are just asking you not to break our working systems because you haven't personally found a need for a feature. >In some cases perhaps it's because people think that resolving an FPI always >gets you the same data, It doesn't. There isn't even a resolution mechanism >defined apart from Write a letter to ISO or the GCA. **WHO CARES?** There isn't a well defined resolution mechanism for "Lee Quin" but I find it useful to refer to you anyways. Only I want is the *right* to name things in addition to pointing to them. >It is true that a way to refer to commonly available files (DTDs, entity sets, >the SunSoft logo :-), notation definitions perhaps) in a way that would >allow a lookaside cache is useful. We include a number of common entity sets >and DTDs with SoftQuad Panorama, so that if you put something on the web >with one of those DTDs, the user doesn't need to download it... and the >software knows that. > >That's useful for SGML and Panorama. >Would it be useful for XML? Why not? Have we come to the end of DTDs? or entity sets? >We have focussed in very closely on FPIs, but I don't think we've >talked enough about what problems we are trying to solve. As David said: "Persistent, globally-unique, resolution mechanism-independent identifiers for internet resources, for use in XML documents." Okay. I have 10 000 students around the world. We all have the same DTD. We all have a common set of entities defined. We may be using common XML software, we may not be. I need a way to send them XML documents that refer to those DTDs and entities in a way that is not dependant on the network, because they will use it off of the network. FPIs solve this problem. URLs *cannot*, because I cannot know where on their machine their DTDs and entities are. I also cannot require them to edit a text file (either a catalog file or a .sgm) to fix it. This is a real problem I have (100 students, not 10 000). Let's say, instead, that we are on an Intranet. FPIs are stored in a relational database and translated to URLs by the database. URLs cannot work, because they are not names, and are bound to particular network protocols. >Hence, I suggest that we need the following: >* a list of required functionality David has provided that. "Persistent, globally-unique, resolution mechanism-independent identifiers for internet resources, for use in XML documents." >* discussion of possible approaches to providing that functionality I think we've already discussed three: "URL or URN, pick one", "as many URLs or URNs as you want" and "FPIs and URLs". URLs are not persistent, or resolution mechanism independent. Paul Prescod From: Ken Holman To: "'w3c-sgml'" Subject: RE: Fried Parrot Intestines in my XML Soup Date: Mon, 2 Dec 1996 17:40:51 -0500 [lee@sq.com] >Hence, I suggest that we need the following: >* a list of required functionality A mechanism is required for authors to create system- and location-independent specification of entities. Users of system- and location-independent identifiers can create read-only XML files with entities whose resolution can be specified (or modified) outside of the XML file. >* discussion of possible approaches to providing that functionality Allow users to specify a system-independent identifier either in conjunction with, or in place of, a system-dependent identifier. Where an entity is defined by both a system-independent identifier and a system-dependent identifer, an attempt is first made to satisfy the system-independent identifier, and when that is not successful, an attempt is then made to satisfy the system-dependent identifier. (This behaviour is implied in SGML Clause 10.1.6 "The system identifier can be omitted if the system can generate it from the public identifier and/or other information available to it"). >* final choice of mechanism(s) Not my place to decide; I vote for the above. >* discussion on syntax, if needed Coming in another post (real soon now!). >* final shoice of syntax Not my place to decide; I vote for my pending straw man (modified, as required, by those more practiced in XML functional specification than me). > >Lee ......... Ken -- G. Ken Holman Tel: +1 613 596-CADE(2233) /\ /\ Chief Technology Officer Fax: +1 613 596-5934 \/ \/ Computer Microstar Software Ltd. WATS: 1 800 267-9975 /\ /\ Aided 3775 Richmond Road Mail: gkholman@microstar.com \/ \/ Document Nepean Ontario Info: cade@microstar.com /\ /\ Engineering CANADA K2H-5B7 Web: http://www.microstar.com \/ \/ -- 1605 Mardick Court, Box 266 Phone: +1 613 489-2987 Kars, Ontario CANADA K0A-2E0 E-mail: gkholman@CanadaMail.com -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6 mQCNAjHOimAAAAEEAK47HbDtZZB8hJBk+r9Zh7m7QxdFHTaz/m200QQ7J9XX4QYs h6hsuP6ZqBJXyLdmIVMEwR6Ry6oxjKMd6BRJ8OGScD89eIghgbrpMX+900NxM0x2 v3yWO9ki2gKRPrn4vlCXznnmcmsyke0G02T2LefXbgZHyVSqDSOLy8nwuN7dAAUR tClHLiBLZW4gSG9sbWFuIDxhYjk0NUBmcmVlbmV0LmNhcmxldG9uLmNhPg== =IN3T -----END PGP PUBLIC KEY BLOCK----- Date: Mon, 02 Dec 96 15:16:29 CST From: Michael Sperberg-McQueen Subject: Re: SGML and XML To: "David G. Durand" , W3C SGML Working Group On Mon, 2 Dec 1996 16:02:35 -0500 David G. Durand said: >First, we have a compriomise on this issue that is far from perfect, >but took us at least a month of intense debate to reach: why are we >re-opening it? Let's leave not-so-well-enough alone! I'm wistful for >a better (different) solution, but given that everyone (including me) >seems to find the compromise livable, shouldn't we leave it in place? I think some people are interested in re-opening it because Charles seems to have found a way to get RE Delenda Est behavior out of SGML processors. At least, it's the behavior I thought was intended for REDE: record ends are always passed through in mixed content, and all white space is eaten in element content. Since many people gave up on REDE solely for compatibility reasons, Charles's new proposal seems to open the possibility of getting something materially better than the compromise we have now. >At 11:04 AM 12/1/96, Michael Sperberg-McQueen wrote: >>I think I agree that simplifying RE/RS handling would be good, but >>in the interests of clarity let me ask: Do you mean we have to make >>the DTD irrelevant to white-space handling >> >> * because that was decided at some point, or > Yes. When was that? I don't remember that decision, and I'd like to refresh my memory on the wording. Actually, I rather suspect that Lee was just failing to distinguish his own opinions from logical necessities and from decisions made in the design to date, and since he has scolded others on that account, I thought I'd ask to make sure. >> * as a direct consequence of some other decision, or > Yes, as well: if this is not the case, then applications will produce >different results depending on whether a DTD is present or not. Yes, they will. How is this incompatible with any decision we've made? I don't remember a decision saying they can't produce different results, only a series of strenuous attempts to minimize the set of differences. It didn't go to zero, and won't while XML has attribute defaults and entities. Since I'd rather have attribute defaults and entities than lose them, this seems like a good deal to me. The goal is to let some useful applications skip the DTD, if they can. Right now, very few useful applications can skip the DTD, because most useful applications are guided in their actions by element boundaries, and these cannot be reliably detected without knowing which elements have declared content. In XML, there are no CDATA or RCDATA elements, and EMPTY elements are detectable without a DTD. The only remaining types of information for which a DTD is essential are: - content models (for validation) - attribute types and default values (for recognition of IDREFs etc. and for full provision of all attributes which apply to each element instance) - entity declarations (for resolution of entity references) - element vs. mixed content (for elimination of white space in element content) I can think of useful applications which require this information; I can also think of useful applications which don't. Removing the last item from the list doesn't add or shift any applications I can think of. >> * because that would be a cleaner design? > I dunno, none of the proposals we've considered leads to a "clean >design" other than making all whitespace significant (a policy that has >been vetoed for compatibility reasons). And brought up again for further discussion precisely because, as Charles explained, we may find a compatible way to do it - at least for mixed content. >>I agree that it would be desirable to make rules which allow >>processors to skip the DTD and still get a wholly correct parse >>tree, er, grove, but I don't think that's possible, since a >>processor cannot resolve entity references or provide correct >>default attribute values without reading the DTD. > >We already determined that resolving entity references is not >required: the XML data structure cannot completely hide entity >references from applications in that way that SGML parsers do. >Default attributes at least have fixed values, so they must offer >relatively little to an application that doesn't care about DTDs. I don't say these things are earth-shattering; I do think they constitute differences between the parse tree produced by processors which read the DTD and those which don't. The parse trees aren't the same and tweaking the rules for white-space handling in element content won't make them the same. So I don't see why we should depart from SGML by saying that white space should be preserved in element content. If that were the only difference between DTD-less parsing and DTD-aware parsing, we might have something to discuss, but it's not and we don't. >>I also believe that if XML makes no distinction between element >>content and mixed content, it will be impossible to build correct >>XML processors on top of SGML parsers. >We did determine, I think, that some differences would exist between >the SGML and XML "data streams" in the case of RE near markup. This >was deemed acceptable compared to the horrendously complex SGML >status quo. Not all REs near markup; only REs in pathological positions, and they won't appear or disappear, they'll only be moved from their actual position by SGML parsers, and left alone by XML processors. >> discussion of validation, production of correct ESIS, and >> simple parsing with almost-correct ESIS >This is far too complex: two kinds of parsing are enough. Which two do you choose? I suggested we distinguish the three concepts of validatiion, generation of correct ESIS, and minimal parsing because we seem to be using all three of these distinct concepts while pretending to ourselves that there are only two concepts involved. If one of these three is unnecessary, by all means tell us which it is so we can do without it. Michael Date: Mon, 2 Dec 1996 18:52:30 -0500 From: gtn@ebt.com (Gavin Nicol) To: U35395@UICVM.UIC.EDU CC: dgd@cs.bu.edu, w3c-sgml-wg@w3.org Subject: Re: SGML and XML Mr. McQueen writes >The only remaining types of information for which a DTD is essential are: > > - content models (for validation) Not necessary for a parser per se. The parser can do it's job without validating it, and validation of content models could be performed using the post-parse data structures (event stream/tree). > - attribute types and default values (for recognition of IDREFs > etc. and for full provision of all attributes which apply to each > element instance) Yes. I would be interested in hearing why people wish to necessarily have the entire attribute set appear on each element instead of just the attributes that had values specified (ie. I wonder if applications couldn't be built for the latter case that are effectively the same as the former). > - entity declarations (for resolution of entity references) Only really needed for the *type* of the entity, which, assuming object labelling in storage managers, may be available elsewhere. Resolution can begin with the name of the entity itself... > - element vs. mixed content (for elimination of white space in > element content) This is not a strictly *necessary* distinction. Date: Mon, 2 Dec 1996 19:02:59 -0400 To: "Christopher R. Maden" From: murray@spyglass.com (Murray Altheim) Subject: Re: MIME type for XML Cc: w3c-sgml-wg@w3.org "Christopher R. Maden" writes: >I think that application/xml is necessary, too. All text/* types, I >believe, are required to have normalized line endings (0x0d0a). This >works for 8-bit encodings, but for 16-bit encodings, 0x000d000a >becomes 0x000d0a000d0a, which isn't quite the same thing... This sounds like a conflict with the MIME specification and Unicode that needs to be resolved, rather than hacking a (albeit, working) solution. XML documents are by nature 'text', so possibly this is something that has been already brought up in MIME discussions. Anyone? I was going to ask Glenn Adams, but he *just* left for the night. I have to agree with Jon that text/x-xml seems logical (ie., the choice that people would guess). Murray ``````````````````````````````````````````````````````````````````````````````` Murray Altheim, Program Manager Spyglass, Inc., Cambridge, Massachusetts email: http: "Give a monkey the tools and he'll eventually build a typewriter." From: Ken Holman To: "'w3c-sgml'" Subject: Straw-man XML Support for PUBLIC Identifiers and XML Catalogs Date: Mon, 2 Dec 1996 19:06:33 -0500 --------- Section 4.2.2 "External Entities" in Working Draft 14-Nov-96 includes the following excerpts: [69] ExternalId := 'SYSTEM' S SystemLiteral [70] SystemLiteral := '"' [^"]* '"' | "'" [^']* "'" ... The Literal that follows the keyword SYSTEM is called the entity's system identifier. It is a URL,... --------- Straw man (please help, this doesn't look precise enough in lingua-XML): [69] ExternalId := 'PUBLIC' S ExternalLiteral ( S ExternalLiteral )? | 'SYSTEM' S ExternalLiteral [70] ExternalLiteral := '"' [^"]* '"' | "'" [^']* "'" ... The ExternalLiteral that follows the PUBLIC keyword is called an entity's public identifier. The ExternalLiteral that may follow the entity's public identifier and the ExternalLiteral that must follow the SYSTEM keyword are called an entity's system identifier. An XML processor may retrieve the content of the entity using the public identifier as an index into an associated XML catalog (see the definition of the catalog for details). If the XML processor cannot successfully use or resolve the public identifer, and a system identifier follows the public identifier, the XML processor shall behave as if the system identifier was the only identifier supplied. If the XML processor cannot successfully use or resolve the public identifier, and a system identifier does not follow the public identifier, the XML processor shall behave as if a system identifier could not be resolved. The system identifier is a URL,... 4.3.1 XML Catalog Specification When an XML processor encounters an external entity defined by a public identifier, the processor may, but need not, determine the associated system identifier required for the tasks described in section 4.3 of this specification from a system-sourced supplemental XML catalog entity defined as follows: [CAT1] XMLCatalog := S? ( ( catComment | catPublic | catEntry ) S )* ( catComment | catPublic | catEntry )? [CAT2] catComment := '--' [^-]* ('-' [^-]+)* '--' [CAT3] catPublic := 'PUBLIC' S catLiteral S catLiteral [CAT4] catEntry := catKeyword ( S catLiteral )* [CAT5] catLiteral := '"' [^"]* '"' | "'" [^']* "'" [CAT6] catKeyword := [^S"'] [^S]* In [CAT3] where the catKeyword is 'PUBLIC', exactly two catLiteral arguments are required: the first is the ExternalLiteral of an entity's or notation's public identifier and the second is information used to locate the entity or recognize the notation. An XML processor may choose to determine a system identifier from this second catLiteral value. Users of an XML processor that determines system identifiers >from the second catLiteral value of the catPublic production must ensure that the syntax of that value is compatible with the processor in use --------- A few things about this straw man ... (1) - it does not recognize the embryonic efforts that have started in SGML Open regarding NOTATION declarations, but I don't think that it can ... we still need to specify the use of ExternalId in XML[77] (2) - "... information used to locate ... or recognize ..." is far too vague, but I wanted to get in that this may just be human information for the reader and not always system information ... it could also be the full FSI specification ... it could just be "" followed by the URL (so that it could be more in the future) ... I've left the exact wording of whan an XML processor can do with the second catLiteral to someone who can wordsmith more quickly than me (3) - the bit about "users of XML processors ... must ensure ..." is there in order to reinforce the "client-side-resolution" and show people that it the catalog is not the resonsibility of the author of the information, but rather, the consumer of the information (though it behooves the author to be helpful here) (4) - it does recognize that the use of the PUBLIC identifier is optional by the author; that the author could also supply a SYSTEM identifier in the event that the consumer cannot resolve PUBLIC identifiers; and that an XML processor is not required to support PUBLIC identifiers (5) - we could define the productions as follows: [69] ExternalId := 'PUBLIC' S PublicId ( S SystemId )? | 'SYSTEM' S SystemId [70] ExternalLiteral := '"' [^"]* '"' | "'" [^']* "'" [70a]PublicId := ExternalLiteral [70b]SystemId := ExternalLiteral Please feel free to discard this all and start from scratch ... I just wanted to get something written down that will show that this need not be onerous. ...... Ken -- G. Ken Holman Tel: +1 613 596-CADE(2233) /\ /\ Chief Technology Officer Fax: +1 613 596-5934 \/ \/ Computer Microstar Software Ltd. WATS: 1 800 267-9975 /\ /\ Aided 3775 Richmond Road Mail: gkholman@microstar.com \/ \/ Document Nepean Ontario Info: cade@microstar.com /\ /\ Engineering CANADA K2H-5B7 Web: http://www.microstar.com \/ \/ -- 1605 Mardick Court, Box 266 Phone: +1 613 489-2987 Kars, Ontario CANADA K0A-2E0 E-mail: gkholman@CanadaMail.com -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6 mQCNAjHOimAAAAEEAK47HbDtZZB8hJBk+r9Zh7m7QxdFHTaz/m200QQ7J9XX4QYs h6hsuP6ZqBJXyLdmIVMEwR6Ry6oxjKMd6BRJ8OGScD89eIghgbrpMX+900NxM0x2 v3yWO9ki2gKRPrn4vlCXznnmcmsyke0G02T2LefXbgZHyVSqDSOLy8nwuN7dAAUR tClHLiBLZW4gSG9sbWFuIDxhYjk0NUBmcmVlbmV0LmNhcmxldG9uLmNhPg== =IN3T -----END PGP PUBLIC KEY BLOCK----- Date: Mon, 02 Dec 1996 18:28:25 -0600 From: Len Bullard To: "David G. Durand" CC: w3c-sgml-wg@w3.org Subject: Re: Fried Parrot Intestines in my XML Soup David G. Durand wrote: > FPIs are a social discipline for naming, independent of mechanisms. Your > "c" syntax is also an acceptable mechanism for FPIs. FPIS are _NOT_ about > mechanism. Correct. Another way to say it is, naming depends on policy for process and enforcement of process. That is, play by the rules, it works. Else, results are error or undefined. > >We have focussed in very closely on FPIs, but I don't think we've > >talked enough about what problems we are trying to solve. > > Persistent, globally-unique, resolution mechanism-independent identifiers > for internet resources, for use in XML documents. Hmm, and other sources. The local disk is where most action takes place. However, should the XML system know the difference? NOTE, as soon as we start talking system, we are talking mechanism, and resolution mechanisms are always, system. A good aspect of the HyTime work on locators was in classifying the types of resolvers by the information they need. > It shows that some people don't understand the difference between names and > locations, nothing more. Read the URN group logs. There is no solution to > this argument, either you agree to let the "namers" have their "harmless > games" or they lose capabilities that they depend on and you don't believe > in. I'm sorry, but there's not much more to it, once we have agreed to > forswear philosophy. Quite so. Besides, those "harmless games" are the way they organize indexes. Reference designators are spatially organized "names" (morphologies), but if you take them away, the airplane falls out of the sky because they are used to locate parts/components, and the writers are trained to apply them. So game or no, everyone plays. > hierarchical authority-based name assignment (like FPIs) is the easiest > name-assignment discipline to implement. And best matches the way humans work. Remember them? > Why not FPIs, they already exist, and are compatible, etc.?? > > -- David, who cannot believe he is having this same discussion again. It is had every time someone sits down to organize or design a hypermedia language. It is fundamental. (I think I learned that from a book with your name on it, David.) The reason is, James Clark says it is too hard to implement. Given this discussion, does he still hold to that opinion as technical lead of the ERB which must draft language and vote on this? We need to get to independent links and locator types. The work I do in technical manuals suggests to me we need these badly and will implement them anyway in some form because they make it much simpler to express and maintain relationships external to the document about the use of a multimedia aggregate. We have to talk about HyTime: The Concepts. Regardless of the IETF drafts, that is the ISO standard for linking and locating. I don't say we have to accept it as is, but at this juncture we have to talk about it. len bullard Date: Mon, 02 Dec 1996 18:35:40 -0600 From: Len Bullard To: Ken Holman CC: "'w3c-sgml'" Subject: Re: Client-side-resolved Indirection Ken Holman wrote > > In another message I'll prepare a straw-man specification of changes to > XML for PUBLIC identifiers. > > Responses to this thread can continue on the merits and drawbacks of the > concept. We use a similar mechanism in our domain files (search here when all else fails). It also makes the use and implementation of link validator/verifiers easier. Link validation is one of the gaping holes in Internet hypermedia which local intranet systems can't live with. (No, I don't choose to always "serve" SGML to the client; the DB solution is not applicable.) So, from the production perspective of working SGML hypermedia systems, the strawman is welcome. len bullard Date: Mon, 2 Dec 1996 16:34:27 -0800 (PST) From: Terry Allen To: crm@ebt.com, murray@spyglass.com Subject: Re: MIME type for XML Cc: w3c-sgml-wg@w3.org RFC 1874 on SGML Media Types defines both text and application for SGML, with some language that may or may not be relevant; the main idea appears to be to provide fallback to text/plain >from text/sgml. ftp://ds.internic.net/rfc/rfc1874.txt Regards, Terry Allen Fujitsu Software Corp. tallen@fsc.fujitsu.com "In going on with these experiments, how many pretty systems do we build, which we soon find outselves obliged to destroy?" - Benjamin Franklin A Davenport Group Sponsor: http://www.ora.com/davenport/index.html >From w3c-sgml-wg-request@w3.org Mon Dec 2 16:10 PST 1996 Date: Mon, 2 Dec 1996 19:02:59 -0400 To: "Christopher R. Maden" From: murray@spyglass.com (Murray Altheim) Subject: Re: MIME type for XML Cc: w3c-sgml-wg@w3.org "Christopher R. Maden" writes: >I think that application/xml is necessary, too. All text/* types, I >believe, are required to have normalized line endings (0x0d0a). This >works for 8-bit encodings, but for 16-bit encodings, 0x000d000a >becomes 0x000d0a000d0a, which isn't quite the same thing... This sounds like a conflict with the MIME specification and Unicode that needs to be resolved, rather than hacking a (albeit, working) solution. XML documents are by nature 'text', so possibly this is something that has been already brought up in MIME discussions. Anyone? I was going to ask Glenn Adams, but he *just* left for the night. I have to agree with Jon that text/x-xml seems logical (ie., the choice that people would guess). Murray ``````````````````````````````````````````````````````````````````````````````` Murray Altheim, Program Manager Spyglass, Inc., Cambridge, Massachusetts email: http: "Give a monkey the tools and he'll eventually build a typewriter." Date: Mon, 02 Dec 1996 19:49:19 -0500 To: "'w3c-sgml'" From: Paul Prescod Subject: Straw-man XML Support for PUBLIC Identifiers and XML Catalogs I think that your proposal is a good balance of the needs of implementors and users. Just a few of notes. * as long as we are going to discuss SGML catalogs (and I'm still not sure if we should), we should discuss how to transmit them for those cases where authors have provided catalogs as a convenient "default mapping." The SGML Open spec has a little to say about that. * instead of saying: "if an XML processor cannot..." why not say "an XML processor can use the PUBLIC..., for instance to redirect to a local copy of the information." * the "real" SGML Open spec is quite clear that other resolution mechanisms (i.e. URN resolvers) may be used before and after the catalog. We must also make that clear. * it isn't clear what the CatKeyword production is for without looking at SGML Open's docs. Since the catalog mechanism is totally optional, and overridable, why don't we move it to an Appendix? I, personally, would be 100% happy with that solution. The "right" catalog mechanism to use for most applications would be obvious (avoiding the pre-SGML Open Catalog situation in the SGML world), but people with legitimate other needs would not be constrained to it. Paul Prescod Date: Mon, 2 Dec 1996 20:50:02 -0400 To: Terry Allen From: murray@spyglass.com (Murray Altheim) Subject: Re: MIME type for XML Cc: crm@ebt.com, w3c-sgml-wg@w3.org Terry Allen writes: >RFC 1874 on SGML Media Types defines both text and application >for SGML, with some language that may or may not be relevant; >the main idea appears to be to provide fallback to text/plain >from text/sgml. > >ftp://ds.internic.net/rfc/rfc1874.txt Got it -- thanks Terry. The fact that the RFC states preference for US-ASCII was kinda my point. Since the default charset for XML is not US-ASCII, I don't think the assumption in RFC 1874 is valid or useful in XML. And yes, I think the RFC should be changed, not XML. From the XML 1.0 spec: This specification depends on the international standard ISO/IEC 10646 and the technically identical Unicode Standard, Version 2.0, which define the encodings and meanings of the characters which make up XML text data. Relevant quotes from RFC 1874 follow. Section 2.1 describes text/sgml as being employed when software is meant to be human-readable: 2.1. Text/SGML MIME type name: Text MIME subtype name: SGML Required parameters: none Optional parameters: charset, SGML-bctf, SGML-boot Encoding considerations: may be encoded Security considerations: see section 4 below Published specification: ISO 8879:1986 Person and email address to contact for further information: E. Levinson The Text/SGML media-type can be employed when the contents of the SGML entity is intended to be read by a human and is in a readily comprehensible form. That is the content can be easily discerned by someone without SGML display software. Each record in the SGML entity, delimited by record start (RS) and record end (RE) codes, must correspond to a line in the Text/SGML body part. SGML entities that do not meet the above requirements should use the Application/SGML media-type. A document in UCS-4 Arabic is certainly intended to be read by a human. The problem doesn't seem to be use of RS and RE per se, it's their transformation into multibyte Unicode equivalents. [...describing the 'charset' parameter...] charset The charset parameter for Text/SGML is defined in [RFC-1521], the valid values and their meaning are registered by the Internet Assigned Numbers Authority (IANA) [RFC-1590]. The default charset value for all Text content-types is "us-ascii" [RFC-1521]. The charset parameter is provided to permit non- SGML capable systems to provide reasonable behavior when Text/SGML defaults to Text/Plain. SGML capable systems will use the SGML-bctf param- eter. What needs changing is the definition of MIME 'text/*' from 646 to 10646, not a UCS-4 document instance into an 'application/*' MIME type. Otherwise, MIME is inextricably bound to US-ASCII, which seems a mistake. I'm sure someone more qualified than I has argued this out in the MIME/SGML WGs. XML may simply be among the first applications requiring this type of i18n modification to what are gradually becoming outdated specs. Murray ``````````````````````````````````````````````````````````````````````````````` Murray Altheim, Program Manager Spyglass, Inc., Cambridge, Massachusetts email: http: "Give a monkey the tools and he'll eventually build a typewriter." Date: Mon, 2 Dec 1996 21:41:59 -0500 From: gtn@ebt.com (Gavin Nicol) To: murray@spyglass.com CC: crm@ebt.com, w3c-sgml-wg@w3.org Subject: Re: MIME type for XML >This sounds like a conflict with the MIME specification and Unicode that >needs to be resolved, rather than hacking a (albeit, working) solution. XML >documents are by nature 'text', so possibly this is something that has been >already brought up in MIME discussions. Anyone? I was going to ask Glenn >Adams, but he *just* left for the night. This has been discussed to death by myself, and others. The MIME folk are ware of the problem, but there seems to be no resolution pending that I am aware of. Date: Tue, 03 Dec 1996 09:41:02 +0000 To: Ken Holman , "'w3c-sgml'" From: Martin Bryan Subject: Re: Straw-man XML Support for PUBLIC Identifiers and XML Catalogs Ken >The system identifier is a URL,... > >4.3.1 XML Catalog Specification You might want an extra paragraph between these two (or just before them) that reads. "The public identifier could be a formal public identifier that conforms to the rules of ISO/IEC 9070:1987." ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ Date: Tue, 3 Dec 1996 11:02:48 -0500 To: Michael Sperberg-McQueen , W3C SGML Working Group From: dgd@cs.bu.edu (David G. Durand) Subject: Re: SGML and XML At 3:16 PM 12/2/96, Michael Sperberg-McQueen wrote: >On Mon, 2 Dec 1996 16:02:35 -0500 David G. Durand said: >>First, we have a compriomise on this issue that is far from perfect, >>but took us at least a month of intense debate to reach: why are we >>re-opening it? Let's leave not-so-well-enough alone! I'm wistful for >>a better (different) solution, but given that everyone (including me) >>seems to find the compromise livable, shouldn't we leave it in place? > >I think some people are interested in re-opening it because Charles >seems to have found a way to get RE Delenda Est behavior out of SGML >processors. At least, it's the behavior I thought was intended for >REDE: record ends are always passed through in mixed content, and >all white space is eaten in element content. Since many people gave >up on REDE solely for compatibility reasons, Charles's new proposal >seems to open the possibility of getting something materially better >than the compromise we have now. Well, that's not what I intended. I have always advocated a first-line approach that says that _all_ characters not in a tag or a comment are significant, and part of the parse tree. This is a _simple rule_. Anything else is a complicated rule, although I agree that the behavior Charles is claiming would provide a much-less complex rule. But I don't like the fact that we would have different answers about the contents of the document depending on the processor. I suppose it's no worse than the current mess. OK. You've convinced me. Now can someone check to see if the standard is unabiguous enought about shortrefs and whitespace handling that this hack will actually work? At least sgmls, nsgmls, and some of the commercial products need to be checked. >>This is far too complex: two kinds of parsing are enough. > >Which two do you choose? > >I suggested we distinguish the three concepts of validatiion, >generation of correct ESIS, and minimal parsing because we seem to >be using all three of these distinct concepts while pretending to >ourselves that there are only two concepts involved. If one of >these three is unnecessary, by all means tell us which it is so we >can do without it. We should still have two kinds; I think three is too many. We should simply abandon the notion of ESIS-equivalent parsing. We have minimal parsing (different ESIS), and validating. If people find that they want only some of the functionality of a validating parser (e.g. whitespace handling only) they will implement it into their minimal parser, but we need not have a formal compliance level for that. We can delcare that implementing some validation features in a minimal parser is _not_ an error in conformance. This is very non-ISO, but quite IETF in intent, I think. -- David Can we start interpreting "RE Delenda est" as a statement, rather than en exhortation? I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ Date: Tue, 3 Dec 1996 11:21:07 -0500 To: w3c-sgml-wg@w3.org From: dgd@cs.bu.edu (David G. Durand) Subject: Re: Straw-man XML Support for PUBLIC Identifiers and XML Catalogs At 4:41 AM 12/3/96, Martin Bryan wrote: >You might want an extra paragraph between these two (or just before them) >that reads. > >"The public identifier could be a formal public identifier that conforms to >the rules of ISO/IEC 9070:1987." This is a very good point. One of my pet-peeves by the way is that transformation from SGML PI syntax to 9070 format does not commute with trnasformation in the opposite direction. So that you can follow the rules in 9070 and generate a different 9070 indentifier by converting to SGML form, and then back out. If I remember correctly, there are also 8879 SGML identifiers that will transform in 9070 identifiers that will transform back into NONSGML 8879 identifiers. Perhaps Charles can dig my comment to the ISO editor out of the files and post it here for the curious. By the way, the 9070 syntax is far superior, omitting as it does those strange SGML syntax qualifiers that make SGML-form FPIs so ugly. -- David _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ Date: Tue, 3 Dec 1996 08:41:19 -0800 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org CC: bosak@atlantic-83.eng.sun.com Subject: Re: Straw-man XML Support for PUBLIC Identifiers and XML Catalogs [Martin Bryan:] | >The system identifier is a URL,... | > | >4.3.1 XML Catalog Specification | | You might want an extra paragraph between these two (or just before them) | that reads. | | "The public identifier could be a formal public identifier that conforms to | the rules of ISO/IEC 9070:1987." I'm not sure that you want 9070 to be the normative text here. We looked into this recently and came to the conclusion that we would rather stick with the 8879 description of FPIs because it allows the (soon-to-be-standard) three-letter language codes, while 9070 (at least on last reading) appears to prohibit them. Jon Date: Tue, 3 Dec 1996 09:06:52 -0800 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org CC: bosak@atlantic-83.eng.sun.com, rdaniel@lanl.gov Subject: FPIs to URNs Perhaps this is the place to introduce some information I've been sitting on for the past couple of weeks. As I've mentioned, at Sun we've put together a system for using FPIs >from the bottom up as a way to get ad hoc location-independent document naming on company intranets using SGML Open catalogs with the proposed DELEGATE extension (which we call "socats" for short). We like the system a lot and think that others will, too. I've also promised you a description of that system, which it now appears I won't be able to get to until this weekend. I recently had an interesting conversation with Ron Daniel about our FPI system and how it could tie in with NAPTR. He seemed immediately to grasp the usefulness of a bottom-up FPI scheme, and suggested that we view NAPTR as the fallback mechanism for FPI-based links that fail within our system (none of Sun's will fail if the user is connected to the Internet, because in our system, all unresolved references to our own documents will simply fall through to our own Web site, but it is certainly conceivable that links put in by others might fail). I took an immediate liking to this proposal in its abstract form, but since I didn't have a clue about NAPTR mechanics, when Ron got into the details I just blanked. Ron followed up with the message below, which I quote with permission. I confess that I'm still not understanding all the details, but it's clear that Ron is suggesting that some kind of FPI resolving authority be built into the NAPTR framework. I would have preferred to hold onto this until I could put a description of the system that Ron is referring to in front of you, but he needs feedback in the short term, and I suspect that there are people on this list who understand his NAPTR descriptions perfectly. I will forward all comments on this to Ron. Jon ======================================================================== Date: Fri, 22 Nov 1996 13:42:04 -0700 From: Ron Daniel Subject: FPIs and URNs - resolution possibility As you know, FPIs are one of the namespaces I would like to be able to handle as URNs. Here are some notes on how we might be able to do that resolution through a combination of SOCATs (SGML Open Catalogs), the NAPTR proposal, and HTTP. First, a note on syntax. When FPI-aware software is using FPIs, they should have no special encoding and will be strings like: -//Sun Microsystems::SunSoft//DOCUMENT Solaris 2.5 User Manual//EN (without the line break, of course) When FPI-aware software is going to emit an FPI to software that is URN-aware, but not necessarily FPI-aware, the FPI should be encoded by: 1) Prepending "fpi:" 2) %encoding any special characters that are illegal in URNs, such as space. 3) The URN syntax draft is currently being reviewed and may make additional requirements, but this is close enough for now. Thus, our example FPI would yield a URN like: fpi:-//Sun%20Microsystems::SunSoft//DOCUMENT%20Solaris%20 2.5%20User%20Manual//EN (again, without the line break) This note addresses three cases. The first case is the ideal world where we have a client that knows about URNs and SGML Open Catalogs. The next two cases then look at how the infrastructure for the ideal world can be used to offer backward compatability to clients that know SOCAT but not URNs and vice versa. Case 1: SOCAT-savvy and URN-savvy client ======================================== Assume we have a client attempting to resolve the FPI: -//Sun Microsystems::SunSoft//DOCUMENT Solaris 2.5 User Manual//EN It consults the local catalog, but finds no appropriate PUBLIC or DELEGATE entries. Rather than throwing an error, it tries to use NAPTR-style URN resolution as a fall-back mechanism. It queries fpi.urn.net for NAPTR records and gets the response: fpi.urn.net IN NAPTR 50 50 "A" "http+N2C" "" bigcat.sgmlopen.org The current drafts for URNs would have the client send a GET request with the full URN, and a little bit of stuff at the beginning to say that we are asking for the N2C operation. So, the client does just that, but uses the Accept: header to ask for a SOCAT. The request might look like: Accept: application/socat GET /uri-res/N2C/fpi:-//Sun%20Microsystems::SunSoft//DOCUMENT %20Solaris%202.5%20User%20Manual//EN HTTP/1.0 (without the line break) The HTTP daemon at bigcat.sgmlopen.org fires up the N2C CGI script. The script looks at the Accept: header, and realizing it is speaking to a SOCAT-savvy client it starts to build a custom SOCAT on-the-fly by trolling through its huge catalog and only returning entries that match. This response might look like: DELEGATE -//Sun Microsystems::SunSoft// http://www.sunsoft.com/sunsoft.socat DELEGATE -//Sun Microsystems http://www.sun.com/suncat.socat (Not sure of the exact SOCAT format, but the idea should be clear). This new mini-catalog is then processed by the client. It fetches the SunSoft catalog and procedes as usual. Actually making this into a reality would require commitment >from some SGML-affiliated organization, such as SGML Open or GCA, to try and maintain a big catalog of registered and unregistered FPIs, and agree to serve that up for the advancement of the community. I think doing such a thing on an experimental basis might not be too hard, but committing to doing so on an ongoing basis is another story. Anyone want to comment on the liklihood of this scenario? Case 2: SOCAT-savvy, URN-unaware client ======================================= If we have a URN-unaware client, we could modify its catalog to say something like DELEGATE -// http://bigcat.sgmlopen.org/fullcatalog.socat This would not require any modifications to the SOCAT-aware client, but the size of the response is likely to be staggering. It might be desirable to have a SOCAT version 2 proposal that allows the magic characters "%!uri" in the URL of a DELEGATE entry to mean "substitute the full FPI here". The SOCAT-2 client would then have a catalog entry of DELEGATE -// http://bigcat.sgmlopen.org/uri-res/N2C/%!uri which, as long as the client can specify application/socat in the Accept: header, would result in the mini-catalog as in Case 1. (In fact, this SOCAT-2 hack might be used in a new version of Case 1 so that when the client asks for the catalog from SunSoft, they could actually ask it for a tailored one. But I digress). Case 3: SOCAT-unaware, URN-savvy client ======================================= This client somehow comes across a FPI URN, perhaps as the HREF in some HTML: Fun Stuff at Sun! Not having a special content handler for the fpi namespace, it goes to fpi.urn.net and gets the NAPTR record from Case 1. Since the client does not know about the application/socat media type, its Accept: header asks for things like text/plain, text/html, etc. (Or no Accept header is sent). Other than that it sends the same HTTP request as in Case 1. The script at SGMLOpen looks at the Accept: header, does the machine equivalent of a sigh, and sends back an HTML response that says: "You should obtain our fine SOCAT Content Handler, but in the meantime, you are about to be automatically redirected to a site that may be able to tell you more about the resource you have requested. If it fails, you must obtain our fine SOCAT handler to locate that resource". The user is then redirected to http://www.sunsoft.com/uri-res/N2C/fpi:-//etc.etc.etc. This is likely to fail. Alternatively, the script could attempt to assist the SOCAT- unaware software by doing the SOCAT work itself. It would fetch the catalog from SunSoft, and follow any PUBLIC and DELEGATE entries until it obtains the answer. It can then send the answer back to the user. This imposes a noticible burden on the SGML Open server, but it has the benefit of automating part of the process of maintaining the big catalog. Comments? Ron Daniel Jr. email: rdaniel@lanl.gov Advanced Computing Lab voice: +1 505 665 0597 MS B287 fax: +1 505 665 4939 Los Alamos National Laboratory http://www.acl.lanl.gov/~rdaniel/ Los Alamos, NM, USA 87545 obscure_term: "hyponym" Date: Tue, 3 Dec 1996 10:13:20 -0800 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org cc: bosak@atlantic-83.eng.sun.com, rdaniel@lanl.gov Subject: Re: FPIs to URNs Reading Ron's message of 11/22 once again, I suddenly realized that one of the major reasons I was having trouble visualizing the scenarios that he describes is that what he calls a "client" is, in our system, a specialized HTTP server process. As you will see in greater detail when I find time to describe it, the Solaris 2.6 AnswerBook2 document system that replaces our traditional AnswerBooks is based on a mutant Web server (a customized version of DynaWeb) implemented as an optionally installed Solaris daemon whose only function is to serve out SGML-based documents to Web clients. It's this daemon that manages all the FPI resolution, not the Web browsers; all they ever see is weird-looking URLs. So when Ron says, for example, that a client consults the local catalog, read "AB2 server" for "client". A consequence of this is that in our FPI scheme, *no* logic is added to the client application itself. (It couldn't be; the whole system is built around the capabilities of current HTML browsers.) The same would hold true if our server were talking to some future XML client. In our system, name resolution is always a server problem, not a client problem. A further consequence (speaking again only for Sun's contemplated use of FPIs) is that, for our purposes, FPIs need not be included in XML at all if XML is used purely for document delivery, just as FPIs are not now included in HTML. They only need to be allowed in XML if XML is going to live in its native state on the server, and then they only need to have the same status that they do now in SGML -- dumb strings with a known syntax. Of course, other people may have completely different ideas of what an XML client should be doing with an FPI, and I'm not denying that it might be very useful for all of us to nail down expected behavior with regard to SGML Open catalogs. I'm only saying that for my own company's purposes, it is not necessary to specify or require any FPI resolution capabilities for XML, it is only necessary to allow FPIs as syntactic objects. Jon Date: Tue, 3 Dec 1996 10:40:42 -0800 (PST) From: "Deborah A. Lapeyre" Subject: FPIs To: "David G. Durand" cc: w3c-sgml-wg@w3.org This is a plea for: Persistent, globally-unique, resolution mechanism-independent identifiers for internet resources, for use in XML documents. (Thank you David) --Debbie ======================================================================= Deborah A. Lapeyre Phone: 301-231-6933 Mulberry Technologies, Inc. Fax: 301-231-6935 6010 Executive Blvd. Suite 608 E-mail: dlapeyre@mulberrytech.com Rockville, MD USA 20852 ======================================================================= From: lee@sq.com Date: Tue, 3 Dec 96 20:34:54 EST To: murray@spyglass.com, tallen@fsc.fujitsu.com Subject: Re: MIME type for XML Cc: crm@ebt.com, w3c-sgml-wg@w3.org > What needs changing is the definition of MIME 'text/*' from 646 to 10646, Note that MIME has to be backwards compatible with RFC822 by default, and with existing mail and netnews gateways -- it isn't only used for HTTP, of course... Other applications get round this by using a MIME Charset: header, I think. But I'm not sure how to get an existing or nearly-existing httpd server to add Charset lines wiothout hacking it, unless you put the MIME headers in the file and use .asis processing (where supported). Lee From: lee@sq.com Date: Tue, 3 Dec 96 21:07:00 EST To: bosak@atlantic-83.eng.sun.com, w3c-sgml-wg@w3.org Subject: Re: FPIs to URNs Cc: rdaniel@lanl.gov Thanks for posting that, Jon -- It's very informative. A couple of questions... > The HTTP daemon at bigcat.sgmlopen.org fires up the N2C > CGI script. The script looks at the Accept: header, and > realizing it is speaking to a SOCAT-savvy client it starts to > build a custom SOCAT on-the-fly by trolling through its huge > catalog and only returning entries that match. If matching entries are based only on ownr identifier, N2C would need to have a list of every ISBN used in an FPI, with corresponding URLs, and also every registered owner. This isn't the same as every ISBN ever issued, if you don't mind it failing on obscure ones that the owner had not registered with the orgnisation running N2C. Right? > This response might look like: > > Content-type: application/socat > DELEGATE -//Sun Microsystems::SunSoft// > http://www.sunsoft.com/sunsoft.socat > DELEGATE -//Sun Microsystems > http://www.sun.com/suncat.socat [...] > This new mini-catalog is then processed by the client. It > fetches the SunSoft catalog and procedes as usual. So how big would a CATALOG be of every SGML or XML public identifier ever issued at Sun? Isn't this likely to be more than a kilobyte? I see problems with performance here. > Actually making this into a reality would require commitment > from some SGML-affiliated organization, such as SGML > Open or GCA, to try and maintain a big catalog of registered > and unregistered FPIs, and agree to serve that up for the > advancement of the community. I think doing such a thing on > an experimental basis might not be too hard, but committing to > doing so on an ongoing basis is another story. Anyone want to > comment on the liklihood of this scenario? Zero. :-) I wonder how many FPIs have been used so far? A million? Ten million? That's a major management problem. It needs to be a more hierarchical namespace, so that noone needs to manage the whole list. Or, better, charge $5,000 to register an FPI, and store only the new ones. > Case 2: SOCAT-savvy, URN-unaware client > ======================================= > [...] > It might be desirable to have a SOCAT version 2 proposal that > allows the magic characters "%!uri" in the URL of a > DELEGATE entry to mean "substitute the full FPI here". The > SOCAT-2 client would then have a catalog entry of > DELEGATE -// > http://bigcat.sgmlopen.org/uri-res/N2C/%!uri If I am not mistaken (I might well be) I think that CATALOG is only processed once -- the DELEGATE entry (Panorama actually uses the keyword CATALOG for that) isn't read multiple times, so this wouldn't work. These schemes are interesting, and I think if the difficulty of a previously flat namespace for informal public identifiers is surmounted they can probably work. I also think that it would be well worth reopening the CATALOG spec (sorry, Paul!) to include such information in it, once it is shown to be workable. Lee Date: Tue, 3 Dec 1996 22:07:57 -0500 From: gtn@ebt.com (Gavin Nicol) To: lee@sq.com CC: murray@spyglass.com, tallen@fsc.fujitsu.com, crm@ebt.com, w3c-sgml-wg@w3.org Subject: Re: MIME type for XML Many current servers allow you to add the charset label on the HTTP Content-Type field. Apache and Netscape certainly do. Date: Wed, 4 Dec 96 10:26:17 CST From: paul@arbortext.com (Paul Grosso) To: w3c-sgml-wg@w3.org Subject: Re: FPIs to URNs Cc: rdaniel@lanl.gov > From: bosak@atlantic-83.Eng.Sun.COM (Jon Bosak) > . . . > > Ron followed up with the message below, which I quote with permission. > . . . > > ======================================================================== > > Date: Fri, 22 Nov 1996 13:42:04 -0700 > From: Ron Daniel > Subject: FPIs and URNs - resolution possibility > > . . . > > Case 1: SOCAT-savvy and URN-savvy client > ======================================== > Assume we have a client attempting to resolve the FPI: > -//Sun Microsystems::SunSoft//DOCUMENT Solaris 2.5 User > Manual//EN > It consults the local catalog, but finds no appropriate PUBLIC > or DELEGATE entries. Rather than throwing an error, it tries to > use NAPTR-style URN resolution as a fall-back mechanism. It queries > fpi.urn.net for NAPTR records and gets the response: > > fpi.urn.net IN NAPTR 50 50 "A" "http+N2C" "" bigcat.sgmlopen.org > > The current drafts for URNs would have the client send a GET > request with the full URN, and a little bit of stuff at the > beginning to say that we are asking for the N2C operation. So, > the client does just that, but uses the Accept: header to ask for > a SOCAT. The request might look like: > > Accept: application/socat > GET /uri-res/N2C/fpi:-//Sun%20Microsystems::SunSoft//DOCUMENT > %20Solaris%202.5%20User%20Manual//EN HTTP/1.0 > (without the line break) The IANA-registered MIME media type for SGML Open catalogs is: Application/SGML-Open-Catalog as opposed to the "application/socat" shown above. > > The HTTP daemon at bigcat.sgmlopen.org fires up the N2C > CGI script. The script looks at the Accept: header, and > realizing it is speaking to a SOCAT-savvy client it starts to > build a custom SOCAT on-the-fly by trolling through its huge > catalog and only returning entries that match. This response > might look like: > > Content-type: application/socat > DELEGATE -//Sun Microsystems::SunSoft// > http://www.sunsoft.com/sunsoft.socat > DELEGATE -//Sun Microsystems > http://www.sun.com/suncat.socat > (Not sure of the exact SOCAT format, but the idea should > be clear). (Pretty close, though each of the two arguments to the DELEGATE entry should be quoted.) > This new mini-catalog is then processed by the client. It > fetches the SunSoft catalog and procedes as usual. > > Actually making this into a reality would require commitment > from some SGML-affiliated organization, such as SGML > Open or GCA, to try and maintain a big catalog of registered > and unregistered FPIs, and agree to serve that up for the > advancement of the community. I think doing such a thing on > an experimental basis might not be too hard, but committing to > doing so on an ongoing basis is another story. Anyone want to > comment on the liklihood of this scenario? While it has not been discussed and is therefore not out of the question, I suspect SGML Open would have resource problems and perhaps jurisdiction problems doing this. Since the GCA is in line to be endorsed as the Owner Identifier Registrar, they may be a more obvious choice. However, I question why the design requires one "big catalog". With the use of DELEGATE and CATALOG entry types (both of which are *not* in the current TR9401:1995, though both of which are extensions currently supported by SP and both on the list of things currently being considered for the next amendment to TR9401), a hierarchy of smaller potentially distributed (and possibly mirrored) catalogs would seem feasible, more effective, and more manageable. Given that "distributed processing and distributed resources" is one of the key models of the internet, it seems only reasonable to adopt this model for FPI resolution too. (I do admit to lack of experience and expertise in the area of web-wide FPI resolution, so I welcome statements of proof that, in fact, large monolithic catalogs are required and/or desirable.) > > Case 2: SOCAT-savvy, URN-unaware client > ======================================= > > If we have a URN-unaware client, we could modify its catalog > to say something like > DELEGATE -// > http://bigcat.sgmlopen.org/fullcatalog.socat > This would not require any modifications to the SOCAT-aware > client, but the size of the response is likely to be staggering. > It might be desirable to have a SOCAT version 2 proposal that > allows the magic characters "%!uri" in the URL of a > DELEGATE entry to mean "substitute the full FPI here". The > SOCAT-2 client would then have a catalog entry of > DELEGATE -// > http://bigcat.sgmlopen.org/uri-res/N2C/%!uri > which, as long as the client can specify application/socat in the > Accept: header, would result in the mini-catalog as in Case 1. > (In fact, this SOCAT-2 hack might be used in a new version of > Case 1 so that when the client asks for the catalog from SunSoft, > they could actually ask it for a tailored one. But I digress). I'm not entirely sure I understand the %!uri bit. While I can certainly appreciate the power of grep and regexp like things happening in catalogs, as the person most responsible for trying to get all SGML Open members to support our Resolutions, I'm also quite nervous about adding complexity to things. If we can avoid these sorts of complexity, it will be much more likely that we can get a timely amendment to TR9401 passed and that there will be a greater level of support of it in the SGML vendor community. In short, though it's certainly not my call alone, for me to be able to support %!uri in SO catalogs, I'd need to be convinced its need is crucial and undeniable and that its implementation will be supported by a majority of vendors. If we can live without it, I'd probably want to live without it. > > Case 3: SOCAT-unaware, URN-savvy client > ======================================= > > . . . > Ron Daniel Jr. email: rdaniel@lanl.gov > Advanced Computing Lab voice: +1 505 665 0597 > MS B287 fax: +1 505 665 4939 > Los Alamos National Laboratory http://www.acl.lanl.gov/~rdaniel/ > Los Alamos, NM, USA 87545 obscure_term: "hyponym" Date: Wed, 04 Dec 1996 10:07:54 -0700 To: lee@sq.com, bosak@atlantic-83.eng.sun.com, w3c-sgml-wg@w3.org From: Ron Daniel Subject: Re: FPIs to URNs Cc: rdaniel@lanl.gov Thus spoke lee@sq.com (at least at 09:07 PM 12/3/96 EST) Ron Daniel(forwarded by Jon Bosak) said: >> the N2C >> CGI script [...] starts to >> build a custom SOCAT on-the-fly by trolling through its huge >> catalog and only returning entries that match. > >If matching entries are based only on ownr identifier, N2C would >need to have a list of every ISBN used in an FPI, with corresponding >URLs, and also every registered owner. This isn't the same as every >ISBN ever issued, if you don't mind it failing on obscure ones that >the owner had not registered with the orgnisation running N2C. >Right? Well, that would be the ideal, but this fallback to the big catalog in the sky should be regarded as a "best effort", not a guarantee. For one thing, there will be a large set of resources identified by an FPI that do not have a URL. >> This response might look like: >> >> Content-type: application/socat >> DELEGATE -//Sun Microsystems::SunSoft// >> http://www.sunsoft.com/sunsoft.socat >> DELEGATE -//Sun Microsystems >> http://www.sun.com/suncat.socat >[...] >> This new mini-catalog is then processed by the client. It >> fetches the SunSoft catalog and procedes as usual. > >So how big would a CATALOG be of every SGML or XML public identifier >ever issued at Sun? Isn't this likely to be more than a kilobyte? >I see problems with performance here. Right. You don't want to do this sort of thing for on-the-fly rendering of pages. You want to use local catalogs for that. This sort of thing is only for when you have an FPI that can't be resolved by way of the local catalog, and you are willing to wait in the hope of getting the resource. >It needs to be a more hierarchical namespace, so that noone needs >to manage the whole list. That can be accomodated pretty easily. bigcat.sgmlopen.org (or wherever the "root" happens to be) could have a catalog with entries of the form: DELEGATE -//U.S. Navy//CALS http://www.navy.mil/foo/cals_catalog.socat DELEGATE -//Sun Microsystems http://www.sun.com/bar/suncat.socat DELEGATE -//DOE//OSTI http://www.osti.doe.gov/baz/ostifpis.socat The root does not have to know ALL FPIs, it would need to know of sites that make their catalogs available and the prefix that is common to the entries in that catalog. The US Navy might well want to split their CALS catalog into smaller pieces, so it could have entries of the form DELEGATE -//U.S. Navy//CALS//nuts-n-bolts http://www.navy.mil/stores/fasteners/nutscat.socat DELEGATE -//U.S. Navy//CALS//paints-n-brushes http://www.navy.mil/stores/preservatives/paints.socat etc. Eventually one would stop being delegated to other catalogs and would finally bottom out at a PUBLIC entry. Ron Daniel Jr. email: rdaniel@lanl.gov Advanced Computing Lab voice: +1 505 665 0597 MS B287 fax: +1 505 665 4939 Los Alamos National Laboratory http://www.acl.lanl.gov/~rdaniel/ Los Alamos, NM, USA 87545 obscure_term: "hyponym" Date: Wed, 4 Dec 1996 12:38:15 -0400 To: paul@arbortext.com (Paul Grosso) From: murray@spyglass.com (Murray Altheim) Subject: Re: FPIs to URNs Cc: w3c-sgml-wg@w3.org paul@arbortext.com (Paul Grosso) writes: > From: bosak@atlantic-83.Eng.Sun.COM (Jon Bosak) >> Actually making this into a reality would require commitment >> from some SGML-affiliated organization, such as SGML >> Open or GCA, to try and maintain a big catalog of registered >> and unregistered FPIs, and agree to serve that up for the >> advancement of the community. I think doing such a thing on >> an experimental basis might not be too hard, but committing to >> doing so on an ongoing basis is another story. Anyone want to >> comment on the liklihood of this scenario? > >While it has not been discussed and is therefore not out of the >question, I suspect SGML Open would have resource problems and >perhaps jurisdiction problems doing this. Since the GCA is in >line to be endorsed as the Owner Identifier Registrar, they may >be a more obvious choice. > >However, I question why the design requires one "big catalog". With >the use of DELEGATE and CATALOG entry types (both of which are *not* in >the current TR9401:1995, though both of which are extensions currently >supported by SP and both on the list of things currently being >considered for the next amendment to TR9401), a hierarchy of smaller >potentially distributed (and possibly mirrored) catalogs would seem >feasible, more effective, and more manageable. Given that "distributed >processing and distributed resources" is one of the key models of the >internet, it seems only reasonable to adopt this model for FPI >resolution too. (I do admit to lack of experience and expertise in the >area of web-wide FPI resolution, so I welcome statements of proof that, >in fact, large monolithic catalogs are required and/or desirable.) [...] This was my first reaction as well. I didn't know about the Owner Identifier Registry becoming available online. This being the case, it seems that rather than a large catalog hosted by GCA, organizations such as IBM, Sun, Microsoft, W3C, IETF, Davenport, etc. would be responsible for hosting that portion of the delegated catalog for their public files. The name resolution would check ownerID and delegate authority to the proper owner site. Smaller organizations, individuals, or those not wishing to set up their own NAPTR-style URN resolution would query a central registry (SGML Open or GCA), whose potential load would be quite smaller, given that many of the organizations who create public files would desire canonical ownership of their namespace. This sounds a great deal like DNS, except that it's on its head: you want to start at the top of the tree and work down only so far as necessary, rather than start at the bottom and work up. One would also have the added advantage of knowing that the downloaded file was genuine. If you download a DTD from Netscape, you know it's Netscape's DTD rather than someone's hack. Digital signatures could certainly play a part here as well. Another question that comes to mind is that this level of name resolution is certainly more complex than a simple local catalog resolution on string comparision of FPIs (I can hear Tim choking in the background). While we can certainly leave the name resolution mechanism unspecified, if we decide to include any of this, how far should the XML specification go? If we don't plan to deal with this in the specification, we probably don't need to be talking about it here, despite its obvious interest to many of us. I'm certainly not trying to shut down the discussion, just trying to understand which portions we can glean for our XML needs. Murray ``````````````````````````````````````````````````````````````````````````````` Murray Altheim, Program Manager Spyglass, Inc., Cambridge, Massachusetts email: http: "Give a monkey the tools and he'll eventually build a typewriter." Date: Wed, 4 Dec 1996 12:42:21 -0400 To: Ron Daniel From: murray@spyglass.com (Murray Altheim) Subject: Re: FPIs to URNs Cc: w3c-sgml-wg@w3.org Ron Daniel writes: [...] >That can be accomodated pretty easily. bigcat.sgmlopen.org (or wherever >the "root" happens to be) could have a catalog with entries of the form: >DELEGATE -//U.S. Navy//CALS http://www.navy.mil/foo/cals_catalog.socat >DELEGATE -//Sun Microsystems http://www.sun.com/bar/suncat.socat >DELEGATE -//DOE//OSTI http://www.osti.doe.gov/baz/ostifpis.socat > >The root does not have to know ALL FPIs, it would need to know >of sites that make their catalogs available and the prefix that is >common to the entries in that catalog. > >The US Navy might well want to split their CALS catalog into smaller >pieces, so it could have entries of the form >DELEGATE -//U.S. Navy//CALS//nuts-n-bolts > http://www.navy.mil/stores/fasteners/nutscat.socat >DELEGATE -//U.S. Navy//CALS//paints-n-brushes > http://www.navy.mil/stores/preservatives/paints.socat >etc. >Eventually one would stop being delegated to other catalogs and >would finally bottom out at a PUBLIC entry. Well, this message beat mine into the hopper by about a minute, so you've answered most of my questions. I'm a little curious about one thing: DELEGATE -//U.S. Navy//CALS http://www.navy.mil/foo/cals_catalog.socat Why isn't the second field quoted? Because whitespace is allowed in FPIs, this seems to be a necessity. Murray ``````````````````````````````````````````````````````````````````````````````` Murray Altheim, Program Manager Spyglass, Inc., Cambridge, Massachusetts email: http: "Give a monkey the tools and he'll eventually build a typewriter." Date: Wed, 04 Dec 1996 11:28:25 -0700 To: paul@arbortext.com (Paul Grosso), w3c-sgml-wg@w3.org From: Ron Daniel Subject: Re: FPIs to URNs Cc: rdaniel@lanl.gov Thus spoke Paul Grosso (at least at 10:26 AM 12/4/96 CST) > From: Ron Daniel >While it has not been discussed and is therefore not out of the >question, I suspect SGML Open would have resource problems and >perhaps jurisdiction problems doing this. Since the GCA is in >line to be endorsed as the Owner Identifier Registrar, they may >be a more obvious choice. One of the things I wanted to achieve was being able to deal with *unregistered* FPI owners. (This would be subject to caveats, of course, but there are a lot of unregistered owners out there that have a lot of good stuff it would be nice to be able to get to). I have purposefully avoided the details of the NAPTR URN resolution mechanism, but it would be easy to send requests for registered owners to GCA and requests for resolution of unregistered FPIs to someplace else. There are limits on the number of such directions, but triggering on the initial '+' vs. '-' to go to GCA vs. someplace else is well within the capabilities of NAPTR. >However, I question why the design requires one "big catalog". In truth, it does not. I just replied to Lee's message which mentioned the same point. >I welcome statements of proof that, >in fact, large monolithic catalogs are required and/or desirable. I'm not a big fan of monolithic solutions, but let me point out two things that make a big "root" catalog fairly nice. First, the number of network accesses and the amount of client-side processing is reduced. Second, maintence of a big catalog could be achieved to a certain degree by use of a web-crawler that was targetted towards finding catalogs and things identified by FPIs. (One of my co-workers would shoot me if he knew I suggested a robot was *ever* a possible solution to a problem, but I digress. :-) >> Case 2: SOCAT-savvy, URN-unaware client >> ======================================= [...] >> It might be desirable to have a SOCAT version 2 proposal that >> allows the magic characters "%!uri" in the URL of a >> DELEGATE entry to mean "substitute the full FPI here". >I'm not entirely sure I understand the %!uri bit. While I can >certainly appreciate the power of grep and regexp like things >happening in catalogs, as the person most responsible for trying >to get all SGML Open members to support our Resolutions, I'm also >quite nervous about adding complexity to things. I certainly understand about the complexity concern. Also, I should make it clear that resolving FPIs as URNs does *not* require such a change. That change was floated as a suggestion on how SOCATs could be modified to deal with the problem of huge catalogs. Someone told me that was a concern about the current DELEGATE proposal and cited the example of the Navy and their 7 million CALS FPIs to show why. Since what I am *really* after is being able to resolve FPIs when they are not reachable from the local catalog, and since the %!uri suggestion is not critical to that, I don't want to pursue it in this "FPIs to URNs" thread. If someone on the w3c-sgml-wg list does want more info on the suggestion, I'll be happy to take it up in another thread. Regards, Ron Daniel Jr. email: rdaniel@lanl.gov Advanced Computing Lab voice: +1 505 665 0597 MS B287 fax: +1 505 665 4939 Los Alamos National Laboratory http://www.acl.lanl.gov/~rdaniel/ Los Alamos, NM, USA 87545 obscure_term: "hyponym" 3245 146th Place S.E., Suite 270, Bellevue, WA 98007, USA Date: Wed, 04 Dec 1996 10:20:41 -0800 To: w3c-sgml-wg@w3.org From: Derek Denny-Brown Subject: Re: FPIs to URNs At 10:07 AM 12/4/96 -0700, you wrote: >Thus spoke lee@sq.com (at least at 09:07 PM 12/3/96 EST) >Ron Daniel(forwarded by Jon Bosak) said: >>> the N2C >>> CGI script [...] starts to >>> build a custom SOCAT on-the-fly by trolling through its huge >>> catalog and only returning entries that match. I would worry about any single, central catalog, such as is being designed. You are asking for all the name-space problems which are plaguing InterNIC etc. currently. How would GCA, etc. administer such a system? Where would the funding come from? If you desire XML to be used in place of HTML, expect a huge load on such a server. A central server such as is being talked about would quickly become a bottleneck. I do like the idea of some method of distributed FPI name lookup, as is being discussed, and think it would be an amazingly useful tool. I also believe it is worthy a discussion-group/mailing-list of its own, if that is what is people are looking for. An issue as complex as that should not be the concern of the XML-wg. We should, for now, just say that the application (or parser?) should provide some means to lookup names, and maybe provide some guidelines, base criteria. The ideas being tossed around are great. I would want them. I just don't think it is reasonable to demand them _right_now_. Behavior such as Panorama where the catalog is the responsibility of the document provider should suffice for XML-1.0. Anything more would just make the first applications even longer in the arriving. -derek "that which is not slightly distorted lacks sensible appeal: from which it follows that irregularity - that is to say, the unexpected, surprise, and astonishment, are an essential part and characteristic of beauty" - Charles Baudelaire Date: Wed, 04 Dec 1996 11:28:25 -0700 To: murray@spyglass.com (Murray Altheim) From: Ron Daniel Subject: Re: FPIs to URNs Cc: w3c-sgml-wg@w3.org Thus spoke Murray Altheim (at least at 12:42 PM 12/4/96 -0400) >I'm a little curious about one thing: > > DELEGATE -//U.S. Navy//CALS http://www.navy.mil/foo/cals_catalog.socat > >Why isn't the second field quoted? Because whitespace is allowed in FPIs, >this seems to be a necessity. You are correct. Paul Grosso has informed me that the true syntax for SGML Open Catalogs has both fields in quotes. Regards, Ron Daniel Jr. email: rdaniel@lanl.gov Advanced Computing Lab voice: +1 505 665 0597 MS B287 fax: +1 505 665 4939 Los Alamos National Laboratory http://www.acl.lanl.gov/~rdaniel/ Los Alamos, NM, USA 87545 obscure_term: "hyponym" Date: Wed, 4 Dec 1996 12:19:11 -0800 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org Subject: [rdaniel@acl.lanl.gov: Re: FPIs to URNs] Date: Wed, 04 Dec 1996 10:07:54 -0700 To: lee@sq.com, bosak@atlantic-83.eng.sun.com, w3c-sgml-wg@w3.org From: Ron Daniel Subject: Re: FPIs to URNs Cc: rdaniel@lanl.gov Thus spoke lee@sq.com (at least at 09:07 PM 12/3/96 EST) Ron Daniel(forwarded by Jon Bosak) said: >> the N2C >> CGI script [...] starts to >> build a custom SOCAT on-the-fly by trolling through its huge >> catalog and only returning entries that match. > >If matching entries are based only on ownr identifier, N2C would >need to have a list of every ISBN used in an FPI, with corresponding >URLs, and also every registered owner. This isn't the same as every >ISBN ever issued, if you don't mind it failing on obscure ones that >the owner had not registered with the orgnisation running N2C. >Right? Well, that would be the ideal, but this fallback to the big catalog in the sky should be regarded as a "best effort", not a guarantee. For one thing, there will be a large set of resources identified by an FPI that do not have a URL. >> This response might look like: >> >> Content-type: application/socat >> DELEGATE -//Sun Microsystems::SunSoft// >> http://www.sunsoft.com/sunsoft.socat >> DELEGATE -//Sun Microsystems >> http://www.sun.com/suncat.socat >[...] >> This new mini-catalog is then processed by the client. It >> fetches the SunSoft catalog and procedes as usual. > >So how big would a CATALOG be of every SGML or XML public identifier >ever issued at Sun? Isn't this likely to be more than a kilobyte? >I see problems with performance here. Right. You don't want to do this sort of thing for on-the-fly rendering of pages. You want to use local catalogs for that. This sort of thing is only for when you have an FPI that can't be resolved by way of the local catalog, and you are willing to wait in the hope of getting the resource. >It needs to be a more hierarchical namespace, so that noone needs >to manage the whole list. That can be accomodated pretty easily. bigcat.sgmlopen.org (or wherever the "root" happens to be) could have a catalog with entries of the form: DELEGATE -//U.S. Navy//CALS http://www.navy.mil/foo/cals_catalog.socat DELEGATE -//Sun Microsystems http://www.sun.com/bar/suncat.socat DELEGATE -//DOE//OSTI http://www.osti.doe.gov/baz/ostifpis.socat The root does not have to know ALL FPIs, it would need to know of sites that make their catalogs available and the prefix that is common to the entries in that catalog. The US Navy might well want to split their CALS catalog into smaller pieces, so it could have entries of the form DELEGATE -//U.S. Navy//CALS//nuts-n-bolts http://www.navy.mil/stores/fasteners/nutscat.socat DELEGATE -//U.S. Navy//CALS//paints-n-brushes http://www.navy.mil/stores/preservatives/paints.socat etc. Eventually one would stop being delegated to other catalogs and would finally bottom out at a PUBLIC entry. Ron Daniel Jr. email: rdaniel@lanl.gov Advanced Computing Lab voice: +1 505 665 0597 MS B287 fax: +1 505 665 4939 Los Alamos National Laboratory http://www.acl.lanl.gov/~rdaniel/ Los Alamos, NM, USA 87545 obscure_term: "hyponym" Date: Wed, 4 Dec 96 15:17:40 CST From: paul@arbortext.com (Paul Grosso) To: w3c-sgml-wg@w3.org Subject: Re: FPIs to URNs Cc: rdaniel@lanl.gov > From: Ron Daniel > > Thus spoke Paul Grosso (at least at 10:26 AM 12/4/96 CST) > > From: Ron Daniel > > >While it has not been discussed and is therefore not out of the > >question, I suspect SGML Open would have resource problems and > >perhaps jurisdiction problems doing this. Since the GCA is in > >line to be endorsed as the Owner Identifier Registrar, they may > >be a more obvious choice. > > One of the things I wanted to achieve was being able to deal with > *unregistered* FPI owners. (This would be subject to caveats, of > course, but there are a lot of unregistered owners out there that have > a lot of good stuff it would be nice to be able to get to). > > . . . I did not mean to suggest that any such central catalog(s) would be for registered FPIs only. I only meant that GCA crosses my mind as a possible host for an FPI resolution catalog because they appear to be taking on some vaguely related tasks such as owner id registration. paul From: lee@sq.com Date: Wed, 4 Dec 96 23:00:54 EST To: bosak@atlantic-83.eng.sun.com, lee@sq.com, rdaniel@acl.lanl.gov, w3c-sgml-wg@w3.org Subject: Re: FPIs to URNs Cc: rdaniel@lanl.gov > >> This new mini-catalog is then processed by the client. It > >> fetches the SunSoft catalog and procedes as usual. > > > >So how big would a CATALOG be of every SGML or XML public identifier > >ever issued at Sun? Isn't this likely to be more than a kilobyte? > >I see problems with performance here. > > Right. You don't want to do this sort of thing for on-the-fly > rendering of pages. You want to use local catalogs for that. > This sort of thing is only for when you have an FPI that > can't be resolved by way of the local catalog, and you are > willing to wait in the hope of getting the resource. I want to see FPI resolution that works when you *don't* already have the file and have never seen the FPI before on your local system. For my part, I would be happy to ditch FPIs and use some other URN scheme if it scaled better: is accessible on the WWW today (using Panorama) if you know how. But I could use just as happily. Note that this meets Debbie's requirement without actually being an SGML Formal Public Identifier. I.e., there are other possible solutions. > >It needs to be a more hierarchical namespace, so that noone needs > >to manage the whole list. > > That can be accomodated pretty easily. bigcat.sgmlopen.org (or wherever > the "root" happens to be) could have a catalog with entries of the form: > DELEGATE -//U.S. Navy//CALS http://www.navy.mil/foo/cals_catalog.socat > DELEGATE -//Sun Microsystems http://www.sun.com/bar/suncat.socat > DELEGATE -//DOE//OSTI http://www.osti.doe.gov/baz/ostifpis.socat The trouble is that there are probably over a million organisations or individuals who have issued PIs at this point... This bigcat would hence be the better (worse) part of 50 MBytes, which won't fit in my Netscape tmp directory, and hence can't be downloaded by Netscape for use with a helper app. This is my biggest concern with the (literally) millions of unregistered public identifiers belonging to tens of thousands or hundreds of thousands or more of issuing "authorities". Minor concerns include the way that there are many cases where the same PI is used by two people to refer to entirely different things, especially with informal PIs that are often set to be the same as the file name... and also the way that there are often many PIs for the same public text, although that doesn't matter as much, if at all. Really, an SGML OPEN catalog could be thought of as a local entity cache; although they are often mantained by hand today, if URN resolution were used, a catalog could point into one's URN cache automatically... But in that case, perhaps it wouldn't be needed any more. Then we'd be down to something simpler, with the CATALOG as a possible implementation optimisation. So, is there a URN scheme that scales in a way that makes it easier to implement than grandfathering SGML FPIs? Lee From: Ken Holman To: "'w3c-sgml'" Subject: RE: FPIs to URNs Date: Thu, 5 Dec 1996 08:53:23 -0500 >---------- >From: lee@sq.com[SMTP:lee@sq.com] >Sent: Wednesday, December 04, 1996 23:00 >To: bosak@atlantic-83.Eng.Sun.COM; lee@sq.com; rdaniel@acl.lanl.gov; >w3c-sgml-wg@w3.org >Cc: rdaniel@lanl.gov >Subject: Re: FPIs to URNs > .... > >I want to see FPI resolution that works when you *don't* already have >the file and have never seen the FPI before on your local system. The supplemental CATALOG, being separate from the markup, is perfect for this. > >For my part, I would be happy to ditch FPIs and use some other URN >scheme >if it scaled better: >v3.4//EN"> >is accessible on the WWW today (using Panorama) if you know how. > >But I could use >Quin//bailey.dtd;version=3.4"> >just as happily. > >Note that this meets Debbie's requirement without actually being an >SGML Formal Public Identifier. I.e., there are other possible >solutions. I don't think this meets my requirement because of the implication of location in the URN. I use FPI's for entity identification and the CATALOG for entity resolution. I don't think my needs are met if both are tied up into a single mechanism. If I want to use a URN for resolution, I would map the FPI to the URN in the CATALOG. To some this seems redundant, but to me "urn.mitra.org" isn't an identifier, its a location. >For example, using PUBLIC "urn:urn.mitra.org//Liam Quin//bailey.dtd;version=3.4": How would I override an XML processor >from accessing "urn.mitra.org" to give me the entity when I would rather use a local file? I want to override this without having to get into the file with my clumsy fingers and risk "breaking" something while I'm in the file. >> >It needs to be a more hierarchical namespace, so that noone needs >> >to manage the whole list. >> >> That can be accomodated pretty easily. bigcat.sgmlopen.org (or wherever >> the "root" happens to be) could have a catalog with entries of the form: >> DELEGATE -//U.S. Navy//CALS http://www.navy.mil/foo/cals_catalog.socat >> DELEGATE -//Sun Microsystems http://www.sun.com/bar/suncat.socat >> DELEGATE -//DOE//OSTI http://www.osti.doe.gov/baz/ostifpis.socat > >The trouble is that there are probably over a million organisations or >individuals who have issued PIs at this point... >This bigcat would hence be the better (worse) part of 50 MBytes, which >won't fit in my Netscape tmp directory, and hence can't be downloaded >by Netscape for use with a helper app. > >This is my biggest concern with the (literally) millions of >unregistered >public identifiers belonging to tens of thousands or hundreds of >thousands or more of issuing "authorities". Good reasons for me to not include the resolution aspects of any kind of identifier in the XML specification. We are defining a meta-language for document markup that identifies information in a file. I've always viewed SGML (and XML) as passive, not active. CATALOGs are active. FPI's are passive. A local URL is, in my opinion, active. SYSTEM identifiers are active. These two very separate problems need two separate, distinct but related, solutions, not a single solution that ties both important concepts inextricably. The tutorial I've been getting in this maillist about global URLs and URNs leaves me the impression that these are very "active" things that "impose" implementation on me, when I'm only responsible for marking up my information. I am more than willing to give you supplemental information to work with my markup, but I do not want to hardwire environmental client-side-resolution issues in my markup. >Really, an SGML OPEN catalog could be thought of as a local entity >cache; >although they are often mantained by hand today, if URN resolution were >used, a catalog could point into one's URN cache automatically... Great flexibility! Just think, I didn't have to change my markup to point into my URN cache. > >But in that case, perhaps it wouldn't be needed any more. But it would be needed, because the next day I may not want my catalog to point into my URN cache (or anybody else's URN cache, or somebody else pointing into my URN cache (I might be sensitive about that!:{)). >Then we'd be down to something simpler, with the CATALOG as a possible >implementation optimisation. > >So, is there a URN scheme that scales in a way that makes it easier >to implement than grandfathering SGML FPIs? In my opinion, these are solving two different problems, hence, no URN scheme would successfully grandfather FPI's. I want my FPI's to be absolutely permanent if I wish, so my sacrosanct document remains inviolable, yet I have the flexibility to work with that document in any environment I have now, or have to work in in the future. Ken -- G. Ken Holman Tel: +1 613 596-CADE(2233) /\ /\ Chief Technology Officer Fax: +1 613 596-5934 \/ \/ Computer Microstar Software Ltd. WATS: 1 800 267-9975 /\ /\ Aided 3775 Richmond Road Mail: gkholman@microstar.com \/ \/ Document Nepean Ontario Info: cade@microstar.com /\ /\ Engineering CANADA K2H-5B7 Web: http://www.microstar.com \/ \/ -- 1605 Mardick Court, Box 266 Phone: +1 613 489-2987 Kars, Ontario CANADA K0A-2E0 E-mail: gkholman@CanadaMail.com -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6 mQCNAjHOimAAAAEEAK47HbDtZZB8hJBk+r9Zh7m7QxdFHTaz/m200QQ7J9XX4QYs h6hsuP6ZqBJXyLdmIVMEwR6Ry6oxjKMd6BRJ8OGScD89eIghgbrpMX+900NxM0x2 v3yWO9ki2gKRPrn4vlCXznnmcmsyke0G02T2LefXbgZHyVSqDSOLy8nwuN7dAAUR tClHLiBLZW4gSG9sbWFuIDxhYjk0NUBmcmVlbmV0LmNhcmxldG9uLmNhPg== =IN3T -----END PGP PUBLIC KEY BLOCK----- Date: Thu, 5 Dec 1996 10:29:03 -0500 To: "'w3c-sgml'" From: dgd@cs.bu.edu (David G. Durand) Subject: Resolution != Naming (was RE: FPIs to URNs) I was going to reply in detail to Ken Holman's note, and the note from Lee that inspired it, and Ron Daniels' notes that have started this thread, but instead I think I should summarize, rather than doing a blow-by-blow. Ron has laid out some strategies for FPI resolution. Note that URNs will have _at least_ several alternative resolution mechnisms (probably including some kind of local filesystem access). Ron is asking about appropriate mechnisms for resolving FPIs within the URN framework. This is great: certainly some variation of catalogs should be useful in the FPI subset of the URN space. Note that URNs (of any sort) do not inherently require the use of DNS or any particular machines, though NAPTR defines a resolution mechanism based on DNS. Eventually, for some documents, I expect that libraries will get into the act, or some groups filling the social function of library catalogs if they don't fill that niche. I think, given that the URN group is willing to figure out how to integrate FPI into URN resolution, that we should encourage and help them on that (but not worry about specifying or discussing those mechanisms here, as it's not the right place). The XML standard should specify the option to have a PUBLIC string, expain that it is unique, persistent, etc., and how to get a GCA or ISBN publisher prefix. It should point out that resolution is application dependent, but that applications are suggested to use catalogs (We might make a simplified catalog format the does not bother with remapping SYSTEM, network resolution, etc, if that makes for 1000's of lines of code), or the Internet's URN mechanisms for FPIs (this latter is being worked by the URN group, and need not be further specified). We also need to note that other resolution mechanisms are also acceptable. That gives an implementor two things to implement if they desire, and a browser (or other simple parser) can simply use system IDs, and treat the FPI as documentation when reporting the failure, if it does not ignore the FPI completely. FPIs can _only_, in principle, guarantee that something has a unique name, not that you can find the thing. (But, of course, having a unique name can be of use in finding things). -- David I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ From: lee@sq.com Date: Thu, 5 Dec 96 10:57:21 EST To: gkholman@microstar.com, w3c-sgml-wg@w3.org Subject: RE: FPIs to URNs I had written that I could use > > >Quin//bailey.dtd;version=3.4"> > >just as happily. > > > >Note that this meets Debbie's requirement without actually being an > >SGML Formal Public Identifier. I.e., there are other possible > >solutions. Ken Holman was confused by my example -- which wasn't clear to those who have not followed URNs at all, I agree. Sorry. He said: > I don't think this meets my requirement because of the implication of > location in the URN. There is no implication of location in the URN. That's why it's a URN. The "urn.mitra.org" mentioned here is not a host, and is not contacted, and is not the location of the resource. It's a URN. N. not L. If it is easier to understand, imagine, if you will, that I am asking for a slightly different syntax for FPIs, with a hierarchy of organisations who can allocate naming authority prefixes. +//NA:US:CA:SUN:SUNSOFT:DIVDOC//DTD XXX YYY//EN for example. This would be easier to arrange for automatic resolution, easier to administer and more robust. Or so it seems to me. > >For example, using PUBLIC "urn:urn.mitra.org//Liam > Quin//bailey.dtd;version=3.4": How would I override an XML processor > from accessing "urn.mitra.org" to give me the entity when I would rather > use a local file? Ignoring the misunderstanding about the domain name.... This is a real issue. A lookaside cache can be called CATALOG if it makes people happy, but that is all it is; you can have a cache for anything. > I want to override this without having to get into the > file with my clumsy fingers and risk "breaking" something while I'm in > the file. So you need a tool. [...] > >This is my biggest concern with the (literally) millions of > >unregistered > >public identifiers belonging to tens of thousands or hundreds of > >thousands or more of issuing "authorities". > > Good reasons for me to not include the resolution aspects of any kind of > identifier in the XML specification. If we don't believe they will work, WE MUST NOT RELY ON THEM, and cannot use them in the spec. Period. No hand waving. No ``we left that out of the spec hoping no-one would notice our whole plan was built on a false premise' :-) > We are defining a meta-language > for document markup that identifies information in a file. I've always > viewed SGML (and XML) as passive, not active. I don't follow this -- we are discussing the underpinnings of a hyperlinking mechanism. Whether you believe hyperlinking to be grammatically male or female, active or passive, it still has to be specified. XML is not the same as SGML. This sounds supiciously like Conformance to Folklore to me. Where in ISO 8879 are these active/passive terms defined? Where does this rule come from" > >Really, an SGML OPEN catalog could be thought of as a local entity > >cache; > >although they are often mantained by hand today, if URN resolution were > >used, a catalog could point into one's URN cache automatically... > > Great flexibility! Just think, I didn't have to change my markup to > point into my URN cache. I am not sure if this is cynicism or genuine (sorry). > >But in that case, perhaps it wouldn't be needed any more. > > But it would be needed, because the next day I may not want my catalog > to point into my URN cache (or anybody else's URN cache, or somebody > else pointing into my URN cache (I might be sensitive about that!:{)). I _really_ don't understand that. If you dont want to use the copy of an FPI's text that's in your CATALOG, you know what to do about it. All I did was change the names, not the mechanism: it's still an SGML OPEN CATALOG. > >So, is there a URN scheme that scales in a way that makes it easier > >to implement than grandfathering SGML FPIs? > > In my opinion, these are solving two different problems, hence, no URN > scheme would successfully grandfather FPI's. > > I want my FPI's to be absolutely permanent if I wish, so my sacrosanct > document remains inviolable, yet I have the flexibility to work with > that document in any environment I have now, or have to work in in the > future. You can't have that. You will already have to change your environment to work with XML. If FPIs are "grandfathered" as has been discussed, they become URNs. Every FPI is then automatically and henceforward a URN. So there is no point in saying that you want FPIs and not URNs, because you can't have that. If it is easier to understand, imagine, if you will, that I am asking for a slightly different syntax for FPIs, with a hierarchy of organisations who can allocate naming authority prefixes. +//NA:US:CA:SUN:SUNSOFT//DTD XXX//EN for example. This would be easier to arrange for automatic resolution, easier to administer and more robust. Or so it seems to me. Lee 3245 146th Place S.E., Suite 270, Bellevue, WA 98007, USA Date: Thu, 05 Dec 1996 08:32:20 -0800 To: w3c-sgml-wg@w3.org From: Derek Denny-Brown Subject: Re: Resolution != Naming (was RE: FPIs to URNs) David G. Durand said: > The XML standard should specify the option to have a PUBLIC string, >expain that it is unique, persistent, etc., and how to get a GCA or ISBN >publisher prefix. It should point out that resolution is application >dependent, but that applications are suggested to use catalogs (We might >make a simplified catalog format the does not bother with remapping SYSTEM, >network resolution, etc, if that makes for 1000's of lines of code), or the >Internet's URN mechanisms for FPIs (this latter is being worked by the URN >group, and need not be further specified). We also need to note that other >resolution mechanisms are also acceptable. > > That gives an implementor two things to implement if they desire, and a >browser (or other simple parser) can simply use system IDs, and treat the >FPI as documentation when reporting the failure, if it does not ignore the >FPI completely. please. discussion on this topic so far has told me that SGML Open Catalogs provide a solution, of sorts, for now. A better solution to this problem could benefit SGML in general, as well as XML. It might thus be better treated as a independent project. -derek "that which is not slightly distorted lacks sensible appeal: from which it follows that irregularity - that is to say, the unexpected, surprise, and astonishment, are an essential part and characteristic of beauty" - Charles Baudelaire Date: Thu, 05 Dec 1996 09:06:07 -0700 To: lee@sq.com, bosak@atlantic-83.eng.sun.com, lee@sq.com, w3c-sgml-wg@w3.org From: Ron Daniel Subject: Re: FPIs to URNs Cc: rdaniel@lanl.gov Thus spoke lee@sq.com (at least at 11:00 PM 12/4/96 EST) >I want to see FPI resolution that works when you *don't* already have >the file and have never seen the FPI before on your local system. I thought that was what I was describing. That is certainly what I am trying for. >For my part, I would be happy to ditch FPIs and use some other URN scheme >if it scaled better: [...] >Note that this meets Debbie's requirement without actually being an >SGML Formal Public Identifier. I.e., there are other possible solutions. I'll be happy to answer any questions I can on alternative URN schemes. [hierarchical example deleted] >This bigcat would hence be the better (worse) part of 50 MBytes, which >won't fit in my Netscape tmp directory, and hence can't be downloaded >by Netscape for use with a helper app. > >This is my biggest concern with the (literally) millions of unregistered >public identifiers belonging to tens of thousands or hundreds of >thousands or more of issuing "authorities". It is not necessary to download the whole catalog. If the client is "URN-savvy" it can issue a request containing the FPI it is trying to resolve and the server is supposed to send back a "tailored" catalog that only contains matching entries. The %!uri substitution thing was to offer the same benefits to non-URN systems. >So, is there a URN scheme that scales in a way that makes it easier >to implement than grandfathering SGML FPIs? Sorry to be difficult here, but let me turn the question around. Do you know of any naming approach that demonstrates the scalability you want? If so, point me at it and we can see about how to make it a URN namespace. Regards, Ron Daniel Jr. email: rdaniel@lanl.gov Advanced Computing Lab voice: +1 505 665 0597 MS B287 fax: +1 505 665 4939 Los Alamos National Laboratory http://www.acl.lanl.gov/~rdaniel/ Los Alamos, NM, USA 87545 obscure_term: "hyponym" Date: Thu, 5 Dec 1996 12:35:50 -0800 Subject: RE: FPIs to URNs From: "David G. Durand" To: w3c-sgml-wg@w3.org lee@sq.com wrote: >Ken Holman was confused by my example -- which wasn't clear to those who >have not followed URNs at all, I agree. Sorry. He said: And some of us who have followed URNs, as Mitra was a vocal advocate of the (incorrect) viewpoint that DNS should be a required part of the URN infrastructure -- and so the name you chose was suggestive to some... >If it is easier to understand, imagine, if you will, that I am asking >for a slightly different syntax for FPIs, with a hierarchy of organisations >who can allocate naming authority prefixes. > +//NA:US:CA:SUN:SUNSOFT:DIVDOC//DTD XXX YYY//EN >for example. This is already part of the FPI syntax (except that they use :: instead of : to assign subauthorities). It is also problematic to base resolution on naming authority in the long run -- as names are supposed to persist, but the control of a name may change over time: There's a URN problem that is equivalent to the URL hell cause by the move from CERN to W3C. So we have the infrastructure you ask for in FPIs already. >This would be easier to arrange for automatic resolution, easier >to administer and more robust. Or so it seems to me. well, robust over a 5-10 year period anyway. >Ignoring the misunderstanding about the domain name.... >This is a real issue. A lookaside cache can be called CATALOG if it >makes people happy, but that is all it is; you can have a cache for anything. If I can hand-edit it (maybe with a nice tool, of course) then it's more than a simple cache, which is restricted to monitoring what's happening. As an aside, we should consider a way to label an XML entitiy with its FPI: This would be a one-sentence change in the standard if we were using something extensible like HTTP headers, rather than processing instructions. This way we can _have_ a cache, if we want. >> >This is my biggest concern with the (literally) millions of >> >unregistered >> >public identifiers belonging to tens of thousands or hundreds of >> >thousands or more of issuing "authorities". >> >> Good reasons for me to not include the resolution aspects of any kind of >> identifier in the XML specification. > >If we don't believe they will work, WE MUST NOT RELY ON THEM, and >cannot use them in the spec. Period. No hand waving. No ``we left >that out of the spec hoping no-one would notice our whole plan was >built on a false premise' :-) This is the URN problem, and as has been said over and over: if you have a name, and the SYSTEM ID is broken, there is no way in hell you will _ever_ be able to guarantee that the name can be resolved. However, if you have a broken SYSTEM ID, you can look to the name for help, and if you have a broken name, you can look to the SYSTEM ID for help. This is redundancy in the reliable- system sense (good), not the authorial sense (bad). >If it is easier to understand, imagine, if you will, that I am asking >for a slightly different syntax for FPIs, with a hierarchy of organisations >who can allocate naming authority prefixes. > +//NA:US:CA:SUN:SUNSOFT//DTD XXX//EN >for example. Since FPIs already have this, maybe we need a real objection. And please remember that guaranteed resolution is impossible for any kind of name, which does _NOT_ make names useless. The reason I am now backing PUBLIC as opposed to URNs in SYSTEM is that the fallback strategy I outlined is not possible if we have a single ID that is only a URN, and no URL. You may think of the URL as a kind of "hint" but it lets people use names (and SGML tools) now, while still be compatible with the future. And if URNs eventually take off, the PUBLIC ID may become a quaint relic -- but if it takes a while they will be useful. -- David Date: Thu, 05 Dec 1996 13:29:24 -0500 To: Derek Denny-Brown , w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: Resolution != Naming (was RE: FPIs to URNs) At 08:32 AM 12/5/96 -0800, Derek Denny-Brown wrote: >discussion on this topic so far has told me that SGML Open Catalogs provide >a solution, of sorts, for now. I think you are missing David's point. He is explicitly not looking for a solution. He is looking for the flexibility to implement his own solutions. If SGML Open Catalogs happen to be the solution that is useful to many people, then maybe it could be mentioned in the standard, or even required (as a default solution, not as a "universal solution"). >A better solution to this problem could >benefit SGML in general, as well as XML. It might thus be better treated as >a independent project. I think that we all agree that we aren't going to solve the global naming "problem" here. As David says, "given that the URN group is willing to figure out how to integrate FPI into URN resolution, that we should encourage and help them on that (but not worry about specifying or discussing those mechanisms here, as it's not the right place)." All I am asking is that I not be tied to a "solution, of sorts, for now." Paul Prescod Date: Thu, 05 Dec 1996 13:29:27 -0500 To: lee@sq.com, gkholman@microstar.com, w3c-sgml-wg@w3.org From: Paul Prescod Subject: RE: FPIs to URNs At 10:57 AM 12/5/96 EST, lee@sq.com wrote: >> >This is my biggest concern with the (literally) millions of >> >unregistered >> >public identifiers belonging to tens of thousands or hundreds of >> >thousands or more of issuing "authorities". >> >> Good reasons for me to not include the resolution aspects of any kind of >> identifier in the XML specification. > >If we don't believe they will work, WE MUST NOT RELY ON THEM, and >cannot use them in the spec. Who is "we?" If you don't want them, don't use them. Names are not for document interchange.. They are for document maintenance in "internal systems", where the "systems" may be as small as a single machine, as large as a corporation, or, eventually, as large as the Internet. I really don't give a fig if *you* can resolve *my* FPIs. That's not why they are there. They are there so that I can regenerate the "correct" URL in my document as the URL changes, or so that within our organization we can redirect to local versions, or so that documents can be cached according to their names or ... Not so that you can "find" the documents I am pointing at. URLS do that. >Period. No hand waving. Both 8879 and the SGML Open catalog spec allow this "hand waving." 8879:"The system identifier can be omitted if the system can generate it >from the public identifier and/or other information available to it." SGML Open Catalog: "This resolution does not dictate when an entity manager should access this catalog; for example, an application may attempt other mapping algorithms before or (if the catalog fails to produce a successful mapping) after accessing this catalog." Of course it is also implicit in the catalog spec that an application may choose to implement or not to implement the catalog itself. The catalog standard is not an SGML TC. >No ``we left >that out of the spec hoping no-one would notice our whole plan was >built on a false premise' :-) You've got it all wrong: "we left that out of the spec because we realize that many people have many different needs and we didn't want to force a solution." Just as XML will presumably not *require* DSSSL-O, it should not require a particular catalog format. Think of public identifiers as "addressing processing instructions." You don't really expect anyone else to understand them unless you two have "worked out a system" (like SGML catalogs or DNS based resolution). But they are useful *without* anyone else understanding them. And if there is a good reason to "publish" them and agree to "share them" then you have that option. >> We are defining a meta-language >> for document markup that identifies information in a file. I've always >> viewed SGML (and XML) as passive, not active. > >I don't follow this -- we are discussing the underpinnings of a >hyperlinking mechanism. Whether you believe hyperlinking to be >grammatically male or female, active or passive, it still has to be >specified. XML is not the same as SGML. This sounds supiciously like >Conformance to Folklore to me. Where in ISO 8879 are these active/passive >terms defined? Where does this rule come from" Ken is precisely right that SGML and XML should not, in general, standardize behaviours. We only stray from that rule where we need to standardize a behaviour in order to get a useful hypermedia/internet system. Since URLs can be used to get that aforementioned useful hypermedia/internet system, FPI resolution is never a requirement for XML document publishing or display on the Internet. (they are VERY useful OFF of the Internet, however). The "active/passive" rule is the basic concept underlying generic markup. Markup is descriptive, not interpretive. Systems "use" it, not "execute" it. In XML we are making exceptions in order to simplify the system and make implementation straightforward. This does not have to be one of those cases because mandatory URLs make implementation straightforward. Furthermore, even when we specify a behaviour, for instance if we make a defined element name a "hyperlink", or require DSSSL-O, we will not restrict the interpretation of other elements as hyperlinks, or disallow other style sheet formats, will we? If your system knows (somehow) that my element is a hyperlink (though it may not conform to the XML hyperlink description), then it is surely allowed to attach hyperlink behaviour to it. In other words, in every case so far, when we have attached "behaviour" to markup, we have allowed (explicitly or implicitly) alternate mechanisms for achieving that behaviour. The one exception that I can think of is that SOIs must be URLs. But since the definition of "URL" is as wide as the definition of "SOI", that is not really a restriction at all. >If it is easier to understand, imagine, if you will, that I am asking >for a slightly different syntax for FPIs, with a hierarchy of organisations >who can allocate naming authority prefixes. > +//NA:US:CA:SUN:SUNSOFT//DTD XXX//EN >for example. > >This would be easier to arrange for automatic resolution, easier >to administer and more robust. Or so it seems to me. Since some FPI usages do not require automatic resolution, or *any* resolution, reorganizing the scheme based on the idea of hierarchical resolution is probably overly restrictive. And as David points out, what happens if Quebec separates (for example), or Sunsoft is "spun out" of Sun? Your hierarchy is obsolete. I am just as resistent to forcing XML public identifiers to conform to what we predict will be the needs of a system which is not yet in existance (URN) as to tie it exclusively to a system that is almost certainly going to be obsolete (for most applications) within a few years (catalogs). Technical restrictions on URNs may require most Public Identifiers to change format in the future. as you suggest. If we leave their resolution open, then users can make that change in their documents in the future. In summary: Please just leave Public Identifiers open, and the community will choose their own solutions, just as they will for stylesheets or meta-data formats. We could specify a "default" (as we intend to with stylesheets), but it should not be exclusive. Paul Prescod Date: Thu, 05 Dec 1996 20:11:25 -0900 To: dgd@cs.bu.edu (David G. Durand), "'w3c-sgml'" From: "W. Eliot Kimber" Subject: Re: Resolution != Naming (was RE: FPIs to URNs) At 10:29 AM 12/5/96 -0500, David G. Durand wrote: [...] > The XML standard should specify the option to have a PUBLIC string, >expain that it is unique, persistent, etc., and how to get a GCA or ISBN >publisher prefix. It should point out that resolution is application >dependent, but that applications are suggested to use catalogs (We might >make a simplified catalog format the does not bother with remapping SYSTEM, >network resolution, etc, if that makes for 1000's of lines of code), or the >Internet's URN mechanisms for FPIs (this latter is being worked by the URN >group, and need not be further specified). We also need to note that other >resolution mechanisms are also acceptable. I think this is a very reasonable proposal. > That gives an implementor two things to implement if they desire, and a >browser (or other simple parser) can simply use system IDs, and treat the >FPI as documentation when reporting the failure, if it does not ignore the >FPI completely. By saying "can simply use system IDs" are you implying that you would always have a system ID even if you also had a public identifier? I'm not sure I'd want to require a system ID in all cases. > FPIs can _only_, in principle, guarantee that something has a unique >name, not that you can find the thing. (But, of course, having a unique >name can be of use in finding things). But of course nothing can guarantee the ability to find a thing in the general case (i.e., a open system, e.g., the Web). Cheers, E. -- W. Eliot Kimber (eliot@isogen.com) Senior SGML Consulting Engineer, Highland Consulting 2200 North Lamar Street, Suite 230, Dallas, Texas 75202 +1-214-953-0004 +1-214-953-3152 fax http://www.isogen.com (work) http://www.drmacro.com (home) "Rats in the morning, rats in the afternoon...if they don't go away, I'll be re-educated soon..." --Austin Lounge Lizards, "1984 Blues" Date: Thu, 5 Dec 1996 22:37:05 -0500 To: Michael Sperberg-McQueen , w3c-sgml-wg@w3.org From: dgd@cs.bu.edu (David G. Durand) Subject: Re: Straw-man XML Support for PUBLIC Identifiers and XML Catalogs I am CCing my response to a query from Michael about 9070 identifiers in case it is of interest to the rest of the list. Of course the only reference of record is the standard: I do have a copy, though. I've attached some very old URN ramblings about FPIs below. In brief, and by example, the basic syntax for ISO 9070 Object Identifiers is a::b::c//d::e::f::g Left // separated field is the name issuing authority. This is confusingly called the "Object owner", but it identifies the owner of the _name_ (not the object). Any "owner" of a prefix can delegate a subspace of their namespace to others, so if I own "a::b" I could have assigned "a::b::c" to you for your Object Identifiers. The "Object name" (the part following the //) has similar hierarchical identifiers, but no semantics or administrative procdedures attach to them. The bug in the 9070 standard is that SGML FPIs map all the fileds other than the authority into successive positions of the Object ID. So if I use an FPI like: -//SUN::SUNSOFT//DTD My weird DTD//EN It becomes: SUN::SUNSOFT//DTD::-::My Weird DTD::EN This is fine, but if I use :: separated items in organizing the object IDs of an SGML FPI, things get messed up: -//SUN::SUNSOFT//DTD dgd::dtds::weird-1//EN becomes: SUN::SUNSOFT//DTD::-::dgd::dtds::weird-1::EN And I can't extract the Language and version specifications dependably: They may be the last, or the last 2 items -- no marker as to which. And, since the count of fields is no longer fixed, I can't do it by counting either. Now this conversion is defined in an informative annex, so we need not be bothered by compatibility with this for XML, should we decide that this has any relevance to XML. One thing I really like about the 9070 syntax is that it is simple and general, lacking many of the odd mandatory fields in 8879 FPIs. On the other hand it is a different syntax, not in wide use. I've attached some old notes on 9070 I made for the URN list, years ago. They duplicate some of this, but touch on a few points not given above. -- David Attached archival material: As an "SGML guy" (though not the sort of SGML-zealot one sometimes sees, I hope), I have also been wondering this. Particularly given the limited use of SGML in WWW, the ISO naming stuff which is integrated with SGML seems a natural (at least for that application). Here's some possibly relevant info: The ISO FPIs are defined in ISO 9070. They are based on a two part structure: + naming authority + object identifier Each of these can be split into multiple hierarchical parts (allowing for complex object names and delegation of naming authorities). Root authorities can be assigned based on ISBN publisher numbers at the moment. The character set that can be used is case-insensitive and also restricted to be highly portable across national character sets. The syntax is character-based, rather than simply describing a sequence of octets. There are only two objections to the FPI standard as far as I can tell, based on the URN requirements doc. The first is a somewhat ugly syntax: "//" to delimit the two major parts, and "::" to delimit fields within each item. The second is an arbitrary length restriction 100 chars for owner name, 100 chars for object name -- seemingly chosen so that the corresponding SGML identifiers come to less than 250 characters. This last restriction is something that could be changed pretty easily through the ISO, I think, especially as the harmonization with the internet would appeal to the part of ISO that developed the FPI standards. (i.e. it was _not_ developed as part of OSI). It seems that the syntactic flexibility of the FPIs is sufficient to handle any naming needs I've seen proposed here, and the hierarchical authority assignment should be pretty scalable and decentralizable. 9070 has a provision for the use of ISBNs themselves as a root authority. This was a revision to the standard in the second edition (ISO/IEC 9070:1991(E)). This means that there is a non-ISO based authority to assign names. This is important since ISO 9070 naming authority (to be administered by ANSI) is not yet in operation. I have been told that the ISBN people are prepared to issue ISBN publisher numbers to anyone wishing to pursue electronic publication. The ISO 9070 character set is Upper/lower case, digits and "'()+,-.:=?/". The standard is defined in terms of a "character repertoire" not a particular encoding, so that national issues are a protocol rather than a naming issue. This lowest common denominator make make the "name" aspect of formal public identifiers less meaningful to Europeans and non-roman script users. Comparison rules (to determine sameness of named objects) are defined by 9070. I am not sure if they are case-sensitive or not, since I'm lacking access to some reference materials at the moment. >2) Public identifiers have a separation between owner-name components and >object-name components which has no equivalent in object identifiers. >(This separation may well prove artificial and lead to errors.) This may also, rather than a drawback, prove critical to enabling easy support for different methods of encoding object names, distinct from the issuing authorities. I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ From: Paul Prescod Subject: Re: Resolution != Naming (was RE: FPIs to URNs) To: eliot@isogen.com (W. Eliot Kimber) Date: Fri, 6 Dec 1996 05:20:24 -0500 (EST) Cc: dgd@cs.bu.edu, w3c-sgml-wg@w3.org > By saying "can simply use system IDs" are you implying that you would > always have a system ID even if you also had a public identifier? I'm not > sure I'd want to require a system ID in all cases. I think that any document that is published on the Web should have system IDs for all entities. That could be in an SGML Open Catalog if we make SOCAT handling a mandatory default resolution mechanism, or inline, if we do not. Lee is right to complain that there must be a URL somewhere, or else we are just handwaving. Within an "internal system", where a resolution mechanism is well defined, URLs may not be useful. I am not sure if there is a clear line between "published on the web" and used in an "internal system", but perhaps authors should make that decision. If they want everyone to be able to read their documents, they must put URLs where the user agents know how to find them: inline or in some widely-implemented catalog mechanism. Paul Prescod Date: Fri, 06 Dec 1996 08:12:01 -0900 To: w3c-sgml-wg@w3.org From: "W. Eliot Kimber" Subject: Re: Resolution != Naming (was RE: FPIs to URNs) At 05:20 AM 12/6/96 -0500, Paul Prescod wrote: >> By saying "can simply use system IDs" are you implying that you would >> always have a system ID even if you also had a public identifier? I'm not >> sure I'd want to require a system ID in all cases. > >I think that any document that is published on the Web should have system >IDs for all entities. That could be in an SGML Open Catalog if we make SOCAT >handling a mandatory default resolution mechanism, or inline, if we do not. >Lee is right to complain that there must be a URL somewhere, or else we are >just handwaving. What about for data content notations? As many notations don't have publicly accessible definition documents (which is what the external identifier of a notation is supposed to refer to first and foremost), it's seems unreasonable to require URLs for them. (If this issue's already been discussed, let me know--I haven't had time to go back and read all the traffic since SGML '96.) Cheers, E. -- W. Eliot Kimber (eliot@isogen.com) Senior SGML Consulting Engineer, Highland Consulting 2200 North Lamar Street, Suite 230, Dallas, Texas 75202 +1-214-953-0004 +1-214-953-3152 fax http://www.isogen.com (work) http://www.drmacro.com (home) "Rats in the morning, rats in the afternoon...if they don't go away, I'll be re-educated soon..." --Austin Lounge Lizards, "1984 Blues" Date: Fri, 6 Dec 1996 07:36:21 -0800 (PST) From: "Deborah A. Lapeyre" Subject: Re: Resolution != Naming (was RE: FPIs to URNs) To: "W. Eliot Kimber" cc: "David G. Durand" , "'w3c-sgml'" > At 10:29 AM 12/5/96 -0500, David G. Durand wrote: > > The XML standard should specify the option to have a PUBLIC string, > >expain that it is unique, persistent, etc., and how to get a GCA or ISBN > >publisher prefix. It should point out that resolution is application > >dependent, but that applications are suggested to use catalogs, [...] > or the Internet's URN mechanisms for FPIs. Seconded -Debbie Lapeyre Date: Fri, 6 Dec 1996 11:03:53 -0500 To: w3c-sgml-wg@w3.org From: dgd@cs.bu.edu (David G. Durand) Subject: Re: Resolution != Naming (was RE: FPIs to URNs) At 8:12 AM 12/6/96, W. Eliot Kimber wrote: >At 05:20 AM 12/6/96 -0500, Paul Prescod wrote: >>I think that any document that is published on the Web should have system >>IDs for all entities. That could be in an SGML Open Catalog if we make SOCAT >>handling a mandatory default resolution mechanism, or inline, if we do not. >>Lee is right to complain that there must be a URL somewhere, or else we are >>just handwaving. I also agree with this. It's best to have a name and a location than one only of each... >What about for data content notations? As many notations don't have >publicly accessible definition documents (which is what the external >identifier of a notation is supposed to refer to first and foremost), it's >seems unreasonable to require URLs for them. (If this issue's already been >discussed, let me know--I haven't had time to go back and read all the >traffic since SGML '96.) We've been discussing entities, levaing NOTATION to the side for the moment. On NOTATION, I think we need to be careful -- Wherever possible, we should use MIME-types in harmony with the rest of the net -- whether we also want to support FPIs on the same optional basis as we should for documents is an open question. Their real use, it seems to me, would be for private types, and since we have the x- mime types (though their use is discouraged officially, I think) we may not even need them for that. For notation, I don't know that we need FPIs because that SYTEM ID (if a MIME type) is already globally-unique and persistent, with a well-defined naming authority: IANA. I think it would be a compatibility issue, more than anything. My impression is that public notations are not very common, but I could be wrong. -- David I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ Date: Fri, 6 Dec 1996 11:10:57 -0800 (PST) From: Terry Allen To: dgd@cs.bu.edu, w3c-sgml-wg@w3.org Subject: Re: Resolution != Naming (was RE: FPIs to URNs) > We've been discussing entities, levaing NOTATION to the side for the moment. On NOTATION, I think we need to be careful -- Wherever possible, we should use MIME-types in harmony with the rest of the net Agreed. Regards, Terry Allen Fujitsu Software Corp. tallen@fsc.fujitsu.com "In going on with these experiments, how many pretty systems do we build, which we soon find outselves obliged to destroy?" - Benjamin Franklin A Davenport Group Sponsor: http://www.ora.com/davenport/index.html Date: Sat, 7 Dec 1996 15:24:05 -0500 To: w3c-sgml-wg@w3.org From: davep@acm.org (Dave Peterson) Subject: Re: SGML and XML Gavin Nicol wrote: > I would be interested in hearing why people wish to necessarily >have the entire attribute set appear on each element instead of just >the attributes that had values specified (ie. I wonder if applications >couldn't be built for the latter case that are effectively the same as >the former). I cannot see any reason they could not. The reason for "real" defaulted values as I see it is so that the DTD designer can direct that all applications get the same default value, which "#IMPLIED" cannot insure. Administrative sanctions on application builders can have the same effect. (Though the may be tough to enforce in some open environments....) Dave Peterson SGMLWorks! davep@acm.org Date: Sun, 8 Dec 1996 19:49:55 -0800 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org cc: bosak@atlantic-83.eng.sun.com Subject: An application of FPIs In this message I'm going to attempt to fulfill promises made in earlier postings to the SGML WG by sketching the FPI system we're putting into place at Sun for Solaris documentation. The point is to give a concrete example of a bottom-up FPI resolution system that is well on its way toward implementation and may serve as an initial test case for FPI/URN resolution. Despite the fact that not everyone on my team at Sun agrees with me that FPIs should be included in XML 1.0, please note that I am speaking here as an interested party and consider what I have to say in that light. The AB2 daemon -------------- To understand why my publishing group is interested in location-independent addressing, you have to know a little bit about how we intend to distribute documents in Solaris 2.6. In that release, we are going to replace our old PostScript-based AnswerBooks with a new SGML-based system. The new system will be HTTP-based rather than NFS-based and will deliver data to generic HTML clients rather than to specialized viewers. We are still looking for a properly slick market-oriented name for this system, but at the moment we are just calling it AnswerBook2, or AB2 for short. AB2 is implemented as a daemon, ab2d, that is (at administrator option) installed on a server, along with some set of compiled SGML documents, and started up when Solaris is booted. The ab2d daemon is not a full Web server but just a lightweight HTTP process that sits in the background and waits for document requests. When a Web client makes such a request (using a complex URL driven by a forms-based interface of our construction), ab2d hands it off to a custom DynaWeb plugin that turns the complex URL into a query against the compiled SGML database, retrieves the data, and converts the requested document -- or by autochunking, an appropriate document fragment -- into HTML on the way out. (DynaWeb implementors will understand that the server may also return a generated table of contents, among other things, and that it is trivially easy for the same server to generate XML instead of HTML from the same compiled SGML files if a client identifies itself as XML-aware.) A distributed document space ---------------------------- This system has a number of pretty cool advantages, not least among which is the ability of system administrators to make their own tradeoffs between performance and disk usage in corporate networks. Let's suppose that in order to conserve local disk space you have installed just the basic Solaris user documentation on your workstation. The ab2d daemon is running on your workstation, and you can consult the documentation you've installed there using HotJava, Navigator, lynx, or whatever HTML browser you happen to have handy. Now suppose that you want to consult some documentation relating to the C compiler. You don't have that available on your system, but you know that Sally down the hall has installed that on the departmental server. If you know the name of that server, a simple request to its ab2d lets you access that server's list of document resources, and you can navigate, read, and bookmark the documents available there just as easily as you can the ones on your own machine. To find even less frequently used documentation, you may have to go as far as the corporate library machine, located at some distant node on the corporate WAN. And to get some really obscure piece about an obsolete version of one of Sun's products, you may have to go through the Internet to Sun's AB2 server, which has copies of everything we publish. But in every case, you are accessing a unified document space. If your AB2 installation has been set up correctly (which we make very easy to do), you are not even aware where specific documents are located; the only perceptible difference between getting a document >from your own machine or some other machine is a performance difference resulting from network latency and the difference in CPU utilization. If you go clear out to Sun's document repository you will, of course, notice a perceptible lag due to Internet transmission delays, but within the kind of corporate network for which this system is primarily designed, such delays will be far less noticable. If you find yourself using a distant collection often enough, you can always request that it be copied to a machine that is faster or closer to you, or you can just access Sun's master server directly and download the compiled SGML files to your own workstation. Location independence --------------------- Anyone who has tried to set up a distributed system like the one I've just described knows that it is virtually impossible to do if URLs are the only way to identify documents. What happens to the cross-reference links between your partial document set and the compiler documentation if Sally's machine goes down? What happens to your bookmarks if she reorganizes the file system on her machine? We realized very early on in the AB2 effort that the whole idea hinges on the use of location-independent document identifiers. Two things are needed to make the AB2 system work: 1. All documents must be referred to through a location-independent naming system, and 2. This system cannot depend on centralized name resolution, because (a) the corporate network cannot be assumed to be connected to the Internet, (b) an internal resolution server cannot be assumed to be available at all times, and (c) users and system adminstrators must be free to install and remove documents from the bottom up, without having to check things in and out of some central tracking database. We also wanted our customers to be able to use AB2 to distribute their own documents with an absolute minimum of organizational overhead. FPIs and socats --------------- The most important design decision we made in constructing a location-independent naming system was a negative one: we were not going to attempt a global, all-encompassing solution, we were just going to implement the simplest possible system that could work for us and our customers. Consequently, a number of features that one would want in a more general solution, such as the ability of the system to find the best possible copy of a resource at any given moment, have been deliberately omitted from the design. The assumption of relatively modest goals has resulted in a system that is simple, robust, and easily managed. The whole thing is based on FPIs and on SGML Open catalogs that use Tauber's proposed DELEGATE extension. We call such catalogs "socats" for short. Whenever a document is checked into our corporate document system, it is assigned a unique identifier of the form -//Sun::SunSoft//DOCUMENT SPARCINSTDESK Version 1//EN The uniqueness of the identifier is verified by the check-in process that all publications have to go through on their way to our master document database. The same database is referred to by the link editor that all SunSoft authors use for making links between books, for example [...] Installation Instructions for Solaris 2.6 (SPARC Platform Edition) Thus, all book authoring, management, compilation, and distribution is done using identifiers that are completely independent of physical location. When a book fragment containing the example link above is finally converted from SGML to HTML at the moment of its transmission to the client browser, the FPI reference in the olink is translated to a URL containing encoding that an AB2 server will understand as a request to resolve the FPI if the user traverses the link. To resolve the link, all AB2 servers maintain a "local socat" consisting only of PUBLIC and DELEGATE entries (other entries are ignored). The PUBLIC entries provide a lookup table for the FPIs of all books installed on the local system, and the DELEGATE entries point to a list of other AB2 servers that have been designated as alternative sources of information by the system administrator. The local socat is automatically updated whenever a documentation package is installed on or removed from the system. By default it only contains one DELEGATE entry, which points to the master document repository at Sun; a utility allows the system administrator to add or remove other DELEGATE entries at will, keeping the default entry always at the bottom of the list (because a reference to one of our FPIs will always find a match there and never fall through to other entries). The local socat is typically cached in server RAM. If there were a requirement to solve the global URN resolution problem with this mechanism, it would obviously fail, but since the local socat consists only of locally installed books and manually entered DELEGATE entries, it remains small and fast, even if (as in the case of our own master document server) it contains entries for every document we publish. If we were to extend the DELEGATE mechanism to allow multiple levels of indirection, as suggested in Tauber's original proposal, we could run into a number of interesting complications, starting with circular references. But again, we're not trying to solve the world's problems with this system, so we feel perfectly comfortable in arbitrarily limiting the DELEGATE process to just one level. If an attempt to resolve a given FPI fails, an AB2 server queries (in list order) the other AB2 servers pointed to by the DELEGATE entries in its local socat. These AB2 servers know how to respond to the special query by returning their own local socats, but their DELEGATE entries are ignored. The result, therefore, is that every AB2 document on the local system and every AB2 document on systems explicitly pointed to in entries made by the system administrator is available to the user in a seamless, unified document space. In a large network, a careful admin will make sure to include servers that have duplicate copies of various document sets. The order in which to seek alternative copies of a resource is not determined by some insanely complex algorithm but simply by the order in which the admin has decided to list them based on his or her own knowledge of the network environment. Thus, if the user has bookmarked a publication that typically comes >from a machine at the other end of the building, and that machine happens to be down at the moment, then the link does not fail but rather continues down the list until it finds another copy. If the user is hooked up to the Internet, then in the worst case the FPI falls through to the bottom of the list and finds resolution on Sun's master document repository. The user will notice a delay, but the link won't fail, and as soon as the usual machine comes back up again, performance will return to normal. There are lots of interesting details that I have left out of this brief description because the folks on the AB2 team who are implementing the system plan to tell you all about it at the WWW6 conference in April and at SGML Europe in May. Suffice it to say that this system solves our problem and will (we think) solve similar problems for Solaris customers who wish to use the same infrastructure for distributing their own documents. Relevance to FPIs in XML ------------------------ As I stated in an earlier posting, nothing in our scheme requires the addition of a single line of code to existing HTML browsers or future XML browsers; the implementation hit on browser vendors is exactly zero. In our system this all takes place on a server. Furthermore, nothing in our scheme requires changes to the existing XML draft if XML is used only as a delivery mechanism, because XML generated from our SGML data can use exactly the same generated URLs that we're using now in HTML generated from that data. The issue of including FPIs in XML arises only if XML is going to be used as an authoring format. Our current DocBook-based publishing system doesn't need FPIs in XML, but at some point I would like to use XML as a migration path for other groups within Sun that are using HTML on an ad hoc basis to provide documentation for unbundled products. At that point, we will need XML to include FPIs as syntactic objects. Some people (including the AB2 developers!) have pointed out to me that FPIs need not be included in XML 1.0 to serve Sun's immediate needs, which is true. But I do think that they have to be included at some point if I'm going to be able to use XML as a way to get current HTML writers into our unified document space, and since the only requirement for an application like ours is that FPIs have a syntactic specification, I think that they should be included now rather than later. As I made clear at the beginning of this message, my argument is far >from disinterested, but I suspect that the application I've described is not unrepresentative of other schemes that use FPIs to uniquely identify documents. Jon Date: Mon, 09 Dec 1996 06:31:05 -0800 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Public Identifiers Wow, for the first time in this process I went away for a week, now I feel the pain of anyone who cares about this but gets a little behind. Anyhow, a flight from Sydney to LA makes an excellent venue for catching up on this stuff. Anyhow, having slogged through all this, I think that a powerful case has been made that since people are using public identifiers, we shouldn't get in the way of so doing in XML documents. The discussion has also made it clear that it would be a huge mistake to wire in a requirement for processing FPI's or URN's or for an understanding of the kozmic chasm between the buddha-natures of Name and Location. Adopting Ken's proposed syntax delta seems like the way to go, minus the explicit references to catalog grammars (we shouldn't overlap with other specs, particularly when they're moving targets) - it also seems sound to point out that there are these things called FPI's and ISBN's and URN's and URI's and FSI's and Socats (love that) and that the implementor should bone up on them before implementing a public identifier resolver. I share Michael's discomfort at specifying a reference capability without including the resolution mechanism, but it is an undeniable fact that [a] people *are* using public identifiers usefully, and [b] there is nothing remotely approaching consensus on how to go about naming and locating things. So to address Michael's concerns, I'd say that the only way that you as an information provider can really control which versions of components get addressed is to use an absolute URL pointing back a machine you operate. But, just because *we* don't all agree on how to achieve permanence, portability, and guaranteed uniqueness in a naming scheme, doesn't mean we shouldn't let people use names. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Mon, 9 Dec 1996 10:41:56 -0500 To: w3c-sgml-wg@w3.org From: dgd@cs.bu.edu (David G. Durand) Subject: SDATA, again I would like to repreat my appeal that we revisit the SDATA decision. I'm sure that no extended rationale was presented for the decision, and I've been unable to find a record of it at all. As XML stands we have the private use character codes, which provide no resolution mechanism for determining which non-unicode character is included in an instance. I am asking only that we repalce the numerical values provided by private-use, with string values. We will still not have a resolution mechanism, but we will have a foundation on which one can be built. I'll also remind people: + in DSSSL (at least the near-final draft which I read) characters are identified primarily by strings, and not chracter codes. + since the ISO entities are no longer pre-defined, we will have to define XML substitutes for them, unless we allow SDATA. + since some of the characters in ISO math (and more importantly, TeX math) are not in Unicode, we may have to assign them private-use code points, if we don't allow SDATA. In concrete, I propose that we allow SDATA entities, and define that they are _only_ to be used to represent undefined characters by a descriptive string. We should also reserve SDATA entities of the form "[XML:" Character* "]" for a future glyph/character resolution mechanism, if one should be devised. -- David I would also invite some of the new list members that I know have character set expertise to comment on the alternative of informally assigned numerical codes versus informally assigned character strings. I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ Date: Mon, 9 Dec 1996 14:35:22 -0500 (EST) From: David J Birnbaum To: "David G. Durand" cc: w3c-sgml-wg@w3.org Subject: Re: SDATA, again On Mon, 9 Dec 1996, David G. Durand wrote: > As XML stands we have the private use character codes, which provide no > resolution mechanism for determining which non-unicode character is > included in an instance. I am asking only that we repalce the numerical > values provided by private-use, with string values. We will still not > have a resolution mechanism, but we will have a foundation on which one > can be built. I joined this list only recently and was not part of the original discussion of SDATA. But wading in now, I do not see any significant disadvantages to David's proposal, let alone any that would outweigh the obvious advantages. Can someone fill me in on the arguments for the other side? And is there any procedural reason why this issue should not be reexamined? Thanks, David ________________________________________________________________________ Professor David J. Birnbaum email: djbpitt+@pitt.edu Department of Slavic Languages url: http://clover.slavic.pitt.edu/~djb/ 1417 Cathedral of Learning voice: 1-412-624-5712 University of Pittsburgh fax: 1-412-624-9714 Pittsburgh, PA 15260 USA Date: Mon, 09 Dec 1996 13:47:42 -0900 To: w3c-sgml-wg@w3.org From: "W. Eliot Kimber" Subject: Re: SDATA, again At 02:35 PM 12/9/96 -0500, David J Birnbaum wrote: >On Mon, 9 Dec 1996, David G. Durand wrote: > >> As XML stands we have the private use character codes, which provide no >> resolution mechanism for determining which non-unicode character is >> included in an instance. I am asking only that we repalce the numerical >> values provided by private-use, with string values. We will still not >> have a resolution mechanism, but we will have a foundation on which one >> can be built. > >I joined this list only recently and was not part of the original >discussion of SDATA. But wading in now, I do not see any significant >disadvantages to David's proposal, let alone any that would outweigh the >obvious advantages. Can someone fill me in on the arguments for the other >side? And is there any procedural reason why this issue should not be >reexamined? I believe our original reasoning against SDATA was as follows: 1. The mechanism for mapping SDATA replacement text to characters/glyphs/whatever was not defined or definable. This makes SDATA entities inherently uninterchangable. 2. You could do everything you needed to do with SDATA by using CDATA entities with 10646 character references (but this appears to not be true). 3. Leaving SDATA out significantly simplified the spec, putting it in significantly complicated it. Simple wins, as a rule. Cheers, E. -- W. Eliot Kimber (eliot@isogen.com) Senior SGML Consulting Engineer, Highland Consulting 2200 North Lamar Street, Suite 230, Dallas, Texas 75202 +1-214-953-0004 +1-214-953-3152 fax http://www.isogen.com (work) http://www.drmacro.com (home) "Rats in the morning, rats in the afternoon...if they don't go away, I'll be re-educated soon..." --Austin Lounge Lizards, "1984 Blues" Date: Mon, 09 Dec 1996 13:31:48 -0800 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Re: SDATA, again At 01:47 PM 12/9/96 -0900, W. Eliot Kimber wrote: >I believe our original reasoning against SDATA was as follows: Eliot is correct. However, the ERB did take up SDATA once again, in response to the high level of concern in the WG; I'm not sure Eliot was at that meeting. At that time our thinking was along the lines of: - there are lots of useful things that people want to use that are not in 10646, and - if you want to use such a character, or your particular display machinery doesn't support some character for any reason, if all you have is a number, there's not much you can do; an ancillary string might well support table lookup, or at the very least enable a coherent error message, so - people use SDATA to provide a string to help out with this, but - this was not what SDATA was meant for, says Charles among others, and - the syntax of said strings, and the manner of their implementation, is wildly different from vendor to vendor and from app to app, and - James pointed out that if what you really need is a string associated with a particular character, then there should be a facility for this, e.g. an attributed character reference, rather than kludging our way through with SDATA, and furthermore - we really didn't have time in the V1.0 timeframe to figure out what the right solution is, and finally - XML was already getting too big. So we [nearly unanimously, as I recall] decided to leave this problem unsolved in V1.0. Sorry. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Mon, 9 Dec 1996 14:04:02 -0800 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: djbpitt+@pitt.edu CC: dgd@cs.bu.edu, w3c-sgml-wg@w3.org Subject: Re: SDATA, again | I joined this list only recently and was not part of the original | discussion of SDATA. But wading in now, I do not see any significant | disadvantages to David's proposal, let alone any that would outweigh the | obvious advantages. Can someone fill me in on the arguments for the other | side? And is there any procedural reason why this issue should not be | reexamined? The technical intricacies are over my head, but the procedural situation is pretty simple: the ERB examined this subject in great detail and came to the conclusion that the SDATA mechanism as it stands in 8879 is not specified well enough to be usable and that we are not the right group to provide an optimum replacement. Our plan is to include SDATA on a list of items that we urgently wish ISO to consider and to include it in XML when ISO has fixed it. Jon From: lee@sq.com Date: Mon, 9 Dec 96 17:17:40 EST To: w3c-sgml-wg@w3.org Subject: Re: SDATA, again > - the syntax of said strings, and the manner of their implementation, is > wildly different from vendor to vendor and from app to app, and SGML OPEN members may already know that I have started a working group to discuss coming up with a standardised syntax; however, it is very unlikely to come to anything in time for XML 2.0. > So we [nearly unanimously, as I recall] decided to leave this problem > unsolved in V1.0. Sorry. I think this isn't bad -- if the SGML Open syntax ends up being practical and satisfactory, it can be adoppted then. In the meantime, you can use a pair of entities as follows: (note that XML doesn't allow comments in the entity definition itself, as is done in the ISO public texts) Lee Date: Mon, 9 Dec 1996 17:47:08 -0500 To: w3c-sgml-wg@w3.org From: davep@acm.org (Dave Peterson) Subject: Re: SDATA, again Liam Quinn recently wrote: In the meantime, you can use >a pair of entities as follows: > > > >(note that XML doesn't allow comments in the entity definition itself, >as is done in the ISO public texts) Presumably if your only intent is to provide a human-readable comment, you'd use: Dave Peterson SGMLWorks! davep@acm.org Date: Mon, 9 Dec 1996 17:47:05 -0500 To: w3c-sgml-wg@w3.org From: davep@acm.org (Dave Peterson) Subject: Re: SDATA, again Jon recently wrote: >The technical intricacies are over my head, but the procedural >situation is pretty simple: the ERB examined this subject in great >detail and came to the conclusion that the SDATA mechanism as it >stands in 8879 is not specified well enough to be usable and that we >are not the right group to provide an optimum replacement. Our plan >is to include SDATA on a list of items that we urgently wish ISO to >consider and to include it in XML when ISO has fixed it. I.e., in the revision, not in any earlier TC. (Personal opinion.) Dave Peterson SGMLWorks! davep@acm.org Date: Mon, 9 Dec 1996 19:47:44 -0800 Subject: Re: SDATA, again From: "David G. Durand" To: w3c-sgml-wg@w3.org >From: Tim Bray >To: w3c-sgml-wg@w3.org >Subject: Re: SDATA, again >Date: Mon, 09 Dec 1996 13:31:48 -0800 >Eliot is correct. However, the ERB did take up SDATA once again, in >response to the high level of concern in the WG; I'm not sure Eliot >was at that meeting. At that time our thinking was along the lines of: > > - there are lots of useful things that people want to use that are > not in 10646, and Sure are! > - if you want to use such a character, or your particular display > machinery doesn't support some character for any reason, if all you > have is a number, there's not much you can do; an ancillary string > might well support table lookup, or at the very least enable a > coherent error message, so Sure does! > - people use SDATA to provide a string to help out with this, but Sure do! > - this was not what SDATA was meant for, says Charles among others, and Given the above, who cares? > - the syntax of said strings, and the manner of their implementation, is > wildly different from vendor to vendor and from app to app, and But if we reserve a syntax for an eventual standard method, the first two points carry more weight and objections about the "best" solution lose force, as we can punt on that safely while improving the ordinary user's lot, by eliminating a class of annoying and unrecoverable behavior. (A number is _never_ better than a string). > - James pointed out that if what you really need is a string > associated with a particular character, then there should be a facility > for this, e.g. an attributed character reference, rather than > kludging our way through with SDATA, and furthermore This might as well be implemented by SDATA, given that SDATA is already used that way, and given that we don't otherwise need SDATA (agreed by all I think). > - we really didn't have time in the V1.0 timeframe to figure out what > the right solution is, and finally But we don't need to.. > - XML was already getting too big. Is this really that much? One keyword, and no mandatory processing other than to display it to the user if you don't choose to implement some more complex strategy. > >So we [nearly unanimously, as I recall] decided to leave this problem >unsolved in V1.0. Sorry. I take the point, but given that I think the arguments against SDATA are specious, I feel compelled to drag us through the damn thing one more time. We don't have to pick an optimal solution to significantly improve XML, in this case; we don't lose compatibility with SGML as defined, and actually retain it with SGML as practiced. >Cheers, Tim Bray Well, no cheers here, I'm afraid. -- David We can't even represent the musician who used to be known as Prince, who can truly claim: "I am not a number, I am an undefined character!" Date: Mon, 09 Dec 1996 21:18:40 -0600 From: Len Bullard To: "David G. Durand" CC: w3c-sgml-wg@w3.org Subject: Re: SDATA, again David G. Durand wrote: > > We can't even represent the musician who used to be known as Prince, who can > truly claim: > "I am not a number, I am an undefined character!" Sure you can: he is a member of the DingBat family. len bullard Date: Mon, 9 Dec 1996 19:32:03 -0800 (PST) From: "Deborah A. Lapeyre" Subject: Re: SDATA, again To: "David G. Durand" cc: w3c-sgml-wg@w3.org Reply to Dave Durand. I second your post. --Debbie ======================================================================= Deborah A. Lapeyre Phone: 301-231-6933 Mulberry Technologies, Inc. Fax: 301-231-6935 6010 Executive Blvd. Suite 608 E-mail: dlapeyre@mulberrytech.com Rockville, MD USA 20852 ======================================================================= Date: Tue, 10 Dec 1996 08:43:33 -0500 From: gtn@ebt.com (Gavin Nicol) To: dgd@cs.bu.edu CC: w3c-sgml-wg@w3.org Subject: Re: SDATA, again >I take the point, but given that I think the arguments against SDATA are >specious, I feel compelled to drag us through the damn thing one more time. >We don't have to pick an optimal solution to significantly improve XML, in >this case; we don't lose compatibility with SGML as defined, and actually >retain it with SGML as practiced. I agree with David here. Even though there are other ways of doing much the same thing, I favor SDATA over all alternatives, and it doesn't cost much to add it to the XML spec. Date: Tue, 10 Dec 1996 09:36:25 -0500 To: w3c-sgml-wg@w3.org From: davep@acm.org (Dave Peterson) Subject: Re: SDATA, again "David G. Durand" recently wrote: >>From: Tim Bray >> - this was not what SDATA was meant for, says Charles among others, and As I understand 8879 and ESIS as currently described, I think it fair to say: Internal SDATA, when encountered, should be passed to the application marked as such and with internal-to-the-document entity name attached. Then the application is free to do with it what it wishes. By definition, the text of the entity is system dependent. If XML wishes to consider all XML implementations one system for this purpose and prescribe the appropriate content and behaviour, this is not inconsistent with 8879. >> - James pointed out that if what you really need is a string >> associated with a particular character, then there should be a facility >> for this, e.g. an attributed character reference, rather than >> kludging our way through with SDATA, and furthermore > >This might as well be implemented by SDATA, given that SDATA is already used >that way, and given that we don't otherwise need SDATA (agreed by all I >think). I've not heard directly from James this "pointing out", but it sounds like he means one of two things: o If he means to bind the character to an unused code point in the document character set, thereby adding the character to the document's character repertoire, this will not work for XML since XML intends to prescribe one universal document character set--one universal character repertoire and one universal interpretation of numeric character references as characters in that repertoire. o If he means to create a new form of character reference, other than the existing numeric and named references (e.g., "{" and "&#RE;"), in which an abstract "character" is tied to the reference via some kind of declaration that associates a description of the character with an internal name, then that sounds like an SDATA entity by another name, with a more closely circumscribed interpretation than SDATA currently has. A "CHAR" entity type has been proposed in one forum or another, but not discussed to any great length yet. Hope this helps. Dave Peterson SGMLWorks! davep@acm.org Date: Tue, 10 Dec 96 08:47:25 CST From: Michael Sperberg-McQueen Subject: Re: SDATA, again To: W3C SGML Working Group On Mon, 9 Dec 1996 10:36:14 -0500 David G. Durand said: > I would like to repreat my appeal that we revisit the >SDATA decision. I'm sure that no extended rationale was >presented for the decision, and I've been unable to find a >record of it at all. As XML stands we have the private use The second half of October was, if I remember correctly, given over largely though not exclusively to the discussion of this question. There seems little point in my contributing to this discussion now, since what I contributed then (I thought they were extended discussions, and they certainly described the rationale for the ERB decision) seems not only not to have persuaded David, but to have been invisible to him. So I'll just ask David to follow the obvious guideline: if you failed to persuade everyone that SDATA is worth having in the spec the first time around, and now want to raise the question again, can you please provide some new arguments? The old arguments have been read, and understood, and discussed. David thinks, I suspect, that I didn't understand his arguments, but understanding is not the same as agreement. His arguments didn't persuade me then, and I for one have enough other things on my plate not to want to discuss an issue for the third time, when nothing new has been added to the discussion. In all the cases I know of where SDATA entities now work without user intervention, the SDATA entities are known characters which occur in Unicode. In cases where non-Unicode characters are successfully handled, all the cases I know of involve user intervention in the application's character tables. Removing SDATA entities does not change this state of affairs one whit: known characters will work, and others will require user intervention. The problem of characters not known to an application is a real and serious one. I'd like to see some discussion of how to solve it, rather than further special pleading for SDATA, which seems to me to be a band-aid for the problem, not a solution to it. We need a real solution, not a band-aid, and I am not convinced that SDATA is even a part of a real solution. Michael Sperberg-McQueen Date: Tue, 10 Dec 1996 11:03:27 -0800 To: Michael Sperberg-McQueen , W3C SGML Working Group From: dgd@cs.bu.edu (David G. Durand) Subject: Re: SDATA, again At 8:47 AM 12/10/96, Michael Sperberg-McQueen wrote: >On Mon, 9 Dec 1996 10:36:14 -0500 David G. Durand said: >The second half of October was, if I remember correctly, given >over largely though not exclusively to the discussion of this >question. There seems little point in my contributing to this >discussion now, since what I contributed then (I thought they were >extended discussions, and they certainly described the rationale for >the ERB decision) seems not only not to have persuaded David, but to >have been invisible to him. I was referring to official disclosure of the ERB discussion: We do not see ERB discussions except as they are revealed to us: and the original ERB decision was taken before the long WG debate you refer to. There may have been a brief post about a revisiting of the issue in the ERB: and it may have been revisited, but this was not clearly apparent to me. I do occasionally delete messages, so I may have lost one, though I generally keep all the ERB decision notes. Certainly the discuussion was not invisible to me, just whether there was a revisiting of the issue in the ERB, and the final rationale for the ERB decision after that process. If I have missed something, I apologize, and I tried in my initial note to leave that possiblity open -- with 1623 messages in my current file, missing, misrembering, or failing to find one in a search is a constant possibility. >So I'll just ask David to follow the obvious guideline: if you >failed to persuade everyone that SDATA is worth having in the spec >the first time around, and now want to raise the question again, can >you please provide some new arguments? The old arguments have been >read, and understood, and discussed. David thinks, I suspect, that >I didn't understand his arguments, but understanding is not the same >as agreement. His arguments didn't persuade me then, and I for one >have enough other things on my plate not to want to discuss an issue >for the third time, when nothing new has been added to the >discussion. I'm not worried about Michael per-se: some people prefer uninterpreted numbers to uninterpreted namespaces. Tim's posting did clearly lay out the rationale for the ERB decision (as the rationale for the ERB decision): whether this was the original discussion or a revisiting, post-WG brouhaha, I do not know. If it was post-brouhaha then I submit my rebuttal, but I will sit down and shut up. If that was the original decision, and it was not revisited, I ask that it be re-examined. I'm sorry for not being clearer about what I was asking in my first post. >In all the cases I know of where SDATA entities now work without >user intervention, the SDATA entities are known characters which >occur in Unicode. In cases where non-Unicode characters are >successfully handled, all the cases I know of involve user >intervention in the application's character tables. Removing SDATA >entities does not change this state of affairs one whit: known >characters will work, and others will require user intervention. Yes, but unknown characters labelled with strings may be user-intervenable, even without a private arrangement between author and reader: a very common case on the Web. Private use characters can _never_ work without an explicit agreement between author and reader. I think this is not a new argument, but it still seems knock-down/drag-out to me. One works better, and the syntactic cost is almost, but not quite, nil. >The problem of characters not known to an application is a real and >serious one. I'd like to see some discussion of how to solve it, >rather than further special pleading for SDATA, which seems to me to >be a band-aid for the problem, not a solution to it. We need a real >solution, not a band-aid, and I am not convinced that SDATA is >even a part of a real solution. Special pleading is a kind of loaded term. I'm arguing that we lay a foundation for the general solution to be built, in the way that is most compatible with SGML practice (and to some extent HTML practice, in that people expect that "funny characters" are represetned by strings starting with an ampersand). Certainly some method of associating string data with undefined characters must be part of any real solution: entities are the logical kind of SGML construct to associate it with. We could decide that much without having to develop the technology to run glyph registries right now. OK. I'm sitting down now. -- David David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ Date: Tue, 10 Dec 1996 09:53:46 -0800 From: jon.bosak@Eng (Jon Bosak) To: w3c-sgml-wg@w3.org CC: jon.bosak@Eng Subject: SDATA, again [Michael Sperberg-McQueen:] | The problem of characters not known to an application is a real and | serious one. I'd like to see some discussion of how to solve it, | rather than further special pleading for SDATA, which seems to me to | be a band-aid for the problem, not a solution to it. We need a real | solution, not a band-aid, and I am not convinced that SDATA is even a | part of a real solution. Agreed, but I hasten to point out that this W3C working group is not the place to develop such a solution. People who are serious about helping to fix this are urged to join the appropriate working group within their national standards body, which in the U.S. is ANSI X3V1. I'm sure that your participation would be happily accepted. Jon Date: Tue, 10 Dec 1996 14:24:49 -0800 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org Subject: XML in Seybold See http://www.media.sbexpos.com/OldHotStories/961210.htm (Thanks to Robin Cover for bringing this to my attention.) Jon Date: Tue, 10 Dec 1996 15:52:22 -0800 (PST) From: Terry Allen To: jon.bosak@Eng, w3c-sgml-wg@w3.org Subject: Re: SDATA, again Jon Bosak writes: | [Michael Sperberg-McQueen:] | | The problem of characters not known to an application is a real and | | serious one. I'd like to see some discussion of how to solve it, | | rather than further special pleading for SDATA, which seems to me to | | be a band-aid for the problem, not a solution to it. We need a real | | solution, not a band-aid, and I am not convinced that SDATA is even a | | part of a real solution. | Agreed, but I hasten to point out that this W3C working group is not | the place to develop such a solution. People who are serious about | helping to fix this are urged to join the appropriate working group | within their national standards body, which in the U.S. is ANSI X3V1. | I'm sure that your participation would be happily accepted. As the XML spec says in relevant part: Users may extend the ISO 10646 character repertoire, in the rare cases where this is necessary, by making use of the private use areas. (admitting that this is at times necessary), and as the Unicode Standard 2.0 says flatly that it defines no way to do so, I think that Jon's response is inadequate. XML must either define a way to use the private use area(s?), or say that a future version of the spec will do so, or say that users must make private agreements to do so. Or rule out use of the private use area for the Internet (which is effectively the case now). And if it is truly contemplated that the private use area (rather than SDATA entities) are to be used for the purpose under discussion, doesn't the EBNF need to reflect that? And is the spec still available at textuality.com? Last time I looked that version was more up to date than the one at the W3C, yet now I can't find it. [.sig spelling correction thanks to Ken Holman] Regards, Terry Allen Fujitsu Software Corp. tallen@fsc.fujitsu.com "In going on with these experiments, how many pretty systems do we build, which we soon find outselves obliged to destroy?" - Benjamin Franklin A Davenport Group Sponsor: http://www.ora.com/davenport/index.html Date: Tue, 10 Dec 96 18:26:22 CST From: Michael Sperberg-McQueen Subject: Re: SDATA, again To: W3C SGML Working Group On Tue, 10 Dec 1996 19:23:25 -0500 Terry Allen said: >As the XML spec says in relevant part: > > Users may extend the ISO 10646 character repertoire, in the rare cases > where this is necessary, by making use of the private use areas. > > ..., and as the Unicode Standard 2.0 says flatly that it defines no >way to do so, I think that ... XML must either define a way to use >the private use area(s?), or say that a future version of the spec >will do so, or say that users must make private agreements to do so. >Or rule out use of the private use area for the Internet (which is >effectively the case now). When the ERB decided not to include SDATA entities in XML 1.0, it placed the topic of non-Unicode characters, glyph identification, and documenting (or making) private agreements for character-set handling on the list of problems to be addressed in future revisions. A posting of 19 October 1996 put it this way: Techniques for dealing with non-Unicode characters, specification of glyphs rather than characters, and related topics (such as possible mechanisms for document private agreements governing the ISO 10646 Private Use Areas) will be addressed in future revisions. I think Jon meant simply that a fully worked out proposal is not our prime business, and could usefully be addressed elsewhere. I hope the SGML Open group Lee Quin is heading will give us something good to work from or adopt. The TEI is another place where discussions of this type might find a home, in the guise of discussing whether and how to revise the TEI Writing System Declaration. I would have no objection to adding a sentence to the spec to observe that using the private use area successfully requires out-of-band agreements between sender and recipient of files, or between user and software. But I thought that was pretty much clear from the start. I also have no objection to publishing, as a separate document, the list of topics we said we want to come back to in version 1.n or 2.0, though I don't think time-dependent information like that belongs in a specification. >And if it is truly contemplated that the private use area (rather than >SDATA entities) are to be used for the purpose under discussion, doesn't >the EBNF need to reflect that? I believe it does -- unless you mean that you think the characters in the private-use area should be allowed in generic identifiers. The EBNF doesn't reflect that, because I believe there is some consensus that private-use characters should be data, not markup. -C. M. Sperberg-McQueen Date: Tue, 10 Dec 1996 17:06:02 -0800 (PST) From: Terry Allen To: U35395@UICVM.UIC.EDU, w3c-sgml-wg@w3.org Subject: Re: SDATA, again Michael writes: | When the ERB decided not to include SDATA entities in XML 1.0, it | placed the topic of non-Unicode characters, glyph identification, | and documenting (or making) private agreements for character-set | handling on the list of problems to be addressed in future revisions. Fair enough. | I think Jon meant simply that a fully worked out proposal is not our | prime business, and could usefully be addressed elsewhere. I hope Perhaps, but what he said is that it's ANSI's business, and the existing Unicode spec does not inspire confidence that they consider it so. | the SGML Open group Lee Quin is heading will give us something good | to work from or adopt. The TEI is another place where discussions of | this type might find a home, in the guise of discussing whether and | how to revise the TEI Writing System Declaration. Okay. | I would have no objection to adding a sentence to the spec to observe | that using the private use area successfully requires out-of-band | agreements between sender and recipient of files, or between user and | software. But I thought that was pretty much clear from the start. Only if you have read the Unicode spec; the (ahem) naive reader of XML will assume that some mechanism already exists. | I also have no objection to publishing, as a separate document, the list | of topics we said we want to come back to in version 1.n or 2.0, though | I don't think time-dependent information like that belongs in a | specification. Surely a list of what is deferred to future consideration is appropriate? It's not properly time-dependent but version-dependent (except in the sense that all things are time-dependent). But so far as I can see, this is the only point on which I'd like such a comment. | >And if it is truly contemplated that the private use area (rather than | >SDATA entities) are to be used for the purpose under discussion, doesn't | >the EBNF need to reflect that? | | I believe it does -- unless you mean that you think the characters in | the private-use area should be allowed in generic identifiers. The Goodness no. | EBNF doesn't reflect that, because I believe there is some consensus | that private-use characters should be data, not markup. I'm tempted to ask, which private-use characters? But won't. Regards, Terry Allen Fujitsu Software Corp. tallen@fsc.fujitsu.com "In going on with these experiments, how many pretty systems do we build, which we soon find outselves obliged to destroy?" - Benjamin Franklin A Davenport Group Sponsor: http://www.ora.com/davenport/index.html Date: Tue, 10 Dec 1996 21:39:59 -0500 To: w3c-sgml-wg@w3.org From: davep@acm.org (Dave Peterson) Subject: Re: SDATA, again Michael Sperberg-McQueen recently said: >On Tue, 10 Dec 1996 19:23:25 -0500 Terry Allen said: >>And if it is truly contemplated that the private use area (rather than >>SDATA entities) are to be used for the purpose under discussion, doesn't >>the EBNF need to reflect that? > >I believe it does -- unless you mean that you think the characters in >the private-use area should be allowed in generic identifiers. The >EBNF doesn't reflect that, because I believe there is some consensus >that private-use characters should be data, not markup. If you use the private use area, then either XML must prespecify the abstract character represented by the encoding, which is impossible by definition as I see it, or XML must make these non-SGML character numbers, which means they cannot be in the document directly but only via numeric character references, or XML must permit users to change (extend) the document character set. You pays your money and you takes your choice; I don't think you want any of the three. ;-| Dave Peterson SGMLWorks! davep@acm.org Date: Wed, 11 Dec 1996 21:15:55 +1100 From: Rick Jelliffe To: "David G. Durand" CC: w3c-sgml-wg@w3.org Subject: SPREAD public entity set for ISO 10646 (Re: SDATA, again) David G. Durand wrote: > In concrete, I propose that we allow SDATA entities, and define that > they are _only_ to be used to represent undefined characters by a > descriptive string. > I would also invite some of the new list members that I know have character > set expertise to comment on the alternative of informally assigned > numerical codes versus informally assigned character strings. Some members of the list probably are not aware of the SPREAD public entity sets, which provide a standard mecanism for encoding ISO 10646 characters. I am just (now) finishing the Unicode 2.0 version. The Unicode 1.1 version has been available for ten months. This entity set is unusual in that it has been examined by East Asian academics, standards and industry people, and endorsed by the CJK DOCP, which has a liason to ISO WG8. I hope that SPREAD, or a version of it, can be included in XML. In order to make sure there is no confusion about what SPREAD actually proposes as far as names and numbers, here are some excepts from the entity set. The first is the hub entity: the SPREAD entities are split into blocks, so that only those needed can be loaded. The identifiers of the entities use the (hex) code number of the Unicode 2.0 set: anyone who has a Unicode 1.1, Unicode 2.0, ISO 10646 book, or the Unicode 2.0 CDROM (I think), can look up the characters. The SDATA value of the entities usually is the ISO 10646 name, except for Han ideograms and Hangul syllables, which just use the recommended Unicode short form. === %arabic; %armenian; %bengali; %bopomofo; %cjk-misc; %cyrillic; %devanagari; %diacrits; %dingbat; %georgian; %greek; %gujarati; %gurmukhi; %han; %hangul; %hebrew; %hiragana; %ipa; %jamo; %kannada; %katakana; %lao; %latin; %malayam; %oriya; %punctuation; %symbol; %tamil; %telugu; %thai; %tibetan; === Next, here is the SPREAD public entity set for Han ideograms. These have no useful romanised names, so the SDATA value just copies their number. === !-- SPREAD - Standardisation Project Regarding East Asian Documents Public Entity Set for Han, 1 November, 1996 --> Standard affixes include: JP Japan KR Korea CN Simplified Chinese (PRC) TW Traditional Chinese (Taiwan) HK Hong Kong SG Singaporean VT Vietnamese If you define entities using this convention, please be careful to put them in their own entity set, so that they are not confused with the SPREAD entities proper. --> === Finally, here is are the SDATA entities for a block for which there are sensible names. === === I hope this clarifies exactly what the SPREAD entities provide, and what XML should provide to support them. They should be used whenever using numeric character references (to the presumed ISO 10646 character set) are not appropriate, I guess. I hope SGML Open vendors will think about providing the SPREAD entities as part of their product distribution support material. In particular, they may provide a convenient way to make 8-bit products useful to Asian and Middle Eastern markets. Personally, I think there are great advantages of having the SPREAD entities as a fallback mechanism. Not the least is that the identifier only is five characters. And you don't need to resort to hashing or multistage lookups to go from identifier to Unicode value. -- Regards Rick Jelliffe email: ricko@allette.com.au _______________________________________________________________ Allette Systems (Australia) email: info@allette.com.au Level 10, 91 York Street www: http://www.allette.com.au Sydney 2000 NSW Australia phone: +61 2 262 4777 fax: +61 2 262 4774 _______________________________________________________________ From: dgd@cs.bu.edu (David Durand) Date: Wed, 11 Dec 1996 09:42:57 -0500 (EST) To: dgd@cs.bu.edu, ricko@allette.com.au Subject: Re: SPREAD public entity set for ISO 10646 (Re: SDATA, again) Cc: w3c-sgml-wg@w3.org Rick's comment reminds me that since we do not have SDATA we have a few follow on work items: + add private use characters to data (Thanks Terry!) + document that the ISO public entity sets (and SPREAD) cannot be used with XML, or redefine equivalent entitie sets using the proper combination of Unicode defined and private-use characters. + Check the status of HTML entities. I think all the original HTML use numeric character references and will work, right? What about the new HTML 3.2 entities? Sorry, I don't have the reference handy. -- David (whose logorrhea is overwhelming his promise to shut up). Date: Wed, 11 Dec 96 13:08:27 CST From: Michael Sperberg-McQueen Subject: simplifying comments in SGML '97 To: W3C SGML Working Group At SGML '96, there was a fair amount of discussion of aspects of 8879 which had forced XML into uncomfortable (or in some views unnatural) positions on this issue or that, and some members of WG8 have indicated an interest in issuing a technical corrigendum to the standard, to change those aspects of SGML, at least in the cases where it's clear and widely agreed what should change and how. In discussing this possibility with members of WG8, the ERB has encountered a question on which we need the guidance of the WG, namely, what to do about the syntax of comments in SGML. The XML spec of 14 November defines as a 'comment' what in 8879 is called a 'comment declaration' -- to reduce confusion, in what follows I am going to try to use the 8879 terminology, not the XML terminology. The current version of the XML spec differs from 8879 in various ways: (a) XML allows comments *only* in comment declarations; 8879 allows them in other markup declarations as well (though not in all locations) (b) XML allows exactly one comment in a comment declaration; 8879 allows zero or more (c) XML defines two delimiters '' which bound the construct in question; 8879 sees three delimiter roles involved here, and allows white space in some locations: XML '' corresponds to 'com, s*, mdc' (again, productions 91-92) N.B. I think this list is complete but may be wrong. The main perceived problem with the XML spec is that the comment itself is barred not only from containing '-->' (its closing delimiter) but '--'. Changing the SGML com delimiter from '--' to something else (e.g. ';;') would change the forbidden string to a less frequently encountered one, and was considered, but it would not eliminate the apparent irrationality and was felt in any case to be unwise. A technical corrigendum to 8879 would ideally eliminate this problem and make it possible for '--' to appear within XML-style comment declarations; it might also make it possible for SGML parsers to enforce XML's rules on (a) number of comments within the comment declaration and (b) absence of white space before the closing mdc. So far so good. The problem is that there seem to be at least two ways of approaching the problem, and it's not clear which is preferable. Your opinions, please. A. The Simple Comment This proposal adds a SIMPLEC (simple-comment) optional feature to the SGML declaration. If SIMPLEC NO is declared, comments behave as they do now. If SIMPLEC YES is declared, comments have an alternative definition. The relevant clause might read like this (thanks to Dave Peterson for the draftsmanship). I have marked additions in ... and substitutions in ...: 10.3 Comment declaration [91] comment declaration = normal comment declaration | simple comment declaration [91a] normal comment declaration = mdo, (comment, (s | comment)*)?, mdc [92] comment = como, SGML character*, comc [92a] simple comment declaration = mdo, com, SGML character*, com, mdc No markup is recognized in a comment, other than the com delimiter that terminates it. No markup is recognized in a simple comment declaration other than the com delimiter immediately followed by an mdc delimiter that terminates it. NOTES 1. A com delimiter not followed by an mdc delimiter will be recognized in a comment (in a comment declaration or other declaration) but not in a simple comment declaration. 2. The SGML declaration specifies whether normal or simple comment declarations are used in a document. No document may use both. Advantages: captures all of XML's rules except the prohibition on comments in other markup declarations. Disadvantages: not clear whether the precise mix of simplifications undertaken by XML is of general enough interest / use to warrant this approach: would other application profiles prefer to impose different rules on comments? B. Splitting the com delimiter. This proposal simply replaces the 8879 com delimiter with a pair of delimiters, como and comc (comment open and comment close). Documents using 8879:1986 syntax have como = comc = com; in the RCS that's como = comc = '--'. XML could allow -- within comments by setting como to '--*' and comc to '*--', to retain the general look and feel of current comments, and still allow '--' in the comment itself. Nested comments might also become possible, in SGML (and then in XML), or not. Production 91 of 8879 could remaing the same as it now is; 92 would change. We might have: 91 comment declaration = mdo, (comment, (s | comment)*)?, mdc 92 comment = como, SGML character*, comc or (to allow nested comments) 92 comment = como, (SGML character | comment)*, comc Note: If this is what we propose to WG8, the XML spec should probably change *now* to use these delimiters, replacing production 21 with [21] Comment ::= '' Advantages: this seems relatively simple and relatively compatible with the look and feel of 8879 as a whole. Disadvantages: it doesn't allow the SGML parser to enforce XML's rules. What does the SGML Work Group think about this problem? -C. M. Sperberg-McQueen Date: Wed, 11 Dec 1996 12:05:51 -0800 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: ERB discussion of public identifiers We spent a lot of time on this question on Dec. 11th, and it is clear we need some more help from the WG. The ERB is, by at least a substantial majority, convinced that there is a real nead for PUBLIC identifiers in XML. The ERB is highly concerned, in at least a significant minority, about the effects of putting this facility in without specifying a resolution mechanism. Doing so would contravene one of the major design goals of XML - that any compliant XML processor should be able to read any compliant XML document. On the other hand, there is also substantial concern about giving an unconditional blessing to any particular name resolution mechanism at this point in history. Thus, there are a variety of options open to us. 1. Leave it as it is 2. Agree that we'll put PUBLIC identifiers into XML *when* we are ready to specify the resolution mechanism; the practical effect is almost certainly that they don't go in for now. 3. Put a slot in the syntax for PUBLIC identifiers... 3* ...and require an accompanying SYSTEM identifier 3a - say nothing about resolution mechanisms, perhaps providing a taxonomy of available technologies in this area 3b - document one resolution mechanism, but not make it required. This is somewhat similar to our i18n approach, where we bless 2 encodings but admit the possibility that people use others. 3c - document one resolution mechanism as before, but make its use mandatory for XML processors. Note that there is a continuum between 3b and 3c; we could place varying strengths of recommendation behind one resolution mechanism, with homilies about document portability. In the area of which resolution mechanism to (perhaps nonexclusively) bless, SGML/Open catalogs (hereinafter Socats) stand out, and would probably be the ERB's choice. On another hand, there has been a lot of work go into the URN effort; on another hand, that work has not yet born practical fruit in terms of ubiquitous implementations; on another hand, the FPI syntax is repellent to some and it is not clear how well it supports internationalization; on another hand, it may be the case that FPI's really are URN's as they stand. I suspect that if a binding vote were taken today, the ERB would either (a) reinstate the PUBLIC keyword, and put in a nonexclusive recommendation for Socat support, or (b) refuse to put PUBLIC in until there was agreement on a required resolution mechanism. Input, please. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 Date: Wed, 11 Dec 1996 12:01:33 -0800 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: RS/RE, again (sorry) On September 11th, the ERB spent considerable time revisiting the RS/RE question. We're just a fun-loving bunch, and it became apparent that the WG is going to have to put some more thought into this. At the moment, we have the -XML-SPACE mechanism, which toggles two behaviors: collapsing of leading and multiple spaces (essentially the HTML semantic), and passing through all the bytes to the application. Some feel the mechanism is unaesthetic and prone to misinterpretation as it stands, and should simply be discarded, with all non-markup bytes being passed to the application to do with as it will. This can be made 8879-compliant in the short term via a mechanism proposed by Charles Goldfarb, and in the medium/long term with a TC via WG8. There are some problems with both the current and revised approaches: o -XML-SPACE, although this is not documented, really only deals with mixed content; many feel it's important to ignore white space in element content; .. .. ^^ e.g. the above but XML, when there's no DTD, doesn't know where element content is and *cannot* be made to do this. o SGML's world-view tripartions the set of characters: those that are text, those that are markup, and those that are insignificant white space. Can XML really afford to discard this distinction? o Many real-world editors, largely to deal with the fact that text (whether or not we like it) is stored in files in what amounts to a series of records, freely insert line breaks and other white space because they know SGML processors will ignore it. Can we afford to make that white space significant? o Some applications, e.g. full-text indexers, really need to know where everything is by byte offset, whether or not the bytes are significant; thus the -XML-SPACE="COLLAPSE" behavior means they can't read the text with an XML processor (unless they can turn off -XML-SPACE processing through the API) So there are a few things we could do, which are not entirely mutually exclusive. 1. Go to the RE Delenda Est model. This has the advantage that it's trivially easy to explain, document, and implement. It has some of the disadvantages listed above; there is some very strong sentiment on the ERB against this - look from a follow-up from other ERB-folk. 2. Expand the -XML-SPACE attribute from two values to three. The third would be named REMOVE or DISCARD or something, and would be designed to signal element content, i.e. all this white space can be safely ignored. 3. Add language to the spec allowing the application to force the processor to pass through all the bytes regardless of the -XML-SPACE setting. Your input is requested. Cheers, Tim Bray tbray@textuality.com http://www.textuality.com/ +1-604-488-1167 From: lee@sq.com Date: Wed, 11 Dec 96 15:33:47 EST To: w3c-sgml-wg@w3.org, U35395@UICVM.UIC.EDU Subject: Re: simplifying comments in SGML '97 I have always used --*......*-- for SGML comments in the hope that one day it would be possible to make these be comment start & end delimiters, and I would very much like to see this done for XML. I think that XML would not be harder to parse if the rule was that a comment could appear anywhere S could appear; this is very close to the SGML rules, with the exception that there are some places (e.g. within a content model) where 8879 forgot (I assume) to allow comments. This approach encourages (but does not enforce) an implementation in which comments are stripped at the tokeniser level. At any rate, I favour the separate delimiters. Lee From: Ken Holman To: "'w3c-sgml-wg@w3.org'" Subject: RE: ERB discussion of public identifiers Date: Wed, 11 Dec 1996 16:19:00 -0500 >---------- >From: Tim Bray[SMTP:tbray@textuality.com] >Sent: Wednesday, December 11, 1996 15:05 >To: w3c-sgml-wg@w3.org >Subject: ERB discussion of public identifiers > ... > >I suspect that if a binding vote were taken today, the ERB would either >(a) reinstate the PUBLIC keyword, and put in a nonexclusive >recommendation > for Socat support, or >(b) refuse to put PUBLIC in until there was agreement on a required > resolution mechanism. > >Input, please. Personally, (a) would satisfy me. To mitigate implementers' concerns about how complex "Socat support" might be, the straw-man proposal posted 1996-12-02 describes (I hope) a syntax compatible with the complete SGML Open Catalog definition, with only the single keyword "PUBLIC" supported. This would allow implementers to work with a supplied Socat without modification, ignoring any specification of function other than the basic PUBLIC indirection. Implementers could then choose to support more than the basic functionality at their leisure ... there would be no fear of complexity that might be there if XML just "pointed" to the Socat spec. ......... Ken -- G. Ken Holman Tel: +1 613 596-2233 Chief Technology Officer Fax: +1 613 596-5934 Microstar Software Ltd. WATS: 1 800 267-9975 3775 Richmond Road Mail: gkholman@microstar.com Nepean Ontario Info: info@microstar.com CANADA K2H-5B7 Web: http://www.microstar.com -- 1605 Mardick Court, Box 266 Phone: +1 613 489-2987 Kars, Ontario CANADA K0A-2E0 E-mail: gkholman@CanadaMail.com -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6 mQCNAjHOimAAAAEEAK47HbDtZZB8hJBk+r9Zh7m7QxdFHTaz/m200QQ7J9XX4QYs h6hsuP6ZqBJXyLdmIVMEwR6Ry6oxjKMd6BRJ8OGScD89eIghgbrpMX+900NxM0x2 v3yWO9ki2gKRPrn4vlCXznnmcmsyke0G02T2LefXbgZHyVSqDSOLy8nwuN7dAAUR tClHLiBLZW4gSG9sbWFuIDxhYjk0NUBmcmVlbmV0LmNhcmxldG9uLmNhPg== =IN3T -----END PGP PUBLIC KEY BLOCK----- From: lee@sq.com Date: Wed, 11 Dec 96 16:26:45 EST To: w3c-sgml-wg@w3.org, tbray@textuality.com Subject: Re: RS/RE, again (sorry) It seems to me that the best thing, if it is possible, would be * the XML parser can pass all whitespace back to the application * the distinction between data and whitespace is retained, so that it is not an error to have whitespace in element content, even though that whitespace is passed back to the application. * RS and RE and space and tab and the places where comments were before they were elided are all whitespace It's important to be careful about what "significant" means. [1] whether the parser discards it [2] whether it is treated as #PCDATA [3] whether it is recognised as S in a production I do not suggest making all whitespace be PCDATA and losing the distinctioon between whitespace and data. Hence, I favour 3. Add language to the spec allowing the application to force the processor to pass through all [whitespace] regardless of the -XML-SPACE setting. in conjunction with removing the -XML-SPACE setting. Lee Date: Wed, 11 Dec 1996 16:33:27 -0500 To: Tim Bray , w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: RS/RE, again (sorry) At 12:01 PM 12/11/96 -0800, Tim Bray wrote: >o Some applications, e.g. full-text indexers, really need to know where > everything is by byte offset, whether or not the bytes are significant; > thus the -XML-SPACE="COLLAPSE" behavior means they can't read the text > with an XML processor (unless they can turn off -XML-SPACE processing > through the API) I don't really think that this is difficult to solve. How is this situation any different than that of ? Just as an application may optionally ask the parser to return comments, it should be able to ask for insignificant whitespace. Paul Prescod Date: Wed, 11 Dec 1996 17:23:07 -0500 From: gtn@ebt.com (Gavin Nicol) To: lee@sq.com CC: w3c-sgml-wg@w3.org, tbray@textuality.com Subject: Re: RS/RE, again (sorry) Very simple summary of my position: 1) Pass back all whitespace. 2) Have the application layers(s) decide what to remove, and what not to remove. A validation layer could be 8879 conformant if so desired. Date: Wed, 11 Dec 1996 17:30:49 -0500 To: Tim Bray , w3c-sgml-wg@w3.org From: Paul Prescod Subject: RS/RE Delenda Est compromise? At 12:01 PM 12/11/96 -0800, Tim Bray wrote: >1. Go to the RE Delenda Est model. This has the advantage that it's > trivially easy to explain, document, and implement. It has some of the > disadvantages listed above; there is some very strong sentiment on the > ERB against this - look from a follow-up from other ERB-folk. Here is why I am against it: as long as we are going to invalidate most existing SGML documents, I would really like to introduce a syntax that will allow people to put back in their pretty-printing-whitespace. I'll elucidate in a moment. >2. Expand the -XML-SPACE attribute from two values to three. The third would > be named REMOVE or DISCARD or something, and would be designed to signal > element content, i.e. all this white space can be safely ignored. Ugh. Can't we find a way to use a nice syntax for this instead?

Significant whitespace

}/TR> }/TABLE> Conceptually this is just a small change from the RE Delenda Est model. It introduces a mechanism for explicitly allowing insignificant whitespace. But it does not use a cumbersome, non-intuitive attribute syntax. The new syntax would have to be introduced through an SGML TC or more SGML declaration wizardry (is there any more magic left in that hat?). Those that want the "strict" RE Delenda Est behaviour can just use the old SGML STAGC syntax and leave out the insignificant whitespace. >3. Add language to the spec allowing the application to force the > processor to pass through all the bytes regardless of the -XML-SPACE > setting. This can be handled as comments are handled. Paul Prescod From: lee@sq.com Date: Wed, 11 Dec 96 17:40:53 EST To: w3c-sgml-wg@w3.org, tbray@textuality.com Subject: Re: ERB discussion of public identifiers > 2. Agree that we'll put PUBLIC identifiers into XML *when* we are ready > to specify the resolution mechanism; the practical effect is almost > certainly that they don't go in for now. I think this is sensible. > 3. Put a slot in the syntax for PUBLIC identifiers... > 3* ...and require an accompanying SYSTEM identifier If you don't specify a resolution mechanism, the SYSTEM identifier will certainly be required. > 3a - say nothing about resolution mechanisms, perhaps providing a > taxonomy of available technologies in this area I think this is a cop-out: "XML has a feature which is not documented in the spec, but here is the telephone number of the ERB members so you can ask how to implement XML properly" reminds me of another markup definition language.... :-) If you expect programmers to write code, the functionality of that code must be in the spec. There isn't anywhere else for it, unless you can include a normative reference to another text (e.g. to the X500 specification for ISO directory services, which obviously (?) should be preferred in an ISO/SGML context over the IETF URN work :-) There is no use (as far as I can see) for PUBLIC "string" being in the XML spec if XML systems all ignore it. It just clutters the language (like the - - omissible tag parameters, or ""). A restricted subset of CATALOG, together with a documented set of mechanisms for resolving the catalog itself, might work. The mechanism for finding a remote catalog over ftp or http must work in the presence of CGI scripts and URLs with / in a query. I can post details of what SoftQuad Panorama does if it will help. It can all be described in under a page, I think, taking us to 28pp, although we're still some 50% over the 20 page goal. > 3b - document one resolution mechanism, but not make it required. > This is somewhat similar to our i18n approach, where we bless 2 > encodings but admit the possibility that people use others. The difficulty with this is that URNs aren't ready yet, and SOcats only defer the problem: you then have to say how to get the catalog. On the other hand, we are going to have the same problem with associating style sheets with documents, and with following hypertext links based on public identifiers for documents. > 3c - document one resolution mechanism as before, but make its use > mandatory for XML processors. I don't think this will fly, ,as we don't have a single mechanism that satisfies everyone's needs yet, I think. I would still favour omitting PUBLIC for XML 1.0. The concerns that have been voiced all seem to relate to using SGML in an SGML context; it's not clear to me that the same concerns and implementation constraints apply to XML. There has been no clear statement in support of PUBLIC saying what functionality is needed and why. Allowing PUBLIC "xxx" to be ignored in XML has (as I see it) only a single advantage -- it is then possible to have an XML file that is also a legal HTML file, as long as it doesn't use any EMPTY elements or entities. It doesn't help you use existing SGML tools on your XML, as unmodified SGML tools don't actually work with XML (they don't interpret the charset processing instruction, for example, and have problems with the NET tag). Yuo have to modify your SGML files to use them with XML, althouth the transformatioon is automatic and simple. That modification can remove PUBLIC. You have to modify XML files to use them with old SGML tools. That modification can add PUBLIC. Lee Date: Wed, 11 Dec 1996 17:35:41 -0500 To: Tim Bray From: "Keith M. Corbett" Subject: Re: ERB discussion of public identifiers Cc: w3c-sgml-wg@w3.org At 12:05 PM 12/11/96 -0800, Tim Bray wrote: >Input, please. Sorry for the long-winded response... I would like to see a 3b type spec for PUBLIC, which I characterize as: hand-wave about name resolution schemes, and allow a range of implementations consistent with common Web and SGML practices. I would like to see some recognition of catalogs, FPIs, and URNs, without requiring them. I'm swayed by all the arguments for allowing PUBLIC in XML. I see no compelling argument against full support per 8879-1, if we allow systems to assume "FORMAL NO" by default in the SGML declaration. This only requires a validating system to report an error if a PUBLIC ID is not a minimum literal. Authors and processing systems are free to interpret IDs as they see fit. On further reflection I believe that, in the interest of fostering the deployment of name resolution schemes for the Web, the XML spec should *recommend* that URL syntax be used for SYSTEM IDs, and URN syntax for PUBLIC IDs. The common naming schemes are relatively easy to explain, and parsing IDs is trivial. The message to implementors is forward looking: XML (the language) does not require recognition or handling of URNs and FPIs, but *does* encourage their use. The message to Web publishers is very positive: XML supports a useful feature for interchange with name resolution systems, and XML will scale better than HTML as industrial strength naming schemes come online. To make this work the spec should provide some concrete examples. The example should be placed in context with real world problems, and should employ naming schemes that are known to work (URL, FPI, ISBN). So we may not offer a formal specification of PUBLIC and SYSTEM IDs for the Web, but we can enable everyone in the Web community to recognize them when they see them. This approach strikes me as a reasonable balance among realism, political correctness, and functionality. It recognizes an important mechanism for name resolution, but does not impose undue hardship on implementors. No authors are forced to use them; no implementors are forced to parse them. And of course customers and vendors will work things out in the marketplace. /kmc Date: 11 Dec 1996 22:38:21 +0000 (GMT) From: Peter Flynn Subject: Re: ERB discussion of public identifiers Tim Bray on 11 Dec 1996 12:05:51 -0800) To: w3c-sgml-wg@w3.org 3. Put a slot in the syntax for PUBLIC identifiers... 3* ...and require an accompanying SYSTEM identifier As a list newcomer, I wasn't party to the previous debate on this, but whichever way it goes, I feel it is essential to specify what the order of acceptance is. The current position in SGML is a serious problem for users and implementors, where one application will use the SYSTEM identifier if it is there, and ignore the PUBLIC identifier even if it cannot locate the resource referred to by the SYSTEM identifier; and another application takes the PUBLIC identifer first and ignores the SYSTEM identifer even if it cannot resolve the PUBLIC identifier; and yet others work only with PUBLIC identifers or SYSTEM identifers but not both. ///Peter Date: Wed, 11 Dec 1996 17:57:36 -0400 To: w3c-sgml-wg@w3.org From: murray@spyglass.com (Murray Altheim) Subject: XML and HTML Interoperability "XML and HTML HTML and XML Live together in per-fect Har-mo-neeee..." (sung to the tune of "Ebony and Ivory") An unnamed browser company wishes to understand the ramifications of XML on the Web, so I thought this might be of interest to this group. If not, please tune out this thread and sorry for its intrusion. I hope to hash out some real answers for the HTML community, as well as my company. This document is also posted online, where it's a bit easier to read (markup! yes!): http://www.cm.spyglass.com/doc/spec/htmlxml.html Obviously, how we answer numbers 1, 4, and 5 below may have a great deal of impact on MS, Netscape, Spyglass, and others, as well as how XML is accepted in the greater Web community. The other questions are equally important to different demographics. While I realize some of this has yet to be decided, I thought I'd get a running start and ask (and therefore try to answer) some questions. People wanna know: 1. What happens when an XML document meets an HTML browser? 2. What happens when an HTML document meets an XML browser? 3. What changes must one make to an HTML document to make it compatible with an XML browser? 4. What changes must one make to an XML document to make it compatible with an HTML browser? 5. What changes must an HTML browser developer make to their product to allow it to correctly parse XML? 6. What changes must an XML browser developer make to their product to allow it to correctly parse valid HTML? 7. What changes must an XML browser developer make to their product to allow it to correctly parse invalid HTML? I'll try to break this down by question: 1. XML documents' behavior in an HTML browser a. the '.xml' extension is not understood, so the HTML browser tries to download it. The user gets to read his XML in emacs. b. the '.xml' extension is understood as text/plain, so the user gets to read his XML in Netscape. Whoopee. c. the HTML browser tries to parse it as HTML. All sorts of strange stuff appears on the screen (what is this "" thingie? why don't any of the images appear, and I see all these "" (or even "") tags?) 2. HTML documents' behavior in an XML browser a. the HTML document is not valid, nor even well-formed. The XML browser dies a thousand violent deaths. (I've seen this happen. Not pretty.) b. the HTML document is either well-formed or even (!) valid. But it is strictly HTML, so the XML browser hits the first IMG tag and just keeps looking for that elusive . The user goes to bed wondering. 3. HTML document compatibility with an XML browser a. This is currently not possible without modification to the declaration and HTML DTD. Now, if one is willing and able to make these changes, we might be able to play. In the SGML declaration, declare the null end tag NET="/>". In the DTD, disallow minimization rules -- require end tags for all elements. For standardized DTDs like DocBook and HTML in wide use, this might be a problem. Also, unquoted attribute values are disallowed. b. Another option, modifying the DTD to include no empty element declarations, wouldn't work, as the installed document base prohibits such a change. 4. XML document compatibility with an HTML browser a. Current HTML browser will produce noise on PIs (and not process the PIs), not handle the modified NET properly, and due to the arbitrary nature of the markup, produce unpredictable results. b. If the XML document is really an HTML document in an XML wrapper (see #3), then it's a matter of modifying the browser as in #5 below. 5. HTML browser parsing XML An HTML browser developer must modify the current HTML parsing code to take into account: a. Both the document character set and encoding are different from HTML. XML uses Unicode and UTF-2/UTF-8 (allowing other encodings), so unless the browser is already i18n-ed, this may be a big problem. b. Processing Instructions I didn't realize we'd also changed PIC for XML. Hmmm. d. Funky End Tag Weirdness d. A currently unspecified hyperlinking mechanism characterized by a different link syntax using IDREFs and IDs (?) e. If we care about validity (as does everybody on the Web), there will be some XML documents that are broken, ie., well formed. (I think we can safely ignore this concern.) f. Marked section handling. Easy: 1. Search for "" All content between #1 and #2 (ignoring whitespace) is the MS keyword. This can be IGNORE or INCLUDE, or an entity that expands to that (With no entity expansion in HTML, it'd better be INCLUDE or IGNORE). If the keyword is not IGNORE, then default to INCLUDE. (If it's CDATA, what to do?) All content between #2 and #3 is the content to be included or ignored. All the rest (including the keyword) is markup to be discarded by the formatter. 6. XML browser parsing valid HTML a. Once the XML browser detects an HTML document, punt. b. ? 6. XML browser parsing invalid HTML a. Once the XML browser detects an HTML document, punt. b. ? ------------ Again, a thousand pardons for the noise this creates, but I think/hope this will be of benefit to all. All comments, criticisms, etc. to me. Comments to me either privately or publicly may/will be incorporated into the online document. If anyone else has already written something like this, please let me know and maybe we can work a deal. I've got ten live chickens in my apartment I might be willing to sell... Murray ``````````````````````````````````````````````````````````````````````````````` Murray Altheim, Program Manager Spyglass, Inc., Cambridge, Massachusetts email: http: "Give a monkey the tools and he'll eventually build a typewriter." Date: Wed, 11 Dec 1996 17:59:57 -0500 To: gtn@ebt.com (Gavin Nicol), lee@sq.com From: "Eve L. Maler" Subject: Re: RS/RE, again (sorry) Cc: w3c-sgml-wg@w3.org, tbray@textuality.com There's one complication to this. If there's no DTD (quite possible), AND no way in the instance (such as with an attribute value) to indicate which is element content and which is mixed content, AND no stylesheet semantic that allows this to be controlled, how can the application layer decide? I'm guessing that DSSSL expects a document to have gone through a classic SGML parser first, which means that element content will have internal whitespace stripped. XML documents might not undergo any such treatment. Somehow, the element-vs.-mixed distinction needs to be made in cases where it's critical for rendering or other processing. Where is the right place to make the distinction, if there's no DTD? Is is the stylesheet? If so, are there any technical barriers to doing this? Eve At 05:23 PM 12/11/96 -0500, Gavin Nicol wrote: >Very simple summary of my position: > >1) Pass back all whitespace. >2) Have the application layers(s) decide what to remove, and what > not to remove. A validation layer could be 8879 conformant if so > desired. Date: Wed, 11 Dec 1996 18:19:55 -0500 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: ERB discussion of public identifiers At 05:40 PM 12/11/96 EST, lee@sq.com wrote: >There is no use (as far as I can see) for PUBLIC "string" being in the >XML spec if XML systems all ignore it. It just clutters the language >(like the - - omissible tag parameters, or ""). XML systems will not "all" ignore it. Some XML systems will ignore it. Just as some XML systems will ignore my organization's or application's processing instructions. Nobody seems to have a problem with processing instructions, so why are public identifiers any more or less dangerous/mysterious/useless? For now, PUBLIC identifiers are "addressing processing instructions." At some point in the future they will be standardized, just as some processing instruction syntaxes are standardized (e.g. Subject: Re: RS/RE, again (sorry) Cc: w3c-sgml-wg@w3.org, tbray@textuality.com At 05:23 PM 12/11/96 -0500, Gavin Nicol wrote: >Very simple summary of my position: > >1) Pass back all whitespace. >2) Have the application layers(s) decide what to remove, and what > not to remove. A validation layer could be 8879 conformant if so > desired. Is this not the same as the current specification? "In editing XML documents, it is often convenient to use "white space" (spaces, tabs, and blank lines, denoted by the nonterminal S in this specification) to set apart the markup for greater readability. Such white space is typically not intended for inclusion in the delivered version of the document. On the other hand, "significant" white space that must be retained in the delivered version is common; for example in poetry or source code. In element content, all white space (S) is ignored; validating XML processors must not pass it to the application. Non-validating processors which do not read the DTD must treat all elements as if they were declared with mixed content; this will in some cases result in a different parse tree >from that produced by processors which do read the DTD." Paul Prescod Date: Wed, 11 Dec 1996 18:19:14 -0400 To: Tim Bray From: murray@spyglass.com (Murray Altheim) Subject: Re: ERB discussion of public identifiers Cc: w3c-sgml-wg@w3.org Tim Bray writes: >We spent a lot of time on this question on Dec. 11th, and it is clear we >need some more help from the WG. > >The ERB is, by at least a substantial majority, convinced that there is a >real nead for PUBLIC identifiers in XML. > >The ERB is highly concerned, in at least a significant minority, about >the effects of putting this facility in without specifying a resolution >mechanism. Doing so would contravene one of the major design goals of >XML - that any compliant XML processor should be able to read any compliant >XML document. This is in contradistinction to other markup specifications. Is XML trying to specify the world around it? Are we specifying the behavior of an XML processor (parser?), or an entire XML environment? Note that while HTML mentions things like URLs and http, implementation details of other protocols, etc. are left to those other specifications. >On the other hand, there is also substantial concern about giving an >unconditional blessing to any particular name resolution mechanism at >this point in history. Given that one is well accepted but changing, the other not ready for prime time just yet. I don't see why there is a need for an "unconditional blessing". This sounds like a red herring. Merely describing a solution as *one option* for resolution without restricting others seems like the path of least resistance. >Thus, there are a variety of options open to us. > >1. Leave it as it is This disallows a feature that *seems* to be demanded by many potential users of XML, forcing them to continue using 8879 unless they are willing to give up use of public text or embed system info in their documents. For example, I imagine an XML-ized HTML DTD available for XML developers might be a good idea. Well, how would that be distributed? A fixed SYSTEM URL? yuck. Not much for standards. >2. Agree that we'll put PUBLIC identifiers into XML *when* we are ready > to specify the resolution mechanism; the practical effect is almost > certainly that they don't go in for now. Again, this begs the question of why the XML spec *needs* to create a complete solution in the first place. Putting off the discussion for later versions seems dangerous. I can see FONT and BLINK on the horizon. >3. Put a slot in the syntax for PUBLIC identifiers... > 3* ...and require an accompanying SYSTEM identifier *Requiring* a SYSTEM identifier is tantamount to requiring authors to hardwire URLs into their documents, which is (at least in my mind) the principal objection to the status quo. > 3a - say nothing about resolution mechanisms, perhaps providing a > taxonomy of available technologies in this area This would be my choice. > 3b - document one resolution mechanism, but not make it required. > This is somewhat similar to our i18n approach, where we bless 2 > encodings but admit the possibility that people use others. This would be fine, but I'm guessing some folks don't want anything to do with putting Socats in the spec, judging by the discussion. I think if we did, it would be best as an appendix. > 3c - document one resolution mechanism as before, but make its use > mandatory for XML processors. This limits the XML specification, as well as shackles development of better approaches in the future. While it may promote interoperability at some level, I suspect there are many who would either break the rule or simply ignore XML. > Note that there is a continuum between 3b and 3c; we could place > varying strengths of recommendation behind one resolution mechanism, > with homilies about document portability. Agreed. >In the area of which resolution mechanism to (perhaps nonexclusively) >bless, SGML/Open catalogs (hereinafter Socats) stand out, and would >probably be the ERB's choice. My choice as well. Particularly if we limit socats to PUBLIC only, disallowing such things as LINKTYPE, SGMLDECL, DTDDECL and the complexities of delegated catalogs. This would reduce it to a simple lookup table, which should be extremely simple to parse for content. >On another hand, there has been a lot >of work go into the URN effort; on another hand, that work has not yet >born practical fruit in terms of ubiquitous implementations; on another >hand, the FPI syntax is repellent to some and it is not clear how well >it supports internationalization; on another hand, it may be the case >that FPI's really are URN's as they stand. These are issues that have yet to be decided. We can't wait until they are, given that timeline on this spec is so short. >I suspect that if a binding vote were taken today, the ERB would either >(a) reinstate the PUBLIC keyword, and put in a nonexclusive recommendation > for Socat support, or My vote. (well, I don't get a vote, but whatever) >(b) refuse to put PUBLIC in until there was agreement on a required > resolution mechanism. As I mentioned above, I think this puts an unnecessary onus on the XML specification to provide a solution, which seems unlike other markup specifications. Murray ``````````````````````````````````````````````````````````````````````````````` Murray Altheim, Program Manager Spyglass, Inc., Cambridge, Massachusetts email: http: "Give a monkey the tools and he'll eventually build a typewriter." Date: Wed, 11 Dec 1996 18:27:52 -0500 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: RS/RE, again (sorry) At 05:59 PM 12/11/96 -0500, Eve L. Maler wrote: Where is >the right place to make the distinction, if there's no DTD? Is is the >stylesheet? If so, are there any technical barriers to doing this? The implication is that you need a different stylesheet (one that ignores whitespace in "element content") if your document is parsed by a non-validating parser (or "just a parser", in Gavin's terminology) vs. a validating parser (or a parser whose output is made 8879 conformant by a post process, in Gavin's terminology). This is, as far as I know, the current situation in XML. I presume we are only discussing the issue because that solution was considered inelegant. I'm presuming that mixed content is no longer a problem. If input newlines are not going to behave like "RE" in mixed content, then those attribute hacks are basically useless. Just pass all of the mixed content whitespace through, all of the time. Paul Prescod From: "Christopher R. Maden" Date: Wed, 11 Dec 1996 23:28:14 GMT To: w3c-sgml-wg@w3.org Subject: Re: XML and HTML Interoperability [Murray Altheim] > 1. XML documents' behavior in an HTML browser > > a. the '.xml' extension is not understood, so the HTML browser tries > to download it. The user gets to read his XML in emacs. > b. the '.xml' extension is understood as text/plain, so the user > gets to read his XML in Netscape. Whoopee. > c. the HTML browser tries to parse it as HTML. All sorts of strange > stuff appears on the screen (what is this "" thingie? why > don't any of the images appear, and I see all these "" (or > even "") tags?) Which happens depends on the browser's configuration. Presumably, the first time encountering text/xml or text/x-xml, the browser will prompt the user. I don't see a. or b. as a problem, since that's a matter of choice for the user. For c., they'll see PIs, but that's really a bug in most browser implementations. The markup (including ) will generally be hidden. They'll see all the content, probably run together in one large paragraph unless the XML happens to include a

element. If one is authoring XML and expecting that HTML browsers may be used with it, one should use a tagset that is a superset of HTML. > 2. HTML documents' behavior in an XML browser > > a. the HTML document is not valid, nor even well-formed. The XML > browser dies a thousand violent deaths. (I've seen this > happen. Not pretty.) If people would validate their HTML, this wouldn't happen. But I would assume that an XML browser wouldn't be used (at least not as such) for text/html; on encountering that, it should act as an HTML user-agent, with the usual error recovery for bogus markup. Even in XML mode, I would expect a user-agent to have reasonable error recovery for dealing with interleaved tags, etc. > b. the HTML document is either well-formed or even (!) valid. But it > is strictly HTML, so the XML browser hits the first IMG tag and > just keeps looking for that elusive . The user goes to bed > wondering. See 2a. for why this shouldn't be an issue, or 3. if it still is. > 3. HTML document compatibility with an XML browser > > a. This is currently not possible without modification to the > declaration and HTML DTD. Now, if one is willing and able to make > these changes, we might be able to play. In the SGML declaration, > declare the null end tag NET="/>". In the DTD, disallow > minimization rules -- require end tags for all elements. For > standardized DTDs like DocBook and HTML in wide use, this might > be a problem. Also, unquoted attribute values are disallowed. Require end-tags for all elements in HTML per Prescod; now valid HTML documents are well-formed XML. No current Web browsers will choke on this (though some insert extra space on
and

). > b. Another option, modifying the DTD to include no empty element > declarations, wouldn't work, as the installed document base > prohibits such a change. Not true - truly valid documents (the minority) declare to which DTD they comply. Ones that weren't valid to begin with don't care what the DTD says. It works in the browser. > 4. XML document compatibility with an HTML browser > > a. Current HTML browser will produce noise on PIs (and not process > the PIs), not handle the modified NET properly, and due to the > arbitrary nature of the markup, produce unpredictable results. They will handle the modified NET (at least most of them will - I haven't actually tried Spyglass's). Anything based on libwww ignores <[^?].*>. > b. If the XML document is really an HTML document in an XML wrapper > (see #3), then it's a matter of modifying the browser as in #5 > below. > 5. HTML browser parsing XML > > An HTML browser developer must modify the current HTML parsing code > to take into account: > > a. Both the document character set and encoding are different from > HTML. XML uses Unicode and UTF-2/UTF-8 (allowing other > encodings), so unless the browser is already i18n-ed, this may be > a big problem. I don't see it as much of a problem. Non-i18n browsers are probably U.S. or European, and will deal with UTF-8 OK. UCS-2 will look bad, but this will be a problem for HTML as well as XML, so I don't see it as an XML issue. > b. Processing Instructions I didn't realize we'd also > changed PIC for XML. Hmmm. Shouldn't change anything for most current Web clients. > c. Funky End Tag Weirdness Not a problem - see above. > d. A currently unspecified hyperlinking mechanism characterized by a > different link syntax using IDREFs and IDs (?) I think hyperlinks are the least of a Web client's worries handling XML. I'd be happy to see the data in a readable way. > e. If we care about validity (as does everybody on the Web), there > will be some XML documents that are broken, ie., well formed. (I > think we can safely ignore this concern.) Did you mean "not well formed"? > f. Marked section handling. Easy: > 1. Search for " 2. Parse forward to next "[" > 3. Parse forward to next "]]>" > All content between #1 and #2 (ignoring whitespace) is the MS > keyword. This can be IGNORE or INCLUDE, or an entity that > expands to that (With no entity expansion in HTML, it'd > better be INCLUDE or IGNORE). If the keyword is not IGNORE, > then default to INCLUDE. (If it's CDATA, what to do?) > All content between #2 and #3 is the content to be included > or ignored. All the rest (including the keyword) is markup to > be discarded by the formatter. Only CDATA marked sections are permitted in the document instance (prod. [19]). Display all marked sections outside . > 6. XML browser parsing valid HTML > > a. Once the XML browser detects an HTML document, punt. My preference for 6. and 7. I don't think a text/xml user agent should attempt to handle text/html. If a single program incorporates a user agent for both types, fine; but keep the distinction. > Again, a thousand pardons for the noise this creates, but I > think/hope this will be of benefit to all. All comments, criticisms, > etc. to me. Comments to me either privately or publicly may/will be > incorporated into the online document. If anyone else has already > written something like this, please let me know and maybe we can > work a deal. I've got ten live chickens in my apartment I might be > willing to sell... -Chris -- http://www.ebt.com +1.401.421.9550 +1.401.521.2030 One Richmond Square, Providence, RI 02906 USA" NDATA SGML.Geek> Date: Wed, 11 Dec 1996 15:31:13 -0800 To: w3c-sgml-wg@w3.org From: Tim Bray Subject: Please please please Can the allegedly technically competent participants in this forum please stop what they are doing RIGHT NOW and review their mailer documentation in order to figure out how to reply to the group, just the group, not the group and the 3 or 4 other participants in the current thread!?!? This weather is heavy enough without getting multiple copies of everything. Thanks thanks thanks. -T. Date: Wed, 11 Dec 1996 18:40:20 -0500 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: public identifiers: "interchange XML" I think that the fundamental problem we are encountering is that many want to use XML internally (within an organization, or even a widely flung community) but others want every XML document to be exactly as useful on every computer on the Internet as on every other computer. We've run into this before. Processing instructions can also "favor" one machine or system over another. So can variant character set encodings. Do we want to make an explicit distinction between "exchange XML" (UTF-8/UCS2, no PIs, URLs) and "archive XML?" (which would allow that stuff) Would this solve people's problems? And if not, why can't we resolve public identifiers the same way we resolved those other questions: in favor of end-user flexibility. Compared to the potential havoc caused by shipping shift-JIS documents across the web encoded using PIs instead of elements, public identifiers are quite harmless. They are trivial to ignore and are easy to use redundantly with URLs. Paul Prescod To: W3C SGML Working Group Subject: Re: simplifying comments in SGML '97 Date: Wed, 11 Dec 1996 16:51:01 -0800 From: Joe English Michael Sperberg-McQueen wrote: > The problem is that there seem to be at least two ways of approaching > the problem, and it's not clear which is preferable. Your opinions, > please. > > A. The Simple Comment This would introduce another case of feature-dependent syntax in SGML, which is an unmitigated evil. See for a long discussion of the hazards of feature-dependent syntax... In particular, this scheme would make it impossible (in the general case) to combine two text entities if one was written with SIMPLEC YES and the other was written with SIMPLEC NO; there is no way to create a "greatest common denominator" SGML declaration that makes both comment syntaxes legal. Also, this scheme would make COM a delimiter-in-context instead of a plain delimiter inside COM declarations, and SGML certainly doesn't need any more D-I-Cs. > B. Splitting the com delimiter. This is a much better solution. (IMHO having distinct COMO and COMC delimiters would be a good idea in any case, even without regard to XML.) --Joe English jenglish@crl.com Date: Wed, 11 Dec 1996 17:00:07 -0800 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org CC: bosak@atlantic-83.eng.sun.com Subject: Re: XML and HTML Interoperability [Chris Maden:] | If one is authoring XML and expecting that HTML browsers may be used | with it, one should use a tagset that is a superset of HTML. The name "HTXML" has been suggested for this (I can't remember by whom now). It's been suggested that the definition of an HTXML might be useful somehow in helping people to get past the kind of transitional situations that Murray has described. Jon Date: Wed, 11 Dec 1996 20:33:21 -0900 To: w3c-sgml-wg@w3.org From: "W. Eliot Kimber" Subject: Public IDs for notations in XML All, As the ERB grapples with the issue of public identifiers in XML, we realized that regardless of the final decision, there probably needs to be a distinction between the rules for entities and the rules for data content notations. In particular, it doesn't really make any sense for the system identifiers of notations to be URLs. It seems to make much more sense to use MIME types. This would make notation declarations mappings from local notation names and (potentially) public identifiers to MIME types, which would be useful both for XML and traditional SGML processors. Thus we are thinking of a solution along the lines outlined in the following straw proposal. In this proposal I'm assuming that public identifiers are allowed (at least for notations), although I realize they might not be or might be URNs or something. Proposed: Notation declarations consist of a notation name followed by an external identifier. The external identifier consists of either the keyword "SYSTEM" followed by a literal containing a MIME type specification or the keyword "PUBLIC" followed by a literal containing a public identifier followed by a literal containing a MIME type specification. The MIME type specification could either be just a MIME type name, i.e., "image/jpeg" (or mime type followed by parameters) or it could, potentially, be an SGML formal system identifier with a storage manager of MIMETYPE: "image/jpeg". Thus, a notation declaration could either be: or If XML does not specify a public identifier resolution mechanism then it would not be allowed to have a public identifier without a corresponding MIME type. If there is such a mechanism, then it would be possible, potentially, to do the mapping to MIME types there. One reason for allowing the public identifier for notations without a resolution mechanism is to make the correlation between existing notations used in SGML documents and the MIME types used for Internet delivery clearer, especially to those who already have SGML documents. Obviously, if you are coming from HTML and only ever thought about MIME types, you don't necessarily need or want a public ID and don't need to specify one. In addition, many SGML processes (including much of architecture processing) depend on notations and public identifiers for notations, so it will make taking advantage of these facilities from XML much easier if we allow public identifiers for notations even if we don't for entities. Cheers, Eliot -- W. Eliot Kimber (eliot@isogen.com) Senior SGML Consulting Engineer, Highland Consulting 2200 North Lamar Street, Suite 230, Dallas, Texas 75202 +1-214-953-0004 +1-214-953-3152 fax http://www.isogen.com (work) http://www.drmacro.com (home) "Rats in the morning, rats in the afternoon...if they don't go away, I'll be re-educated soon..." --Austin Lounge Lizards, "1984 Blues" Date: Wed, 11 Dec 1996 21:56:00 -0600 From: Len Bullard To: w3c-sgml-wg@w3.org CC: cbullard@HiWAAY.net Subject: Re: Public IDs for notations in XML W. Eliot Kimber wrote: > In addition, many SGML processes (including much of architecture > processing) depend on notations and public identifiers for notations, so it > will make taking advantage of these facilities from XML much easier if we > allow public identifiers for notations even if we don't for entities. since the question has arisen on CTS, and it will again, would it make sense to use this facility to identify the type of an XML document? yes, doctype is there to be used as well. len bullard Date: Wed, 11 Dec 1996 23:21:55 -0500 From: gtn@ebt.com (Gavin Nicol) To: elm@arbortext.com CC: lee@sq.com, w3c-sgml-wg@w3.org, tbray@textuality.com Subject: Re: RS/RE, again (sorry) >There's one complication to this. If there's no DTD (quite possible), >AND no way in the instance (such as with an attribute value) to indicate >which is element content and which is mixed content, AND no stylesheet >semantic that allows this to be controlled, how can the application layer >decide? I'm guessing that DSSSL expects a document to have gone through >a classic SGML parser first, which means that element content will have >internal whitespace stripped. XML documents might not undergo any such >treatment. Somehow, the element-vs.-mixed distinction needs to be made >in cases where it's critical for rendering or other processing. Where is >the right place to make the distinction, if there's no DTD? Is is the >stylesheet? If so, are there any technical barriers to doing this? I'd like to hear from people the cases that they can think of where the distinction is truly necessary. In cases where it really *is* necessary, I see no problem at all in having/requiring a "validation" layer that *requires* a DTD so that element and mixed content can be distinguished. This layer would validate the post-parse representation using whatever rules need be applied (one might be to use SGML's "normal" rules). I cannot see why a *parser* for an essentially trivial syntax need be bothered by this. Conceptual model: input-->parser-->events/grove------------------+-->application \ / +-->filter-->events/grove+ where "filter" can perform some rules-driven (ie. DTD, DSSSL spec. etc.) transformation of the parser output. Date: Wed, 11 Dec 1996 23:29:13 -0500 From: gtn@ebt.com (Gavin Nicol) To: papresco@calum.csclub.uwaterloo.ca CC: lee@sq.com, w3c-sgml-wg@w3.org, tbray@textuality.com Subject: Re: RS/RE, again (sorry) >In element content, all white space (S) is ignored; validating XML >processors must not pass it to the application. Non-validating processors >which do not read the DTD must treat all elements as if they were declared >with mixed content; this will in some cases result in a different parse tree >from that produced by processors which do read the DTD." I am trying to distinguish between *parser* proper, and validator. A validation processor does not need to be *in* the parser proper, but instead can work on the output from it. This would lead to a model in which all *parsers* would return the same results, making it easier to validate/prove that the parsers are correct. Date: Wed, 11 Dec 1996 23:39:46 -0500 From: gtn@ebt.com (Gavin Nicol) To: murray@spyglass.com CC: w3c-sgml-wg@w3.org Subject: Re: XML and HTML Interoperability >a. Both the document character set and encoding are different from HTML. > XML uses Unicode and UTF-2/UTF-8 (allowing other encodings), so unless > the browser is already i18n-ed, this may be a big problem. The actual model is almost identical, though the emphasis on Unicode is stronger. I think you find HTML following this path too (more people are commiting to Unicode etc.). Would that the "Unicode Lingua Franca" idea had taken off... From: lee@sq.com Date: Wed, 11 Dec 96 23:46:25 EST To: eliot@isogen.com, w3c-sgml-wg@w3.org Subject: Re: Public IDs for notations in XML Eliot wrote: > Using a MIME media type as a system identifier. I think this is very sensible, although I would urge a list: This would allow for content negotiation in an interesting and clean way. It would mean I like jpeg no less than gif (and maybe a lot more) and I like gif no less than I like png, so send me the image in the first matching format in this list that you can, please. Eliot mentioned that FPIs are used in Architectures (presumably in the HyTime TC, which I think is not yet generally available); I should point out that it may very well be necessary to use an entirely different mechanism for architectures in XML, not just because FPIs might not have the same semantics, but very possibly for other reasons (e.g. no FIXED attributes), but that I don't think that matters very much. I haven't comented on Eliot's other points, which are very sensible. Lee To: gtn@ebt.com (Gavin Nicol) cc: w3c-sgml-wg@w3.org Subject: Re: RS/RE, again (sorry) <199612120421.XAA02778@nathaniel.ebt> Date: Thu, 12 Dec 1996 06:56:17 -0500 From: Eve Maler >>Somehow, the element-vs.-mixed distinction needs to be made >>in cases where it's critical for rendering or other processing. Where is >>the right place to make the distinction, if there's no DTD? Is is the >>stylesheet? If so, are there any technical barriers to doing this? > >I'd like to hear from people the cases that they can think of >where the distinction is truly necessary. > >In cases where it really *is* necessary, I see no problem at all in >having/requiring a "validation" layer that *requires* a DTD so that >element and mixed content can be distinguished. This layer would >validate the post-parse representation using whatever rules need be >applied (one might be to use SGML's "normal" rules). > >I cannot see why a *parser* for an essentially trivial syntax need be >bothered by this. > >Conceptual model: > > input-->parser-->events/grove------------------+-->application > \ / > +-->filter-->events/grove+ > >where "filter" can perform some rules-driven (ie. DTD, DSSSL >spec. etc.) transformation of the parser output. The most obvious case is plain old rendering. If you've just invented a DTD on the fly that contains a list that itself can contain some list items, and the input looks something like this: Blah <-- Blah <-- Blah ... ...do you render those extra lines, or what? In this case, validation isn't even on the author's or renderer's mind, and yet you need to know the distinction. Eve Date: Thu, 12 Dec 1996 12:25:41 +0000 To: Joe English , W3C SGML Working Group From: Martin Bryan Subject: Re: simplifying comments in SGML '97 > >This would introduce another case of feature-dependent syntax >in SGML, which is an unmitigated evil. > >See for >a long discussion of the hazards of feature-dependent syntax... ISO has this week passed a resolution asking its SC's to avoid preparing standards that allow multiple incompatible configurations of products conforming to them to be created!! ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ Date: Thu, 12 Dec 1996 12:25:39 +0000 To: Michael Sperberg-McQueen , W3C SGML Working Group From: Martin Bryan Subject: Re: simplifying comments in SGML '97 If you define comment as: > [92] comment =3D como, SGML character*, comc > > > [92a] simple comment declaration =3D > mdo, com, SGML character*, com, mdc > you cannot say > > No markup is recognized in a comment, other than the com > delimiter that terminates it. You must change com to comc, or retain com in place of como and comc. (I'm not clear that como and comc buy you enough to invent two more delimiters, with their associated parsing contexts) The following note does not make sense if you use como and comc to delimi= t comments: =20 > 1. A com delimiter not followed by an mdc delimiter will be > recognized in a comment (in a comment declaration or other > declaration) but not in a simple comment declaration. > Then there is option B! >B. Splitting the com delimiter. > >This proposal simply replaces the 8879 com delimiter with a pair of >delimiters, como and comc (comment open and comment close). Documents >using 8879:1986 syntax have como =3D comc =3D com; in the RCS that's >como =3D comc =3D '--'. XML could allow -- within comments by setting >como to '--*' and comc to '*--', to retain the general look and feel >of current comments, and still allow '--' in the comment itself. >Nested comments might also become possible, in SGML (and then in XML), >or not. Why go to all this fuss: simply change com in XML to "*--" or, better sti= ll "**", and then -- becomes legal! > >Production 91 of 8879 could remaing the same as it now is; 92 would >change. We might have: > > 91 comment declaration =3D mdo, (comment, (s | comment)*)?, mdc > 92 comment =3D como, SGML character*, comc How does this approach restrict XML to a single comment? >or (to allow nested comments) > > 92 comment =3D como, (SGML character | comment)*, comc So now I can mix text with comments e.g. How very clever! >Note: If this is what we propose to WG8,=20 it will be rejected! >the XML spec should probably >change *now* to use these delimiters, replacing production 21 with > > [21] Comment ::=3D '' > >Advantages: this seems relatively simple and relatively compatible >with the look and feel of 8879 as a whole. Disadvantages: it >doesn't allow the SGML parser to enforce XML's rules. And won't work as it fails to make -- legal, which seemed to be the whole point of this useless exercise.! ---- Martin Bryan, The SGML Centre, Churchdown, Glos. GL3 2PU, UK=20 Phone/Fax: +44 1452 714029 WWW home page: http://www.u-net.com/~sgml/ Date: Thu, 12 Dec 1996 09:25:10 -0500 From: gtn@ebt.com (Gavin Nicol) To: eve@doctools.com CC: w3c-sgml-wg@w3.org Subject: Re: RS/RE, again (sorry) > > >Blah > <-- >Blah > <-- >Blah >... > > >...do you render those extra lines, or what? In this case, validation >isn't even on the author's or renderer's mind, and yet you need to know >the distinction. You can handle this with a rule saying that all PCDATA psuedo-elements that are direct descendant of need not be displayed. You should be able to do this with DSSSL, I believe. Date: Thu, 12 Dec 1996 09:54:26 -0500 To: gtn@ebt.com (Gavin Nicol), eve@doctools.com From: Paul Prescod Subject: Re: RS/RE, again (sorry) Cc: w3c-sgml-wg@w3.org At 09:25 AM 12/12/96 -0500, Gavin Nicol wrote: >You can handle this with a rule saying that all PCDATA psuedo-elements >that are direct descendant of need not be displayed. You should >be able to do this with DSSSL, I believe. But existing DSSSL scripts will break, because they would not expect these pseudo-elements to be significant. The problem is that you get a different parse tree if you use an XML parser (which doesn't discard any whitespace), or an SGML parser (which must). And of course, DSSSL scripts are only the beginning. Every application that is currently based on an SGML parser will get a different parse tree from an XML parser. For some (mostly browsers) it will not be a big deal. For others, it will be. The current XML specification does the same thing, but it divides the parser world into two categoreies: validating parsers, which will produce 8879 parse trees, and non-validating parsers, which may not. Presumably you use non-validating parsers for browsers, and hard-coded apps, where it doesn't matter, and validating parsers for other applications. Your proposal, though, divides the world up into thousands of different categories: applications which keep all whitespace, applications which use 8879's rules, and applications which use totally proprietary whitespace elimination rules. In practice, that means that we'll all end up using Microsoft and Netscape's whitespace elimination rules (whatever they might end up being). No thanks, I'd rather provide a mechanism for having totally meaningless whitespace, just as we have a mechanism for totally meaningless . Just as comments *may not* appear in the text of a conforming browser's parse tree, the spaces between list items should be defined to disappear. I have suggested that perhaps we should use STAGC syntax to make this easily possible. Paul Prescod From: Ken Holman To: "'w3c-sgml-wg@w3.org'" Subject: RE: Public IDs for notations in XML Date: Thu, 12 Dec 1996 10:16:40 -0500 The ISO/IEC JTC 1/SC 18/WG 8 project 15.07.02.09.00 is creating ISO TR9573 Part 9 regarding the use of FPI's for standardized notations. >---------- >From: W. Eliot Kimber[SMTP:eliot@isogen.com] >Sent: Thursday, December 12, 1996 00:33 >To: w3c-sgml-wg@w3.org >Subject: Public IDs for notations in XML > ..... >The MIME type specification could either be just a MIME type name, >i.e., >"image/jpeg" (or mime type followed by parameters) or it could, >potentially, be an SGML formal system identifier with a storage manager >of >MIMETYPE: "image/jpeg". > >Thus, a notation declaration could either be: > >Group//NOTATION >JPEG Image Format//EN" "image/jpeg" > > >or > > ..... According to the draft of this technical report, the above example with both PUBLIC and SYSTEM identifiers would be (long but meaningful and derivable): As I noted before (1996-12-02) there are embryonic efforts in SGML Open to create a Technial Report for a method of specifying what to do in a particular environment with notation data based on PUBLIC and SYSTEM identifiers. It might be prudent to suggest the above method as an interim interpretation until an XML revision can incorporate a meaningful subset of the SGML Open TR. ...... Ken -- G. Ken Holman Tel: +1 613 596-2233 Chief Technology Officer Fax: +1 613 596-5934 Microstar Software Ltd. WATS: 1 800 267-9975 3775 Richmond Road Mail: gkholman@microstar.com Nepean Ontario Info: info@microstar.com CANADA K2H-5B7 Web: http://www.microstar.com -- 1605 Mardick Court, Box 266 Phone: +1 613 489-2987 Kars, Ontario CANADA K0A-2E0 E-mail: gkholman@CanadaMail.com -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6 mQCNAjHOimAAAAEEAK47HbDtZZB8hJBk+r9Zh7m7QxdFHTaz/m200QQ7J9XX4QYs h6hsuP6ZqBJXyLdmIVMEwR6Ry6oxjKMd6BRJ8OGScD89eIghgbrpMX+900NxM0x2 v3yWO9ki2gKRPrn4vlCXznnmcmsyke0G02T2LefXbgZHyVSqDSOLy8nwuN7dAAUR tClHLiBLZW4gSG9sbWFuIDxhYjk0NUBmcmVlbmV0LmNhcmxldG9uLmNhPg== =IN3T -----END PGP PUBLIC KEY BLOCK----- Date: Thu, 12 Dec 1996 12:51:40 -0800 To: lee@sq.com, w3c-sgml-wg@w3.org, tbray@textuality.com From: dgd@cs.bu.edu (David G. Durand) Subject: Re: ERB discussion of public identifiers At 5:40 PM 12/11/96, lee@sq.com wrote: >> 2. Agree that we'll put PUBLIC identifiers into XML *when* we are ready >> to specify the resolution mechanism; the practical effect is almost >> certainly that they don't go in for now. > >I think this is sensible. Since there will _never_ be a single resolution mechanism, this is not sensible. >I think this is a cop-out: >"XML has a feature which is not documented in the spec, but > here is the telephone number of the ERB members so you can ask > how to implement XML properly" reminds me of another markup >definition language.... :-) This straw man has been burned repeatedly. >There is no use (as far as I can see) for PUBLIC "string" being in the >XML spec if XML systems all ignore it. It just clutters the language >(like the - - omissible tag parameters, or ""). But we have seen that not _all_ systems will ignore it. Just the ones that you write, I guess. That's OK. >A restricted subset of CATALOG, together with a documented set of >mechanisms for resolving the catalog itself, might work. The mechanism >for finding a remote catalog over ftp or http must work in the presence >of CGI scripts and URLs with / in a query. > >I can post details of what SoftQuad Panorama does if it will help. >It can all be described in under a page, I think, taking us to 28pp, >although we're still some 50% over the 20 page goal. > > >> 3b - document one resolution mechanism, but not make it required. >> This is somewhat similar to our i18n approach, where we bless 2 >> encodings but admit the possibility that people use others. > >The difficulty with this is that URNs aren't ready yet, and SOcats only >defer the problem: you then have to say how to get the catalog. >On the other hand, we are going to have the same problem with >associating style sheets with documents, and with following hypertext >links based on public identifiers for documents. CATALOG has default places to look that extend trivially to URLs and relative URLs -- this is another straw man. It is common to defer complex mechanisms one needs to protocols that provide those mechanisms -- And commonplace for schedules to be less-than perfectly aligned in such development. More straw. >> 3c - document one resolution mechanism as before, but make its use >> mandatory for XML processors. >I don't think this will fly, ,as we don't have a single mechanism that >satisfies everyone's needs yet, I think. There is no _single_ mechanism that can meet the need to have multiple mechanisms. This point has been rather frequently repeated. >The concerns that have been voiced all seem to relate to using SGML in >an SGML context; it's not clear to me that the same concerns and >implementation constraints apply to XML. There has been no clear statement >in support of PUBLIC saying what functionality is needed and why. I can repeat my quote from the URN list yet again, but I will let you look it up: (Hint: it's short and starts with "Mechanism independent") -- David I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ Date: Thu, 12 Dec 1996 12:51:12 -0800 To: w3c-sgml-wg@w3.org From: dgd@cs.bu.edu (David G. Durand) Subject: Re: Public IDs for notations in XML At 8:33 PM 12/11/96, W. Eliot Kimber wrote: Good suggestion. A few quibbles below (basically lose PUBLIC on NOTATION, FSI on SYSTEM): >Proposed: > >Notation declarations consist of a notation name followed by an external >identifier. The external identifier consists of either the keyword >"SYSTEM" followed by a literal containing a MIME type specification or the >keyword "PUBLIC" followed by a literal containing a public identifier >followed by a literal containing a MIME type specification. > >The MIME type specification could either be just a MIME type name, i.e., >"image/jpeg" (or mime type followed by parameters) or it could, >potentially, be an SGML formal system identifier with a storage manager of >MIMETYPE: "image/jpeg". Since we made the broken decision on SYTEM IDs we should preserve the brokenness on Notations -- FSI syntax should not be allowed one place and not the other. I think this good idea should be deep-sixed for compatibility with other SYSTEM IDs. >If XML does not specify a public identifier resolution mechanism then it >would not be allowed to have a public identifier without a corresponding >MIME type. If there is such a mechanism, then it would be possible, >potentially, to do the mapping to MIME types there. >One reason for allowing the public identifier for notations without a >resolution mechanism is to make the correlation between existing notations >used in SGML documents and the MIME types used for Internet delivery >clearer, especially to those who already have SGML documents. Obviously, >if you are coming from HTML and only ever thought about MIME types, you >don't necessarily need or want a public ID and don't need to specify one. I could live with this, but given the extensibility of mime-types (the "x-" prefix), and the relative paucity of well-known PUBLIC notation names in the SGML world, I'm inclined to drop PUBLIC identifiers altogether for NOTATION. -- David I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ Date: Thu, 12 Dec 1996 12:51:20 -0800 To: w3c-sgml-wg@w3.org From: dgd@cs.bu.edu (David G. Durand) Subject: Re: RS/RE, again (sorry) At 12:01 PM 12/11/96, Tim Bray wrote: I'll try to be brief, as I've said enough on this previously, but since my position is maybe slightly different this time around, I'll comment. I will also refer to some other responses, to this note. >Some feel the mechanism is unaesthetic and prone to misinterpretation >as it stands, and should simply be discarded, with all non-markup >bytes being passed to the application to do with as it will. This >can be made 8879-compliant in the short term via a mechanism proposed by >Charles Goldfarb, and in the medium/long term with a TC via WG8. as noted, documents parsed w/ and w/out DTDs will be different in this approach. Paul P. and Eve Mahler have claimed that ignoring whitespace in element content will be a great hardship -- but I've not seen the proof of this yet. Validating parsers could ignore those spaces for applications. >There are some problems with both the current and revised approaches: > >o -XML-SPACE, although this is not documented, really only deals with > mixed content; many feel it's important to ignore white space in > element content; .. .. > ^^ > e.g. the above > but XML, when there's no DTD, doesn't know where element content is and > *cannot* be made to do this. If this is the case, the only good justification for this hack has died, I think. > >o SGML's world-view tripartions the set of characters: > those that are text, those that are markup, and those that are > insignificant white space. Can XML really afford to discard this > distinction? I think it must. See below for one reason why. The other is even simpler. This tripartite division confuses almost everybody -- If we have trouble keeping it straight, without dedicating significant effort, how could we propagate it further to a more naive, less-motivated public? >o Many real-world editors, largely to deal with the fact that > text (whether or not we like it) is stored in files in what amounts > to a series of records, freely insert line breaks and other white > space because they know SGML processors will ignore it. Can we > afford to make that white space significant? Since those editors will already require a minor facelift to work with XML anyway, removing a few "\n"s in the code is likely to be easy. I think this issue is less-important than some make it. Yes, we might have to change software, but no, it is not a hard change, even added to the few others we have already admitted. >o Some applications, e.g. full-text indexers, really need to know where > everything is by byte offset, whether or not the bytes are significant; > thus the -XML-SPACE="COLLAPSE" behavior means they can't read the text > with an XML processor (unless they can turn off -XML-SPACE processing > through the API) This explanation of byte-offset requirements is a bit confusing: we could always add an interface to parsers to give the current byte-offset within the underlying entity. The real problem is that in many linking applications we would like to address characters within an element: and here the distinction becomes more problematic, as the user's view of the element differs from the system's view by some number of "insignificant" characters. That's why "insignificance" should die. >So there are a few things we could do, which are not entirely >mutually exclusive. > >1. Go to the RE Delenda Est model. This has the advantage that it's > trivially easy to explain, document, and implement. It has some of the > disadvantages listed above; there is some very strong sentiment on the > ERB against this - look from a follow-up from other ERB-folk. I doubt the pretty-printing contingent will accept this (although CRLF before the end ">" of tags actually allows almost-pretty printing). I still like this, but given its overall unpopularity, don't expect it to live. (Just to simple to survive, I guess). >3. Add language to the spec allowing the application to force the > processor to pass through all the bytes regardless of the -XML-SPACE > setting. 4. Pass all whitespace through in DTD-free parsing mode. For the pretty-printing contingent, we make validation parsing mode different: It will strip all whitespace in element content. I guess this is "RE delenda est" with a failover. In addition, optionally allow 3: validating parsers may be made to pass all space (and comments) to applications (like indexers) that require it. We should add a note that processors that depend on such space for any form of human-presentation or formatting violate the intent of the standard. -- David I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ Date: Thu, 12 Dec 1996 12:51:30 -0800 To: w3c-sgml-wg@w3.org From: dgd@cs.bu.edu (David G. Durand) Subject: comments The issues of syntactic simplicity and compatibility with the style of almost all modern programming languages argue strongly for a single comment start and comment end string. are already used that way by most people, so should remain. I think that a comment should be valid anywhere an s is, except in DTD delarations (where compatiblity dictates that we not have any comments of this form). I don't have a problem adding "-- comment --" inside declarations, but only if comment and space are equivalent. We should not preserve the current confusion between syntactic and lexical analysis in XML whenever possible. Since it would be user and not just implementor visible in declaration syntax, I think that such comments are right out unless we decide to be incompatible. We should really make sure that comments can be treated like whitespace by a lexical analyzer. We still have Delimiter in context woes, but if there are only two contexts, with transitions triggered by "<" and ">", it is still tirivial to implement in a lex equivalent or very short hand-coded lexer. I would rather that the syntax were also described in two levels to make this strategy more self-evident, but as longs as it's possible, I think we are OK. -- David I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ Date: Thu, 12 Dec 1996 12:52:00 -0800 To: w3c-sgml-wg@w3.org From: dgd@cs.bu.edu (David G. Durand) Subject: Re: ERB discussion of public identifiers At 12:05 PM 12/11/96, Tim Bray wrote: >We spent a lot of time on this question on Dec. 11th, and it is clear we >need some more help from the WG. > >The ERB is, by at least a substantial majority, convinced that there is a >real nead for PUBLIC identifiers in XML. yay! >The ERB is highly concerned, in at least a significant minority, about >the effects of putting this facility in without specifying a resolution >mechanism. Doing so would contravene one of the major design goals of >XML - that any compliant XML processor should be able to read any compliant >XML document. They'll get over it. "Mechanism independent" means that a requirement of a specific mechanism is not productive. We _should_ suggest URNs and SOCAT as possibilities and leave it to the market. 2 key points: 1. PUBLIC IDs, if resolvable by a parser, should take precedence over a SYSTEM ID. This lets people who want to count on naming do so... 2. SYSTEM IDs must be required. This means that there is always _one_ resolution mechanism that can be tried even if you do nothing with names. People with internal use of PUBLIC and active opposition to SYSTEM can just put in an empty string. >On the other hand, there is also substantial concern about giving an >unconditional blessing to any particular name resolution mechanism at >this point in history. Given the state of implementation and acceptance it would be foolish. >Thus, there are a variety of options open to us. > >1. Leave it as it is Please, no! >2. Agree that we'll put PUBLIC identifiers into XML *when* we are ready > to specify the resolution mechanism; the practical effect is almost > certainly that they don't go in for now. Since a mandatory resolution mechanism is an anti-requirement, I think this is a bad approach. >3. Put a slot in the syntax for PUBLIC identifiers... > 3* ...and require an accompanying SYSTEM identifier > > 3a - say nothing about resolution mechanisms, perhaps providing a > taxonomy of available technologies in this area > 3b - document one resolution mechanism, but not make it required. > This is somewhat similar to our i18n approach, where we bless 2 > encodings but admit the possibility that people use others. 3b* please: We suggest the two most likely, allowing for doing nothing, and require a SYSTEM ID for those who find it disturbing not being able to tell if you can resolve a name or not. [ Note that this is the real difference between URL and URN resolution: you can tell that a URL is broken at time T, by trying to fetch the resource and seeing if you get it. Given a URN, you can _never_ tell if it is really broken, or you are just trying to wrong mechanism to resolve the URN. ] > 3c - document one resolution mechanism as before, but make its use > mandatory for XML processors. > > Note that there is a continuum between 3b and 3c; we could place > varying strengths of recommendation behind one resolution mechanism, > with homilies about document portability.\ I think 3c is actively unhelpful, as it removed one of the key kinds of flexibility that make URNs/FPIs attractive to those who find them attractive. >In the area of which resolution mechanism to (perhaps nonexclusively) >bless, SGML/Open catalogs (hereinafter Socats) stand out, and would >probably be the ERB's choice. On another hand, there has been a lot >of work go into the URN effort; on another hand, that work has not yet >born practical fruit in terms of ubiquitous implementations; on another >hand, the FPI syntax is repellent to some and it is not clear how well >it supports internationalization; on another hand, it may be the case >that FPI's really are URN's as they stand. FPIs probably will become URNs, with some kind of prefix. URN syntax is already divergent from FPI syntax (and in the initial characters, too). Maybe we shoul follow Keith Corbett's excellent suggestions: + FORMAL NO + URN, FPI both legal (I would add 9070 Object identifiers too, but I'm a weirdo). Keith's comment sums it up nicely: > The message to Web publishers is very positive: XML supports a useful > feature for interchange with name resolution systems, and XML will scale > better than HTML as industrial strength naming schemes come online. Tim, again: >I suspect that if a binding vote were taken today, the ERB would either >(a) reinstate the PUBLIC keyword, and put in a nonexclusive recommendation > for Socat support, or Acceptable. >(b) refuse to put PUBLIC in until there was agreement on a required > resolution mechanism. Wrong, because it eliminates one of the reasons to _use_ PUBLIC. >Input, please. more than you want, I'm sure. You're welcome. -- David I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ To: w3c-sgml-wg@w3.org Subject: Element vs. mixed content (Was Re: RS/RE, again (sorry)) Date: Thu, 12 Dec 1996 10:42:43 -0800 From: Joe English gtn@ebt.com (Gavin Nicol) wrote: > >elm@arbortext.com (?): > >Somehow, the element-vs.-mixed distinction needs to be made > >in cases where it's critical for rendering or other processing. Where is > >the right place to make the distinction, if there's no DTD? Is is the > >stylesheet? If so, are there any technical barriers to doing this? > > I'd like to hear from people the cases that they can think of > where the distinction is truly necessary. The only case I can think of is an application that uses HyTime or HyTime-like addresses for hyperlinks. In order for treeloc, pathloc, and most of the relloc forms to work, the application has to know whether a sequence of syntactic whitespace should be counted as a pseudo-element node or ignored. Note that TEI-style XPTRs do not have this problem, so they could be used instead of the HyTime addressing forms. Another possible solution is to specify that SSEP nodes are included in the XML grove plan. DTD-less parsers would create PELEMENT nodes in a few places where DTD-aware ones would create SSEP nodes, but at least they would still count the same number of nodes. --Joe English jenglish@crl.com From: lee@sq.com Date: Thu, 12 Dec 96 13:46:10 EST To: w3c-sgml-wg@w3.org Subject: Re: RS/RE, again (sorry) > From: gtn@ebt.com (Gavin Nicol) > I am trying to distinguish between *parser* proper, and validator. A > validation processor does not need to be *in* the parser proper, but > instead can work on the output from it. > > This would lead to a model in which all *parsers* would return the > same results, making it easier to validate/prove that the parsers are > correct. I think this is more formal than is needed for XML. I'd like to see databases offer save as XML, load XML, without having to buy an XML parser or worry about parser conformance to that extent. It's much easier to write an ad-hoc parser than to use someone else's for a language this small in many cases, and I don't want to forbid that. By parser, I mean "code that reads XML and does something with it". Nothing more formal than that. Conformance tests would be interesting, but they should be at the input level, not internal to applications, I think. I agree with you that validation can be handled separately from parsing, and should be, and in fact generally is: an "SGML parser" is not a parser in the normal sense of the word... Lee From: lee@sq.com Date: Thu, 12 Dec 96 14:09:38 EST To: w3c-sgml-wg@w3.org Subject: Re: RS/RE, again (sorry) > From w3.org!w3c-sgml-wg-request Thu Dec 12 10:12:23 1996 > At 09:25 AM 12/12/96 -0500, Gavin Nicol wrote: > >You can handle this with a rule saying that all PCDATA psuedo-elements > >that are direct descendant of need not be displayed. You should > >be able to do this with DSSSL, I believe. Paul Prescod said: > But existing DSSSL scripts will break, because they would not expect these > pseudo-elements to be significant. Well, both of the two existing DSSSL style sheets in the world :-) are written for SGML DTDs that are not directly XML compatible, so that's not a problem. Furthermore, if you use DSSSL with an SGML parser, you'll get SGML whitespace rules. Since we haven't discussed the use of DSSSL with XML, it could do anything it liked... So this, although a good point, isn't an issue for XML. > Every application that is currently based on an SGML parser will > get a different parse tree from an XML parser. Yes. And that's fine. Lee From: lee@sq.com Date: Thu, 12 Dec 96 14:30:30 EST To: w3c-sgml-wg@w3.org Subject: Re: comments I agree with David Durand's comments on comments completely, with one small exception: > The issues of syntactic simplicity and compatibility with the style of > almost all modern programming languages argue strongly for a single comment > start and comment end string. are already used that way by > most people, so should remain. In fact, "most people" don't use computers at all :-), and there is not yet a large community of XML users. XML comments can be whatever we say they are. EMPTY element syntax and PI syntax is already different from SGML, and OMITTAG NO means XML files look nothing like most HTML files. I don't see this as a problem at all. I have come to think that if (as David said) a comment is equivalent to a space, it would be possible to explain that comments can only appear between , and hence SGML-style comments could be explained easily. In that case, --* ... *-- would work fine, with , although admittedly it's rather a lot of syntax for a comment. Martin said why not use **, which would allow the usual programming-language style (PL/1, C, Java etc) of as long as you always had a multiple of 4 stars, but I'd really rather not explain that restriction! I think we should ask for separate start & end comment delimiters, and use them. David also said: > I would rather that the syntax were also described in two levels to make > this strategy more self-evident, but as longs as it's possible, I think we > are OK. and I agree. Lee Date: Thu, 12 Dec 1996 15:13:34 -0500 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: RS/RE, again (sorry) At 02:09 PM 12/12/96 EST, lee@sq.com wrote: >Paul Prescod said: >> But existing DSSSL scripts will break, because they would not expect these >> pseudo-elements to be significant. > >Well, both of the two existing DSSSL style sheets in the world :-) are >written for SGML DTDs that are not directly XML compatible, so that's >not a problem. It isn't really the legacy issue I'm worried about. It is requiring every new DSSSL script to include code to detect and destroy whitespace that the author thought (reasonably) was only for pretty-printing. It is even more than that...it is the idea of having pretty-printing whitespace be something that is handled by the application *at all*. That's out of whack with most people's understanding of a parser's job. C++ parsers don't return "whitespace nodes" between tokens that the C++ "back end" must detect and delete. >Furthermore, if you use DSSSL with an SGML parser, you'll get SGML >whitespace rules. Since we haven't discussed the use of DSSSL with XML, >it could do anything it liked... I don't really understand that point. >> Every application that is currently based on an SGML parser will >> get a different parse tree from an XML parser. > >Yes. And that's fine. Well, that's the XML status quo. I hoped that we could do better. I think it will be rather annoying to have to know when I code my style sheets whether the application uses an XML parser or an SGML parser. I might just instruct authors to avoid whitespace between elements altogether, since it will not be reliably interpreted (which, I guess, is what some people want). Paul Prescod From: streich@austin.sar.slb.com Date: Thu, 12 Dec 1996 15:17:08 -0600 To: w3c-sgml-wg@w3.org Subject: Re: RS/RE, again (sorry) Paul Prescod > It isn't really the legacy issue I'm worried about. It is requiring every > new DSSSL script to include code to detect and destroy whitespace that the > author thought (reasonably) was only for pretty-printing. It is even more > than that...it is the idea of having pretty-printing whitespace be something > that is handled by the application *at all*. That's out of whack with most > people's understanding of a parser's job. C++ parsers don't return > "whitespace nodes" between tokens that the C++ "back end" must detect and > delete. Paul, I don't think there's any worthwhile analogy between parsing SGML and parsing C, C--, or any other "programming" language. The purposes and expectations on the output are entirely different. If you're accustomed to publishing, you're accustomed to worrying about white-space. > >> Every application that is currently based on an SGML parser will > >> get a different parse tree from an XML parser. > > > >Yes. And that's fine. > > Well, that's the XML status quo. I hoped that we could do better. I think it > will be rather annoying to have to know when I code my style sheets whether > the application uses an XML parser or an SGML parser. I might just instruct > authors to avoid whitespace between elements altogether, since it will not > be reliably interpreted (which, I guess, is what some people want). I would hope that anytime you write a DSSSL stylesheet, you wouldn't make any assumptions about what an SGML parser would do with white-space. If you do, you still have more faith in software than I do. Call me paranoid, but I would always build in rules for collapsing white-space where I didn't want. I'd rather be safe than sorry. bob From: "Christopher R. Maden" Date: Thu, 12 Dec 1996 21:08:28 GMT To: w3c-sgml-wg@w3.org Subject: Re: RS/RE, again (sorry) [Paul Prescod] > It isn't really the legacy issue I'm worried about. It is requiring > every new DSSSL script to include code to detect and destroy > whitespace that the author thought (reasonably) was only for > pretty-printing. It is even more than that...it is the idea of > having pretty-printing whitespace be something that is handled by > the application *at all*. That's out of whack with most people's > understanding of a parser's job. C++ parsers don't return > "whitespace nodes" between tokens that the C++ "back end" must > detect and delete. This is already a necessity. The 8879 rules don't eliminate any internal whitespace, and they don't eliminate all leading and trailing whitespace. If I have a DSSSL stylesheet on top of an 8879 parser, I am going to have routines to strip and compress whitespace. RE delenda est does not introduce a new pain here. (I wouldn't strip whitespace if I expected the element to be preformatted, but in that case, I'd just as soon that the parser give me *all* of my whitespace, not *most* of it.) > Well, that's the XML status quo. I hoped that we could do better. I > think it will be rather annoying to have to know when I code my > style sheets whether the application uses an XML parser or an SGML > parser. I might just instruct authors to avoid whitespace between > elements altogether, since it will not be reliably interpreted > (which, I guess, is what some people want). If every parser passed all the whitespace, then all parsers would give the same parse tree. As long as we have DTD-less parsing, this is the *only* option that will give the same parse tree. Without a DTD, everything must be assumed to be mixed content, and with a DTD, any heuristic that distinguishes between element and mixed content will result in a different parse tree. -Chris -- http://www.ebt.com +1.401.421.9550 +1.401.521.2030 One Richmond Square, Providence, RI 02906 USA" NDATA SGML.Geek> From: dgd@cs.bu.edu (David Durand) Date: Thu, 12 Dec 1996 16:30:17 -0500 (EST) To: w3c-sgml-wg@w3.org Subject: Re: RS/RE, again (sorry) To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: RS/RE, again (sorry) It isn't really the legacy issue I'm worried about. It is requiring every new DSSSL script to include code to detect and destroy whitespace that the author thought (reasonably) was only for pretty-printing. It is even more than that...it is the idea of having pretty-printing whitespace be something that is handled by the application *at all*. That's out of whack with most people's understanding of a parser's job. C++ parsers don't return "whitespace nodes" between tokens that the C++ "back end" must detect and delete. Yes, but such languages have explicit quoting for contexts where spaces matter. Things are a little more tricky in a document markup language where there is a reasonable expectation that the file is _literal data_ to which a minimumm of additional structural information (markup) has been added. Now the notion of "pretty printing" is not so straightforward. But this is philosophy... and your original comment was also philosophy. >> Every application that is currently based on an SGML parser will >> get a different parse tree from an XML parser. > >Yes. And that's fine. Well, that's the XML status quo. I hoped that we could do better. I think it will be rather annoying to have to know when I code my style sheets whether the application uses an XML parser or an SGML parser. I might just instruct authors to avoid whitespace between elements altogether, since it will not be reliably interpreted (which, I guess, is what some people want). Or you will just write stylesheets that don't assume some whitespace will be automatically deleted. (and leave your users alone...). The problem remains that there is no way to tell element content from mixed content given only an instance. The problem with your delimiter proposal is the same as the problem with Charles' explicit quoting proposal -- too ugly. Worse, it's not even easy to explain: "all tags look like this, except that if they can't contain data, they instead this other way". And if you change a DTD to turn element content into mixed content (or, God forbid, have a parameter entity controlling this), you will have to change a giant mass of delimiters in all your instances -- very unfriendly... -- David Date: Thu, 12 Dec 1996 16:41:00 -0500 To: streich@austin.sar.slb.com, w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: RS/RE, again (sorry) At 03:17 PM 12/12/96 -0600, streich@austin.sar.slb.com wrote: >I would hope that anytime you write a DSSSL stylesheet, you wouldn't make >any assumptions about what an SGML parser would do with white-space. If you >do, you still have more faith in software than I do. Call me paranoid, but >I would always build in rules for collapsing white-space where I didn't >want. I'd rather be safe than sorry. In *element* content? I have never written a stylesheet that explicitly removed whitespace between list items, for instance, or between titles and paragraphs, or between paragraphs or between bibliography items, or... egad...am I misunderstanding or are we talking about a *massive* DSSSL script? According to my reading of the DSSSL spec, the standard behaviour for #PCDATA is to pass the characters on to the output. That means you must explicitly blow them away, unless, as Lee says, we do some special interaction between XML and DSSSL-O, or there is some way to turn them off globally. Uck. As I mentioned before, it would be better off leaving out whitespace between elements. We can't have *both* all whitespace significant *and* reliable whitespace removal for pretty printing. Paul Prescod Date: Thu, 12 Dec 1996 16:33:44 -0500 From: gtn@ebt.com (Gavin Nicol) To: papresco@calum.csclub.uwaterloo.ca CC: eve@doctools.com, w3c-sgml-wg@w3.org Subject: Re: RS/RE, again (sorry) >Your proposal, though, divides the world up into thousands of >different categories: applications which keep all whitespace, >applications which use 8879's rules, and applications which use >totally proprietary whitespace elimination rules. I would expect that any application that *requires* SGML semantics in whitespace elimination to include some form of "validator". Date: Thu, 12 Dec 1996 16:41:02 -0500 To: "Christopher R. Maden" , w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: RS/RE, again (sorry) At 09:08 PM 12/12/96 GMT, Christopher R. Maden wrote: >This is already a necessity. The 8879 rules don't eliminate any >internal whitespace, and they don't eliminate all leading and trailing >whitespace. They *do* eliminate whitespace in element content. There's a heck of a lot of that. Most of the newlines in most of my documents are in element content. >If I have a DSSSL stylesheet on top of an 8879 parser, I >am going to have routines to strip and compress whitespace. Why? If the author put whitespace in mixed content, they probably want it there. I would just leave it. Anyhow, Jade seems to Do The Right Thing in the backend. Which is totally different than having to deal with whitespace "nodes" in the front end. >(I wouldn't strip whitespace if I expected the element to be >preformatted, but in that case, I'd just as soon that the parser give >me *all* of my whitespace, not *most* of it.) Again, this is all mixed content stuff. I'm talking about element content. >If every parser passed all the whitespace, then all parsers would give >the same parse tree. Sure, as long as you are willing to a) do away with existing SGML parsers and b) forgo reliable whitespace removal in element content, which will lead to c) no whitespace in element content, or only whitespace in element content according to the rules that Microsoft and Netscape dictate. >As long as we have DTD-less parsing, this is the *only* option that >will give the same parse tree. Not so. We could put information in the instance to differentiate mixed content from element content. Paul Prescod Date: Thu, 12 Dec 1996 16:45:42 -0500 From: gtn@ebt.com (Gavin Nicol) To: lee@sq.com CC: w3c-sgml-wg@w3.org Subject: Re: RS/RE, again (sorry) >> This would lead to a model in which all *parsers* would return the >> same results, making it easier to validate/prove that the parsers are >> correct. > > >I think this is more formal than is needed for XML. Perhaps. One thing that bothers me with SGML is that it is very hard to *prove* that an SGML parser is actually parsing correctly, and the standard doesn;t help to make it obvious that they are.. Date: Thu, 12 Dec 1996 17:09:48 -0500 To: dgd@cs.bu.edu (David Durand), w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: RS/RE, again (sorry) At 04:30 PM 12/12/96 -0500, David Durand wrote: > Or you will just write stylesheets that don't assume some whitespace >will be automatically deleted. (and leave your users alone...). The >problem remains that there is no way to tell element content from mixed >content given only an instance. I can't presume that I will always use applications where I have control over "extraneous nodes" as I may in a stylesheet language. > The problem with your delimiter proposal is the same as the problem >with Charles' explicit quoting proposal -- too ugly. Worse, it's not >even easy to explain: "all tags look like this, except that if they >can't contain data, they instead this other way". It is easy to explain. There are structural tags, that can contain only other structural tags. And there are content tags, that hold text -- simple. I think that this *simplifies* SGML by making it clear where text can and can't go. "Why do I get an error message when I put text between my table tag and my first row? I just wanted a caption." Furthermore, there is no "all tags look like this except these", there is " >And if you change a >DTD to turn element content into mixed content (or, God forbid, have a >parameter entity controlling this), you will have to change a giant mass of >delimiters in all your instances -- very unfriendly... As in "regular" SGML (or two phase XML processing to emulate SGML), changing an element's content model to mixed content changes all whitespace into content. Therefore, if you make that change you should go looking at your instances anyways. They now mean something different than they did before you changed the DTD. That kind of subtle change is, in my opinion, the most dangerous to document maintainability. Paul Prescod Date: Thu, 12 Dec 1996 16:54:28 -0500 From: gtn@ebt.com (Gavin Nicol) To: papresco@calum.csclub.uwaterloo.ca CC: w3c-sgml-wg@w3.org Subject: Re: RS/RE, again (sorry) >It is even more than that...it is the idea of having pretty-printing >whitespace be something that is handled by the application *at >all*. That's out of whack with most people's understanding of a >parser's job. C++ parsers don't return "whitespace nodes" between >tokens that the C++ "back end" must detect and delete. Then again, we aren't talking about C++ are we? We're talking about a markup language for text. A language that doesn't have delimiters around strings... We seem to be confusing parsing XML, and parsing the grammar defined by the DTD is you ask me... From: "Christopher R. Maden" Date: Thu, 12 Dec 1996 22:12:57 GMT To: w3c-sgml-wg@w3.org Subject: Re: RS/RE, again (sorry) [Paul Prescod] > In *element* content? I have never written a stylesheet that > explicitly removed whitespace between list items, for instance, or > between titles and paragraphs, or between paragraphs or between > bibliography items, or... > > egad...am I misunderstanding or are we talking about a *massive* > DSSSL script? You're going to have to do this for XML regardless of what rules we settle on, since you can't guarantee that the receiving user agent will parse your DTD. > According to my reading of the DSSSL spec, the standard behaviour > for #PCDATA is to pass the characters on to the output. That means > you must explicitly blow them away, unless, as Lee says, we do some > special interaction between XML and DSSSL-O, or there is some way to > turn them off globally. Uck. And without a DTD, everything is going to be #PCDATA. Unless we add new delimiters, which I think will be prohibitively confusing to new users. > As I mentioned before, it would be better off leaving out whitespace > between elements. We can't have *both* all whitespace significant > *and* reliable whitespace removal for pretty printing. That's just gross. I can easily think of documents that I would not be able to edit in vi or in xedit due to line or record length restrictions. -Chris -- http://www.ebt.com +1.401.421.9550 +1.401.521.2030 One Richmond Square, Providence, RI 02906 USA" NDATA SGML.Geek> From: lee@sq.com Date: Thu, 12 Dec 96 17:47:58 EST To: w3c-sgml-wg@w3.org Subject: Re: RS/RE, again (sorry) > As I mentioned before, it would be better off leaving out whitespace between > elements. We can't have *both* all whitespace significant *and* reliable > whitespace removal for pretty printing. Without a DTD, there is no such thing as element content, since #PCDATA is allowed and recognised everywhere. With a DTD, the distinction is possible. With a DTD, you can also distinguish, as SGML does, between whitespace that does not match anything in the content model, and #PCDATA that happens to contain only whitespace characters. Multiple successive whitespace characters can be collapsed into either zero or one space, depending on the context, or passed through as-is to the receiver (an SGML system or application). In SGML, this is done within a document depending on the element's content model, and within a DTD based on relatively complex but fixed rules. (e.g. multiiple spaces between a GI and an attribute name in are collapsed into a single space (or SEPCHAR), but outside the angle brackets spaces are retained or collapsed into zero spaces depending on the content model; there is no explicit control over this behaviour in SGML) Unfortunately, this is not sufficient in practice. HTML, for example, uses additional application-defined rules, and many people on this list wanted the ability to have an element in which RS/RE processing was not done, all whitespace matched PCDATA and space was not collapsed. Other people wanted to be able to have multiple successive spaces collapsed in certain places but not others, either for "pretty printing" or in at least one case because they were using editors that put trailing spaces on the input lines. I agree that for those not using SGML-aware tools, pretty-printing is very useful. Unfortunately, although you can do this if you like, SGML doesn't actually say that the spaces and/or tabs at the start of this line are not significant, and has no way of doing so. You can arrange to surround each input line with a tag automatically with the "B" feature of shortref, but then the space hasn't been ignored or collapsed,and you have extra markup. Although SGML gurus can say that they only want to indent the tags, and not the content, and that it is intuitively obvious where spaces are ignored and where they are not, the length of this debate has showed clearly that it is not obvious. If we are going to support this kind of pretty-printing in XML, we must support it properly. That is, we must support it in a way that is simple to understand and explain. It is clear to me that the SGML rules for RS/RE handling are too complex for XML. Charles' proposal would be a good short-term way of disabling RS/RE processing, and adding "RSRE NONE" to the SGML declaration (or some other syntax) to disable special treatment of RS and RE would be very helpful. We may need to add ASCII NL and CR to SEPCHAR as a result of that, depending on how it's worded, but obviously that's no problem. Entirely separate from that are the questions (1) can a validating XML parser ignore whitespace in element content? I think we all agree that the answer should be Yes -- in other words, a space or tab or newline or carriage return between two elements where #PCDATA is not allowed should not be an error. (an XML validator might have an option to produce a warning, though, as if the file is processed by something not looking at a DTD, that space will obviously and necessarily be treated as #PCDATA) (2) in a well-formed XML document, whether or not there is a DTD, should multiple successive spaces that are matched as PCDATA be collapsed into a single space? I think this should depend on the application. For database input & output, I might need spaces retained, for example. Therefore, an XML reader should pass all whitespace through to its client. In other words, white space should be retained by the XML reader, but should be treated as whitespace and not PCDATA by a validating parser checking a content model. Lee Long Note: I have used the term "XML reader" to mean the code that reads the sequence of input characters and turns it into a stream of tokens or potential tree nodes for a program that will build a tree or do whatever else it wants with the information; in computer science this is called a parser, but an SGML parser does something quite different. An SGML parser such as SP would, when reading an XML document, include (conceptually) both an XML reader and also content validation and other processing code. The term "XML reader" is not proposed here as a term for the XML spec, but only to try and clarify this message. Date: Thu, 12 Dec 1996 19:17:26 -0500 To: gtn@ebt.com (Gavin Nicol) From: Paul Prescod Subject: Re: RS/RE, again (sorry) Cc: w3c-sgml-wg@w3.org At 04:54 PM 12/12/96 -0500, Gavin Nicol wrote: >>It is even more than that...it is the idea of having pretty-printing >>whitespace be something that is handled by the application *at >>all*. That's out of whack with most people's understanding of a >>parser's job. C++ parsers don't return "whitespace nodes" between >>tokens that the C++ "back end" must detect and delete. > >Then again, we aren't talking about C++ are we? We're talking about a >markup language for text. A language that doesn't have delimiters >around strings... > >We seem to be confusing parsing XML, and parsing the grammar defined >by the DTD is you ask me... But one of the important points about SGML (of which XML is a subset) is a contract between the parser and the application: "I will not hand you data which does not conform to the DTD." This is *central*. Without it, we can seldom do intelligent things with documents. XML's barely acceptable solution is to divide parsers down the middle into those that respect this contract and those that do not. (validating and non-validating parsers) But at least the whitespace handling of both is explicit in the XML standard. Your solution would leave it up entirely to applications, which will (IMO) almost inevitably lead to incompatibility. Paul Prescod Date: Fri, 13 Dec 1996 08:51:23 -0500 From: gtn@ebt.com (Gavin Nicol) To: papresco@calum.csclub.uwaterloo.ca CC: crm@ebt.com, w3c-sgml-wg@w3.org Subject: Re: RS/RE, again (sorry) >Sure, as long as you are willing to a) do away with existing SGML parsers >and b) forgo reliable whitespace removal in element content, which will lead >to c) no whitespace in element content, or only whitespace in element >content according to the rules that Microsoft and Netscape dictate. This is not really true: the rules to eliminate unwanted whitespace would not be overly complex, and besides, if you are really so keen on doing exactly the same thing as current SGML tools, you can either: a) Use them by using the declarations for XML b) Add a validator/grove transformer that *requires* a DTD, and which will give you what you want. >>As long as we have DTD-less parsing, this is the *only* option that >>will give the same parse tree. > >Not so. We could put information in the instance to differentiate mixed >content from element content. Sure, but this has it's own set of (large) usability problems as people have pointed out. The easiest thing: use LISP syntax. Date: Fri, 13 Dec 1996 08:57:56 -0500 From: gtn@ebt.com (Gavin Nicol) To: papresco@calum.csclub.uwaterloo.ca CC: dgd@cs.bu.edu, w3c-sgml-wg@w3.org Subject: Re: RS/RE, again (sorry) >>And if you change a >>DTD to turn element content into mixed content (or, God forbid, have a >>parameter entity controlling this), you will have to change a giant mass of >>delimiters in all your instances -- very unfriendly... > >As in "regular" SGML (or two phase XML processing to emulate SGML), changing >an element's content model to mixed content changes all whitespace into >content. Therefore, if you make that change you should go looking at your >instances anyways. They now mean something different than they did before >you changed the DTD. That kind of subtle change is, in my opinion, the most >dangerous to document maintainability. Sounds like a good reason for finding some way of making such changes easier on users to me.... Call me a fool, but I think an overriding goal for software design is to make is as easy for people to do the things they want as possible. This generally means making the processing model as intuitive as possible... so people don't get surprised. Date: Fri, 13 Dec 1996 09:13:28 -0500 To: gtn@ebt.com (Gavin Nicol), papresco@calum.csclub.uwaterloo.ca From: Paul Prescod Subject: Re: RS/RE, again (sorry) Cc: dgd@cs.bu.edu, w3c-sgml-wg@w3.org At 08:57 AM 12/13/96 -0500, Gavin Nicol wrote: >>As in "regular" SGML (or two phase XML processing to emulate SGML), changing >>an element's content model to mixed content changes all whitespace into >>content. Therefore, if you make that change you should go looking at your >>instances anyways. They now mean something different than they did before >>you changed the DTD. That kind of subtle change is, in my opinion, the most >>dangerous to document maintainability. > >Sounds like a good reason for finding some way of making such changes >easier on users to me.... > >Call me a fool, but I think an overriding goal for software design >is to make is as easy for people to do the things they want as >possible. This generally means making the processing model as >intuitive as possible... so people don't get surprised. But the parsing model is only one part of the processing model. Your plan would simplify the whitespace parsing model (which users don't really know or care about) and complicate the whitespace processing model by leaving the choice of what to do with whitespace in the hands of vendors. SGML is *explicit* about what happens when you make this change...the content changes, for every application. According to your plan, it would depend on what "filter" the application used internally. Why not just standardize two filters: SGML style, and no filter at all (as XML does). Though this seems suboptimal to me, it is better than an open ended, vendor-controlled choice. Paul Prescod Date: Fri, 13 Dec 1996 09:18:12 -0500 From: gtn@ebt.com (Gavin Nicol) To: lee@sq.com CC: w3c-sgml-wg@w3.org Subject: Re: RS/RE, again (sorry) Paul wrote: > As I mentioned before, it would be better off leaving out whitespace between > elements. We can't have *both* all whitespace significant *and* reliable > whitespace removal for pretty printing. I have a really simple idea: if you want to keep absolute SGML compatability *and* allow DTD-less parsing. Why not write a filter that will rewrite your instances such that they produce exactly the same parse tree in XML and SGML parsers? It'd probably take less time to do than the time spent arguing.... >Long Note: I have used the term "XML reader" to mean the code that reads the >sequence of input characters and turns it into a stream of tokens or >potential tree nodes for a program that will build a tree or do whatever >else it wants with the information; in computer science this is called a >parser Right, and this corresponds to my use of the term "parser", though it might also be close to traditional "scanners" too. SGML is interesting because it has a parser for it's input stream, and another parser for the grammar defined by the DTD... Date: Fri, 13 Dec 1996 09:34:46 -0500 From: gtn@ebt.com (Gavin Nicol) To: papresco@calum.csclub.uwaterloo.ca CC: w3c-sgml-wg@w3.org Subject: Re: RS/RE, again (sorry) >>We seem to be confusing parsing XML, and parsing the grammar defined >>by the DTD is you ask me... > >But one of the important points about SGML (of which XML is a subset) is a >contract between the parser and the application: "I will not hand you data >which does not conform to the DTD." Really? I've never seen it explicitly stated that this is true. Given that SGML is largely silent about error-handling, and grey on a lot of other fronts, it seems like this would be difficult to achieve at best. >This is *central*. Without it, we can seldom do intelligent things >with documents. I'll confess that I quite often do not use a DTD at all during document creation, simply because I find that in my writing, DTD's are a nuisance. I generally *do* go back and validate my documents, however, and in later processing, ignore it again. >Your solution would leave it up entirely to applications, which will (IMO) >almost inevitably lead to incompatibility. Depends. At least all the applications will know *exactly* what they'll be handed. Date: Fri, 13 Dec 1996 10:14:43 -0500 From: gtn@ebt.com (Gavin Nicol) To: papresco@calum.csclub.uwaterloo.ca CC: papresco@calum.csclub.uwaterloo.ca, dgd@cs.bu.edu, w3c-sgml-wg@w3.org Subject: Re: RS/RE, again (sorry) >Why not just standardize two filters: SGML style, and no filter at all (as >XML does). Though this seems suboptimal to me, it is better than an open >ended, vendor-controlled choice. I'd prefer to toss the SGML filter as well, then things would *always* be the same. Date: Fri, 13 Dec 1996 11:04:39 -0500 To: w3c-sgml-wg@w3.org From: dgd@cs.bu.edu (David G. Durand) Subject: Musing on the philosophy of FPIs In a private note to Lee, I came up with a way of explaining it that I'd never used before. FPIs are like SGML in that they are a late-binding mechanism with undefined semantics. We use tags because we may need to process the same document many times and tags help us to late-bind processing to document features. We name the tags because there are fuzzy human-level semantics associated with those textual phenomena that can't be formally described. We use FPIs because we may need to resolve names differently as we re-use the names items on many different systems. We require certain well-defined semantics -- uniqueness, arbitrary lifetime without name reassignment (persistence) -- and late-bind the resolution mechanism for those names at processing times. We give them human0-intepretable names because that helps humans to assign and manipulate them, and we know that this manipulation is not likely to be fully automatic any time soon. I don't know if this helps to make the positions clearer -- it helped me to think about the different problems together, though. -- David I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ Date: Fri, 13 Dec 1996 11:04:32 -0500 To: w3c-sgml-wg@w3.org From: dgd@cs.bu.edu (David G. Durand) Subject: RS/RE: Decision point? We have determined that, for most of the WG, most positions on the ideal whitespace model have not changes significantly... The original proposal was that we examine the current compromise in XML (which like most compromises is sub-optimal to many of us) and the proposed "RE SHORTREF est" proposal that Charles made, and see if, on balance that is less sub-optimal. Michael convinced me that the extra whitespace nodes in tables are not that damaging, and the elimination of whitespace "modes" makes implementation easier, so I think it is better than what we have. I think we should restrict the discussion to the choice between XML's current state, and the alternative proposed (and maybe minor variations of that, if they are crucially important), since we will probably reach closure on the ideal method as soon as we did the last time (i.e. never, given the incompatible demands different groups seem to be making). That's unless anyone has a _new_ argument to make on the topic. -- David I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ 3245 146th Place S.E., Suite 270, Bellevue, WA 98007, USA Date: Fri, 13 Dec 1996 08:33:38 -0800 To: w3c-sgml-wg@w3.org From: Derek Denny-Brown Subject: Re: RS/RE, again (quite sorry) At 09:34 AM 12/13/96 -0500, Gavin wrote: >>>We seem to be confusing parsing XML, and parsing the grammar defined >>>by the DTD is you ask me... >> >>But one of the important points about SGML (of which XML is a subset) is a >>contract between the parser and the application: "I will not hand you data >>which does not conform to the DTD." Is it expected that the parser will parse the instance differently if it has a DTD vs. if it does not? i.e. if I were to construct a grove from the result of the parse, would the portion of the grove representing the instance be equivalent (isomorphic?) or would the presence of the DTD imply the (strong) likelihood of a differing parse? It would seem to me that it would be best if I could expect the result from the parses to be the same, regardless of DTD, for some applications. Parsing with regard to the DTD could be viewed as a filter of the parse w/o the DTD (a subset, almost). It may be easier to handle this issue if the parse is treated as a (potential) 2nd step which requires a DTD. >>This is *central*. Without it, we can seldom do intelligent things >>with documents. (Gavin cut this from the above, but I assume it went with the previous quote.) The problem, as I see it is that the application may have an idea what the DTD would be, but the parse does not. So long as the application know exactly what it is going to get, this should not be a problem. >>Your solution would leave it up entirely to applications, which will (IMO) >>almost inevitably lead to incompatibility. > >Depends. At least all the applications will know *exactly* what >they'll be handed. Unfortunately, a agree on both fronts. Since we do have a number of concrete ideas about how the parser should report the document to the application, if it has a DTD, why not consolidate those ideas, define what the parser should return if it is parsing a document relative to a DTD, then say it is up to the application to treat an instance which it knows to be conforming to a DTD as if it were parsed with regard to that DTD. Going back to my 2 step parser model, an application with a fixed set of DTDs would always take the raw parse and then have hard coded into it some procedures which filtered the parsers events appropriately (with regard to the DTD). A generic DSSSL style sheet might include some extension which would tell the parser whether to return the raw parse or to require a DTD.... I see an almost unanimous agreement that there is no clean way to tell how to handle white space in the document without a DTD. I also hear people pounding that they want DTDless operation. I don't see any easy way to resolve this short of making all white-space relevant, which throws everyone who wants readable XML into fits (with very good reason, if that is your criteria). The only way to know how to deal with white space is to have the DTD, so one end of the transaction needs to know the DTD in order to normalize white space. The only other solution, i can seriously think of, is to say that all white space is significant in a document, unless you have a DTD. This opens the door for all sorts of mess (why does it look different over there vs here....) though and I hestitate even on that. The view of a document as a formatted (for human readability) text file vs. a hierarchical encoding of some data are fundamentally different views, which happened to correspond exactly to the view of the human author/viewer vs the parser/application. Reconciling these views is one of the problems I have had with SGML all along... -derek "that which is not slightly distorted lacks sensible appeal: from which it follows that irregularity - that is to say, the unexpected, surprise, and astonishment, are an essential part and characteristic of beauty" - Charles Baudelaire Date: Fri, 13 Dec 96 10:49:37 CST From: Michael Sperberg-McQueen Subject: Re: simplifying comments in SGML '97 To: W3C SGML Working Group On Wed, 11 Dec 96 13:08:27 CST I said: >A. The Simple Comment > ... > [92] comment = como, SGML character*, comc Which is a cut-and-paste error for the correct production [92] comment = com, SGML character*, com This eliminates, I think, the inconsistencies to which Martin Bryan objected, though it does not affect some other arguments concerning this proposal. Apologies to Dave Peterson for mistranscribing his draft, and thanks to Martin Bryan for calling the problem to my attention. -C. M. Sperberg-McQueen From: Jean Paoli To: "'w3c-sgml-wg@w3.org'" Subject: RE: RS/RE, again (sorry) Date: Fri, 13 Dec 1996 10:30:48 -0800 Encoding: 90 TEXT the problem I have with RE Delenda is the one pointed by Prescod: there is no mechanism provided for having totally meaningless whitespaces. It is a fact that editors or batch applications which read XML, manipulate it and then save it need a way to freely insert CR/LF inside the XML stream in order to cut it in lines because a line has a limited number of characters. It need also a place to freely insert whitespaces for indentation purpose. It need also to know at read-time that those characters could be removed : if not, the document will continue to grow indefinitely. This is the tribute we have to pay to enable XML document be text, which means human readable without tools. In principle, the only safe place to insert or delete such characters is in element-content but element-content cannot be detected in a DTD-less environment so we have a problem. Instead of talking immediately on what a parser should do, I will discus a little bit about application behavior and come back later to the parser output. In the following, * represent a CR/LF/whitespace and are numbered *1, *2 :
    *1*2
  • *3*4
    foo*5*6bar
  • *1 and *2 are meaningless *3 and *4 are white space in mixed content. *5 and *6 are white spaces used for formatting purpose. When an application saves XML, we can realize that it *could* be acceptable to collapse *3 and *4 or to add or delete CR/LF near *3 or *4 in order to do pretty printing or to cut the XML stream in lines. It *could* also be acceptable to collapse *1 and *2 or to add or delete CR/LF near *1 and *2. But it is certainly inacceptable to let the application modify anything for *5 and *6 or insert CR/LF inside of the
     in order to cut the XML stream in lines.
    
    I propose :
    
    1/ XML Parser output: use RE Delenda Est in order to respect the data
    integrity (and for example to permit to full-text indexers to know where
    everything is by byte offset)  
    2/ Change the current -XML-SPACE meaning: instead of having -XML-SPACE
    changing
    the output of the parser, let us define -XML-SPACE as information for
    the application 
    (basically XML is an application profile of SGML, so hey, we go!): we
    define a single value
    -XML-SPACE=PRESERVE in order to indicate to the application that it
    should not mess
    *arbitrarely* with the content of the element for prettyprinting or line
    cut purpose.
    
    Styles are not appropriate to carry this information. Such information
    is related 
    to the content and does not have anything to do with the presentation.
    Using styles in this context means that when you apply two different
    styles sheets
    to the same XML, some whitespaces/CR/LF could be inserted arbitrarely
    inside
    a 
    -like tag, depending of the style-sheet it happened you were
    using at save time.
    This is wrong. Such information should be in the XML data itself.
    
    Note:
    
    My take is that this proposal is less convenient than the current XML
    spec.
    It gives full access to byte offset out of the parser but permit to
    different applications 
    -even if there is some hints to protect regions- to collapse in a ton of
    different ways
    white spaces (such as *3*4 in my example) inside of PCDATA.The only
    remaining 
    problem in my eyes to the current spec is that white spaces/RS/RE like
    *1*2 are collapsed
    and not deleted. But style sheets could do that and there is no perfect
    solution anyway. 
    
    
    
    
    
    Date: Fri, 13 Dec 1996 12:13:11 -0800
    To: w3c-sgml-wg@w3.org
    From: Tim Bray 
    Subject: RE: RS/RE, again (sorry)
    
    I support Jean's position, that we leave -XML-SPACE in, and either
    cut it back to one value (COLLAPSE) or expand it to three values
    (PRESERVE, COLLAPSE, REMOVE), but change its meaning so that it acts
    as a signal from the processor to the application, but that the
    processor *always* in all cases passes all the bytes through to
    the app.
    
    Cheers, Tim Bray
    tbray@textuality.com http://www.textuality.com/ +1-604-488-1167
    
    
    
    From: lee@sq.com
    Date: Fri, 13 Dec 96 14:58:53 EST
    To: w3c-sgml-wg@w3.org, jeanpa@microsoft.com
    Subject: RE: RS/RE, again (sorry)
    
    
    I like Jean Pauli's proposal.
    
    I have two suggestions.
    
    (1)
    the space at *3*4 in
    
    
      *1*2
    • *3*4
      foo*5*6bar
    • Lee Date: Fri, 13 Dec 1996 15:44:18 -0800 To: From: dgd@cs.bu.edu (David G. Durand) Subject: RE: RS/RE, again (sorry) Summary: A few quibbles with Jean, followed by endorsement of Tim's 3-value proposal -- which I was busily independently deriving as his mail arrived. One last quibble -- we can't forget the -xml- prefix on the values unless we're going to revisit the awttribute value decision too. -- David At 10:30 AM 12/13/96, Jean Paoli wrote: >the problem I have with RE Delenda is the one pointed by Prescod: >there is no mechanism provided for having totally meaningless >whitespaces. so far I'm with you. >It is a fact that editors or batch applications which read XML, >manipulate it >and then save it need a way to freely insert CR/LF inside the XML stream >in order to cut it in lines because a line has a limited number of >characters. This is a fact only if we make it so. We are defining the standard. XML already requires _not_ breaking lines in mixed content. Any editor shenanigams in such places will be application-visible, so the flexibility being gained is in fact rather small, for most documents which are composed mostly of mixed-content. >It need also a place to freely insert whitespaces for indentation >purpose. >It need also to know at read-time that those characters could be removed >: >if not, the document will continue to grow indefinitely. In mixed content and in element content under "RE delenda est", whitespace can be added as long as it's added inside markup. So Editors that want to try to shorten lines always have an option. At issue is the stickier human issue of whether, and to what extent, humans need to add insignificant whitespace in documents. >This is the tribute we have to pay to enable XML document be text, which >means >human readable without tools. This is the issue. Given the changes in viewing software (browsers), editing environments (line-free editors like emacs are relatively commonplace), and file management (fixed-record-length filesystems are becoming a memory except for limited legacy contexts), the issue of igorable whitespace seems less critical than it did a decade ago. >In principle, the only safe place to insert or delete such characters is >in element-content >but element-content cannot be detected in a DTD-less environment so we >have a problem. Or in markup. In XML it's always safe to insert any combination of whitespace before any occurence of >. >I propose : > >1/ XML Parser output: use RE Delenda Est in order to respect the data >integrity (and for example to permit to full-text indexers to know where >everything is by byte offset) >2/ Change the current -XML-SPACE meaning: instead of having -XML-SPACE >changing >the output of the parser, let us define -XML-SPACE as information for >the application >(basically XML is an application profile of SGML, so hey, we go!): we >define a single value >-XML-SPACE=PRESERVE in order to indicate to the application that it >should not mess >*arbitrarely* with the content of the element for prettyprinting or line >cut purpose. Does this mean that the application is not allowed to change the space characters, or is allowed to change them? I don't understand the qualification "arbitrary". I like the idea of being able to mark elements with mixed content as -xml-space="-xml-preserve". We should have an DTD automatically mark any elements which have element content, as -xml-space="-xml-ignore". Default handling of mixed content (-xml-space="-xml-collapse") would fold whitespace as under the current proposal. All of these whitespace signals would be application notifications of acceptable behaviour: the parse tree would always be the same, and include the original whitespace, but the application would have clear notification of authorial intent. Jean says this is less-convenient than the other proposals, but I disagree -- because it's definitely convenient to have a single construct for each behavior that you need, and the discussion shown that there are 3 behaviors that we need. -- David I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ Date: Fri, 13 Dec 96 14:24:37 CST From: Michael Sperberg-McQueen Subject: RE: RS/RE, again (sorry) To: Tim Bray , W3C SGML Working Group On Fri, 13 Dec 1996 15:14:05 -0500 Tim Bray said: >I support Jean's position, that we leave -XML-SPACE in, and either >cut it back to one value (COLLAPSE) or expand it to three values >(PRESERVE, COLLAPSE, REMOVE), but change its meaning so that it acts >as a signal from the processor to the application, but that the >processor *always* in all cases passes all the bytes through to >the app. Does this mean a validating XML processor should both pass through all the bytes (n.b. this *still* isn't all the bytes, unless you want me to pass you the white space and quotation marks in the interior of start-tags, which you just aren't going to get, not >from my parser), *and* use its knowledge of the DTD to add an appropriate -XML-SPACE attribute spec before passing the data to the app? So an XML processor which emits sgmls-output-format data might read this is the first item this is the second and emit (I assume some tabs and space in invisible locations) A-XML-SPACE TOKEN REMOVE (LIST -\n\n AN CDATA 1 A-XML-SPACE TOKEN PRESERVE (ITEM -\nthis is the first item )ITEM -\n\n AN CDATA 2 A-XML-SPACE TOKEN PRESERVE (ITEM -this is the \nsecond\n )ITEM -\n\009\009\009\n )LIST A processor which doesn't read the DTD would emit the wrong values for -XML-SPACE some of the time (it would always say #IMPLIED or something). Is this what you have in mind, or something different? -C. M. Sperberg-McQueen Date: Fri, 13 Dec 1996 15:50:34 -0800 To: lee@sq.com, w3c-sgml-wg@w3.org, jeanpa@microsoft.com From: dgd@cs.bu.edu (David G. Durand) Subject: RE: RS/RE, again (sorry) At 2:58 PM 12/13/96, lee@sq.com wrote: >I like Jean Pauli's proposal. >(2) rather than using an attribute called -XML-SPACE, which looks odd and > perhaps even ugly, it would be better to have a PI that says > the attribute TYPE=LIT on the element SAMPLE > has this meaning: > > Please! No more processing instructions! We've reserved some namespace for XML, so let's use it, not multiply PIs. It's bad enough that we've got one in the header, instead of a reasonable syntax. (Did the ERB ever discuss the HTTP header proposal, BTW?) For a feature intended for ugly formatting hacks, PIs are awfully popular, even among the 8879 strict constructionists on the list. -- David "PI delenda est, too" Durand I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ Date: Fri, 13 Dec 1996 15:59:56 -0800 To: w3c-sgml-wg@w3.org From: dgd@cs.bu.edu (David G. Durand) Subject: Whitespace details When I endorsed the 3-states proposal I unwittingly fell into agreement with the current proposal on a point where I disagree strongly: Initial whitespace, and final whitespace should collapse to a single space, not disappear altogether. This is the way HTML browsers currently work (try it in in Netscape, for instance). I know this makes it impossible to ignore any RE after a tag start, but there was never an argument offered to support that behavior that did not beg the conclusion.... and otherwise, the whitespace folding now has two special cases, at the beginning and end of the string... -- David I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ Date: Fri, 13 Dec 1996 16:05:51 -0800 To: w3c-sgml-wg@w3.org From: dgd@cs.bu.edu (David G. Durand) Subject: Things we should not forget... I keep posting long things, so these may have been lost in the noise, but I don't think we can ignore them. We need to use -XML- in front of all attribute values, unless we change our minds on the attribute-value restriction. We need to document that the ISO entity sets will not work with XML, as SDATA has been eliminated, and either develop replacements, or let people know that they will have to develop them themselves. -- David I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ From: Jean Paoli To: "'w3c-sgml-wg@w3.org'" Subject: RE: RS/RE, again (sorry) Date: Fri, 13 Dec 1996 13:45:36 -0800 Encoding: 75 TEXT What I had in mind is a *manual* process: the user marks his element and this is why I was proposing *one* value: PRESERVE. in the manual process, using 3 values becomes cumbersome. >---------- >From: Michael Sperberg-McQueen[SMTP:U35395@UICVM.UIC.EDU] >Sent: Friday, December 13, 1996 12:24 PM >To: Tim Bray; W3C SGML Working Group >Subject: RE: RS/RE, again (sorry) > >On Fri, 13 Dec 1996 15:14:05 -0500 Tim Bray said: >>I support Jean's position, that we leave -XML-SPACE in, and either >>cut it back to one value (COLLAPSE) or expand it to three values >>(PRESERVE, COLLAPSE, REMOVE), but change its meaning so that it acts >>as a signal from the processor to the application, but that the >>processor *always* in all cases passes all the bytes through to >>the app. > >Does this mean a validating XML processor should both pass through >all the bytes (n.b. this *still* isn't all the bytes, unless you >want me to pass you the white space and quotation marks in the >interior of start-tags, which you just aren't going to get, not >from my parser), *and* use its knowledge of the DTD to add an >appropriate -XML-SPACE attribute spec before passing the data to the >app? So an XML processor which emits sgmls-output-format data might >read > > > > > ]> > > > > this is the first item > > this is the > second > > > > >and emit (I assume some tabs and space in invisible locations) > > A-XML-SPACE TOKEN REMOVE > (LIST > -\n\n > AN CDATA 1 > A-XML-SPACE TOKEN PRESERVE > (ITEM > -\nthis is the first item > )ITEM > -\n\n > AN CDATA 2 > A-XML-SPACE TOKEN PRESERVE > (ITEM > -this is the \nsecond\n > )ITEM > -\n\009\009\009\n > )LIST > >A processor which doesn't read the DTD would emit the wrong values >for -XML-SPACE some of the time (it would always say #IMPLIED or >something). > >Is this what you have in mind, or something different? > >-C. M. Sperberg-McQueen > > From: Jean Paoli To: "'w3c-sgml-wg@w3.org'" Subject: RE: RS/RE, again (sorry) Date: Fri, 13 Dec 1996 14:02:21 -0800 Encoding: 124 TEXT >---------- >From: dgd@cs.bu.edu[SMTP:dgd@cs.bu.edu] >Sent: Friday, December 13, 1996 3:44 PM >To: w3c-sgml-wg@w3.org >Subject: RE: RS/RE, again (sorry) > >Summary: A few quibbles with Jean, followed by endorsement of Tim's 3-value >proposal -- which I was busily independently deriving as his mail arrived. >One last quibble -- we can't forget the -xml- prefix on the values unless >we're going to revisit the awttribute value decision too. > > -- David > > >At 10:30 AM 12/13/96, Jean Paoli wrote: >>the problem I have with RE Delenda is the one pointed by Prescod: >>there is no mechanism provided for having totally meaningless >>whitespaces. > >so far I'm with you. > >>It is a fact that editors or batch applications which read XML, >>manipulate it >>and then save it need a way to freely insert CR/LF inside the XML stream >>in order to cut it in lines because a line has a limited number of >>characters. > >This is a fact only if we make it so. We are defining the standard. XML >already requires _not_ breaking lines in mixed content. Any editor >shenanigams in such places will be application-visible, so the flexibility >being gained is in fact rather small, for most documents which are composed >mostly of mixed-content. Look, an application *has* to cut lines where it can and if there is a long stream of text, it is going to cut it. > >>It need also a place to freely insert whitespaces for indentation >>purpose. >>It need also to know at read-time that those characters could be removed >>: >>if not, the document will continue to grow indefinitely. > >In mixed content and in element content under "RE delenda est", whitespace >can be added as long as it's added inside markup. So Editors that want to >try to shorten lines always have an option. At issue is the stickier human >issue of whether, and to what extent, humans need to add insignificant >whitespace in documents. > >>This is the tribute we have to pay to enable XML document be text, which >>means >>human readable without tools. >This is the issue. Given the changes in viewing software (browsers), >editing environments (line-free editors like emacs are relatively >commonplace), and file management (fixed-record-length filesystems are >becoming a memory except for limited legacy contexts), the issue of >igorable whitespace seems less critical than it did a decade ago. > >>In principle, the only safe place to insert or delete such characters is >>in element-content >>but element-content cannot be detected in a DTD-less environment so we >>have a problem. > >Or in markup. In XML it's always safe to insert any combination of >whitespace before any occurence of >. > >>I propose : >> >>1/ XML Parser output: use RE Delenda Est in order to respect the data >>integrity (and for example to permit to full-text indexers to know where >>everything is by byte offset) >>2/ Change the current -XML-SPACE meaning: instead of having -XML-SPACE >>changing >>the output of the parser, let us define -XML-SPACE as information for >>the application >>(basically XML is an application profile of SGML, so hey, we go!): we >>define a single value >>-XML-SPACE=PRESERVE in order to indicate to the application that it >>should not mess >>*arbitrarely* with the content of the element for prettyprinting or line >>cut purpose. > >Does this mean that the application is not allowed to change the space >characters, or is allowed to change them? I don't understand the >qualification "arbitrary". this is the issue: anywhere inside a -xml-preserve, a white space is significant so the application is not allowed to change them for pretty printing or cutting lines (the term significant has to be defined but the long stream of mail on this precise subject prove that at least the concept is real). any where else, the application could do whatever it decide. > I like the idea of being able to mark elements with mixed content as >-xml-space="-xml-preserve". We should have an DTD automatically mark any >elements which have element content, as -xml-space="-xml-ignore". Default >handling of mixed content (-xml-space="-xml-collapse") would fold >whitespace as under the current proposal. > All of these whitespace signals would be application notifications of >acceptable behaviour: the parse tree would always be the same, and include >the original whitespace, but the application would have clear notification >of authorial intent. > > Jean says this is less-convenient than the other proposals, but I >disagree -- because it's definitely convenient to have a single construct >for each behavior that you need, and the discussion shown that there are 3 >behaviors that we need. > > -- David > >I am not a number. I am an undefined character. >_________________________________________ >David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com >Boston University Computer Science \ Sr. Analyst >http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams >--------------------------------------------\ http://dynamicDiagrams.com/ >MAPA: mapping for the WWW \__________________________ > > > From: Jean Paoli To: "'w3c-sgml-wg@w3.org'" Subject: RE: RS/RE, again (sorry) Date: Fri, 13 Dec 1996 14:11:25 -0800 Encoding: 42 TEXT >---------- >From: lee@sq.com[SMTP:lee@sq.com] >Sent: Friday, December 13, 1996 11:58 AM >To: Jean Paoli; w3c-sgml-wg@w3.org >Subject: RE: RS/RE, again (sorry) > > >I like Jean Pauli's proposal. > >I have two suggestions. > >(1) >the space at *3*4 in > >
        *1*2
      • *3*4
        foo*5*6bar
      • >may be collapsed to a single space perhaps, but certainly must not be >deleted, as otherwise the Book title in the B and the Institution in the I >(guessing at the meanings of B and I) will run together! this is why I was saying that the current XML spec was more convenient that the one I proposed in my mail: every application can decide how to deal with this. The authorial intent has not been expressed here. Also, I think that 3 values for XML-SPACE is cumbersome. You would have to write for each paragraph if you take as default -XML-SPACE=DELETE or you woud have to write
          for each list item if you take as default -XML-SPACE=COLLAPSE. > >(2) rather than using an attribute called -XML-SPACE, which looks odd and > perhaps even ugly, it would be better to have a PI that says > the attribute TYPE=LIT on the element SAMPLE > has this meaning: > > > >Lee > > Date: Fri, 13 Dec 1996 14:07:26 -0800 To: W3C SGML Working Group From: Tim Bray Subject: RE: RS/RE, again (sorry) At 02:24 PM 12/13/96 CST, Michael Sperberg-McQueen wrote: >On Fri, 13 Dec 1996 15:14:05 -0500 Tim Bray said: >>...leave -XML-SPACE in ... but the >>processor *always* in all cases passes all the bytes... > >Does this mean a validating XML processor should both pass through >all the bytes (n.b. this *still* isn't all the bytes Uh, all the bytes that aren't markup. >...use its knowledge of the DTD to add an >appropriate -XML-SPACE attribute spec before passing the data to the >app? No; my thinking was that -XML-SPACE was designed for humans to use in the case that they (a) care about how apps handle whitespace and (b) don't trust them to do the right thing without help. In all cases, the parser hands the app all the non-markup bytes. In all cases, the app does what it wants; the XML spec obviously has no normative role as far as apps go. -XML-SPACE is just a message from the author to the apps. This is a good reason why it's not a PI, which a message to *an* app; -XML-SPACE conveys information about the document which may be material to any app; it is necessary because XML lacks SGML's explicit (if difficult) rules specifying exactly which whitespace is data and which isn't. - Tim Date: Fri, 13 Dec 1996 17:51:33 -0400 To: dgd@cs.bu.edu (David G. Durand) From: murray@spyglass.com (Murray Altheim) Subject: Re: Whitespace details Cc: w3c-sgml-wg@w3.org dgd@cs.bu.edu (David G. Durand) writes: >When I endorsed the 3-states proposal I unwittingly fell into agreement >with the current proposal on a point where I disagree strongly: > Initial whitespace, and final whitespace should collapse to a single >space, not disappear altogether. This is the way HTML browsers currently >work (try it in in Netscape, for instance). Leading whitespace (any space between a start tag and element content) in lists and tables in MSIE, Netscape, etc. is always removed, at least as far as any installation I've ever seen. Trailing whitespace is collapsed to a single space in accord with RFC 1866, which makes no statement whatsoever about leading whitespace. > I know this makes it impossible to ignore any RE after a tag start, but >there was never an argument offered to support that behavior that did not >beg the conclusion.... and otherwise, the whitespace folding now has two >special cases, at the beginning and end of the string... Only the end. Try running this fragment through your HTML browser:
          Cell one.Cell two.
          Cell one. Cell two.
          Cell one. Cell two.
          Cell one. Cell two.
          You'll note that apart from the first table (which includes no leading or trailing whitespace), the last three tables are identically formatted, with no leading whitespace in any, and all trailing whitespace collapsed to a single space. Murray ``````````````````````````````````````````````````````````````````````````````` Murray Altheim, Program Manager Spyglass, Inc., Cambridge, Massachusetts email: http: "Give a monkey the tools and he'll eventually build a typewriter." Date: Fri, 13 Dec 1996 14:48:28 -0800 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org cc: bosak@atlantic-83.eng.sun.com Subject: XML papers at WWW6 As you all know, we are planning to roll out XML-with-hypertext at the Sixth World Wide Web Conference in April. The official introduction will be a report in the W3C activity track of the conference, but some of you might wish to submit papers to the technical track independent of the W3C report. For example, I'm planning to submit my paper on XML applications. The original deadline for submissions was today, but that has now been moved back to noon California time on Monday (December 16). Papers can be changed after submission up until February 21, 1997. The theme of the conference is "accessibility", so papers that relate to that have a better chance of being accepted. For further details, see http://logic.stanford.edu/www6/cfp.html Jon Date: Fri, 13 Dec 1996 19:01:07 -0500 From: gtn@ebt.com (Gavin Nicol) To: jeanpa@microsoft.com CC: w3c-sgml-wg@w3.org Subject: RE: RS/RE, again (sorry) >>This is a fact only if we make it so. We are defining the standard. XML >>already requires _not_ breaking lines in mixed content. Any editor >>shenanigams in such places will be application-visible, so the flexibility >>being gained is in fact rather small, for most documents which are composed >>mostly of mixed-content. > >Look, an application *has* to cut lines where it can and if there is a >long stream of text, it is going to cut it. Perhaps I'm just stupid, but I really can't see why... care to explain? From: Jean Paoli To: "'w3c-sgml-wg@w3.org'" Subject: RE: RS/RE, again (sorry) Date: Fri, 13 Dec 1996 16:32:48 -0800 Encoding: 26 TEXT well, for example because it is a very common behavior of a *lot* of text applications to let the user choose the number of characters which are allowed in a line. Even :-) SGML tools such as Grif or Adept allows that. >---------- >From: gtn@ebt.com[SMTP:gtn@ebt.com] >Sent: Friday, December 13, 1996 4:01 PM >To: Jean Paoli >Cc: w3c-sgml-wg@w3.org >Subject: RE: RS/RE, again (sorry) > >>>This is a fact only if we make it so. We are defining the standard. XML >>>already requires _not_ breaking lines in mixed content. Any editor >>>shenanigams in such places will be application-visible, so the flexibility >>>being gained is in fact rather small, for most documents which are composed >>>mostly of mixed-content. >> >>Look, an application *has* to cut lines where it can and if there is a >>long stream of text, it is going to cut it. > >Perhaps I'm just stupid, but I really can't see why... care to explain? > > > From: Jean Paoli To: "'W3C SGML Working Group'" Subject: RE: RS/RE, again (sorry) Date: Fri, 13 Dec 1996 16:56:42 -0800 Encoding: 38 TEXT I totally agree with Tim. >---------- >From: Tim Bray[SMTP:tbray@textuality.com] >Sent: Friday, December 13, 1996 2:07 PM >To: W3C SGML Working Group >Subject: RE: RS/RE, again (sorry) > >At 02:24 PM 12/13/96 CST, Michael Sperberg-McQueen wrote: >>On Fri, 13 Dec 1996 15:14:05 -0500 Tim Bray said: >>>...leave -XML-SPACE in ... but the >>>processor *always* in all cases passes all the bytes... >> >>Does this mean a validating XML processor should both pass through >>all the bytes (n.b. this *still* isn't all the bytes > >Uh, all the bytes that aren't markup. > >>...use its knowledge of the DTD to add an >>appropriate -XML-SPACE attribute spec before passing the data to the >>app? > >No; my thinking was that -XML-SPACE was designed for humans to use >in the case that they (a) care about how apps handle whitespace >and (b) don't trust them to do the right thing without help. > >In all cases, the parser hands the app all the non-markup bytes. >In all cases, the app does what it wants; the XML spec obviously >has no normative role as far as apps go. -XML-SPACE is just a >message from the author to the apps. This is a good reason why >it's not a PI, which a message to *an* app; -XML-SPACE conveys >information about the document which may be material to any app; >it is necessary because XML lacks SGML's explicit (if difficult) >rules specifying exactly which whitespace is data and which isn't. > > - Tim > > Date: Fri, 13 Dec 1996 20:11:43 -0500 To: papresco@calum.csclub.uwaterloo.ca From: Paul Prescod Subject: Re Delenda est summary Cc: w3c-sgml-wg@w3.org At 09:34 AM 12/13/96 -0500, Gavin Nicol wrote: >>>We seem to be confusing parsing XML, and parsing the grammar defined >>>by the DTD is you ask me... >> >>But one of the important points about SGML (of which XML is a subset) is a >>contract between the parser and the application: "I will not hand you data >>which does not conform to the DTD." > >Really? I've never seen it explicitly stated that this is true. Given >that SGML is largely silent about error-handling, and grey on a lot of >other fronts, it seems like this would be difficult to achieve at >best. If a validating parser can pass on data that doesn't conform to the DTD, then what's the point of a validating parser? It always seemed to me that a huge part of the reason for a validating parser was to allow application writers to avoid special case code (e.g. "If there is a title in the footnote, ignore it."). Forcing applications to discard whitespace is just a special case of special case code. >>Your solution would leave it up entirely to applications, which will (IMO) >>almost inevitably lead to incompatibility. > >Depends. At least all the applications will know *exactly* what >they'll be handed. Sure. But when did application writers knowing exactly what the application would process become more important than authors knowing? Moving all whitespace elimination into the application domain leaves authors up to the mercy of application writers. > Why not write a filter that will rewrite your instances such that they > produce exactly the same parse tree in XML and SGML parsers? If we are presuming a preprocessor, then we can make the grammar arbitrarily user-hostile. But preprocessing is user hostile. At 08:51 AM 12/13/96 -0500, Gavin Nicol wrote: >>Sure, as long as you are willing to a) do away with existing SGML parsers >>and b) forgo reliable whitespace removal in element content, which will lead >>to c) no whitespace in element content, or only whitespace in element >>content according to the rules that Microsoft and Netscape dictate. > >This is not really true: the rules to eliminate unwanted whitespace >would not be overly complex, and besides, if you are really so keen on >doing exactly the same thing as current SGML tools, you can either: > a) Use them by using the declarations for XML > b) Add a validator/grove transformer that *requires* a DTD, and > which will give you what you want. But this implies that as an author, I can control the parsing technologies and conventions that my application uses. The way that authors typically assert this form of control is through standardization. That's why we should standardize one or two typical handlings for whitespace elimination, instead of leaving it open to vendor interpretation. Otherwise, we are trading off processing predictability for parsing simplicity. As Lee says: "In other words, white space should be retained by the XML reader, but should be treated as whitespace and not PCDATA by a validating parser checking a content model." Paul Prescod Date: Fri, 13 Dec 1996 20:11:45 -0500 To: "'w3c-sgml-wg@w3.org'" From: Paul Prescod Subject: Whitespace and full-text indexers At 10:30 AM 12/13/96 -0800, Jean Paoli wrote: >I propose : > >1/ XML Parser output: use RE Delenda Est in order to respect the data >integrity (and for example to permit to full-text indexers to know where >everything is by byte offset) I don't understand this byte offset concern. There are other places to put information that is totally meaningless: whitespace within tags and all data within comments, for example. Full-text indexers have totally different needs than "applications" in the parser-client sense. Most parser-clients want a simplification of the data, not just a tokenization of it. If we take support for full-text indexers (and other tools that want all of the bytes) to the extreme, then we must build a grove that contains all of the whitespace within tags, etc. Tim Bray says that it is useful to pass on all of the content that is not markup, but I don't see how. What does this "buy?" How is it any easier to work with an unnormalized version of this:

          Whitespace

          than this:

          Comments

          If we can agree that they are equally difficult to use in a text processing system, then we can agree that whitespace removal can be handled in the same way that comment removal is, by an optional "extra" communication line between the parser and the application. Paul Prescod Date: Fri, 13 Dec 1996 20:35:32 -0500 To: "'w3c-sgml-wg@w3.org'" From: Paul Prescod Subject: Understanding the Paoli proposal At 02:11 PM 12/13/96 -0800, Jean Paoli wrote: >this is why I was saying that the current XML spec was more convenient >that the one I proposed in my mail: every application can decide how to >deal with this. The authorial intent has not been expressed here. I'm getting lost in all of these proposals and amendments. If I want this paragraph to look "right" in all applications, what do I do:

          If I want this paragraph to look right in all applications

          or

          If I want this paragraph to look right in all applications

          My understanding is the latter, because the former gives applications the right to "arbitrarily" remove whitespace, and between the B and the I might well look like a reasonable place to do so. What happens if I turn "on" PRESERVE mode in element content. Should a validating parser tell me that my authorial intent is in contravention of my DTD? Should it just silently eat the whitespaces despite my authorial intent? Should it return a grove that is not valid according to the DTD? Thanks for any enlightenment, Paul Prescod Date: Fri, 13 Dec 1996 18:08:33 -0800 (PST) From: Terry Allen To: w3c-sgml-wg@w3.org Subject: re ERB discussion of public identifiers I fear all the discussion about FPIs as URNs has sailed right over many heads. Tim Bray writes: | The ERB is highly concerned, in at least a significant minority, about | the effects of putting this facility in without specifying a resolution | mechanism. Doing so would contravene one of the major design goals of | XML - that any compliant XML processor should be able to read any compliant | XML document. Understanding a name ("reading" it) is different from being able to resolve it and obtain the object it names. If the design goal is really that all reference must always work, be they SYSTEM or PUBLIC, you'll have to discard it anyway. | On the other hand, there is also substantial concern about giving an | unconditional blessing to any particular name resolution mechanism at | this point in history. If name resolution lies outside the scope of the XML spec, as I think it ought, then no particular mechanism at all should even be mentioned. Your choice seems to lie between treating PUBLIC IDs as URNs, which means "hand it off to your URN resolver"; defining a mechanism for resolving some larger class of vaguely "public" identifiers peculiar to XML; and defining some particular way to resolve FPIs for the Internet. The last course would preclude using URNs that aren't FPIs, which is not a very appealing or sensible course. If the decision is "hand it off to your URN resolver," then no resolution mechanism need be recommended or required; URNs are names, you will resolve by whatever means suit you best. | Thus, there are a variety of options open to us. Not those you list (see above). The ERB's in a pretty deep hole on this one, having deleted from SGML an essential capacity for indirection. | 1. Leave it as it is And appear hidebound, ignoring the example of peer specs such as VRML, the existence of IETF activity on URNs, and the utility of indirection. Will that appeal to your target audiences? will that persuade the great unparsed public that SGML experts have a clue and that their XML spec can be regarded seriously? | 2. Agree that we'll put PUBLIC identifiers into XML *when* we are ready | to specify the resolution mechanism; the practical effect is almost | certainly that they don't go in for now. Equivalent to 1. | 3. Put a slot in the syntax for PUBLIC identifiers... | 3* ...and require an accompanying SYSTEM identifier Why require? Just say that the author has the two choices of how to refer, by URL (SYSTEM) or URN (PUBLIC), or both. If you like, point out that URN resolvers are no more widely deployed than user agents that read XML, and *recommend* use of a SYSTEM identifier if the author isn't confident that the PUBLIC ID will be resolved by his target audience. To the extent it doesn't work right now, people won't use it exclusively. | 3a - say nothing about resolution mechanisms, perhaps providing a | taxonomy of available technologies in this area Bingo. If you define PUBLIC IDs as URNs, resolution mechanisms are entirely out of scope. Which is a big win for the size and simplicity of the XML spec and of XML applications. But I think you must say that PUBLIC IDs are to be resolved as URNs to grasp the laurels. | 3b - document one resolution mechanism, but not make it required. | This is somewhat similar to our i18n approach, where we bless 2 | encodings but admit the possibility that people use others. That is, add unnecessary verbiage. Your i18n approach is not a good parallel; it does make requirements on user agents. | 3c - document one resolution mechanism as before, but make its use | mandatory for XML processors. Meaningless if PUBLIC IDs are URNs. | Note that there is a continuum between 3b and 3c; we could place | varying strengths of recommendation behind one resolution mechanism, | with homilies about document portability. Not really. So far as implementation conformance is concerned, there is only SHOULD and MUST. Jawboning counts for nothing in the Internet standards game. | In the area of which resolution mechanism to (perhaps nonexclusively) | bless, SGML/Open catalogs (hereinafter Socats) stand out, and would | probably be the ERB's choice. On another hand, there has been a lot Are you really willing to rest XML on the admittedly incomplete ("80% solution") and increasingly crufty (viz. the well-meaning proposals at the technical committee meeting in Boston to add more and more facilities to it) socat approach -- necessary as it may be off the Net -- instead of relying on independent Internet name resolution services and relieving yourselves of the necessity to deal with name resolution entirely? | of work go into the URN effort; on another hand, that work has not yet | born practical fruit in terms of ubiquitous implementations; on another This is as bogus as Lee's "URNs aren't ready yet". The URN work, while as undeployed as XML, is farther advanced, and XML isn't due to be finished for a year. I am unaware of any XML implementation that is even biquitous, much less ubiquitous. You are writing the XML spec in an environment of very rapid development, and you want to be able to sell it to people who work in that environment. Yesterday's URN session at IETF was eerily harmonious, especially for that group, *as if people understood that they were close to acheiving something they all think is extremely valuable and which they want to use as soon as possible*. The syntax draft is very nearly complete (it doesn't have whitespace problems). That's really about all that's necessary for people to start constructing resolution systems (beyond the current deployed demonstration of a scalable resolution mechanism). Nobody is worried that URNs won't work. | hand, the FPI syntax is repellent to some and it is not clear how well | it supports internationalization; on another hand, it may be the case | that FPI's really are URN's as they stand. Literally, no, there is a distinction between the name in its native name space and that name as encoded as part of a URN. E.g., spaces must be hex encoded (%20). They can be converted to URNs and resolved as such. The ERB must specify whether PUBLIC is to contain an FPI or a URN, e.g., urn:fpi:-//Davenport//DTD%20DocBook%V3.0//EN (I've forgotten whether the slashes must be escaped also.) | I suspect that if a binding vote were taken today, the ERB would either | (a) reinstate the PUBLIC keyword, and put in a nonexclusive recommendation | for Socat support, or | (b) refuse to put PUBLIC in until there was agreement on a required | resolution mechanism. which is to say, maybe never? Regards, Terry Allen Fujitsu Software Corp. tallen@fsc.fujitsu.com "In going on with these experiments, how many pretty systems do we build, which we soon find outselves obliged to destroy?" - Benjamin Franklin A Davenport Group Sponsor: http://www.ora.com/davenport/index.html Date: Fri, 13 Dec 1996 21:56:02 -0500 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: RE: RS/RE, again (sorry) At 02:58 PM 12/13/96 EST, lee@sq.com wrote: >(2) rather than using an attribute called -XML-SPACE, which looks odd and > perhaps even ugly, it would be better to have a PI that says > the attribute TYPE=LIT on the element SAMPLE > has this meaning: > > While I don't like your syntax, I support the sentiment of making this global somehow. Can't we specify that an application should presume that the whitespace handling attribute value is "inherited" from the last element of the same type? Should we even consider a more general mechanism for inheriting attributes? XML users cannot modify their DTD to get the default attributes behaviours that they want. Paul Prescod Date: Fri, 13 Dec 1996 19:12:19 -0800 (PST) From: Terry Allen To: eliot@isogen.com, w3c-sgml-wg@w3.org Subject: Re: Public IDs for notations in XML Eliot writes: | Notation declarations consist of a notation name followed by an external | identifier. The external identifier consists of either the keyword | "SYSTEM" followed by a literal containing a MIME type specification or the | keyword "PUBLIC" followed by a literal containing a public identifier | followed by a literal containing a MIME type specification. Executive summary: you don't need either. If the "MIME type specification" is a registered MIME type, then PUBLIC is appropriate. If not, SYSTEM is. But really, you shouldn't need either so long as you say that notation declarations must be MIME types: are equally comprehensible and both values are completely ready to be passed to the application that takes the output of the XML parser. | Thus, a notation declaration could either be: | | or | or, as noted, | If XML does not specify a public identifier resolution mechanism then it | would not be allowed to have a public identifier without a corresponding | MIME type. If there is such a mechanism, then it would be possible, | potentially, to do the mapping to MIME types there. ? MIME types are registered at a well known place. No pointer is required if they are registered; if not, local knowledge is required anyway, and it is required by applications outside the XML parser, not within the process of parsing XML. How do you set up a new unregistered MIME type in browsers now? Not with a URL in the HTML document that uses the unregistered MIME type, but by feeding the appropriate information to the (supra-HTML) browser application. What would you expect to obtain by resolving a URL anyway? a description of a MIME type in term of other MIME types? an applet that automatically plugs itself into your browser and registers itself locally with any arbitrary XML user agent? | One reason for allowing the public identifier for notations without a | resolution mechanism is to make the correlation between existing notations | used in SGML documents and the MIME types used for Internet delivery | clearer, especially to those who already have SGML documents. Obviously, | if you are coming from HTML and only ever thought about MIME types, you | don't necessarily need or want a public ID and don't need to specify one. If the reason for allowing something is educational, get rid of it. Your needs are the same regardless of your background. If you don't need to specify something, then you don't need to specify it. | In addition, many SGML processes (including much of architecture | processing) depend on notations and public identifiers for notations, so it | will make taking advantage of these facilities from XML much easier if we | allow public identifiers for notations even if we don't for entities. The need for public identifiers is obviated if you use MIME types, which are by definition labels for well known semantics. And like URNs, the semantics for MIME types must be resolvable outside of HTML user agents, so you might as well avoid creating an XML-specific way of pointing at MIME semantics. Lee responded to Eliot: | I think this is very sensible, although I would urge a list: | | | This would allow for content negotiation in an interesting and clean way. Content negotiation is a completely separate layer of the Internet architecture. Notation declarations (in an XML DTD) should not be bound to the preferences of any particular user agent. | It would mean | I like jpeg no less than gif (and maybe a lot more) | and I like gif no less than I like png, | so send me the image in the first matching format in this | list that you can, please. This is backwards. The user agent cannot express its preferences through the preestablished markup of a document it receives! Regards, Terry Allen Fujitsu Software Corp. tallen@fsc.fujitsu.com "In going on with these experiments, how many pretty systems do we build, which we soon find outselves obliged to destroy?" - Benjamin Franklin A Davenport Group Sponsor: http://www.ora.com/davenport/index.html Date: Fri, 13 Dec 1996 21:53:59 -0600 From: Len Bullard To: Terry Allen CC: eliot@isogen.com, w3c-sgml-wg@w3.org Subject: Re: Public IDs for notations in XML Terry Allen wrote: > > Eliot writes: > | Notation declarations consist of a notation name followed by an external > | identifier. The external identifier consists of either the keyword > | "SYSTEM" followed by a literal containing a MIME type specification or the > | keyword "PUBLIC" followed by a literal containing a public identifier > | followed by a literal containing a MIME type specification. > > Executive summary: you don't need either. > > If the "MIME type specification" is a registered MIME type, then > PUBLIC is appropriate. If not, SYSTEM is. But really, you > shouldn't need either so long as you say that notation declarations > must be MIME types: > > > Happy until the "must be". Notation declarations are more than MIME types in SGML. To me, MIME type means internet, not intranet. How does an XML handler know that an XML document is of a different type in the sense of DOCTYPE if a To: "'w3c-sgml-wg@w3.org'" Subject: RE: Understanding the Paoli proposal Date: Fri, 13 Dec 1996 20:47:37 -0800 Encoding: 69 TEXT You cannot expect a paragraph to look right in all applications because an application, by definition, has the right to do whatever it want with your data. However, our job is to define properly the input we are giving to such applications: such input comes as the output of the XML parser. The proposal that I did defines the output of the XML parser: -1/ All CR/LF/whitespaces are restituted or transformed using Delenda Est and go as the output of the XML parser -2/In addition, the author can communicate its intend, where he think whitespaces/CR/LF are important (such in PRE elements) by writing
          PRESERVE could also be used in semantical elements corresponding for
          example
          to quotations or poem where you do not want the text to be munged for
          formatting purpose.
          
          Response to your first question:
          
          I would not use PRESERVE as you used it in
          >

          If I want this paragraph to look right >in all applications

          because you expect the formatter of your application to "take care" of formatting of your paragraph. Your second question was about turning PRESERVE on an element-content. We have not discussed yet, if, in the case of adopting RE Delenda, we still -in XML- delete white spaces in element-content in a validating parser. >---------- >From: Paul Prescod[SMTP:papresco@calum.csclub.uwaterloo.ca] >Sent: Friday, December 13, 1996 5:35 PM >To: 'w3c-sgml-wg@w3.org' >Subject: Understanding the Paoli proposal > >At 02:11 PM 12/13/96 -0800, Jean Paoli wrote: >>this is why I was saying that the current XML spec was more convenient >>that the one I proposed in my mail: every application can decide how to >>deal with this. The authorial intent has not been expressed here. > >I'm getting lost in all of these proposals and amendments. If I want this >paragraph to look "right" in all applications, what do I do: > >

          If I want this paragraph to look right in all >applications

          > >or > >

          If I want this paragraph to look right >in all applications

          > >My understanding is the latter, because the former gives applications the >right to "arbitrarily" remove whitespace, and between the B and the I might >well look like a reasonable place to do so. > >What happens if I turn "on" PRESERVE mode in element content. Should a >validating parser tell me that my authorial intent is in contravention of my >DTD? Should it just silently eat the whitespaces despite my authorial >intent? Should it return a grove that is not valid according to the DTD? > >Thanks for any enlightenment, > > Paul Prescod > > From: lex@www.copsol.com (Alex Milowski) Subject: Re: ERB discussion of public identifiers To: w3c-sgml-wg@w3.org Date: Sat, 14 Dec 1996 11:42:01 -0600 (CST) > 1. Leave it as it is Please, no.... > 2. Agree that we'll put PUBLIC identifiers into XML *when* we are ready > to specify the resolution mechanism; the practical effect is almost > certainly that they don't go in for now. Maybe. Doing it this way would keep XML system from fragmenting and returning to the current state of "I do it better--do it my way!" marketing. > > 3. Put a slot in the syntax for PUBLIC identifiers... > 3* ...and require an accompanying SYSTEM identifier No. This would break existing documents. It is also philosophically wrong. If I give a public identifier, I really don't want to embed system identifiers. ... not to say that it wouldn't work in some cases. Now, I could write perl script that filters.... slap! sorry, I been writing too much perl/SGML filters lately. > 3a - say nothing about resolution mechanisms, perhaps providing a > taxonomy of available technologies in this area Yes! Resolution technologies do not belong in a document spec. > 3b - document one resolution mechanism, but not make it required. > This is somewhat similar to our i18n approach, where we bless 2 > encodings but admit the possibility that people use others. No. > 3c - document one resolution mechanism as before, but make its use > mandatory for XML processors. No. I would seem to me that we need *two* standards. One that specifies how the document is encoded and one that specifies the technical-environment-of-the-day-that-makes-it-go standard. Why couldn't we have XML standard and the XML-web environment standard for resolutions and other such things. One of the powers of SGML in a good number of the systems that I have built is the ability to replace the technology underneath and *not* change the documents. It would seem that in XML we want this as well. If we have a core XML document encoding standard and ancillary standards that define the environment for transport, resolution, etc. we can change the environment later and still use the document encoding standard. In addition, it can well define the difference between representation of information, understanding that representation, and environmental processing of such information. Such well defined components will keep people from developing systems in which encoding understanding is mixed with environment processing--as the current implementation of the web has done in some cases. It is my proposal that we have the following *separate* but related standards: * XML encoding as a subset of SGML - defines the encoding much as XML 1.0 does today. * XML-Web environment - defines resolution mechanisms, transport, etc. ...later on down the road we might want: * XML-CDROM environment * XML-MAIL environment ... etc. XML encoded documents *will* outlast the technology we use to transport and process it today. ============================================================================== R. Alexander Milowski http://www.copsol.com/ alex@copsol.com Copernican Solutions Incorporated (612) 379 - 3608 From: lex@www.copsol.com (Alex Milowski) Subject: An example of socats and nsgmls To: w3c-sgml-wg@w3.org Date: Sat, 14 Dec 1996 11:58:13 -0600 (CST) I have know that there was URL storage manager in SP for quite awhile. In lookup for a better construct that networked-mapped drives and SGML processing I decided to test it. James has, again, done a wonderful job and the URL storage manager gets things correct and it knows how to understand base interpretation from *where* the socat came from. I setup a small test document on our web server. If you are interested in seeing the SGML, go to: http://www.copsol.com/~lex You will find three files: CATALOG (obvious) blah.txt (an entity) simple.sgml (an extremely simple document) If you want to try it and you have nsgmls do the following: nsgmls -m "http://www.copsol.com/~lex/CATALOG" "http://www.copsol.com/~lex/simple.sgml" This is a basic example of relative paths to where the socat came from. I believe panorama does the same thing. Please don't beat on my server. Try it once. Its not a very exciting document. ============================================================================== R. Alexander Milowski http://www.copsol.com/ alex@copsol.com Copernican Solutions Incorporated (612) 379 - 3608 Date: 15 Dec 1996 00:43:09 +0000 (GMT) From: Peter Flynn Subject: Re: re ERB discussion of public identifiers Allen on 13 Dec 1996 18:08:33 -0800 (PST)) To: w3c-sgml-wg@w3.org Terry Allen writes: | 3. Put a slot in the syntax for PUBLIC identifiers... | 3* ...and require an accompanying SYSTEM identifier Why require? Because to go by current performance, processors and apps on the Web neither know nor care about PUBLIC indentifiers (with one or two obvious exclusions), and certainly wouldn't know where to go to broker a request for resolution. Just say that the author has the two choices of how to refer, by URL (SYSTEM) or URN (PUBLIC), or both. This is exactly the problem I pointed out the other day. It is insufficient to allow both without indicating the sequence in which they should be tried, regardless of how any resolution is to be performed. ///Peter Date: 15 Dec 1996 00:31:11 +0000 (GMT) From: Peter Flynn Subject: Re: Understanding the Paoli proposal from Paul Prescod on 13 Dec 1996 20:35:32 -0500) To: w3c-sgml-wg@w3.org

          If I want this paragraph to look right in all applications

          My understanding is the latter, because the former gives applications the right to "arbitrarily" remove whitespace, and between the B and the I might well look like a reasonable place to do so. But I thought whitespace in #pcdata was implicitly preserved (eg here in paragraphs where mixed content is not in use): it's only whitespace in locations where #pcdata is NOT permitted which can be removed (eg the TABs, newlines, and spaces inserted by psgml to indent the structure while editing). ///Peter Date: Sat, 14 Dec 1996 17:15:26 -0800 (PST) From: Terry Allen To: pflynn@curia.ucc.ie, w3c-sgml-wg@w3.org Subject: Re: re ERB discussion of public identifiers Peter Flynn writes: | Terry Allen writes: | | 3. Put a slot in the syntax for PUBLIC identifiers... | | 3* ...and require an accompanying SYSTEM identifier | Why require? | Because to go by current performance, processors and apps on the Web | neither know nor care about PUBLIC indentifiers (with one or two | obvious exclusions), and certainly wouldn't know where to go to broker | a request for resolution. If they cannot handle PUBLIC right now, then SYSTEM should be used by authors. But that is different from requiring conformant docs to include SYSTEM. What happens when people start to find URN resolution services to be useful and reliable? Do they then have to include a bogus SYSTEM value just to conform? (Today I found this PUBLIC placeholder: -//asdfasdf//asfdsf//EN.) | Just say that the author has the two choices of | how to refer, by URL (SYSTEM) or URN (PUBLIC), or both. | This is exactly the problem I pointed out the other day. It is | insufficient to allow both without indicating the sequence in which | they should be tried, regardless of how any resolution is to be performed. And I meant to reply to your point, but forgot. - why is order significant to correct resolution? - why does the XML spec need to specify the order rather than the user's configuration file? - why shouldn't the user have the option of trying both, and doing something further if the two results don't match? Regards, Terry Allen Fujitsu Software Corp. tallen@fsc.fujitsu.com "In going on with these experiments, how many pretty systems do we build, which we soon find outselves obliged to destroy?" - Benjamin Franklin A Davenport Group Sponsor: http://www.ora.com/davenport/index.html From: dgd@cs.bu.edu (David Durand) Date: Sat, 14 Dec 1996 20:44:12 -0500 (EST) To: w3c-sgml-wg@w3.org Subject: Re: re ERB discussion of public identifiers From: Peter Flynn Subject: Re: re ERB discussion of public identifiers Terry Allen writes: Why require? (a SYSTEM id) Because to go by current performance, processors and apps on the Web neither know nor care about PUBLIC indentifiers (with one or two obvious exclusions), and certainly wouldn't know where to go to broker a request for resolution. I agree with Terry that this is theoretically suspect, but practical considerations have made me side with Peter up till now. However, Terry made a good point -- URNs will not be widely used for publication until thereS is a widely implemented resolution base, so there will be no need to require SYSTEM IDs as people will need them and include them. Once at least one URN scheme is widely deployed, then there may be no point in requiring a SYSTEM ID. I find this market logic pretty convincing, but I can accept the compromise as require SYSTEM IDs even though they may not be necessary. Just say that the author has the two choices of how to refer, by URL (SYSTEM) or URN (PUBLIC), or both. This is exactly the problem I pointed out the other day. It is insufficient to allow both without indicating the sequence in which they should be tried, regardless of how any resolution is to be performed. I think this point is important: even more importantly, the URN (FPI) should take priority, if it is resolvable -- since URN resolution, when applicable, has more options to produce the "best" copy -- since it is constrained only by the identity restrictions of URNs, rather than the location restrictions of SYSTEM identifiers. ///Peter Date: Sun, 15 Dec 1996 08:11:16 -0500 To: "'w3c-sgml-wg@w3.org'" From: Paul Prescod Subject: RE: Understanding the Paoli proposal At 08:47 PM 12/13/96 -0800, Jean Paoli wrote: >Your second question was about turning PRESERVE on an element-content. >We have not discussed yet, if, in the case of adopting RE Delenda, we >still -in XML- delete white spaces >in element-content in a validating parser. Well, this is what I'm really interested in. It is clear to me that the non-validating parsers' whitespace handling will not really be useful for mission critical data handling. When I care, such as in authoring tools and databases, I will have to ask my vendors to supply validating parsers. I need to know how whitespace there will be handled. Paul Prescod Date: Sun, 15 Dec 1996 11:05:26 -0500 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: XML and stylesheets I don't think we want to revisit this whitespace issue after we attack stylesheets. We are currently defining the input to the stylesheet engine, so we whould discuss it now. If we presume that most or all occurances of whitespace in element content will be unlabelled and undifferentiated from whitespace in mixed content, then only the stylesheet writer can properly remove source-formatting whitespace. We've said this before, but I want to make sure that we all have the same understanding. If I understand correctly, * the stylesheet writer will explicitly have to have a declaration for every element with an element-content content-model to surpress the whitespace * OR we could make whitespace elimination the default in the stylesheet language in which case they would have to have an explicit declaration for every element that has mixed content so that whitespace will not be removed erroneously * OR the author must avoid all whitespace in element content. The first will make much larger stylesheets, with lots of repetition. The second will make larger (but perhaps not much larger) stylesheets, but is incompatible with DSSSL as it exists. The last is user-hostile. Have I forgotten any options? Are we satisfied with these ones? Paul Prescod Date: Sat, 14 Dec 1996 18:23:40 -0800 From: bosak@atlantic-83.eng.sun.com (Jon Bosak) To: w3c-sgml-wg@w3.org CC: bosak@atlantic-83.eng.sun.com Subject: Re: Things we should not forget... [Dave Durand:] | We need to use -XML- in front of all attribute values, unless we | change our minds on the attribute-value restriction. We're working to see whether ISO can be convinced to change this in 8879 (it's backward-compatible with existing SGML documents). | We need to document that the ISO entity sets will not work with XML, | as SDATA has been eliminated, and either develop replacements, or let | people know that they will have to develop them themselves. I'd like to consider this in the time following the April Phase II deadline. Jon Date: Sun, 15 Dec 1996 14:11:39 -0500 To: w3c-sgml-wg@w3.org From: dgd@cs.bu.edu (David G. Durand) Subject: Attempt at listing requirements, constraints and proposal fragments Summary: I think we are arguing at cross purposes, because we are not being explicit enough about requirements and strategies. This message attempts to summarize key points, and does not advocate (or intend to) slant towards any conclusion. One of the reasons that Jean's posting has created confusion is that he is intorducing a new requirement: easy of whitespace insertion for editors to break long "lines". I'm just going to try liting the requirements as I think they are: if we can get a list that we can agree on, maybe we can grease ourselves up and get detached from this tar-baby. 1. Almost everyone desires that the parse tree be identical (or as similar as possible) when parsing with and without a DTD. 2. Some people believe very strongly that the ignoring of whitespace in elment content is important because: a) current SGML editors indent in element content mode. b) human readers of documents find whitespace around tags useful for readability. c) stylesheets and applications should no be made responsible for removing whitespace, as they will then not handle things consistently. 3. Jean Paoli has identified a need for future editing software to insert linebreaks as desired (This might, in some cases even occur in mixed content). 4. Most people seem to agree that mixed content should preserve almost all whitespace. There is now even an SGML-compatible way to preserve intial RE in mixed content. Some factors constrain solutions: A. SGML, as defined, ignores whitespace in element content. B. XML, when parsing without a DTD, cannot detect element content reliably. C. SGML does not normally preserve initial RE in mixed content. Some possible techniques may address some of these factors in reaching some of the goals. 1. Charles' shortref RE hack can disable C. without affecting anything else. This would allow SGML-compatible application processing of all whitespace in _mixed_ content. 2. Explicit flags of some sort on elements could signal to an application that it should apply a particular whitespace strategy. Thus, we could pass whitespace to the application, and make the author responsible for marking whether or not they are to be significant. 3. The application could set these flags automatically, in some cases, when parsing with a DTD (element content automatically -XML-SPACE=IGNORE, unless the _author_ requests otherwise, for instance). 4. We could take the Perl approach and have the parser guess and set any missing WS flags. FOr instance, it would set the -XML-SPACE=IGNORE flag for elements containing only WS and other subelements. It could set the -XML-SPACE=COLLAPSE flag for elements containing non-space characters, and (optionally) other markup. The drawback would be complicated rules for the default processing, that can at least be defeated by explicitly tagging elements. This also leads to more-complex implementations. 5. We could take a strict RE delenda est approach: needing no flags, but also only satisfying requirements 1 and 4. 6. Since we can put whitespace before TAGC in any of these proposals. This may satisfy requirement 3. Many feel that this technique utterly fails to satisfy requirements 2a, and 2b, because editors don't work that way now, and the syntax is ugly, respectively. ] [ Ed: Personally I think this technoque completely solves requirement 3 without any work by us at all. It is trivial to implement, and also works in mixed content, providing a better solution to the algorithmic problem.] There are probably missing points and misrepresentations, but can we try to get a list like this finalized before we continue arguing specific proposals? If you like this idea, and want to make a correction, edit the list and re-post it, so that we can converge on a single list without too much pain. -- David I am not a number. I am an undefined character. _________________________________________ David Durand dgd@cs.bu.edu \ david@dynamicDiagrams.com Boston University Computer Science \ Sr. Analyst http://www.cs.bu.edu/students/grads/dgd/ \ Dynamic Diagrams --------------------------------------------\ http://dynamicDiagrams.com/ MAPA: mapping for the WWW \__________________________ Date: Sun, 15 Dec 96 13:53:39 CST From: paul@arbortext.com (Paul Grosso) To: w3c-sgml-wg@w3.org Subject: Re: ERB discussion of public identifiers > From: Tim Bray > > Thus, there are a variety of options open to us. > > 1. Leave it as it is > > 2. Agree that we'll put PUBLIC identifiers into XML *when* we are ready > to specify the resolution mechanism; the practical effect is almost > certainly that they don't go in for now. > > 3. Put a slot in the syntax for PUBLIC identifiers... > 3* ...and require an accompanying SYSTEM identifier > > 3a - say nothing about resolution mechanisms, perhaps providing a > taxonomy of available technologies in this area > 3b - document one resolution mechanism, but not make it required. > This is somewhat similar to our i18n approach, where we bless 2 > encodings but admit the possibility that people use others. > 3c - document one resolution mechanism as before, but make its use > mandatory for XML processors. Even though I was editor of the SGML Open "catalog" Resolution, I have read all the email and considered all the options. I do want public ids and the indirection they give. That means there has to be some resolution mechanism (not that it has to be in the XML spec or there is only one, but merely that resolution will be a necessary step when using public ids). Since SGML Open catalogs have been accepted, implemented, and used as one resolution mechanism, it is important to me to be able to continue to use them to resolve XML public ids. Since I can see benefit in allowing other resolution mechanisms, I am led to choices 3a or 3b. In the interest of having interoperable tools, I lean toward 3b. In fact, I would support having the XML spec require support for some (possibly simplified as per Ken Holman's posting) catalog resolution. Almost all the postings I have seen against mandating a resolution mechanism give good reasons why users/authors/tools shouldn't be restricted to a single resolution mechanism, but I have not seen arguments (except the "it's too hard to implement" argument that I have not quite accepted) to suggest that there is a downside to requiring XML tools to support catalog resolution. What I'm saying differs from 3c in that the use of catalogs would not be mandatory, but *if* the user/author wants to use catalogs, support for its use would be mandatory for all XML tools. Finally, I'd be happier with any of the 3-ish choices than not having public ids in XML 1.0. > From: Peter Flynn > > 3. Put a slot in the syntax for PUBLIC identifiers... > 3* ...and require an accompanying SYSTEM identifier > > As a list newcomer, I wasn't party to the previous debate on this, but > whichever way it goes, I feel it is essential to specify what the > order of acceptance is. The current position in SGML is a serious > problem for users and implementors, where one application will use the > SYSTEM identifier if it is there, and ignore the PUBLIC identifier > even if it cannot locate the resource referred to by the SYSTEM > identifier; and another application takes the PUBLIC identifer first > and ignores the SYSTEM identifer even if it cannot resolve the PUBLIC > identifier; and yet others work only with PUBLIC identifers or SYSTEM > identifers but not both. I would not want XML to require a SYSTEM identifier. I also wouldn't want it to require a PUBLIC id. I'd like to leave it as it is now in 8879 in that respect. This gives maximum flexibility to the user. As far as precedence of the PUBLIC versus SYSTEM id when both are given, the SGML Open technical committee had long discussions on this topic in 1993 and 94 when drafting TR9401. Some SGML luminaries argued that it was obvious that the SYSTEM id should always be preferred while others argued that certainly the PUBLIC id should always be preferred. After much heated discussion, if nothing else was clear it was that neither position is obvious. TR9401 decided to say that all implementations must support the user's ability to say whether they wanted to prefer the PUBLIC or SYSTEM id. In fact, the same user may wish at one time to give preference to the PUBLIC ids and at another time to give preference to the SYSTEM ids. Provided that all catalog implementations comply with the requirement to give the user this choice, I see no problem. (Peter, perhaps you can tell me more off-line about: The current position in SGML is a serious problem for users and implementors, where one application will use the SYSTEM identifier if it is there, and ignore the PUBLIC identifier even if it cannot locate the resource referred to by the SYSTEM identifier; and another application takes the PUBLIC identifer first and ignores the SYSTEM identifer even if it cannot resolve the PUBLIC identifier; and yet others work only with PUBLIC identifers or SYSTEM identifers but not both. It sounds to me like you must be looking at non-compliant implementations.) paul Date: Sun, 15 Dec 96 13:05:36 CST From: paul@arbortext.com (Paul Grosso) To: w3c-sgml-wg@w3.org Subject: Re: simplifying comments in SGML '97 > From: Michael Sperberg-McQueen > > The XML spec of 14 November defines as a 'comment' what in 8879 is > called a 'comment declaration' -- to reduce confusion, in what > follows I am going to try to use the 8879 terminology, not the XML > terminology. > > The current version of the XML spec differs from 8879 in various ways: > > (a) XML allows comments *only* in comment declarations; 8879 > allows them in other markup declarations as well (though not > in all locations) > (b) XML allows exactly one comment in a comment declaration; > 8879 allows zero or more > (c) XML defines two delimiters '' which bound the > construct in question; 8879 sees three delimiter roles involved > here, and allows white space in some locations: XML '' corresponds to 'com, s*, mdc' (again, productions 91-92) > > N.B. I think this list is complete but may be wrong. > > The main perceived problem with the XML spec is that the comment itself > is barred not only from containing '-->' (its closing delimiter) but '--'. While I'll admit to being somewhat inclined to use mdashes in my writing--and have occasionally been caught doing so in an SGML comment declaration--I do not see the benefits of being able to put "--" in a comment as being so great as to warrant doing something that is either particular complicated or different from what SGML parsers do today. When people start getting too concerned about what they can put into comments, they are probably putting too much into comments, and should probably consider creating elements in their DTD to hold the supposed "comments." I think each time we propose changing XML's syntax from that which is the syntax of the SGML in common usage for the last 10 years, we need to ask if the preceived benefits are worth it. In this case, I prefer to leave XML comments as defined in the XML draft. Date: 15 Dec 1996 20:37:51 +0000 (GMT) From: Peter Flynn Subject: Re: re ERB discussion of public identifiers Allen on 14 Dec 1996 17:15:26 -0800 (PST)) To: w3c-sgml-wg@w3.org Terry Allen writes: And I meant to reply to your point, but forgot. - why is order significant to correct resolution? Not so much the order, as bothering to see if the other reference is given. The problem arises when you carefully provide a SYSTEM identifier so that if the user's host cannot provide PUBLIC resolution, the SYSTEM one can be used...and the app just looks at the PUBLIC one, says "I can't do this" and gives up, never looking to see if there was a SYSTEM identifier that might be usable. [And by the same token, apps which grab the SYSTEM identifier first, find it doesn't work (or they can't make it work) and then don't bother trying the carefully-provided PUBLIC identifier.] - why does the XML spec need to specify the order rather than the user's configuration file? If the user has a config file that specifies it, sure, use it, no problem. - why shouldn't the user have the option of trying both, and doing something further if the two results don't match? They should. But it may be the app that won't let them do this. ///Peter Date: 15 Dec 1996 21:20:49 +0000 (GMT) From: Peter Flynn Subject: Re: XML and stylesheets from Paul Prescod on 15 Dec 1996 11:05:26 -0500) To: w3c-sgml-wg@w3.org Paul Prescod writes: If we presume that most or all occurrences of whitespace in element content will be unlabelled, and undifferentiated from whitespace in mixed content, then only the stylesheet writer can properly remove source-formatting whitespace. We've said this before, but I want to make sure that we all have the same understanding. That's the most succinct I've seen yet. * the stylesheet writer will explicitly have to have a declaration for every element with an element-content content-model to surpress the whitespace * OR we could make whitespace elimination the default in the stylesheet language in which case they would have to have an explicit declaration for every element that has mixed content so that whitespace will not be removed erroneously * OR the author must avoid all whitespace in element content. Have I forgotten any options? Are we satisfied with these ones? What's the real take on (b)'s incompatibility with DSSSL? I still don't grasp why we can't make white-space removal the default for mixed-content elements, and retention the default for all others (I think Panorama does it this way: at least, it did it to me this evening, and I had to insert more markup in an abstruse area of the TEI header to overcome it :-) This approach also has the [dis?]advantage of acting as a gentle reminder to authors that mixed content is evil :-) ... or is that seen as too user-hostile? ///Peter Date: Sun, 15 Dec 1996 16:39:23 -0500 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: XML and stylesheets At 09:20 PM 12/15/96 +0000, Peter Flynn wrote: >What's the real take on (b)'s incompatibility with DSSSL?=20 b) * OR we could make whitespace elimination the default in the stylesheet language in which case they would have to have an explicit declaration for every element that has mixed content so that whitespace will not be removed erroneously The result returned by the default construction-rule shall depend on the type of node to which it is applied:=20 =B7 for a node with a char property, it shall return (make character). We would have to change that to: "for a node with a char property, it shall return (make character), if the node is not entirely composed of whitespace" or "for a node with a char property, it shall rreturn (empty-sosofo)." >I still >don't grasp why we can't make white-space removal the default for >mixed-content elements, and retention the default for all others We're presuming that the input to the stylesheet engine is the output of a parser that doesn't have a DTD. So it must presume that all content is mixed content (or make guesses!). >This approach also has the [dis?]advantage of acting as a gentle >reminder to authors that mixed content is evil :-) ... or is that seen >as too user-hostile? To be clear, I am using the definition of mixed content on page 320 of the SGML handbook, any mix of data characters, elements and "other stuff" (pis, comments, etc.). If that's evil, then SGML lends itself to nefarious= purposes... Paul Prescod Date: 15 Dec 1996 21:54:19 +0000 (GMT) From: Peter Flynn Subject: Re: ERB discussion of public identifiers To: w3c-sgml-wg@w3.org Paul writes: I would not want XML to require a SYSTEM identifier. I also wouldn't want it to require a PUBLIC id. I'd like to leave it as it is now in 8879 in that respect. This gives maximum flexibility to the user. As far as precedence of the PUBLIC versus SYSTEM id when both are given, the SGML Open technical committee had long discussions on this topic I seem to have opened an inadvertent can of worms here, sorry. It's not even the precedence which concerns me, so much as making sure apps try the other identifier if both are provided and one fails to resolve. I know this is an app issue, but I do feel the spec should make the point that this performance is expected (trying the other), otherwise we risk misleading authors and users into believing that in providing both identifiers they are somehow improving reliability. ///Peter Date: 15 Dec 1996 22:05:32 +0000 (GMT) From: Peter Flynn Subject: Re: XML and stylesheets from Paul Prescod on 15 Dec 1996 16:39:23 -0500) To: w3c-sgml-wg@w3.org >I still >don't grasp why we can't make white-space removal the default for >mixed-content elements, and retention the default for all others Paul Prescod replied: We're presuming that the input to the stylesheet engine is the output of a parser that doesn't have a DTD. So it must presume that all content is mixed content (or make guesses!). Sorry, I missed this condition (before I arrived). If there's no DTD, you simply throw all remaining (non-markup) bytes at the app, which should pop up a big red flag if it wants to be nice, otherwise just use some Microsoft/Netscape guesses about what the author must have had in mind... >reminder to authors that mixed content is evil :-) ... or is that seen To be clear, I am using the definition of mixed content on page 320 of the SGML handbook, any mix of data characters, elements and "other stuff" (pis, comments, etc.). If that's evil, then SGML lends itself to nefarious= purposes... I agree with you 100%...I've seen some pretty evil DTDs... :-) ///Peter Date: Sun, 15 Dec 1996 17:07:19 -0500 To: w3c-sgml-wg@w3.org From: Paul Prescod Subject: Re: ERB discussion of public identifiers At 09:54 PM 12/15/96 +0000, Peter Flynn wrote: >I seem to have opened an inadvertent can of worms here, sorry. It's >not even the precedence which concerns me, so much as making sure apps >try the other identifier if both are provided and one fails to >resolve. I know this is an app issue, but I do feel the spec should >make the point that this performance is expected (trying the other), >otherwise we risk misleading authors and users into believing that in >providing both identifiers they are somehow improving reliability. I think you are absolutely right, Peter. Providing both *should* improve reliability. I would say that apps that don't try both are not doing the Right Thing (no matter what the spec says), but maybe there are extenuating circumstances. After all, isn't it the job of an app to use all of the information at its disposal to try to solve the user's problems? If a line in the spec can clarify this situation, then I certainly support it. On the other hand, I'm surprised that Paul says that the ordering issue is contentious. If the reader has overridden a public identifier, shouldn't that always take precedence over the author's system id, from his machine?? What is the counter argument? Paul Prescod Date: 15 Dec 1996 22:14:06 +0000 (GMT) From: Peter Flynn Subject: Re: ERB discussion of public identifiers from Paul Prescod on 15 Dec 1996 17:07:19 -0500) To: w3c-sgml-wg@w3.org On the other hand, I'm surprised that Paul says that the ordering issue is contentious. If the reader has overridden a public identifier, shouldn't that always take precedence over the author's system id, from his machine?? What is the counter argument? I think it's more along the lines that the SYSTEM identifier is not seen as an override by authors, but as a failsafe because of the reputed poor resolution of FPIs. It took me weeks to work out how they resolved, when I first encountered them, and they still won't work on some systems (W95 won't allow colon in long filenames; some users can't get write access to the right directory to install them :-), so I feel some sympathy for users who consign them to resource-broker hell and stick to SYSTEM identifiers. If we include PUBLIC identifiers, the resolution must be documented and must work. Much more useful would be a URL-style resolution that would try a fully-qualified URL, but also strip down to the directories and filename and try the local disk as well (someone mentioned the intranet problem), but that's outside the scope of the current discussion. ///Peter Date: Sun, 15 Dec 1996 18:42:10 -0600 From: Len Bullard To: Paul Grosso CC: w3c-sgml-wg@w3.org Subject: Re: simplifying comments in SGML '97 Paul Grosso wrote: > > I think each time we propose changing XML's syntax from that which > is the syntax of the SGML in common usage for the last 10 years, we > need to ask if the preceived benefits are worth it. In this case, > I prefer to leave XML comments as defined in the XML draft. Agree. Len Bullard Date: Sun, 15 Dec 1996 18:45:20 -0600 From: Len Bullard To: Paul Grosso CC: w3c-sgml-wg@w3.org Subject: Re: ERB discussion of public identifiers Paul Grosso wrote: > > TR9401 decided to say that all implementations > must support the user's ability to say whether they wanted to prefer the > PUBLIC or SYSTEM id. In fact, the same user may wish at one time to give > preference to the PUBLIC ids and at another time to give preference to the > SYSTEM ids. > > Provided that all catalog implementations comply with the requirement to > give the user this choice, I see no problem. Agree. Len Bullard