by Rohit Khare and Adam Rifkin
June 26, 1997
$Id: xml.html,v 1.38 1997/06/27 00:40:23 adam Exp $
Draft: Not for Redistribution. Charles: Can you check the information to see if there's anything to add, and if so, please send it to Rohit (khare@mci.net). Meredith: I will be out of town, and Rohit wants to do one more edit of this and then it's all yours. Please let him know if, after he sends you his edit, there is any more information you need about this article (khare@mci.net). Thanks! I will be offline until July 11, but I will be calling Rohit regularly. -- Adam
Just over five centuries ago, European cartographers portrayed the world as flat in its structure and form. A small but dedicated team navigating a triad of ocean-crossing vessels liberated the world from its seemingly insurmountable two dimensions. Half a millenium later, we enjoy the fruits of a three-dimensional, well-rounded globe.
Unfortunately, the World Wide Web in its present incarnation suffers from the same characteristics the European world experienced 500 years ago: its designers originally mapped it out to be flat in its structure and form. And now, a small but dedicated team of architects is navigating a sea of change to liberate the world wide web from its seemingly insurmountable two dimensions.
When we say that The World Wide Web is flat, we mean that the Web lacks the ability to understand itself. That is, it only has the ability to present information; it does not have the ability to know what that information means and use that knowledge to facilitate tasks employing the information.
The key to helping the Web to understand itself is self-description. Using self-description, document contents can be supplemented with codifications of structured metadata, or information about the information. If the metadata is machine-readable, programs can be written to process that information automatically, using the metadata as a roadmap.
The present-day Web has a few facilities for marking up the information within a document with metadata: the HTML tags. This small tagset is elegant in its simplicity, allowing you as a Web author to say what you mean. Saying what you mean is important for human readers of the information, because it helps to present the information in a more meaningful way to those readers. But, saying what you mean is even more important for computer readers of the information, because computers can automate tasks such as generating reports, answering questions, and updating data.
HTML was fine for what it was designed to do: mark up physics papers with tags both to let authors better say what they mean, and to allow authors to connect papers to other related work. It was invented by Tim Berners-Lee around 1990 as part of a holy triad --- along with URLs and HTTP --- that allowed CERN scientists to write physics papers and link them together.
The problem is, the World Wide Web went worldwide. In the past seven years, many flavors of document have surfaced in every nook of the Web: stock-watching pages, news columns, PhD theses, recipe books, diaries, poems, executable computer programs, movie reviews, and those seemingly-ubiquitous opinion pieces, to name a few. Wouldn't it be wonderful if each type of document could have its own custom ontology for describing its content?
That is, wouldn't it be great if each kind of document could have specialized metadata tags appropriate to all documents of that kind? For example, in our PhD theses, as is common with all PhD theses, we might want tags to denote citations, footnotes, quotations, figures, and tables. This is why many people pushed HTML to evolve to include tags that could be used for many of these features.
However, extending HTML with new tags is cumbersome; how can authors be expected to keep up with continual changes? Furthermore, extending HTML with new tags is unnecessary; most tags would go unused in most documents. Fear not, Web author: a solution to generalizing metadata tags in documents has existed for more than 35 years...
XML's family tree extends back three and a half decades to the Generalized Markup Language (GML) that was later standardized by ISO (SGML); this distinguished ancestry demonstrates that the same philosophy has been in place since the mid 1960s. This philosophy has resurfaced now to meet the emerging demands of Web developers and users, who increasingly want the content as well as the form of the Web to be self-describing.
XML's self-description facilities allow the creation of data structures far richer than vanilla HTML documents afford. For example, consider a "business card" document in both HTML and XML. With HTML, you can only mark up items common to every business card document --- such as your first and last names, company, work address, email address, fax number, and phone number --- with tags that are useful for indicating structure (here a <p align=center>, there a <dl> ... </dl>) but say nothing as to what the content actually means.
With XML, you can develop your own personal business card document that conforms to the Business Card document type definition espoused by the American Businesscard Consortium. That is, the ABC publishes a Business Card document specification on the Web, which includes the requisite tags (and their meanings), syntactic structure, linking, and style, for a document to conform to their "business card standard." Once you develop my business card document --- perhaps using a program that understands the format and generates it automatically from a snappy User Interface --- you can use a software tool to verify that my business card document is "ABC-compliant."
Note that ABC-compliance is oh-so more than just a clever sticker that I can affix to my document: it means that you have marked up your document with tags that have a commonly understood meaning. As a result, programs can parse the tags to use the document's information contents in specialized ways. Imagine, for instance, that instead of having to fill out Web forms manually as is presently the case, you could drag-and-drop your business card document into that form and have the fields of that form (name, address, and so forth) be filled out automatically. Better yet, you could feed your business card document into a Java applet, which could then parse the information content with a specific goal in mind, such as confirming you for software beta testing status, forwarding your contact information to a company's HR division, or signing your name to an electronic freedom and privacy petition.
Hence, by leaving your content data in a declarative format --- in this case, supplementing the information bits with business card-specific metainformation --- you allow a wide suite of programs to use that data because the tags are machine-readable and have accepted semantics. Even if there were no other compelling reason to use XML, its metadata properties make it far more powerful than static, proprietary formats such as troff or postscript or Java's serialization format.
Business card documents change only when their authors have switched jobs; as a result, the information contained within such documents is mostly static. What is interesting about XML is that it does not just provide hooks to these static types of information: it also interfaces to dynamically-updated information. Metainformation provided by XML tags can specify the automation of what kinds of information to pull into a document, from where, using what content filters, for every specified period of time.
This dynamic document creation feature can be used for customized information pull, for automated database report generations, or for a debuggable, archival serialization format in object systems. The extensibility facilities of XML will allow Web data formats to evolve as needed as the system grows and changes. Many present state-of-the-art tools add specialized tags to HTML for specific purposes which the Web infrastructure as a whole does not accomodate; as a simple dialect of SGML, XML is establishes guidelines and mechanisms for adding meaningful extensions or for using a completely separate set of tags.
XML's relationship with SGML and HTML has really been evolving for the past decade. HTML is the HyperText Markup Language standardized by World Wide Web Consortium (W3C) for storing and exchanging documents on the World Wide Web. HTML was designed to be simple enough to support ease of authoring Web pages, rich enough to support multimedia embedding in documents, and flexible enough to support hypertext linking.
HTML is an SGML format. SGML, the Standard Generalized Markup Language standardized by ISO for defining and using portable document formats, was designed to be formal enough to allow proofs of document validity, structured enough to handle complex documents, and extensible enough to support management of large information repositories.
W3C's SGML working group, with present efforts given in their activity page, is attempting to standardize the delivery (in Web documents) of self-describing data structures with arbitrary depth and complexity. To that end, they are simplifying SGML for use with the Web (and Web technologies such as Java).
XML, the Extensible Markup Language [Connolly and Bosak, 1997], is a simplified (but strict) subset of SGML that maintains the SGML features of validation, structure, and extensibility. XML is a standardized text format designed specifically for transmitting structured data to Web applications. In addition, XML's goals of being easier to learn, use, and implement than full SGML will have clear benefits for World Wide Web users, making it easier to define and validate document types, to author and manage SGML-defined documents, and to transmit and share them across the Web.
The Extensible Markup Language specification describes XML documents, a class of data objects stored on computers, and partially describes the behavior of XML processor programs used to read XML documents and provide access to their content and structure. XML allows generic SGML to be served, received, and processed on the Web in a manner similar to what is done with HTML today. XML has been designed for ease of implementation and for interoperability with both SGML and HTML.
XML documents are composed of entities, which are storage units containing text and/or binary data [Bos, 1997a]. Text is composed of character streams that form both the document character data and the document markup. Markup describes the document's storage layout and logical structure. XML also provides a markup mechanism to impose constraints on the storage layout and logical structure of documents [Bos, 1997b], and it provides mechanisms that can be used for strong typing [Bray, 1997c].
XML, like SGML, is a meta-language for describing the markup of different types of documents. However, its specification is 26 pages (versus 500 for SGML!). The W3C hopes that offering a simplified version of SGML will make implementing SGML much more palatable to vendors of Web authoring and browsing tools.
XML is not a replacement for SGML. Many features of SGML were left out to keep XML simple. Current SGML users may choose XML for network delivery, and since XML is a valid subset of SGML, the translation from SGML to XML is straightforward. XML was developed as an easy on-ramp to SGML for people who are not yet using it.
To simplify SGML, the W3C working group dropped support for certain features that put a heavy processing burden on SGML client software. For example, a well-formed XML document is unambiguous, so a browser or editor can read the tags and create a tree of the hierarchical structure without having to read its document type definition. XML also does not allow markup minimization, requires that empty elements be self-identifying, and does not support several other complex SGML standard features.
XML is not a replacement for HTML, either: HTML is a useful tool for storing and exchanging small hypermedia documents across the Internet. Furthermore, it is easy to generate HTML documents on the fly from XML (or SGML) documents. XML is designed to complement HTML by enabling different kinds of data to be exchanged over the Web.
For example, current limitations in World Wide Web technologies do not allow the extensibility, structure, and data checking necessary for large-scale commercial Web publishing. Jon Bosak's excellent paper XML, Java and the Future of the Web [Bosak, 1997] explains how XML can enable advanced Web applications, allowing Java applets to embed powerful, automatable data manipulation facilities directly into Web clients.
Unlike HTML, which has a fixed (though seemingly ever-changing) set of tags, XML lets you define your own tags and attributes. Support for XML by the Internet community would open up vast new possibilities for Internet publishing: instead of shoehorning all documents into HTML, or having to invent a browser to handle non-HTML documents, XML would enable a wide array of documents with user-defined tagsets to be handled by generic Web application software. As XML guru Tim Bray once pointed out, "[XML allows us to] finally get off the HTML treadmill."
Presently, an author can create rich documents with an application, and then use a Java applet viewer to attach those documents to Web pages. As long as the browsers continue to provide only crude formatting, such measures are unfortunate but inevitable, much in the same way people use desktop publishing applications to get better typography than can be done with off-the-shelf word processors.
But there is no reason why the concept of a "basic Web page" needs to be limited to a single tag set! The appeal of the Web is its simple hypertext scheme, which provides a simple, unambiguous method of pointing to files with unique names. Although it is handy that HTML is also simple, the success of word processors has demonstrated that consumers can cope with multiple document types.
When XML becomes more widespread, Web authoring tools will become much more flexible in handling basic document constructs. WordPerfect and Word will export directly into XML, using the style names as tags instead of filtering everything into 90 (or however many currently exist) predefined tags.
In such a brave new World Wide Web, Java's role will be to do interesting things with the content, such as mediation between formats, computation and event handling, automation of tasks and dynamic content, presentation of different views to different viewers, and even intelligent filtering of content. XML specification co-editor Tim Bray succinctly put it, "XML gives Java something to chew on."
The W3C is a vendor-neutral international industry consortium founded in 1994 to develop common protocol specifications for the evolving World Wide Web. W3C team members pursue many different Activities, or technical directions. A W3C Activity starts when a new issue relevant to the Web is discovered; then, W3C team members organize a workshop to collect information, opinions, and ideas. This leads to the formation of a working group with specific short-term goals, watched over by an Editorial Review Board elected from nominated experts by the Members, who review the resulting work as expert consultants. W3C results may then enter the IETF standards track; for example, W3C and the IETF are currently in partnership on the development of HTTP 1.1 and Distributed Authoring and Versioning.
XML was invented by a small band of renegades, most of whom had been longtime SGML supporters. The W3C SGML Working Group and Editorial Review board were chartered for a single year (July 1996 through July 1997) with Jon Bosak of Sun Microsystems as chair, and Dan Connolly as the W3C staff contact person. XML's definition was expedited because the Web is amenable to fast, easy communication and collaboration. The result has been a snowball effect, allowing XML to gain momentum and exposure rapidly as its specification is refined by the working group. The following timeline denotes some of the milestones in the XML effort.
XML provides a standard way for information providers to add custom markup to information-rich documents, so that complex documents can be rendered (and published) in a dynamic way. Compare this with HTML: although HTML is easy to use, its simplicity significantly constrains how publishers and users can represent and use documents and databases. XML provides the means to publish and receive any information, regardless of format or origin, in any way desired.
With XML, users will be able to manage documents dynamically in a Web browser, allowing the presentation of personalized Web-based information. Content providers will be able to distribute structured databases that users can manipulate at will, due to several features XML (like SGML) provides that are not available in HTML:
The working draft for XML 1.0 provides a complete specification in two parts: the extensible markup language itself [Bray and Sperberg-McQueen, 1997], and methods for associating hypertext linking [Bray and DeRose, 1997] and stylesheet mechanisms with XML. From this specification, we observe that expressive power, teachability, and ease of implementation, were all major design considerations. And although XML is not backward-compatible with existing HTML documents, we note that documents that are HTML 3.2-compliant can easily be converted to XML, as can documents conforming to ISO 8879 (SGML) and documents generated from databases.
As far as XML acceptance is concerned, Microsoft has embraced XML for future releases of its Internet Explorer, and Netscape is considering doing the same for its browser. XML will most likely gain widespread acceptance once the power of XML-enabled applications is realized. Please see [Bosak, 1997] for detailed descriptions of some XML-enabling applications; according to Bosak, these driving applications will include:
Although scripts embedded in HTML documents may provide some solutions to these problems, XML does not commit authors to individual script languages, authoring tools, and delivery engines. Instead, XML offers an "open standards" solution that is vendor-independent, giving information providers greater control over their content and context.
In style and structure, XML documents look quite similar to HTML documents. However, when Web servers with XML content prepare data for transmission, they typically must generate a context wrapper with each XML fragment, including pointers to an associated Document Type Definition (DTD) and one or more stylesheets for formatting. Web clients that process the XML must be able to unpackage the fragment, parse it in context according to the DTD (if needed), render it (if needed) in accordance with the specified stylesheet, and correctly interpret its hypertext semantics (such as links) associated with the different document tags.
The XML specification has several parts: syntax for specifying DTDs, semantics for specifying hypertext links, and structures for specifying the presentation styles. Together, these provide a unified way to develop new tags with attribute-value pairs, to link meanings to those tags, and to customize the presentation of the information marked up with those tags.
Let's write a simple XML document that only contains tags annotated with attribute-value pairs; that is, there will be no content in the document other than the tags themselves. These tags can then be parsed and processed by software programs.
Our simple example will be a document that maintains a list of people and their email addresses. Suppose, then, we want each "email list tag" to contain three attributes: a person's first name, that person's surname, and that person's email address.
We can specify default values to attributes to guarantee that every tag has the same number of attribute-value pairs (although some values may be null). The declaration of default attributes is lexically scoped by the EMAIL-LIST element (although in this case it has no effect, since none of the elements omit an attribute).
<!DOCTYPE EMAIL-LIST "http://www.infospheres.caltech.edu/schemas/email-list">
<EMAIL-LIST>
<?xml default email-list
firstname = ""
lastname = ""
email = ""
?>
<EMAIL-LIST
firstname = "Rohit"
lastname = "Khare"
email = "khare@mci.net"
/>
<EMAIL-LIST
firstname = "Adam"
lastname = "Rifkin"
email = "adam at xent dot com"
/>
</EMAIL-LIST>
Special pains were taken to make XML human-readable as well as machine-readable. As a result, empty lines immediately following a ">" or immediately preceding a "<" in the document are ignored by the parser, and whitespace inside tags is ignored (note this is different from HTML), so the document can take on a human-understandable format.
As a text-based format, XML is designed for storing and transmitting data. This can either be done through arbitrary attribute-value pairs, as demonstrated in example 1, or it can be done by strategically embedding tags around content to give that content more meaning.
For example, consider the following XML snippet:
<EMAIL-LIST> <FIRSTNAME> Adam </FIRSTNAME> <LASTNAME> Rifkin </LASTNAME> <EMAIL> adam at xent dot com </EMAIL> </EMAIL-LIST> <EMAIL-LIST> <FIRSTNAME> Rohit </FIRSTNAME> <LASTNAME> Khare </LASTNAME> <EMAIL> khare@mci.net </EMAIL> </EMAIL-LIST>
By binding a meaning to the XML tag <EMAIL-LIST>, we understand what is contained in that element (the start tag, the end tag, plus the contents in between).
The forms of metadata provided in example 1 (attribute-value pairs) and example 2 (start-end tags) present different ways for documents can mark up their information with metadata which can then be used to search for information in the document, generate information from the document, or filter the content of the document. Metadata spans a wealth of information, from digital signatures and authentication seals, to prices and timestamps, to links to related information.
In this example, metadata tags supplement the information content of a full, valid document. Note that there is not much difference between the HTML and XML versions of the document, and that observation indicates part of the beauty of XML: it builds on something simple with which Web developers are already familiar.
<!doctype html>
<html version="-//W3C//DTD HTML Experimental 970324//EN">
<head>
<title> A Full XML Document </title>
</head>
<body>
<h1> A Full XML Document </h1>
<p> This is information within an XML document. Notice how
similar it looks to an HTML document. Some good places to find out more
about XML are:
</p>
<ol>
<li><p><a href="http://www.w3.org/XML/">W3.Org's XML</a>
page.</p></li>
<li><p><a href="http://www.ucc.ie/xml/">Commonly
Asked Questions</a> about XML.</p></li>
</ol>
<p> Notice the special XML tags below. </p>
<hr/>
<address><a href="mailto:khare@mci.net">Rohit Khare</a></address>
<!-- Created: Wed Jun 25 12:22:32 MET DST 1997 -->
<!-- hhmts start -->
Last modified: Wed Jun 25 22:32:42 MET DST
<!-- hhmts end -->
</body>
</html>
<!-- Keep this comment at the end of the file
Local variables:
mode: sgml
sgml-declaration:"~/SGML/html.decl"
sgml-default-doctype-name:"html"
sgml-minimize-attributes:t
sgml-nofill-elements:("pre" "style" "br")
sgml-live-element-indicator:t
End:
-->
All information in XML --- including the contents of elements and element names themselves --- is Unicode text, which supports the representation of all international character sets. XML supports a variety of encodings of Unicode (the default is UTF-8); however, an entire document must share a single encoding.
If nested, XML elements must be strictly nested: each start tag must have a corresponding end tag, and elements cannot overlap.
XML's shorthand for an empty element (that is, one without contents), is ending the tag with a "/>".
As with HTML, three characters are reserved: < is < and & is & and > is >
Since HTTP/1.1 uses compression techniques in its transmission, the efficiency of an otherwise verbose XML data transfer improves under HTTP/1.1 .
XML's structuring mechanisms allow the easy addition of digital signature and encryption tags, both to individual parts of a document, and to entire documents, for added security.
In theory, if we have a XML metagrammar, we can create new document types on the fly. In practice, however, we note several potential limitations to XML:
Despite these possible limitations to XML, its extensibility is a powerful enough feature to bring several new kinds of application to the Web --- such as archiving and automation of tasks --- as well as allowing specific application domains (such as mathematics and chemistry) to create tags specially suited to the respective community of document publishers and readers.
Face it: XML is cool because you can extend it. Actually, XML is cool because you are not really extending anything at all, because there is no XML to which to add new tags! XML is a contruction kit for making new languages, not a simplistic competitor to HTML.
As such, an XML document interface is a structured view of information just like CORBA's Interface Definition Language (IDL) is a structured view of methods. Both can be exported for use in catering program behavior to the needs of the data involved. In fact, the distinctions between XML and IDL are further blurred when you realize that IDL is just a Document Type for specifying methods, their arguments, and their return values: in this brave new World Wide Web, XML is IDL!
However, XML is not just an IDL for CORBA objects; XML is an IDL for XML-embedded objects as well. Let's follow the logic: XML itself provides the automation facilities for plucking XML metainformation, which in turn provides greater ease in creating new report formats. With the addition of linking, XML could become a source format (for example, for XML-based databases).
XML philosophy naturally drives ontologies of tags for each information-specific domain; although these domains often overlap and divergence in their claims toward the semantic meaning of the tagged information, AI completeness means we can draw implications by ascertaining to which domain each particular PART belongs.
We could argue semantics all day and not get anywhere, but WE MUST go forward, for the Web is a great interleaving of barista and human brains at Cafe liberty. If I do run over something, after all, can I be assured that insurance will pay my damages?
Today, the Web's protocols affect the adopt-ability of XML as much as the language itself: being able to download parts of a document makes XML-linking more valuable. The ability to link an entire XML document chain makes XML-metadata even more useful. The Web's market acceptance is finally catalyzing lessons about "everyone has their own DTDs" to help groups handling the same kinds of data to develop open, extensible standards for interoperable information formats and semantics.
These efforts have already lent themselves to several new Web applications.
One endeavor involves adding some extensions to XML so it can be used as a metadata format for applications in the technology and society domain. For example, Netscape's Meta Content Framework Using XML [Guha and Bray, 1997] provides the specification for a data model for describing information organization structures (meta content) for collections of networked information, using XML syntax to represent instances of this data model.
The Handheld Device Markup Language (HDML) [Unwired Planet, 1997] addresses the constraints of pocket-size devices: a few lines of display, a limited keypad, tens of kilobytes of memory and a wireless connection to the Internet. HDML, like HTML, is an information publishing and interaction description language, but it took the wrong approach: it extended HTML with new tags, the semantics of which were only clear to Unwired Planet. However, after some pontification, Unwired Planet has corrected its model: although HDML was designed before XML was available, they are presently looking into revising HDML to be based on XML. Such an open solution bodes well for them and for HDML. This will enable them to use device-specific cascading style sheets and preloaded binary compression dictionaries to separately settle the "pocket-size" constraints issue.
Microsoft's proposed Channel Definition Format (CDF) [Ellerman, 1997] lets a Web site use XML to publish existing HTML content in a channel for desktop CDF-compliant push client browsers (from vendors such as Microsoft, PointCast, AirMedia, and BackWeb). XML also provides a way to embed arbitrary data and annotations within the broadcasted HTML, for use with scripts. CDF permits a Web publisher to offer frequently-updated collections of information from any Web server for automatic delivery to compatible receiver programs on PCs or other information appliances.
Mathematical Markup Language (MathML) [Ion and Miner, 1997], is an XML application for describing the structure and content of mathematical expressions, allowing the markup of complex formulas, something mathematicians and computer scientists have been clamoring for since the earliest days of HTML. Just as HTML has enabled the serving and processing of text on the Web, the goal of MathML is to enable mathematics to be served, received, and processed on the Web.
Sophisticated mathematical notation is highly symbolic in nature, and the relation between meaning and notation is often subtle. This has ramifications for the say what you mean aspect of semantic markup. To keep in line with the philosophy behind mathematical expressions, MathML describes expression structure together with its mathematical context. About two dozen MathML tags describe abstract notational structures, and another four dozen provide a way of unambigously specifying the intended meaning of an expression. MathML content and presentation tags can interact to capture the nuances of meaning in traditional equations. The MathML working draft also discusses how renderers might be implemented and how they should interact with browsers.
CML (Chemical Markup Language) [Murray-Rust, 1997] uses XML to manage molecular information. Actually, CML is based strictly on SGML, to be not "just another file format": it is capable of holding extremely complex information structures, acting as an interchange mechanism or for archival, and interfacing easily with modern relational and object-oriented database architectures.
CML takes advantage of the fact that XML documents need not be valid, and can simply be well-formed. Essentially, this means that a document is syntactically correct (for example, the start and end tags balance, ATTRIBUTEs are quoted, and so on), but the document itself might not be valid (for example, it might contain an unknown tag). XML is therefore very well suited to situations where the documents have already been validated, for instance because the authoring software is authenticated, or because the documents have already passed through a validating parser. Although all CML documents must be validatable against the CML Document Type Defintion, but it is possible to manipulate them without necessarily having to validate them.
XML seems to be an ideal substrate for archiving the state of distributed systems. In this case, we mean distributed in the sense of "across organizational boundaries" more than we mean "across address spaces," though there is some of that, too.
Consider the most basic case: archiving a network of dependent objects within one application. Suppose we have a Human Resources application with Employee, Department, and Manager::Employee classes and assorted instances thereof. Let's consider what happens during archiving, at first, without considering HTML/XML at all.
As an initial attempt, you might pack-and-store (known in lingo as pickling) the Department component data by simply writing down its data structure in memory. Immediately, though, we see that there are pointers to a Deparment's Manager, so we actually need to package both objects together to make a complete statement. If you wanted to get all of the Employees within a department, too, then you discover the general case. Since there is a Web of employees, all related to each other because some are managers, we cannot simply write down the Department, the Manager, and then each Employee reporting to that manager: we could fall off the edge into a cyclic loop. In fact, the general case is mark-and-sweep: first you trace out every object connected to the one at hand; and then you collect and serialize every affected object in that subset.
But this is expensive! Before you can even put the first byte on the wire, you have to plan out the entire pickle. It is fragile, too, because two subsequent snapshots may yield separate values for some subsets of the archive: some reporting roles change, and so on. That is, you will end up with duplicate copies of the state of an object in two archives, which is clearly unacceptable.
But hold on, because you actually know a radically different way of solving this problem under your nose: transferring a Web page! A page has many subsidiary resources, some of which load other subparts in turn, some of which are shared with other pages, and so on. But WE do not have to do the actual pickling: HTTP servers do not grovel over home pages and send out neatly packaged bundles of HTML-with-all-embedded-images-and-sounds-in-one-MIME-multipart.
This is because we have the miracle of names at our access! Instead of expensive marshalling burdens on the server (writer), we just send over the one object at hand with names as pointers to other resources. Then, let the client (reader) pulls what it needs to build a complete map... the delay bottleneck goes away, so we can really stream these items as soon as we want. Now, of course, as a performance optimization, we can pipeline the next few employee records you will need to mask the underlying latency, but this flows naturally from the Web's cache push features. And there you have it: optimized Web archival using XML to tag the saved data.
We have saved the best for last: using XML to be an intermediary language between automated tasks. This application of XML, the Web Interface Definition Language (WIDL) [webMethods, 1997], was developed by webMethods specifically for the automation of tasks using the Web.
For its intents and purposes, WIDL is fully compatible with HTML and works really well with XML. It is currently in the process of being submitted to the W3C, and it supports all Web technologies, including cookies, authentication, proxies, caching.
WIDL serves as a schema language for the Web, in much the same way that SQL provides access to relational databases. An HTML-Java mapping provides a binding between XML document elements and Java object instance variables, defines access methods for data elements and functional services, and makes it possible to access XML elements by name, by index, or by text search.
In addition, WIDL provides transparency and fault-tolerance mechanisms for tracking XML and HTML pages, even when the data on those pages are controlled and modified by other people and programs. You use WIDL to specify the type of page monitoring and automatic updates you would like. With WIDL, it becomes possible for content providers to establish information contracts with their users, and to provide stable data interfaces that are not affected by changes in Web site design and content structure. This is particularly applicable to inter-departmental and inter-agency content exchange.
In this way, WIDL is like a easy-to-understand version of CORBA and DCE IDL --- think of it as an enabling API for the Web --- complete with a directory service and exception handling. It has facilities for using HTML files as self-describing data containers suitable for both browser and automated access, incorporating human-readable hooks that are also machine-readable so that they can be understood by data processing applications. With WIDL's well-defined templates, a properly built container can be constructed once and then used again and again.
The primary functions of a WIDL file are location mapping and data binding --- making WIDL conceptually similar to, yet simpler than, CORBA IDL. As an extension to HTML, WIDL is XML, though it can certainly be used in conjunction with HTML and XML documents, too. WIDL Location Mappings allows WIDL <DOCUMENT> and <SERVICE> tags to map long, ungainly URLs into simple service and document names, such as "StockQuote" or "OrderStatus." WIDL Data Bindings describe how output data items are transferred from HTML documents to Java instance variables. For Web services that take inputs via either the HTTP GET and POST methods, an input WIDL binding describes each of the input parameters. WIDL <FIELD> tags bind each HTTP/CGI input parameter to Java class constructor input arguments, and HTML elements in the output are bound to Java instance variables using a Javascript-compatible object model and notation. In addition, a <CONDITION> tag can be used to catch error conditions. These can be expressed either as preconditions for success, or as failure conditions detected at runtime that throw Java exceptions. And, directory service functions can be readily implemented by storing WIDL files at well-known locations on the network.
An example application from webMethods demonstrates how a Web tracking service might design a WIDL package tracking service.
<WIDL NAME="PackageTracking">
<SERVICE NAME=TrackPackage
INPUT=InputData OUTPUT=OutputData METHOD=GET
URL="http://www.packages_r_us.com/cgi-bin/AirbillTrace">
<BINDING NAME=InputData>
<FIELD NAME=trackNum>
</BINDING>
<BINDING NAME=OutputData>
<FIELD NAME=package
VALUE=doc.tables[1].tr[0].td[0].value>
<FIELD NAME=deliveredOn
VALUE=doc.tables[2].tr[3].td[1].value>
<FIELD NAME=signedForBy
VALUE=doc.tables[3].tr[2].td[1].value>
</BINDING>
</WIDL>
Of course, there are many other potential applications for WIDL. For example, a university student was able to easily set up a WIDL page that tracked the prices of computer chips at nine different Web locations --- none of whose data the student controlled access. The simple WIDL page generated a table that extracted the prices for the exact component the student desired from the 9 Web sites; furthermore, this update was done automatically every time he pulled up the page so he always had access to the most current data when he viewed the page in his Web browser! Such is the power of automation...
Taken one level deeper, one could use WIDL to define a single user interface that composes the services provided by twenty different shipping companies like the one in the example above. Or, one could use a software tool to generate custom WIDL pages that watch particular news sites on the Web, combing them for bits through a filter mechanism, and then updating the news on the Web page, presented together in a united interface, from fifteen sources at multiple sites --- automatically!
One third-party integration group has developed a middleware product using WIDL to integrate other third-party packages for Human Resource applications. Essentially, instead of taking the middleware and coding a product to each specific package's API, the integration group used WIDL to integrate the packages with the Web server interface directly! You can intuit how much faster this is than coding against every product's API individually.
Now Web developers can integrate diverse systems, as long as those systems have a Web interface --- in fact, the developers need not control anything on those other systems they want to integrate! WIDL allows you to manage complexity --- through a unified object model and common object repository --- and gives you ability to query that repository with its document-object model. You can define a function with input parameters submitted to to a particular URL, and then use the output return values in the Web pages presented as a result.
Quintessentially, WIDL provides an abstract definition for services that can be provided to C, C++, ActiveX, and Java stubs at runtime to provide access to remote data dynamically. This dynamism allows a program to remap on-the-fly where the data is, regardless of who controls the site from where the data is arriving.
This centralized management of change illustrates the enormous power behind the Web's flexibility: with the Internet, people can provide access to specified parts of internal organzation without violating the security of the rest of the organization. The browser unifies this capability in a common user interface. With automation becoming easier, e-commerce will get a tremendous boost, too.
We have seen now how XML is a simple, flexible, open way to extend the tags that can be used and transferred in Web documents. This creates a wide variety of new applications for the web, from domain-specific markups like the Mathematical Markup Language, to push applications using a Channel Definition Format. But those applications just scratch the surface: XML will open the door to new methods for archiving persistent state across the Web as a universal storage format, and provide the hooks to allow more and more tasks to be automated on the Web.
Rohit Khare, khare@alumni.caltech.edu
Rohit Khare is a member of the MCI
InternetArchitecture staff in Boston, MA. He was previously on the
technical staff of the World Wide Web Consortium at MIT, where he
focused on security and electronic commerce issues. He has been involved
in the development of cryptographic software tools and Web-related
standards development. Rohit received a B.S. in Engineering and Applied
Science and in Economics from California Institute of Technology in
1995. He expects to join the Ph.D. program in computer science at the
University of California, Irvine in Fall 1997.
Adam Rifkin, adam at xent dot com
Adam Rifkin received his B.S. and M.S. in Computer Science from the College of William and Mary. He is presently pursuing a Ph.D. in computer science at the California Institute of Technology, where he works with the Caltech Infospheres Project on the composition of distributed active objects. His efforts with infospheres have won best paper awards both at the Fifth IEEE International Symposium on High Performance Distributed Computing in August 1996, and at the Thirtieth Hawaii International Conference on System Sciences in January 1997. He has done Internet consulting and performed research with several organizations, including Canon, Hewlett-Packard, Griffiss Air Force Base, and the NASA-Langley Research Center.
Modification information:
$Id: xml.html,v 1.38 1997/06/27 00:40:23 adam Exp $
This paper will appear in IEEE Internet Computing.
This paper represents the authors' third publication together. Their second publication, Weaving a Web of Trust, is available for review.