In considering the future of "compositional software architectures" as rendered through distributed object systems and on the World Wide Web, it is useful to set aside the hype of new technologies and consider what is already being accompished with existing infrastructures. On the Web, users and developers have already adopted two powerful ways to compose active processing with information distribution: active pages ("cgi-bin") and active proxies. In this position paper, we focus on the latter as a tool for parties beyond the original developer to externalize extensions to a software or information architecture.
Independent extensibility is a critical affordance of compositional software architectures. To realize the full potential of concurrent evolution of systems by all the system's stakeholders, architects should be encouraged to support externalized, component-oriented hooks. In particular, active proxies on the Web demonstrate the power of independent evolution and the serendipitous synergy of orthogonal services. Soon, HTTP in conjunction with PEP will systematize this power and bring it to clients and servers as well.
When fetching a resource through the HyperText Transport Protocol, clients can contact the origin server or an intermediate server that will fetch it on their behalf. One of the most familiar uses of proxied HTTP is caching: many users behind a single caching proxy can benefit from a local copy of their most frequently accessed resources. The caching proxy operates on behalf of users to maintain up-to-date copies from the origin server. Sometimes, it also acts on behalf of the publisher to collect and report usage statistics for its cached resources; the entire cache may also be filled on behalf the publisher (a "mirror" proxy).
There are many other species of proxies, though. Caches merely relay the original resource; active proxies have free reign to extend and emend these resources. Consider these applications:
This service was developed by Ka-Ping Yee, a student at Waterloo, based on design work by Eric Drexler at the Foresight Institute -- the ideas can be traced back to Ted Nelson's Xanadu. Alexa is a similar community annotation tool that collects feedback from users about document quality (and also acts as a cache -- in this case, from the multiterabyte Internet Archive of extinct pages).
This service was developed by several Bell Labs cryptography and Web protocol researchers. Two related services are the Anonymizer, which intercepts cookies, Java, JavaScript, and strips out Referer: and User-Agent: request headers; and NoShit, which strips out graphics characteristic of Web advertising. Conversely, some advertisers weave ads onto public pages and track end-users with the same technology.
The challenge is that only first-party stakeholders can extend their software architectures in these ways today. The information-oriented extensions listed above were not; they were developed by outsiders and deployed by outsiders in service of outsiders. Active proxies hold the promise that soon, outsiders will write new health-plan comparators and mutual-fund trading interfaces and ...
Want to annotate a Japanese page without advertisements from a HTTP-NG server? Want to book a plane ticket and a hotel room in a single transaction? Active proxies can be neatly reused as black-box components when chained together via HTTP. However, we can envision neater, more efficient ways to enable reuse. The HTTP Protocol Extension Protocol (PEP) transcends the welter of competing APIs to offer a single syntax for naming, specializing, and applying active proxies with finer-grained control. PEP also affords reasoning about compatible extensions and composite extensions.
We are already familiar with many analogues to active proxies as reusable filters. The difference is in the the affordances of the interchange format. UNIX filters operate on ASCII streams; SQL queries operate on relational tables; active proxies and pages operate on Web hypermedia (HTML/XML + HTTP).
The affordances for composition are also similar: manual, sequential composition only. The specification clearly allows HTTP proxy chains to apply several transformations along the way, but in practice none of the services sampled above allows for onward chaining (they just fetch the actual content from the origin server, rather than branching to yet another proxy server specified by the end-user). The downside of packaging these extensions as a proxy is the assumption that all users and all destinations are treated the same -- that is, in applying the same function to all inputs and outputs.
This is the universe PEP was designed for -- each PEP module has the same executable power as a proxy, but can be selectively applied to portions of Web space, on behalf of certain users, with known urgency (required or optional), in concert with other extensions (or exclusively of conflicting ones), in sequence or in parallel, on selected hops of the HTTP proxy chain. Most significantly, since PEP modules are identified by the URI of the protocol they implement, PEP-aware Web tools can negotiate common sets of compatible modules and settings.
PEP enshrines a philosophy of decentralization. Anyone can publish an extension by maintaining a Web page describing it. Any such module has as much expressive power to rewrite its input as an active page or active proxy. Any resource can be bound to require an extension ("those .quicken files require an http://pep.w3.org/SEA/Encryption/-compatible filter"). Any extension can express its own policy (hop-by-hop or end-to-end; requisite and incompatible co-extensions).
Its designers developed applications for content-filtering, electronic-payment selection, and a modular security architecture -- all of which could be composed to, say, purchase encrypted PICS labels. Many of these applications are actually more powerful than active proxies, since PEP allows their functions to be moved into the origin client and server; security decisions can be made at the desktop rather than the firewall.
These benefits are not free, though. Selectively applying active proxy extensions requires more logic in the server to select compatible PEP modules and enforce users' and publishers' policies. In truth, it has been easier for extenders to deploy active pages and active proxies than to design for the future. PEP is still on the IETF standards track two years after its debut. Decentralized extensibility is a tough sell, but we believe it is essential.
Internal extensibility exists and is well supported *within* the black box -- OOA&D and software architectures research gives us reason to hope. Outside the box, though, the rise of open information systems on the geodesic network of the Web heralds a political shift in the constituency for extensiblity: there are many, many more actors with an interest in the extensiblity of your architectures!
For more information about the ideas and systems we have discussed in this document...
Thanks to Jim Whitehead for reviewing a draft of this position paper.