                                                        David M. Kristol

INTERNET DRAFT                                    AT&T Bell Laboratories

<draft-kristol-http-extensions-00.txt>

January 1995                                           Expires July 1995





                A Proposed Extension Mechanism for HTTP







                          Status of this Memo



     This document is an Internet-Draft.  Internet-Drafts are

     working documents of the Internet Engineering Task Force

     (IETF), its areas, and its working groups.  Note that other

     groups may also distribute working documents as Internet-

     Drafts.



     Internet-Drafts are draft documents valid for a maximum of six

     months and may be updated, replaced, or obsoleted by other

     documents at any time.  It is inappropriate to use Internet-

     Drafts as reference material or to cite them other than as

     ``work in progress.''



     To learn the current status of any Internet-Draft, please

     check the ``1id-abstracts.txt'' listing contained in the

     Internet- Drafts Shadow Directories on ftp.is.co.za (Africa),

     nic.nordu.net (Europe), munnari.oz.au (Pacific Rim),

     ds.internic.net (US East Coast), or ftp.isi.edu (US West

     Coast).



     This is author's draft 2.9.  The previously available author's

     draft was 1.3.





1.  ABSTRACT



HTTP, the hypertext transfer protocol, underpins the World-Wide Web

(WWW).  As the Web has grown, pressures have mounted to add a variety of

facilities to HTTP.  Some of the new features that have been proposed

include:  keep-alive, packetized data, compression, security, and

payment.  This memo offers an alternative:  well-defined hooks in a

slightly modified HTTP framework that make it possible to add extensions

to the basic protocol in a way that will retain compatible behavior

between clients and servers, yet allow both clients and servers to

discover and use extended capabilities.  The goal is to use HTTP as just

a transport mechanism, leaving other, higher-level (session) activities

to extensions.





2.  MOTIVATION



One virtue of HTTP is that it is easy to modify: just add more request

or response headers.  Unrecognized headers will be ignored by agents

(client or server) that don't understand them.  Why is this approach

unacceptable?  The following paragraphs will attempt to justify a







Kristol                                                         [Page 1]















INTERNET DRAFT     Proposed HTTP Extension Mechanism        January 1995







different approach.



2.1  Generality



A common, well-defined framework by which to introduce extensions

reduces the danger of uncontrolled proliferation of incompatible

extensions.  Vendors that want to add extensions can do so in a

predictable way.  Client and server software will be better able to

predict what kinds of headers they will encounter.



2.2  Simplicity and Modularity



The server and client architectures should be kept simple.  If

extensions can be recognized easily, it becomes possible to posit the

following architecture.  A client (server) comprises a core part.

Extensions are handed off to an extensions manager, which dispatches

extension requests to the appropriate handler, such as a security

manager, payment manager, etc.  If the interface between the core part

and the extensions manager, and between the extensions manager and the

individual extension managers, is well-defined, the core part of a

client (server) can be quite ignorant of what is actually being done by

the various extensions.  Thus this architecture leads to a highly

modular design into which it is possible to ``plug'' new extensions,

while the core part remains simple.  Vendors of extensions could supply

matched plug-in parts for them to client and server implementors, to

incorporate them in their products.



2.3  Recognizable Extensions



The scheme proposed here makes it easy to identify requests for

extensions quickly.  Requests for, or acceptance of, extensions is

signaled by Extension: request/response headers.  Because they are

easily identified, a caching server can recognize headers for extensions

and store them as part of the cached information.



(Caching requires further consideration.  It may be necessary for

caching servers not to cache information that was obtained using

extensions, since those extensions often entail security or payment.)



Using the HTTP version number to determine what extensions are present

is a bad idea.  Extensions are often disjoint, and clients and servers

should be able to ``mix and match'' the ones they can support.  The HTTP

version number is too crude a discriminant and should be reserved for

true changes to the base protocol.  The extension mechanism proposed

here merely uses HTTP for transport.



















Kristol                                                         [Page 2]















INTERNET DRAFT     Proposed HTTP Extension Mechanism        January 1995







2.4  Who's in Charge?



HTTP headers are likely to require approval from IANA.  Thus, using

multiple headers for different extensions may impede new applications of

the World-Wide Web.  It is easier to get two headers approved that

enable a whole range of extensions, and it carves out a hunk of name

space that can be controlled separately, possibly by W3C (World-Wide Web

Consortium).



2.5  Wrapping Better Than MIME



Wrapping, described below, has capabilities that MIME cannot supply.

Specifically, security may require that the actual request be obscured.

The WRAPPED method in this proposal makes it possible to encrypt the

actual request.  A snooper would see only the WRAPPED request method and

the extension header (with, presumably, enough information to describe

how to decrypt the actual request).





3.  CONCEPTS



The proposed extension mechanism has two fundamental concepts: wrapping

and negotiation.



3.1  Wrapping



Wrapping implies that a core set of information has information added

before and after it.  In some cases the information added may comprise

just headers.  The core information may itself be changed as well, such

as when it is encrypted.  The information added as the pre-wrapper must

convey enough information to the recipient so it can unwrap the

information.  Either a client or server can do the wrapping, although I

assume that the server more often does the wrapping.



3.2  Negotiation



Before a sender wraps information, it must be sure the receiver can

unwrap it.  Thus the two parties must negotiate what kind of wrapping is

desirable.  Therefore a (prospective) receiver (initiator) tells a

sender (responder) what forms of wrapping it accepts, and whether they

are acceptable always, or sometimes.  The sender responds with a

suitably wrapped response.  ``One-of'' wrappings must always be used,

but the sender can choose at will from a group of such wrappings.

``Sometimes'' wrappings may be used at the discretion of the responder.



3.3  Recursion



Wrappings can be recursive.  To give but one example, a core message

might be wrapped thus:  packetize(security(payment(core))).  The

notation implies that, given a core message, first a payment wrapping is

applied, followed by a security wrapper, followed by packetization.  The







Kristol                                                         [Page 3]















INTERNET DRAFT     Proposed HTTP Extension Mechanism        January 1995







receiver must unwrap the message from outside in, i.e., packetization

first.





4.  NOTATIONAL CONVENTIONS



This proposal uses the notational conventions of the (draft)

specification, Hypertext Transfer Protocol - HTTP/1.0, Berners-Lee,

Fielding, Frystyk Nielsen.





5.  EXTENSIONS TO HTTP 1.0



The proposed extension mechanism requires small changes in the existing

HTTP, to add two methods and two request headers.



5.1  Methods



5.1.1  GETEXT  get extensions

The HEAD method in HTTP only provides a limited amount of information

about how the server would respond to a GET request.  GETEXT provides

information about how the server would respond (at least for extensions)

for any method.



<getext>        ::=     "GETEXT" <url> "HTTP/1.0"

<url>           ::=     *

                /       <path part of URL>



A client sends a request with the GETEXT method to a server to learn

what extensions the server supports.



The GETEXT method may have an optional Extension: request header with

the following form:



<getext-hdr>    ::=     "Extension:" "HTTP/Method" "method=" <method>

<method>        ::=     "GET" / "PUT" / "POST" / ...



Semantics



The server reports the extensions as response headers, identical to the

request headers that are described in the next section, that apply.

There are four cases, depending on whether an explicit <url> is present

and whether an Extension: header is present.  In each case the result is

the intersection set of extensions for the method(s) and URL(s)

specified by the GETEXT request.



<url> ::= *, no Extension: request header

          The server returns the set of Extension: response headers that

          apply to any URL and any method on the server.











Kristol                                                         [Page 4]















INTERNET DRAFT     Proposed HTTP Extension Mechanism        January 1995







<url> ::= *, Extension: request header present

          The server returns the set of Extension: response headers that

          would apply if the server got a request comprising the

          <method> in the Extension: request header and any URL.



<url> ::= URL, no Extension: request header

          The server returns the set of Extension: response headers that

          would apply if the server got a request comprising any

          acceptable method applied to the selected URL.



<url> ::= URL, Extension: request header present

          The server returns the set of Extension: response headers that

          would apply if the server got a request comprising the

          <method> in the Extension: request header and the specified

          URL.



5.2.1  WRAPPED wrapped request



<wrapped>       ::=      "WRAPPED" "*" "HTTP/1.0"



A client sends a request with this method to tell a server that the

request is wrapped.  The request headers (next section) specify exactly

how the request is wrapped.  The server must peel the wrappings one

layer at a time until it encounters a normal request, which it can then

process.  The value of what would normally be the URL field, shown above

as *, is ignored.



5.3  Request Headers



Two new request headers specify extensions.  Their syntax and semantics

are given below.



5.3.1  Extension: Header



<extension>     ::=     "Extension:"  <cat-class> [<av-pairs>]

<cat-class>     ::=     <category> "/" <class>

<category>      ::=     <alpha-numeric string>

                                        ; kind of generic extension

<class>         ::=     <alpha-numeric string>

                                        ; class within generic extension

<av-pairs>      ::=     <av-pair> [";" <av-pair>]

<av-pair>       ::=     <attr> "=" <value>

<attr>          ::=     <alpha-numeric string>

<value>         ::=     <alpha-numeric string>



The Extension: request header may wrap, RFC822-style, onto multiple

lines.















Kristol                                                         [Page 5]















INTERNET DRAFT     Proposed HTTP Extension Mechanism        January 1995







Semantics



<category> describes the generic extension, for example security,

payment.  (Should the list of extensions be controlled by W3C?)  A

responder that receives an unrecognized <category> responds with an

error if the <av-pair> required=oneof is present and ignores the header

otherwise.



<class> specifies a member of the <category>.  For example, <category>

payment might have <class>s Visa, MasterCard, Ecash, etc.  If a

<category> has no <class>s, * must be used as the <class>.



Attribute-value pairs (<av-pairs>) are optional.  The only attribute

that is defined for all <category>s is required, with possible values

oneof and sometimes.  If the required attribute is present for an

Extension: header, it must be part of the first <av-pair>.



The algorithm for choosing among Extension: headers is described in

NEGOTIATION, below.



5.4.1  Extension-Order: Header



<extension-order>       ::=     "Extension-Order:" 1#<cat-class>



The Extension-Order: header provides a way for a client or server to

specify the order in which extensions must be applied.  (MIME headers

are unordered by definition.)



5.5  Wrapped Response



A wrapped response comprises a status code of 207 (Wrapped), a single

Extension: response header that describes the outermost wrapping, and a

blank line.  The body of the response comprises the wrapped response.

The wrapping must be defined in such a way that the body can be

unwrapped to produce a new response, complete with status line, response

header line(s), blank line, and new body.  The specifics of that

wrapping are outside the scope of this document and are specific to a

given extension.  The resulting inner response may also be a wrapped

response, in which case the unwrapping occurs recursively.



A response code of 409 by a server denotes a Wrapping Required response.

The response headers specify both the type and order of wrapping that

the server requires from the client.  (Note that in this case the server

acts as an initiator and should provide required= <av-pair>s for the

Extension: response headers it returns.)



5.5.1  Response Headers  An Extension: response header is identical in

form and content to a corresponding request header.  A responder's

header does not contain a required= <av-pair>, because the presence of

the Extension: header means the extension request has been honored,

whether it was optional or mandatory.







Kristol                                                         [Page 6]















INTERNET DRAFT     Proposed HTTP Extension Mechanism        January 1995







6.  NEGOTIATION



This section describes in more detail the negotiation process.  It

describes the roles of initiators and responders and how they should use

and respond to Extension: and Extension-Order: headers.  Either clients

or servers may be initiators or responders.



[This first attempt will undoubtedly be mushy.]



6.1  Initiators



An initiator informs a responder of extension capabilities that are

either mandatory or optional to use.  Each capability is described by an

Extension: (request or response) header.  Mandatory extensions must have

required=oneof as the first <av-pair> of the header.  If no required=

<av-pair> is present in an initiator's header, the default is

required=sometimes.



An initiator can advertise to a responder many <class>s for a particular

<category>, and many extension <category>s.  Based on the negotiation

selection algorithm (below), a responder may (and sometimes must) choose

among them.  How the initiator chooses which extensions to advertise is

outside the scope of this proposal.



6.2  Responders



A responder informs an initiator of which extensions it has selected.

If the initiator has specified multiple <class>s for a particular

<category>, the responder must choose to honor either one or none of the

choices.  When the initiator has specified mandatory extensions

(required=oneof) for a <category>, the responder must choose one.  For

optional extensions (required=sometimes), the responder informs the

initiator of which ones, if any, it has chosen to use.



What happens when a responder cannot honor a mandatory extension depends

on whether it is a client or server.  A client responder would typically

inform a user that it cannot complete a request because it lacks some

capability that the server requires (such as authentication or a

suitable payment method).  A server responder would return an error

response code to the client, preferably with an informative HTML message

for the client to display to describe the failure.



6.3  Negotiation Selection Algorithm



A responder uses the negotiation selection algorithm to choose among

possible <class>s in a given <category>, and to define the order in

which extensions in different <category>s are applied.



For expository purposes, assume each <cat-class> part of an Extension-

Order: header is assigned an integer index from 1 (first) to N (last),

based on its lexical order in the header.  If there is no Extension-







Kristol                                                         [Page 7]















INTERNET DRAFT     Proposed HTTP Extension Mechanism        January 1995







Order: header, the indexes are assigned arbitrarily by the responder.

Note that if there is only one Extension: header, the Extension-Order:

header is redundant.



6.3.1  Selection  All the extensions in the same <category> (call them

category-options) are treated as a group.  If any category-option in the

group has the <av-pair> required=oneof (a mandatory category), then any

category-option in the group that does not have required=oneof is

discarded.  The responder then considers each remaining category-option

in turn, choosing the one it prefers from the group.  Note that at this

point all the members of the group must have the same required= value,

either oneof or sometimes.  (How the responder chooses the one ``it

prefers'' is an implementation issue, outside the scope of this

proposal.)



If the responder cannot support any of the category-options for a

mandatory category, it responds with an error, as indicated earlier.

For an optional category, if the responder cannot support, or chooses

not to honor, an extension, the responder simply does not apply the

extension.



6.3.2  Order  After the responder has identified the extensions it will

honor, based on the initiator's Extension: headers, it applies them in

the order of the index numbers of the honored <cat-class>s.



6.3.3  Exceptional Conditions



  1.  If an Extension-Order: header is present, any Extension: header to

      be considered in the selection process must have its <cat-class>

      listed by the Extension-Order: header.



  2.  If more than one Exception: header has the same <cat-class>, the

      responder should reject the request.



  3.  If a <cat-class> appears in an Exception-Order: header, and there

      is no Exception: header with the same <cat-class>, the responder

      should behave as though the <cat-class> were omitted from the

      Exception-Order: header.





7.  EXAMPLE 1:  SIMPLE AUTHENTICATION



The Basic authentication scheme can be recast in the framework of the

HTTP extension mechanism.  Assume the requested (local) URL on the

server is /foo/bar.



















Kristol                                                         [Page 8]















INTERNET DRAFT     Proposed HTTP Extension Mechanism        January 1995







7.1  Client -> Server



GET /foo/bar HTTP/1.0

Accept: text/html, ...

[blank line]



7.2  Server -> Client        [Server is initiator]



HTTP/1.0 409 Wrapping Required

Date: Wed, 28 Dec 1994 10:59:13 GMT

Server: HTD/0.9

MIME-version: 1.0

Extension: Security/Basic required=oneof;Realm="Demonstration"



7.3  Client -> Server        [Client is responder]



WRAPPED * HTTP/1.0

Extension: Security/Basic data=ZG1rOnBhc3N3b3Jk

[blank line]

====

GET /foo/bar HTTP/1.0

Accept: text/html, ...

====



(Note:  pretend ==== is used to ``wrap'' the inner request.)



7.4  Server -> Client



HTTP/1.0 200 OK

Date: Wed, 28 Dec 1994 10:59:15 GMT

Server: HTD/0.9

MIME-version: 1.0

Content-type: text/html

Last-modified: Thu, 17 Nov 1994 19:35:21 GMT

Content-length: 3729



[body]





8.  EXAMPLE 2:  PAYMENT AND ENCRYPTION



This more extended example demonstrates how the extension mechanism can

be used to support both security and payment.  In the example, the

payment manager relies on an existing security manager to provide the

encryption that secures a credit card number.



















Kristol                                                         [Page 9]















INTERNET DRAFT     Proposed HTTP Extension Mechanism        January 1995







8.1  Client -> Server



GET /foo/bar HTTP/1.0

Accept: text/html, ...

[blank line]



8.2  Server -> Client        [Server is initiator]



The server requires both payment and security (encryption).  The payment

information must be supplied first, followed by the encryption.  Note

that there is no need for an actual ``wrapping'' by the payment piece.



HTTP/1.0 409 Wrapping Required

Date: Wed, 28 Dec 1994 10:59:13 GMT

Server: HTD/0.9

MIME-version: 1.0

Extension: Security/SimpleDES required=oneof;

        keyname=OpenSesame; nonce=873955

Extension: Security/SHTTP required=oneof;

        (S-HTTP parameters)

Extension: Payment/MasterCharge required=oneof;

        cost=2; units=USD; mult=0

Extension: Payment/GreenCard required=oneof;

        cost=2; units=USD; mult=0

Extension: Payment/American_Excess required=oneof;

        cost=25; units=USD; mult=-1

Extension-Order: Security/SHTTP,

        Security/SimpleDES,

        Payment/MasterCharge,

        Payment/American_Excess,

        Payment/GreenCard

[blank line]



8.3  Client -> Server        [Client is responder]



Client provides server with payment information.  It chooses (presumably

with the user's help) which of the three payment methods, MasterCharge,

GreenCard, American_Excess, to use.  It also chooses between the two

security methods.  Note that the ``wrapping'' for the payment part just

comprises adding a header.  The security piece does wrap the request,

however.



























Kristol                                                        [Page 10]















INTERNET DRAFT     Proposed HTTP Extension Mechanism        January 1995







WRAPPED * HTTP/1.0

Extension: Security/SimpleDES keyname=OpenSesame; nonce=873955

[blank line]

====

GET /foo/bar HTTP/1.0

Accept: text/html, ...

Extension: Payment/GreenCard cost=2; units=USD; mult=0;

        cardno=11112223333444

====



Pretend ==== is used to ``wrap'' the inner request, and that the inner

request is encrypted using simple DES with a key named OpenSesame, a

shared secret between the client and server.  The wrapping could just as

well use MIME multipart syntax, but that's a decision to be made by the

particular extension manager.



8.4  Server -> Client



Server accepts payment and returns a receipt to the client.  The server

could, optionally, wrap the response to protect the receipt.



HTTP/1.0 200 OK

Date: Wed, 28 Dec 1994 11:01:56 GMT

Server: HTD/0.9

MIME-version: 1.0

Content-type: text/html

Last-modified: Thu, 17 Nov 1994 19:35:21 GMT

Content-length: 3729

Extension: Payment/GreenCard cost=2; units=USD; mult=0;

        Receipt="You paid $2.  Thank you for using AT&T."



[body]





9.  EXAMPLE 3: GETEXT



This example shows how the GETEXT method might be used.



9.1  Client -> Server



GETEXT /foo/bar HTTP/1.0

Extension: HTTP/Method method=GET

[blank line]



9.2  Server -> Client



















Kristol                                                        [Page 11]















INTERNET DRAFT     Proposed HTTP Extension Mechanism        January 1995







HTTP/1.0 200 OK

Date: Wed, 28 Dec 1994 10:59:13 GMT

Server: HTD/0.9

MIME-version: 1.0

Extension: Security/SimpleDES required=oneof;

        keyname=OpenSesame; nonce=873955

Extension: Security/SHTTP required=oneof;

        (S-HTTP parameters)

Extension: Payment/MasterCharge required=oneof;

        cost=2; units=USD; mult=0

Extension: Payment/GreenCard required=oneof;

        cost=2; units=USD; mult=0

Extension: Payment/American_Excess required=oneof;

        cost=25; units=USD; mult=-1

Extension-Order: Security/SHTTP,

        Security/SimpleDES,

        Payment/MasterCharge,

        Payment/American_Excess,

        Payment/GreenCard

[blank line]





10.  <category> HTTP Extensions



The proposed mechanism can be used for extensions to HTTP itself, as

well as to what might be considered ``outboard'' matters, like security

and payment.  Such extensions would comprise the <category> HTTP.  Two

extensions to HTTP/1.0 have been frequently discussed.  Both can be

accommodated by this proposed mechanism.



10.1  Packetization   HTTP/Packetize



Packetization divides a message into hunks of not more than a fixed

size.  It is well-suited for transmitting the output of a script for

which the total size of the message is unknown when the script begins to

run.



<packet-hdr>    ::=     "Extension:" "HTTP/Packetize"

                        "required=sometimes" [<size>]

<size>          ::=     ";" "size=" <positive, non-zero decimal number>



The initiator (usually a client) announces its willingness to accept

responses in packetized form, with packets no larger than <size>, but

packetization is optional.  The initiator must supply <size>.



The responder (usually a server) announces that it plans to use

packetization in its response by returning a similar Extension: header,

omitting the <size> and required=.



I have omitted details of what a packetized response looks like.  A

typical implementation produces packets that comprise a decimal byte







Kristol                                                        [Page 12]















INTERNET DRAFT     Proposed HTTP Extension Mechanism        January 1995







count, followed by that many bytes (line terminators included).  A zero

or negative count signifies end of data.



10.2  Keep-connection HTTP/KeepConnection



One of the performance defects of HTTP is that clients often open many

connections to the same server to get images for a document they just

fetched.  Connections are expensive in time and resources.  It would be

better to pass follow-up requests over an already-open connection.



<keep-conn>             ::=     "Extension:" "HTTP/KeepConnection"

                                "required=sometimes" [<timeout>]

<timeout>               ::=     ";" "timeout=" <timeout in seconds>



A client (initiator) would signal its ability and willingness to hold a

connection open by passing this request header.  The <timeout> advises

the server how long it would like the connection held open.



A server (responder) would indicate it had honored the request by

returning the same header, minus the required= part.  The server can

respond with a different <timeout> to say how long it plans to hold open

the connection, but that information is strictly advisory.



Note that, as a practical matter, HTTP/KeepConnection must be used with

HTTP/Packetize, because in some instances (e.g., output from scripts),

the server doesn't know how long its output to the client will be, and

it currently signals the end of data by closing the connection.

Packetization lets a server pass back data to a client in manageable

chunks, rather than buffer the entire response and send a correct

Content-length header.





11.  ACKNOWLEDGEMENTS



Jeff Hostetler, Spyglass, Inc., originally suggested the idea of

``wrapping'' to me.





12.  AUTHOR'S ADDRESS



David M. Kristol

AT&T Bell Laboratories

600 Mountain Ave.  Room 2A-227

Murray Hill, NJ  07974

Phone: (908) 582-2250

FAX: (908) 582-5809

Email: dmk@research.att.com















Kristol                                                        [Page 13]









