IRC log

[Doug Davis <du...@us.ibm.com>]

IRC log from tonight:
(See attached file: log)
-Dug

[AXIS ARCH] SOAP Specification Issues (Was: RE: IRC log)

[James Snell <js...@lemoorenet.com>]

Wow... just read the chat... sounds like it was fun ;-)

A *couple* of comments (just a few ;-) ..):

(note: you may want to read through the entire message first before
responding to any single part of it... sorry it is so long!)

First, regarding the SOAP specifications requirements for how a message is
to be processed.  Steve touched on this in the chat but I'd like to
summarize.  The SOAP specification has the following to say about how a
message is processed:

<snip>
   2. The SOAP Message Exchange Model

   SOAP messages are fundamentally one-way transmissions from a sender
   to a receiver, but as illustrated above, SOAP messages are often
   combined to implement patterns such as request/response.

   SOAP implementations can be optimized to exploit the unique
   characteristics of particular network systems. For example, the
   HTTP binding described in section 6 provides for SOAP response
   messages to be delivered as HTTP responses, using the same
   connection as the inbound request.

   Regardless of the protocol to which SOAP is bound, messages are
   routed along a so-called "message path", which allows for processing
   at one or more intermediate nodes in addition to the ultimate
   destination.

   A SOAP application receiving a SOAP message MUST process that message
   by performing the following actions in the order listed below:

   * Identify all parts of the SOAP message intended for that application
     (see section 4.2.2)

   * Verify that all mandatory parts identified in step 1 are supported
     by the application for this message (see section 4.2.3) and process
     them accordingly. If this is not the case then discard the message
     (see section 4.4). The processor MAY ignore optional parts identified
     in step 1 without affecting the outcome of the processing.

   * If the SOAP application is not the ultimate destination of the
     message then remove all parts identified in step 1 before forwarding
     the message.

   Processing a message or a part of a message requires that the SOAP
   processor understands, among other things, the exchange pattern being
   used (one way, request/response, multicast, etc.), the role of the
   recipient in that pattern, the employment (if any) of RPC mechanisms
   such as the one documented in section 7, the representation or encoding
   of data, as well as other semantics necessary for correct processing.

   While attributes such as the SOAP encodingStyle attribute (see section
   4.1.1) can be used to describe certain aspects of a message, this
   specification does not mandate a particular means by which the recipient
   makes such determinations in general. For example, certain applications
   will understand that a particular <getStockPrice> element signals an RPC
   request using the conventions of section 7, while another application
   may infer that all traffic directed to it is encoded as one way messages.
</snip>

OK, now, let's tear this apart to see if we can't come to some kind of
understanding to solve the single v multiple "pivot point" issue.

Now, in Paragraph 1 (P1), SOAP messages are defined as one-way messages sent
from a sender to a receiver.  Paragraph 3 (P3) asserts that these messages
may be routed through multiple intermediate processing nodes on its way to
the receiver.  So something like:

   S = Sender, I = Intermediary, R = Receiver

   S -> I1 -> I2 -> ... -> R

Ok, now, also in P1, there is the statement that messages may be combined to
form various combinations of interaction patterns, RPC is given as an
explicit example.  In RPC we have a two entities E1 and E2.  E1 first sends
a Request to E2, and E2 sends a Response to E1.

This may look like:

   E1 -> E2
   E1 <- E2

Or, because messages may follow message paths, this may look like:

   E1 -> I1 -> I2 -> ... -> E2
   E1 <- I1 <- I2 <- ... <- E2

To be clear, there is also no requirement in the SOAP specification that
states that the Response message MUST follow the exact same message path as
the request message, so you could very well end up with something like:

   E1 -> I1 -> I2 -> ... -> E2
   E1 <-------------------- E2

Or even stranger:

   E1 -> I1 -> I2 -> E2
   E1 <- I4 <- I3 <- E2

All of these are legal SOAP RPC messaging models even though some of them
don't fit classic RPC models.

The SOAP RPC Model requires ONLY that a Response message be returned to the
original sender of the Request message.  It does not say anything about the
composition of the intermediate message path through which those messages
are sent.

Now, there is the question as to which part of the actual message each
intermediary in the message path is allowed to mess around with.  Let's look
at the three processing requirements in greater detail.  Again, the
requirements are:

<snip>
   Section 2...

   A SOAP application receiving a SOAP message MUST process that message
   by performing the following actions in the order listed below:

   * Identify all parts of the SOAP message intended for that application
     (see section 4.2.2)

   * Verify that all mandatory parts identified in step 1 are supported
     by the application for this message (see section 4.2.3) and process
     them accordingly. If this is not the case then discard the message
     (see section 4.4). The processor MAY ignore optional parts identified
     in step 1 without affecting the outcome of the processing.

   * If the SOAP application is not the ultimate destination of the
     message then remove all parts identified in step 1 before forwarding
     the message.
</snip>

First of all, let it be acknowledged that the SOAP specification offers
absolutely no definition of what a "SOAP Application" is.  Therefore, I
offer this simple definition: A SOAP Application is any code capable of
generating or consuming SOAP Messages.  By this definition, SOAP
Applications MUST reside on the Sender, the Intermediary and the Receiver.

Next, let's focus on the second requirement.  Namely, the last sentence in
that paragraph "The process MAY ignore optional parts identified in step 1
without affecting the outcome of the processing."  The use of the word "MAY"
is the most important aspect of this sentence.

Let's explore this a little bit.  Let's say that we have the following SOAP
Envelope:

    <Envelope>
        <Header>
           <t:transaction actor="http://acme.com" />
           <m:security actor="http://creditcheck.com" />
        </Header>
        <Body>
           <getStockQuote />
        </Body>
    </Envelope>

Now, let's say that this Envelope has just been delivered to the SOAP
Application resident at: http://acme.com.

According to the SOAP Specification, the SOAP Application must first scan
the message for all message elements addressed to it using the SOAP actor
attribute.  This scan will return the t:transaction element.  The second
requirement is that the SOAP Application at http://acme.com must decide
whether or not it can properly process the t:transaction element and if not,
it must throw an error.  Now, the question is, is the SOAP Application at
http://acme.com allowed to do anything with the rest of the message or must
is simply pass it on to the next intermediary in the message path?  The
answer is yes, the http://acme.com SOAP Application can do whatever it wants
with the rest of the message because the SOAP Specification says that it MAY
ignore the rest of the message or not.  It is completely up to the SOAP
Application to make this decision.  The last thing that the SOAP Application
must do is remove all of the elements in the message that were identified as
having been addressed directly to it (namely, in the above example, the
t:transaction element).

Ok... so what does this mean in terms of our current issue with whether or
not we can have multiple "Pivot Points" or just a single "Pivot Point"?

Well, first of all, let's identify exactly what we mean by "Pivot Point".
First of all, let's say that the Pivot Point discussion centers completely
around the RPC (Request/Response) model, so, going back to my earlier
examples, we're looking at something like:

    E1 -> I1 -> I2 -> E2
    E1 <- I1 <- I2 <- E2

    Where E1 and E2 are the Requestor and Respondent and I1 and I2 are
intermediaries through which the messages are passed.

Ok, in this instance, if I understand Yuhichi correctly, E2 is the "Pivot
Point".  Or, more specifically, E2 is the SOAP Application to which E1 sends
it's RPC Request.  E2 is the SOAP Application that sends a Response to that
Request back to E1.  Both of these messages, the Request and the Response
are passed through I1 and I2 on their way to the messages final destination.

Now, keep in mind that there are SOAP Applications installed at: E1, I1, I2
and E2.  Each application is basically allowed to do whatever it wants with
the message in terms of processing as long as it meets the three basic
processing requirements established by Section 2 of the SOAP specification.
What this means specifically, is that given the case where the Request
message sent from E1 to E2 may contain a Header element addressed to I1,
when I1 receives the message, while it is required absolutely to process the
Header element addressed directly to it, it MAY ALSO CHOOSE TO DO ADDITIONAL
PROCESSING ON THE ENTIRE MESSAGE.  I add the emphasis to make this point:
Intermediary SOAP Applications are not limited to processing only Headers,
they may do any type of processing on the entire message if they so desire
to do so.

So let's break this down in terms of Axis.  We have defined Axis as a
Message processing application.  That is, Axis is an application that is
capable of receiving a message in some arbitrary format (according to the
requirements doc, multiple protocols must be supported in order to account
for future growth) and processing it according to the specific requirements
of that message specification.  Ok.. now, we intend to make it so that Axis
supports the processing of SOAP Messages.  In other words, we will make Axis
capable of Generating or Consuming SOAP Messages.  This makes Axis a SOAP
Application.  This means that Axis (the SOAP specific implementation of Axis
at least) must be capable of processing messages according to the guidelines
set forth in Section 2 of the SOAP specification.

Now, we have decided that Axis will process messages through a simple
framework of chained application components called handlers.  Each handler
is tasks with performing some type of action such as logging, verifying a
digital certificate, updating some type of database, etc.

Now, the SOAP Specification states that the SOAP Application must follow the
guidelines in Section 2 of the Specification, but here is a question.  Given
the following diagram (sorry for the ascii art):

      ----------------
      | SOAP Message |
      ----------------
             ||
             ||
             \/
      ----------------
      |     Axis     |
      | ------------ |
      | |  Handler | |
      | ------------ |
      |      ||      |
      |      \/      |
      | ------------ |
      | |  Handler | |
      | ------------ |
      |      ||      |
      |      \/      |
      | ------------ |
      | |  Handler | |
      | ------------ |
      |      ||      |
      |      \/      |
      ----------------

What we basically have here is a SOAP Message being received by Axis which
in turn forwards that exact same SOAP message on to each individual Handler
in the chain.  Once all of the Handlers have received and processed the
message, the Axis node forwards the message on to the next SOAP Application
in the Message path if one exists.

Now, given this architecture, isn't it true that each individual Handler is
itself a SOAP Application?  Considering the fact that each individual
Handler is capable of Consuming and Generating a SOAP Message?  This makes
sense to me.  If so, then each Handler is itself responsible for each of the
three basic processing requirements defined by Section 2 of the SOAP
specification.

So, in other words, what we end up with here is that both the Axis Node (the
engine that invokes the handlers) and the handlers themselves are BOTH SOAP
Applications.  And, according to the second requirement in Section 2, either
the Axis engine OR any of the handlers invoked by that engine are capable of
doing whatever they want with the entire contents of the message so long as
they meet all three processing requirements adequately.

What exactly does this mean?  It means essentially that there is no logical
distinction between a "Header Handler" and a "Payload Handler" because
Handlers are themselves complete SOAP Applications that MUST process the
entire Message according to the Section 2 requirements.  Make sense?

By drawing on my earlier message path examples, by building Axis in this
way, we end up with the following assertion:

Given the messaging path:

   E1 -> I1 -> E2

   Where E1 is the Requestor, I1 is the Intermediary and E2 is the
Responder.

   I1 is an Axis node that invokes three handlers for every SOAP message it
receives.

   The true message path is:

   E1 -> H1 -> H2 -> H3 -> E2

   Where H1, H2 and H3 represent the three handlers invoked by I1.  H1, H2
and H3 are all Intermediary SOAP Applications.

Now, back to the issue with Pivot Points.  First of all, "Pivot Point" is
kind of a bad term to use here.  I'd prefer to use the term Responder for
E2.

Here is a use case scenario.  Let's say that E2 is a Stock Quote service.
I1 is an Intermediary (Implemented with Axis) that validates the credentials
of the user to perform the action they are wanting to perform.  E1 is the
Requestor that wants to use the Stock Quote service.  The Envelope that E1
will be sending to E2 through I1 is:

    <Envelope>
       <Header>
          <c:credentials actor="http://I1" userID="jon" pass="doe" />
       </Header>
       <Body>
          <getQuote symbol="ACME" />
       </Body>
    </Envelope>

When the Axis engine at I1 receives the SOAP Message, it will invoke two
handlers.  One will verify that userID="jon" pass="doe" represents a valid
user authorized to perform the "getQuote" operation on E2.  The second will
create an entry in a log that userID="jon" submitted a getQuote request for
symbol "ACME".

Once both of the handlers at I1 have finished their job, the Axis engine at
I1 will forward the request message on to E2 which is an Axis engine that
invokes two handlers to actually process the getQuote request.  The first
handler invoked will verify that the ACME symbol is even a valid symbol.
The second handler invoked will perform the actual look up of the ACME
symbol.  Once both of these handlers have been invoked, E2 will send a
response message directly back to E1 without that message being routed
throug I1.

The basic message path for this looks like:

   E1 -> I1 -> E2
   E1 <------- E2

The message path expanded to include Axis handlers looks like:

   E1 -> H1 -> H2 -> H3 -> H4
   E1 <------------------- H4

   Where H1 = The I1 credentials verification handler
         H2 = The I1 logger handler
         H3 = The E2 symbol verification handler
         H4 = The E2 symbol query handler

Now, let's take a look at what exactly each handler has to do with the SOAP
Message in order to do its job.

H1 (The credentials verification handler) has to make sure that the user
specified in the c:credentials header is authorized to invoke the operation
specified in the message Body.  This means that H1 must be capable of
processing both the header and the body of the message.  (It must process
the body of the message in order to determine the name of the operation
being requested.  It must know the name of the operation being requested in
order to verify that the User has permissions to perform that operation).

H2 (the logger handler) has to record that the User validated by H1 has
requested the getQuote operation.  This means that H2 must be capable of
reading the contents of the SOAP Body in order to determine which operation
and which parameters have been passed along with the SOAP Message.  In this
instance, H2 doesn't have anything to do with Headers.  H2 ONLY deals with
the Payload of the message.

H3 (the symbol validation handler) is designed to take the value of the
symbol attribute in the Payload and verify that the value represents a valid
stock symbol.

H4 (the symbol query handler) is designed to take the value of the symbol
attribute and perform the actual getQuote operation by delegating to an
external application that does the actual work of the stock quote lookup.
H4 is also responsible for constructing the response message that is finally
sent back to E1.

In this instance, we have a perfectly valid SOAP RPC operation even though
the application logic for the getQuote operation is spread out over two
different handlers (H3 and H4).  Specifically, H3 validates the symbol, H4
queries the symbol (the first is as equally important as the second).  All
four handlers in the message path are allowed to play around with the entire
contents of the message.

What can we glean from all of this:

1. There is no logical distinction between Header Handlers and Payload
Handlers since ALL HANDLERS ARE SOAP APPLICATIONS IN AND OF THEMSELVES AND
ARE RESPONSIBLE FOR PROCESSING THE ENTIRE CONTENTS OF THE SOAP MESSAGE
ACCORDING TO SECTION 2 OF THE SOAP SPECIFICIATION.

2. The SOAP RPC specification does not preclude the construction of a SOAP
Message Path in which Intermediary SOAP Applications can do whatever they
want to do with the received message so long as that SOAP Application
adheres to Section 2 of the SOAP Specification.  Specifically, in terms of
Axis, this means that having multiple Handlers that spread out the
processing of an RPC Operation is not illegal so long as there is a single
Handler that can be targeted as the "Responder" or E2 in my example above
(in which H4 becomes the "Responder")

3. Axis Handlers MUST BE FULLY COMPLIANT SOAP APPLICATIONS IN AND OF
THEMSELVES!  In other words, Handlers must be capable of determining
mustUnderstand and actor, AND, each handler must be capable of validating
that a valid SOAP message has been received.

4. I need some more coffee.

A couple of quick comments:

Yuhichi, everything I just explained is exactly how I have always viewed the
architecture up till now.  I believe that this is also the view Steve and
Doug share (correct me if I'm wrong Steve and Doug).  I've tried my best to
lay out this viewpoint in terms of the SOAP messaging model and have tried
to make the following point:

The term "Pivot Point" really has no meaning as far as this viewpoint is
concerned.  We have a Requestor and Responder and Intermediaries in between.
Each Intermediary can do whatever it wants with the message.  In Axis, we
have the concept of Handlers, each Handler is itself an Intermediary, and so
each handler can do whatever it wants with the message.  This is what WE
mean by "multiple pivot points".  There is still a single Responder which is
what YOU mean by "pivot point".  I hope that I've made this point clear.


Now... let's move on to a couple of other issues.

First, mustUnderstand and actor support.  There have been several comments
passed around and I just want to see if there is some kind of rudimentary
consensus yet.  Here's what I'm thinking:

Each Handler will be told what it's actor URI is (whether it is one or many
actor URI's is irrelevant at this point).  Each Handler will be required to
scan the SOAP message for headers that belong to it by using the actor URI
that it is given.

Does everybody agree to this?

If so, let's move on.  The actor URI given to the handler can come from one
of two places (both of which have been proposed on the mailing list): the
transport listener (which determines the actual physical network address of
the Axis engine) or the WSDD file used to deploy the service.  My vote is
for having this information come from the tranport listener.  The reason
(for me) is obvious: we want to make the WSDD file fairly portable so that a
single WSDD file can be used at multiple locations, by embedding the actor
uri in the WSDD we trump this portability.

If we are still all +1 up till this point, let's continue:

I proposed earlier that the basic actor URI for each individual handler be
derived by combining the network location at which a service is hosted with
an ID assigned to that handler within the WSDD deployment file.  In other
words, something like:

    http://www.acme.com/axis/soapservlet/handlerID

If each handler is to have multiple actor uri's, then we would assign at
least one handler specific actor uri then whatever other actor uri's it is
going to need. One use case for multiple actor uri's is the instance where a
single instance of a handler is used by two different services hosted at two
different network endpoints deployed on the same Axis node.  In other words,
we might have something like:

    http://www.acme.com/axis/soapservlet/POService/loggerHandler

    And

    http://www.acme.com/axis/soapservlet/QuoteService/loggerHandler

Where loggerHandler in both instances refers to the exact same instance of a
single handler deployed on the www.acme.com server.  In this case,
loggerHandler will have to know both actor URI's so that it can handle
messages accordingly.

Still with me?  If so, and if we're still all +1 then I think we can move on
to the next issue.

Which is the Reader/Writer model.  All I have to say about this is that I am
currently working on a prototype that builds off the codebases that Doug and
Yuhichi have both produced.  It will be forthcoming shortly.  I'm somewhat
short on time this week (although you probably wouldn't know it by the
length of this message!) so I'll post it as soon as I can.

Well... that's all I gotta say about that. ;-)

- James

> -----Original Message-----
> From: Doug Davis [mailto:dug@us.ibm.com]
> Sent: Monday, January 15, 2001 7:48 PM
> To: soap-dev@xml.apache.org
> Subject: IRC log
>
>
> IRC log from tonight:
> (See attached file: log)
> -Dug