[DISCUSS] Faults and Exceptions in Camel

[Hadrian Zbarcea <hz...@gmail.com>]

Hi,

As we approach the 2.0, there is one more hanging issue I would like  
addressed, if possible.  It's the thorny issue of Faults and  
Exceptions that started in
http://issues.apache.org/activemq/browse/CAMEL-316 (see also the  
related nabble thread linked in the issue description).

I am less concerned about how the DefaultExchange is implemented and I  
hope to reach an agreement on what the Exchange api should be (please  
find the list of Exchange methods below).

As far as faults/exceptions are concerned, Roman thinks that the whole  
concept of in/out/fault/exception is artificial, and only one payload  
(message) api should be enough (Roman please correct me if I  
misinterpret your position).  My opinion is that we *must* distinguish  
between persistent (fault) and transient (exception) errors for the  
simple reason that they have different semantics.  As Roman correctly  
points out, faults are more like outputs, have more of application  
level semantics and are normally handled by the client, where  
exceptions (transient errors) are something camel could try to recover  
from, without much knowledge about the application.  I think that the  
presence of fault in the camel api is not due to it being explicitly  
modeled by jbi and wsdl, as Roman suggests, and Camel simply copying  
that, but it's modeled in Camel for the same reason it's modeled in  
jbi and wsdl, to differentiate transient from persistent errors in a  
non ambiguous way.

If we were to go with only get/setMessage() api, we would still need  
methods (or some ways) to distinguish between the kind of message we  
are dealing with (in/out/fault/exception) so we'd only move the  
problem somewhere else.

So the question becomes if we leave the api the way it is, or we  
replace the get/setFault apis with get/setOut, in which case we'll  
need something like:
     boolean isFault();
method in the Message api or keep the hasFault() method on the Exchange.

Thoughts?


     ExchangePattern getPattern();
     void setPattern(ExchangePattern pattern);

     Object getProperty(String name);
     <T> T getProperty(String name, Class<T> type);
     void setProperty(String name, Object value);
     Object removeProperty(String name);
     Map<String, Object> getProperties();

     Message getIn();
     void setIn(Message in);

     Message getOut();
     boolean hasOut();
     Message getOut(boolean lazyCreate);
     void setOut(Message out);

     Message getFault();
     boolean hasFault();
     Message getFault(boolean lazyCreate);
     void removeFault();
// removeFault() is only used in one place

     Exception getException();
     <T> T getException(Class<T> type);
     void setException(Exception e);
     boolean isFailed();

     boolean isTransacted();
     boolean isRollbackOnly();

     CamelContext getContext();

     Exchange newInstance();
     Exchange copy();
     Exchange newCopy(boolean handoverOnCompletion);
     void copyFrom(Exchange source);

     Endpoint getFromEndpoint();
     void setFromEndpoint(Endpoint fromEndpoint);

     UnitOfWork getUnitOfWork();
     void setUnitOfWork(UnitOfWork unitOfWork);

     String getExchangeId();
     void setExchangeId(String id);

     void addOnCompletion(Synchronization onCompletion);

Re: [DISCUSS] Faults and Exceptions in Camel

[Claus Ibsen <cl...@gmail.com>]

Just to point out how Camel route an exception / fault
(before grokking on james latest post)

Whenever the Camel pipeline router see that the Exchange has either an
Exception set
or a FAULT message then it breaks current routing and back out, eg
returns to the input = the consumer.

I think that is great that you can deliverable set a FAULT message and
Camel will know its a "persistent fault"
and not attempt any redeliveries but back out immediately.

So instead of throwing an OrderNotFoundException (that Camel could
potential attempt to redeliver, if redelivery is configured)
that is doomed to fail all the time, we could set it as a FAULT
instead, and redelivery will newer happen.

The story of transient vs. persistent exception, where we can leverage
the fact that FAULT is persistent per nature in Camel.
And all other exceptions are transient, and a redelivery attempt can
be made, if configured to do so.

Its lunch time for me, keep the discussion warm.


On Thu, Jul 9, 2009 at 12:06 PM, James Strachan<ja...@gmail.com> wrote:
> A few random thoughts.
>
> So we definitely need a way to determine if the output on an exchange
> is an OUT, fault, exception. We could have an OUT and a Fault Message;
> or have a single OUT Message with a boolean fault property.
>
> We could store an Exception as a body of the OUT maybe, though I can't
> help feel that an Exception is a bit different to an OUT/Fault (which
> are messages). e.g. you might want to clear the exception but keep the
> OUT?
>
> To process the output in a pipeline we could do something like...
>
>  Throwable e = exchange.getException();
>  if (e != null) {
>    // process the exception
>  }
>  else {
>     // we should now have now valid output
>    Message out = exchange.getOut();
>    if (out == null) {
>      // no output created, so reuse input?
>      out = exchange.getIn();
>    }
>
>    // we might not care if its a fault or out, we might just wanna
> return it anyway
>   if (out.isFault()) {
>     // do some fault specific coding here...
>   }
> }
>
> So we could use the OUT for a fault or a regular OUT (as you could say
> its an output message, whether a fault or real response) so code might
> not care if its a fault or not? So maybe a flag on OUT is neater?
>
> Exceptions seem different though; its typically something you'd wanna
> look at (and might wanna know what the OUT was when the exception
> threw), so having that as a property on Exchange feels right to me.
>
>
> Main problem seems to be the lazy create stuff. (Bad James!). Maybe we
> just need OUT to be null and if someone wants to create an OUT there's
> a factory method...
>
> Message out = exchange.createOut();
>
> (it could maybe be smart enough that if there's already an OUT defined
> it returns that one?). Once someone calls createOut() then the OUT is
> set and getOut() returns a non null value?
>
> Then if its a fault...
>
> out.setFault(true);
>
>
> Then if folks call exchange.getOut() it will typically return null and
> not do any lazy creation or gobble up messages etc as Claus says?
>
>
> 2009/7/9 Claus Ibsen <cl...@gmail.com>:
>> On Thu, Jul 9, 2009 at 9:03 AM, Guillaume Nodet<gn...@gmail.com> wrote:
>>> In web services world, faults are not exceptions, but usually an xml
>>> payload.  In the java world, faults would be like checked exceptions
>>> and errors runtime exceptions.  However, distinguishing a fault from
>>> an out message by (instanceof Exception) is imho not sufficient
>>
>> Yeah I was about to say the same. I think the FAULT message makes
>> sense. Fits better with the web service if you have declared faults in
>> the wsdl.
>> Then you can construct a XML message and set it as getFault().setBody().
>>
>> And I would also think it is much more confusing to set a fault
>> message, wrapped as an exception on the OUT message as opposed to use
>> setException on the exchange instead. That would be odd.
>>
>> So the API is good, my only griefs are
>> a) the OUT creating empty messages.
>> b) and that people might think as if they can during routing processes
>> piece by piece assemble an OUT message.
>>
>> a)
>> See previous mails about this.
>>
>> b)
>> An example to illustrate this. For instance in the route below A, B, C
>> is independent steps that enrich a message with order details.
>> Suppose the input message is an order id. And the expected message is
>> details about this order.
>>
>> from("direct:start").process(A).process(B).process(C);
>>
>> from
>> IN = 123
>> OUT = null
>>
>> A
>> IN = 123
>> OUT = Orderid: 123.
>>
>> B
>> IN = 123
>> OUT = Orderid: 123. Status = IN PROGRESS
>>
>> C
>> IN = 123
>> OUT = Orderid: 123. Status = IN PROGRESS. ETA: 2009/08/13
>>
>> Client receives
>> OUT: Orderid: 123. Status = IN PROGRESS. ETA: 2009/08/13
>>
>>
>> But the nature of pipes and filter, this is what happens
>>
>> from
>> IN = 123
>> OUT = null
>>
>> A
>> IN = 123
>> OUT = Orderid: 123.
>>
>> B
>> IN = Orderid: 123.
>> OUT = null
>>
>> Kabom!!! now we got a partly OUT message as IN instead of the original
>> IN message.
>>
>> This is right according to the pipes and filters, where previous OUT
>> should be next IN.
>>
>>
>> But people should just be aware with this in relation to IN and OUT -
>> that the OUT is not something that you can piece by piece assemble.
>> And OUT is really not that useable.
>>
>>
>>
>>
>>>
>>> On Thu, Jul 9, 2009 at 07:35, Hadrian Zbarcea<hz...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> Comments inline.
>>>>
>>>> Hadrian
>>>>
>>>> On Jul 9, 2009, at 12:54 AM, Claus Ibsen wrote:
>>>>
>>>>> On Wed, Jul 8, 2009 at 9:52 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> As we approach the 2.0, there is one more hanging issue I would like
>>>>>> addressed, if possible.  It's the thorny issue of Faults and Exceptions
>>>>>> that
>>>>>> started in
>>>>>> http://issues.apache.org/activemq/browse/CAMEL-316 (see also the related
>>>>>> nabble thread linked in the issue description).
>>>>>>
>>>>>> I am less concerned about how the DefaultExchange is implemented and I
>>>>>> hope
>>>>>> to reach an agreement on what the Exchange api should be (please find the
>>>>>> list of Exchange methods below).
>>>>>>
>>>>>> As far as faults/exceptions are concerned, Roman thinks that the whole
>>>>>> concept of in/out/fault/exception is artificial, and only one payload
>>>>>> (message) api should be enough (Roman please correct me if I misinterpret
>>>>>> your position).  My opinion is that we *must* distinguish between
>>>>>> persistent
>>>>>> (fault) and transient (exception) errors for the simple reason that they
>>>>>> have different semantics.  As Roman correctly points out, faults are more
>>>>>> like outputs, have more of application level semantics and are normally
>>>>>> handled by the client, where exceptions (transient errors) are something
>>>>>> camel could try to recover from, without much knowledge about the
>>>>>> application.  I think that the presence of fault in the camel api is not
>>>>>> due
>>>>>> to it being explicitly modeled by jbi and wsdl, as Roman suggests, and
>>>>>> Camel
>>>>>> simply copying that, but it's modeled in Camel for the same reason it's
>>>>>> modeled in jbi and wsdl, to differentiate transient from persistent
>>>>>> errors
>>>>>> in a non ambiguous way.
>>>>>
>>>>> I am one of the persons that would love the Camel Exchange / Message
>>>>> API to be a bit simpler. It has a fair shares of methods.
>>>>>
>>>>> Having listening and discussing with Hadrian on this and doing my own
>>>>> investigations and whatnot I do belive that Hadrian is absolutely
>>>>> right when it comes to FAULT. It has a good place in the API.  I am +1
>>>>> on having FAULT as we do now.
>>>>>
>>>>> The grief I have left is that the IN and OUT. It makes sense to have
>>>>> them and they provide a good value. However they have a big drawnback
>>>>> in how they are routed in Camel with the Pipeline processor, that
>>>>> mimics the pipes and filters EIP. And as a result the OUT will be used
>>>>> as IN
>>>>> in the next step in the route. So its not like you can steadily build
>>>>> up an OUT message on-the-fly during many steps in the route path.
>>>>>
>>>>> Example
>>>>> from("direct:start").process(new Processor()).to("log:foo");
>>>>>
>>>>> a) From
>>>>> IN = Hello World
>>>>> OUT = null
>>>>>
>>>>> b) Processor
>>>>> IN Hello World
>>>>> OUT = Bye World
>>>>>
>>>>> c) Log
>>>>> IN = Bye World
>>>>> OUT = null
>>>>>
>>>> Yes, from an external observer's perspective, this is precisely what
>>>> happens.  How we decide to store it, how many fields we need, is an
>>>> implementation detail of the DefaultExchange.  I don't think the copy from
>>>> out/in is too expensive, but I would be ok with having only one field to
>>>> store the current message in DefaultExchange (I assume that's what you
>>>> propose).  However, my question is about what the api should be.
>>>>
>>>>
>>>>>
>>>>> And then the getOut() method that lazy creates a new empty OUT message
>>>>> is also a pita, as it can lead to people loosing their messages if
>>>>> they do some System out logging of their own
>>>>>
>>>>> public void process(Exchange e) {
>>>>>  System.out.println(exchange.getIn());
>>>>>  System.out.println(exchange.getOut());
>>>>>  // boom you lost your message when its routed to next node in route
>>>>> path, as getOut() created a new empty OUT message that will by used in
>>>>> the pipes and filters EIP routed with the Pipeline
>>>>> }
>>>>>
>>>>> We had this IN OUT discussion a while back and at that time we ended
>>>>> up with a compromise of having a hasOut() method so you should do, to
>>>>> be safe:
>>>>>  System.out.println(exchange.getIn());
>>>>>  if (exchange.hasOut()) {
>>>>>      System.out.println(exchange.getOut());
>>>>>  }
>>>>>
>>>>> Still a pita with the lazy creation IMHO.
>>>>
>>>> The lazyCreate methods are actually deprecated, and imho should be removed
>>>> now.  This would eliminate the confusion.
>>>>
>>>>
>>>>>>
>>>>>> If we were to go with only get/setMessage() api, we would still need
>>>>>> methods
>>>>>> (or some ways) to distinguish between the kind of message we are dealing
>>>>>> with (in/out/fault/exception) so we'd only move the problem somewhere
>>>>>> else.
>>>>>>
>>>>>> So the question becomes if we leave the api the way it is, or we replace
>>>>>> the
>>>>>> get/setFault apis with get/setOut, in which case we'll need something
>>>>>> like:
>>>>>>   boolean isFault();
>>>>>> method in the Message api or keep the hasFault() method on the Exchange.
>>>>>
>>>>> Good question
>>>>>
>>>>> If you use OUT instead then we need to add a isFault() on the
>>>>> org.apache.camel.Message API that
>>>>> the IN message also implements.
>>>>>
>>>>> It could make sense to use OUT as well for FAULT.
>>>>> But how should the API look like to set an OUT as Fault?
>>>>>
>>>>> Something a like this?
>>>>>
>>>>> getOut().setBody("Unknown bank account number.");
>>>>> getOut().setFault(true);
>>>>
>>>> Not quite, I had something like this in mind (there is no setFault()
>>>> method):
>>>> getOut().setBody(java-lang-Exception-derivedObject);
>>>>
>>>> boolean isFault() {
>>>>    return getBody() instanceof Exception;
>>>> }
>>>>
>>>> Personally I am ok with the limitation not being able to have an out of a
>>>> java.lang.Exception type (that would then be a Fault).  I can't imagine a
>>>> case where an Exception would be an expected out, and in such cases one
>>>> could always serialize or wrap it.  The fact that the Fault would be an
>>>> Exception type would be a camel convention that needs to be followed by all
>>>> components.
>>>>
>>>> Another option would be add  header HAS_FAULT or something like that, in
>>>> which case both the out and the fault could be of any type.
>>>>
>>>>
>>>>>
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>>
>>>>>>   ExchangePattern getPattern();
>>>>>>   void setPattern(ExchangePattern pattern);
>>>>>>
>>>>>>   Object getProperty(String name);
>>>>>>   <T> T getProperty(String name, Class<T> type);
>>>>>>   void setProperty(String name, Object value);
>>>>>>   Object removeProperty(String name);
>>>>>>   Map<String, Object> getProperties();
>>>>>>
>>>>>>   Message getIn();
>>>>>>   void setIn(Message in);
>>>>>>
>>>>>>   Message getOut();
>>>>>>   boolean hasOut();
>>>>>>   Message getOut(boolean lazyCreate);
>>>>>>   void setOut(Message out);
>>>>>>
>>>>>>   Message getFault();
>>>>>>   boolean hasFault();
>>>>>>   Message getFault(boolean lazyCreate);
>>>>>>   void removeFault();
>>>>>> // removeFault() is only used in one place
>>>>>
>>>>> +1 to remove it. You can just do setFault(null) instead. I will fix it
>>>>> asap.
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>   Exception getException();
>>>>>>   <T> T getException(Class<T> type);
>>>>>>   void setException(Exception e);
>>>>>>   boolean isFailed();
>>>>>>
>>>>>>   boolean isTransacted();
>>>>>>   boolean isRollbackOnly();
>>>>>>
>>>>>>   CamelContext getContext();
>>>>>>
>>>>>>   Exchange newInstance();
>>>>>>   Exchange copy();
>>>>>>   Exchange newCopy(boolean handoverOnCompletion);
>>>>>>   void copyFrom(Exchange source);
>>>>>>
>>>>>>   Endpoint getFromEndpoint();
>>>>>>   void setFromEndpoint(Endpoint fromEndpoint);
>>>>>>
>>>>>>   UnitOfWork getUnitOfWork();
>>>>>>   void setUnitOfWork(UnitOfWork unitOfWork);
>>>>>>
>>>>>>   String getExchangeId();
>>>>>>   void setExchangeId(String id);
>>>>>>
>>>>>>   void addOnCompletion(Synchronization onCompletion);
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Cheers,
>>> Guillaume Nodet
>>> ------------------------
>>> Blog: http://gnodet.blogspot.com/
>>> ------------------------
>>> Open Source SOA
>>> http://fusesource.com
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[James Strachan <ja...@gmail.com>]

2009/7/14 Roman Kalukiewicz <ro...@gmail.com>:
> 2009/7/14 James Strachan <ja...@gmail.com>:
>> 2009/7/13 Claus Ibsen <cl...@gmail.com>:
>>>>
>>>> Reading between the lines; are you really just trying to make folks
>>>> use (what is currently called) "getOut()" and never try mutate what is
>>>> currently called getIn()?
>>>>
>>>> i.e. so by default the "OUT" property is defaulted to a copy of IN
>>>> that folks can change/mutate.
>>>>
>>>> (what we call these 2 methods is a separate discussion, whether its
>>>> "in,out" or "originalMessage,message" or whatever
>>>>
>>>
>>> Hadrian and I had a chat today and we are clearing up some bits.
>>> I am more on line with him now on the IN and OUT thing.
>>>
>>> So lets keep them.
>>>
>>> And use the time to fix the tiny bits such as the getOut() doing its
>>> lazy creating a new empty message.
>>> And whether its possible to let IN be immutable.
>>
>> I think we're kinda mostly on the same page (though saying it in
>> different ways). BTW I'm taking off my devils advocate hat now :)...
>>
>> What we're agreeing on I think is that;
>>
>> * getIn() should be immutable (when folks try to change it we can
>> throw the exception and describe how getOut() should be used to change
>> the message) - to prevent folks using the old code (which will require
>> code changes).
>
> Definitely agree.
>
>> * having the original immutable message available is handy; but mostly
>> folks should concentrate on the 'out' (current name today)
>
> It is handy.But the question is if we should add another message while
> both client and processor implementor can store original message (body
> or headers he wants) in a variable if they only want to store them. Do
> we have to store those things for them no mater if they need it or
> not? This comment applies only if we want to have IN message on
> processor level (every processor receives its new IN message).
>
> I personally don't see a huge difference between:
> Object oldBody = exchange.getMessage().getBody();
> template.send(exchange);
> Object newBody = exchange.getMessage.getBody();
>
> and
> template.send(exchange);
> Object oldBody = exchange.getOriginalMessage().getBody();
> Object newBody = exchange.getMessage().getBody();
>
> while the second one complicates an API by having additional method on
> the exchange.
>
>> * the out should be automatically populated with a clone of the IN (to
>> avoid all that pain with checking if there's an out or an in, or the
>> possible loss of headers etc. Internally we can use a CopyOnWrite
>> facade to reduce the cost of potentially copying a message which is
>> not actually mutated.
>
> True
>
>> Given that; I think we're mostly agreeing. However given the confusion
>> of getIn() and getOut() I'm wondering if actually your previous idea
>> of changing the api of exchange to have a getMessage() which returns
>> the thing a processor should be changing; then having a
>> getOriginalMessage() (which can be null if you are the first in the
>> chain) might reduce the amount of confusion?
>
> Because of previous comment about IN messages I believe that maybe we
> should have getOriginalMessage() that returns an original message as
> JMS Message, JBI Exchange and so on. This assures that users can
> always reach full information they received. It would look like
> from("jms:q1")  // Original message contains JMS message received
> .transform(expression) // Still original JMS message - no endpoint involved
> .setHeader("foo", expression) // Still original JMS message - still I
> can reach some JMS specific things
> .to("jbi:service") // Now I have JBI exchange in original message
> .bean(myBean) // My Bean reaches exchange directly to reach some stuff
> from ServiceMix specific JBI exchange
> .to("jms:q2") // JMS response message in original message
> .to("direct:end") // Camel Exchange in original message as it is now
> endpoint specific type
> .to("seda:foo") // InOnly operation. I'm not sure if it should be null
> in original message, or maybe nothing modified.
>
> I propose it because I don't really like our custom Message
> subclasses. They tend to mirror Camel message model and some
> underlying technology's model. But sometimes they are not compatible
> really and strange things happen. But maybe this is the discussion for
> another thread?
>
>> i.e. after sleeping on it, I'm warming to the idea of renaming getIn()
>> -> getOriginalMessage() and getOut() -> getMessage(). (Or maybe
>> getInputMessage() for getIn()?)
>>
>> Thoughts?
>
> I definitely agree that for the moment we can add those methods,
> deprecate getIn()/getOut() and let them simply return getMessage().
>
> To summarize my ideal model is (helpers like getBody(Class) omitted):
> CamelMessage
> Object get/setBody() //obvious
> Map<String, Object> getHeaders() // for technology specific headers
> received/sent
> Map<String, Object> getProperties() // for business specific
> information that is generally not touched by endpoints - user can
> store there any information he wants to be sent through the flow and
> not be lost at the endpoint
> Object get/setOriginalMessage() // For technology specific message
> representation like JMS Message/JBI Exchange
> Exception get/setException() // For exception handling
> boolean is/setFault() // For business fault detection
>
> In a model where only one message exists, exchange is not even needed.
>
> Heh - I feel guilty for enlisting all those things now while it looks
> that an agreement on API changes exists ;)
> Maybe a proposal for 2.1? ;)

FWIW I still like the Exchange being there as a holder of the Message;
as folks can then, if they want, create a totally new, empty Message
and set it on the Exchange (rather than just mutating the incoming
one).

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Semantics of IN and OUT

[Willem Jiang <wi...@gmail.com>]

Roman Kalukiewicz wrote:
> 2009/7/14 James Strachan <ja...@gmail.com>:
>> 2009/7/13 Claus Ibsen <cl...@gmail.com>:
>>>> Reading between the lines; are you really just trying to make folks
>>>> use (what is currently called) "getOut()" and never try mutate what is
>>>> currently called getIn()?
>>>>
>>>> i.e. so by default the "OUT" property is defaulted to a copy of IN
>>>> that folks can change/mutate.
>>>>
>>>> (what we call these 2 methods is a separate discussion, whether its
>>>> "in,out" or "originalMessage,message" or whatever
>>>>
>>> Hadrian and I had a chat today and we are clearing up some bits.
>>> I am more on line with him now on the IN and OUT thing.
>>>
>>> So lets keep them.
>>>
>>> And use the time to fix the tiny bits such as the getOut() doing its
>>> lazy creating a new empty message.
>>> And whether its possible to let IN be immutable.
>> I think we're kinda mostly on the same page (though saying it in
>> different ways). BTW I'm taking off my devils advocate hat now :)...
>>
>> What we're agreeing on I think is that;
>>
>> * getIn() should be immutable (when folks try to change it we can
>> throw the exception and describe how getOut() should be used to change
>> the message) - to prevent folks using the old code (which will require
>> code changes).
> 
> Definitely agree.
> 
>> * having the original immutable message available is handy; but mostly
>> folks should concentrate on the 'out' (current name today)
> 
> It is handy.But the question is if we should add another message while
> both client and processor implementor can store original message (body
> or headers he wants) in a variable if they only want to store them. Do
> we have to store those things for them no mater if they need it or
> not? This comment applies only if we want to have IN message on
> processor level (every processor receives its new IN message).
> 
> I personally don't see a huge difference between:
> Object oldBody = exchange.getMessage().getBody();
> template.send(exchange);
> Object newBody = exchange.getMessage.getBody();
> 
> and
> template.send(exchange);
> Object oldBody = exchange.getOriginalMessage().getBody();
> Object newBody = exchange.getMessage().getBody();
> 
> while the second one complicates an API by having additional method on
> the exchange.
> 
>> * the out should be automatically populated with a clone of the IN (to
>> avoid all that pain with checking if there's an out or an in, or the
>> possible loss of headers etc. Internally we can use a CopyOnWrite
>> facade to reduce the cost of potentially copying a message which is
>> not actually mutated.
> 
> True
> 
>> Given that; I think we're mostly agreeing. However given the confusion
>> of getIn() and getOut() I'm wondering if actually your previous idea
>> of changing the api of exchange to have a getMessage() which returns
>> the thing a processor should be changing; then having a
>> getOriginalMessage() (which can be null if you are the first in the
>> chain) might reduce the amount of confusion?
> 
> Because of previous comment about IN messages I believe that maybe we
> should have getOriginalMessage() that returns an original message as
> JMS Message, JBI Exchange and so on. This assures that users can
> always reach full information they received. It would look like
> from("jms:q1")  // Original message contains JMS message received
> .transform(expression) // Still original JMS message - no endpoint involved
> .setHeader("foo", expression) // Still original JMS message - still I
> can reach some JMS specific things
> .to("jbi:service") // Now I have JBI exchange in original message
> .bean(myBean) // My Bean reaches exchange directly to reach some stuff
> from ServiceMix specific JBI exchange
> .to("jms:q2") // JMS response message in original message
> .to("direct:end") // Camel Exchange in original message as it is now
> endpoint specific type
> .to("seda:foo") // InOnly operation. I'm not sure if it should be null
> in original message, or maybe nothing modified.
> 
> I propose it because I don't really like our custom Message
> subclasses. They tend to mirror Camel message model and some
> underlying technology's model. But sometimes they are not compatible
> really and strange things happen. But maybe this is the discussion for
> another thread?
In most of Camel 2.0's components , we don't create the custom Message 
subclasses anymore, we just copy the component specify message context 
into the DefaultMessage header. In this way we could avoid the some 
component specify message copy work which will cause some trouble when 
routing the message between different components.

And we also did some exchange clean up work on Camel 2.0 recently  :).

> 
>> i.e. after sleeping on it, I'm warming to the idea of renaming getIn()
>> -> getOriginalMessage() and getOut() -> getMessage(). (Or maybe
>> getInputMessage() for getIn()?)
>>
>> Thoughts?
> 
> I definitely agree that for the moment we can add those methods,
> deprecate getIn()/getOut() and let them simply return getMessage().
> 
> To summarize my ideal model is (helpers like getBody(Class) omitted):
> CamelMessage
> Object get/setBody() //obvious
> Map<String, Object> getHeaders() // for technology specific headers
> received/sent
> Map<String, Object> getProperties() // for business specific
> information that is generally not touched by endpoints - user can
> store there any information he wants to be sent through the flow and
> not be lost at the endpoint
> Object get/setOriginalMessage() // For technology specific message
> representation like JMS Message/JBI Exchange
> Exception get/setException() // For exception handling
> boolean is/setFault() // For business fault detection
> 
> In a model where only one message exists, exchange is not even needed.
> 
> Heh - I feel guilty for enlisting all those things now while it looks
> that an agreement on API changes exists ;)
> Maybe a proposal for 2.1? ;)
> 
> Roman
> 

Willem

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Roman Kalukiewicz <ro...@gmail.com>]

2009/7/14 James Strachan <ja...@gmail.com>:
> 2009/7/13 Claus Ibsen <cl...@gmail.com>:
>>>
>>> Reading between the lines; are you really just trying to make folks
>>> use (what is currently called) "getOut()" and never try mutate what is
>>> currently called getIn()?
>>>
>>> i.e. so by default the "OUT" property is defaulted to a copy of IN
>>> that folks can change/mutate.
>>>
>>> (what we call these 2 methods is a separate discussion, whether its
>>> "in,out" or "originalMessage,message" or whatever
>>>
>>
>> Hadrian and I had a chat today and we are clearing up some bits.
>> I am more on line with him now on the IN and OUT thing.
>>
>> So lets keep them.
>>
>> And use the time to fix the tiny bits such as the getOut() doing its
>> lazy creating a new empty message.
>> And whether its possible to let IN be immutable.
>
> I think we're kinda mostly on the same page (though saying it in
> different ways). BTW I'm taking off my devils advocate hat now :)...
>
> What we're agreeing on I think is that;
>
> * getIn() should be immutable (when folks try to change it we can
> throw the exception and describe how getOut() should be used to change
> the message) - to prevent folks using the old code (which will require
> code changes).

Definitely agree.

> * having the original immutable message available is handy; but mostly
> folks should concentrate on the 'out' (current name today)

It is handy.But the question is if we should add another message while
both client and processor implementor can store original message (body
or headers he wants) in a variable if they only want to store them. Do
we have to store those things for them no mater if they need it or
not? This comment applies only if we want to have IN message on
processor level (every processor receives its new IN message).

I personally don't see a huge difference between:
Object oldBody = exchange.getMessage().getBody();
template.send(exchange);
Object newBody = exchange.getMessage.getBody();

and
template.send(exchange);
Object oldBody = exchange.getOriginalMessage().getBody();
Object newBody = exchange.getMessage().getBody();

while the second one complicates an API by having additional method on
the exchange.

> * the out should be automatically populated with a clone of the IN (to
> avoid all that pain with checking if there's an out or an in, or the
> possible loss of headers etc. Internally we can use a CopyOnWrite
> facade to reduce the cost of potentially copying a message which is
> not actually mutated.

True

> Given that; I think we're mostly agreeing. However given the confusion
> of getIn() and getOut() I'm wondering if actually your previous idea
> of changing the api of exchange to have a getMessage() which returns
> the thing a processor should be changing; then having a
> getOriginalMessage() (which can be null if you are the first in the
> chain) might reduce the amount of confusion?

Because of previous comment about IN messages I believe that maybe we
should have getOriginalMessage() that returns an original message as
JMS Message, JBI Exchange and so on. This assures that users can
always reach full information they received. It would look like
from("jms:q1")  // Original message contains JMS message received
.transform(expression) // Still original JMS message - no endpoint involved
.setHeader("foo", expression) // Still original JMS message - still I
can reach some JMS specific things
.to("jbi:service") // Now I have JBI exchange in original message
.bean(myBean) // My Bean reaches exchange directly to reach some stuff
from ServiceMix specific JBI exchange
.to("jms:q2") // JMS response message in original message
.to("direct:end") // Camel Exchange in original message as it is now
endpoint specific type
.to("seda:foo") // InOnly operation. I'm not sure if it should be null
in original message, or maybe nothing modified.

I propose it because I don't really like our custom Message
subclasses. They tend to mirror Camel message model and some
underlying technology's model. But sometimes they are not compatible
really and strange things happen. But maybe this is the discussion for
another thread?

> i.e. after sleeping on it, I'm warming to the idea of renaming getIn()
> -> getOriginalMessage() and getOut() -> getMessage(). (Or maybe
> getInputMessage() for getIn()?)
>
> Thoughts?

I definitely agree that for the moment we can add those methods,
deprecate getIn()/getOut() and let them simply return getMessage().

To summarize my ideal model is (helpers like getBody(Class) omitted):
CamelMessage
Object get/setBody() //obvious
Map<String, Object> getHeaders() // for technology specific headers
received/sent
Map<String, Object> getProperties() // for business specific
information that is generally not touched by endpoints - user can
store there any information he wants to be sent through the flow and
not be lost at the endpoint
Object get/setOriginalMessage() // For technology specific message
representation like JMS Message/JBI Exchange
Exception get/setException() // For exception handling
boolean is/setFault() // For business fault detection

In a model where only one message exists, exchange is not even needed.

Heh - I feel guilty for enlisting all those things now while it looks
that an agreement on API changes exists ;)
Maybe a proposal for 2.1? ;)

Roman

Re: [DISCUSS] Semantics of IN and OUT

[James Strachan <ja...@gmail.com>]

2009/7/14 Willem Jiang <wi...@gmail.com>:
> James Strachan wrote:
>>
>> 2009/7/13 Claus Ibsen <cl...@gmail.com>:
>>>>
>>>> Reading between the lines; are you really just trying to make folks
>>>> use (what is currently called) "getOut()" and never try mutate what is
>>>> currently called getIn()?
>>>>
>>>> i.e. so by default the "OUT" property is defaulted to a copy of IN
>>>> that folks can change/mutate.
>>>>
>>>> (what we call these 2 methods is a separate discussion, whether its
>>>> "in,out" or "originalMessage,message" or whatever
>>>>
>>> Hadrian and I had a chat today and we are clearing up some bits.
>>> I am more on line with him now on the IN and OUT thing.
>>>
>>> So lets keep them.
>>>
>>> And use the time to fix the tiny bits such as the getOut() doing its
>>> lazy creating a new empty message.
>>> And whether its possible to let IN be immutable.
>>
>> I think we're kinda mostly on the same page (though saying it in
>> different ways). BTW I'm taking off my devils advocate hat now :)...
>>
>> What we're agreeing on I think is that;
>>
>> * getIn() should be immutable (when folks try to change it we can
>> throw the exception and describe how getOut() should be used to change
>> the message) - to prevent folks using the old code (which will require
>> code changes).
>> * having the original immutable message available is handy; but mostly
>> folks should concentrate on the 'out' (current name today)
>> * the out should be automatically populated with a clone of the IN (to
>> avoid all that pain with checking if there's an out or an in, or the
>> possible loss of headers etc. Internally we can use a CopyOnWrite
>> facade to reduce the cost of potentially copying a message which is
>> not actually mutated.
>>
>> Given that; I think we're mostly agreeing. However given the confusion
>> of getIn() and getOut() I'm wondering if actually your previous idea
>> of changing the api of exchange to have a getMessage() which returns
>> the thing a processor should be changing; then having a
>> getOriginalMessage() (which can be null if you are the first in the
>> chain) might reduce the amount of confusion?
>>
>> i.e. after sleeping on it, I'm warming to the idea of renaming getIn()
>> -> getOriginalMessage() and getOut() -> getMessage(). (Or maybe
>> getInputMessage() for getIn()?)
>>
>> Thoughts?
>>
> +1 for this change.
>
> For the InOnly Message Exchange Pattern, when we want to change the message
> , we just call the exchange's getMessage() and don't not care about if it's
> a in message or an out message.
>
> For the ProduceTemplate sendMessage() method we can get the response message
> back from the exchange, by using exchange.getMessage() method.

Good point.

When sending a message, the getOriginalMessage() will be null (we
should document that its an optional property that might be null etc).


-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Semantics of IN and OUT

[Willem Jiang <wi...@gmail.com>]

James Strachan wrote:
> 2009/7/13 Claus Ibsen <cl...@gmail.com>:
>>> Reading between the lines; are you really just trying to make folks
>>> use (what is currently called) "getOut()" and never try mutate what is
>>> currently called getIn()?
>>>
>>> i.e. so by default the "OUT" property is defaulted to a copy of IN
>>> that folks can change/mutate.
>>>
>>> (what we call these 2 methods is a separate discussion, whether its
>>> "in,out" or "originalMessage,message" or whatever
>>>
>> Hadrian and I had a chat today and we are clearing up some bits.
>> I am more on line with him now on the IN and OUT thing.
>>
>> So lets keep them.
>>
>> And use the time to fix the tiny bits such as the getOut() doing its
>> lazy creating a new empty message.
>> And whether its possible to let IN be immutable.
> 
> I think we're kinda mostly on the same page (though saying it in
> different ways). BTW I'm taking off my devils advocate hat now :)...
> 
> What we're agreeing on I think is that;
> 
> * getIn() should be immutable (when folks try to change it we can
> throw the exception and describe how getOut() should be used to change
> the message) - to prevent folks using the old code (which will require
> code changes).
> * having the original immutable message available is handy; but mostly
> folks should concentrate on the 'out' (current name today)
> * the out should be automatically populated with a clone of the IN (to
> avoid all that pain with checking if there's an out or an in, or the
> possible loss of headers etc. Internally we can use a CopyOnWrite
> facade to reduce the cost of potentially copying a message which is
> not actually mutated.
> 
> Given that; I think we're mostly agreeing. However given the confusion
> of getIn() and getOut() I'm wondering if actually your previous idea
> of changing the api of exchange to have a getMessage() which returns
> the thing a processor should be changing; then having a
> getOriginalMessage() (which can be null if you are the first in the
> chain) might reduce the amount of confusion?
> 
> i.e. after sleeping on it, I'm warming to the idea of renaming getIn()
> -> getOriginalMessage() and getOut() -> getMessage(). (Or maybe
> getInputMessage() for getIn()?)
> 
> Thoughts?
> 
+1 for this change.

For the InOnly Message Exchange Pattern, when we want to change the 
message , we just call the exchange's getMessage() and don't not care 
about if it's a in message or an out message.

For the ProduceTemplate sendMessage() method we can get the response 
message back from the exchange, by using exchange.getMessage() method.


Willem

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Claus Ibsen <cl...@gmail.com>]

> Yeah.
>
> BTW we could leave getIn() and getOut() & try implement the
> ImmutableWrapper and CopyOnWriteFacade for In/Out respectively and try
> them out inside the current API - before we go ahead with any kind of
> renaming of getIn() and getOut().
>

> Maybe we could even keep getIn() / getOut() around for 2.0 (and zap in
> 2.1) and leave deprecated and make them just delegate to the new
> method names (say getInputMessage() and getMessage()). The deprecated
> message could then describe how folks typically should switch
> getIn()/getOut() to using getMessage() unless they really really want
> the immutable input message getInputMessage().
>

Brilliant idea and a good migration path as well. And it wont delay
Camel 2.0 as much as a full blown migration would have done.

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Hadrian Zbarcea <hz...@gmail.com>]

Awesome.  This is very similar to what we agreed upon yesterday with  
Claus.

I personally much prefer the in/out terminology as it has pretty  
clearly defined meaning in a few specs.
The 'original' term is a bit confusing and I would avoid it.  It could  
mean either that the original coming into the consumer, or original  
coming into a processor, and I think we mean the latter.  I am not  
totally opposed to changing the method names, but this seems to me  
just a renaming and since it would incur extra work for the users, I'd  
rather keep it as is.

With cleaning up the implementation and the documentation, which could  
be done now or post 2.0, I am pretty happy.

Cheers,
Hadrian


On Jul 14, 2009, at 5:06 AM, James Strachan wrote:

> 2009/7/14 Claus Ibsen <cl...@gmail.com>:
>>> I think we're kinda mostly on the same page (though saying it in
>>> different ways). BTW I'm taking off my devils advocate hat now :)...
>>>
>>> What we're agreeing on I think is that;
>>>
>>> * getIn() should be immutable (when folks try to change it we can
>>> throw the exception and describe how getOut() should be used to  
>>> change
>>> the message) - to prevent folks using the old code (which will  
>>> require
>>> code changes).
>>> * having the original immutable message available is handy; but  
>>> mostly
>>> folks should concentrate on the 'out' (current name today)
>>> * the out should be automatically populated with a clone of the IN  
>>> (to
>>> avoid all that pain with checking if there's an out or an in, or the
>>> possible loss of headers etc. Internally we can use a CopyOnWrite
>>> facade to reduce the cost of potentially copying a message which is
>>> not actually mutated.
>>>
>>> Given that; I think we're mostly agreeing. However given the  
>>> confusion
>>> of getIn() and getOut() I'm wondering if actually your previous idea
>>> of changing the api of exchange to have a getMessage() which returns
>>> the thing a processor should be changing; then having a
>>> getOriginalMessage() (which can be null if you are the first in the
>>> chain) might reduce the amount of confusion?
>>>
>>> i.e. after sleeping on it, I'm warming to the idea of renaming  
>>> getIn()
>>> -> getOriginalMessage() and getOut() -> getMessage(). (Or maybe
>>> getInputMessage() for getIn()?)
>>>
>>> Thoughts?
>>
>> Very good thoughts James. I think all this will removes many  
>> obstacles
>> and confusing for new users to Camel.
>>
>> Now even the AggregationStrategy is less tricky to use as you will
>> know that using getMessage gets what you are looking for.
>>
>> Maybe we should create a wiki page at Camel where we can outline the
>> API proposal and ideas for the message facade.
>
> Yeah.
>
> BTW we could leave getIn() and getOut() & try implement the
> ImmutableWrapper and CopyOnWriteFacade for In/Out respectively and try
> them out inside the current API - before we go ahead with any kind of
> renaming of getIn() and getOut().
>
> Maybe we could even keep getIn() / getOut() around for 2.0 (and zap in
> 2.1) and leave deprecated and make them just delegate to the new
> method names (say getInputMessage() and getMessage()). The deprecated
> message could then describe how folks typically should switch
> getIn()/getOut() to using getMessage() unless they really really want
> the immutable input message getInputMessage().
>
> -- 
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[James Strachan <ja...@gmail.com>]

2009/7/14 Claus Ibsen <cl...@gmail.com>:
>> I think we're kinda mostly on the same page (though saying it in
>> different ways). BTW I'm taking off my devils advocate hat now :)...
>>
>> What we're agreeing on I think is that;
>>
>> * getIn() should be immutable (when folks try to change it we can
>> throw the exception and describe how getOut() should be used to change
>> the message) - to prevent folks using the old code (which will require
>> code changes).
>> * having the original immutable message available is handy; but mostly
>> folks should concentrate on the 'out' (current name today)
>> * the out should be automatically populated with a clone of the IN (to
>> avoid all that pain with checking if there's an out or an in, or the
>> possible loss of headers etc. Internally we can use a CopyOnWrite
>> facade to reduce the cost of potentially copying a message which is
>> not actually mutated.
>>
>> Given that; I think we're mostly agreeing. However given the confusion
>> of getIn() and getOut() I'm wondering if actually your previous idea
>> of changing the api of exchange to have a getMessage() which returns
>> the thing a processor should be changing; then having a
>> getOriginalMessage() (which can be null if you are the first in the
>> chain) might reduce the amount of confusion?
>>
>> i.e. after sleeping on it, I'm warming to the idea of renaming getIn()
>> -> getOriginalMessage() and getOut() -> getMessage(). (Or maybe
>> getInputMessage() for getIn()?)
>>
>> Thoughts?
>
> Very good thoughts James. I think all this will removes many obstacles
> and confusing for new users to Camel.
>
> Now even the AggregationStrategy is less tricky to use as you will
> know that using getMessage gets what you are looking for.
>
> Maybe we should create a wiki page at Camel where we can outline the
> API proposal and ideas for the message facade.

Yeah.

BTW we could leave getIn() and getOut() & try implement the
ImmutableWrapper and CopyOnWriteFacade for In/Out respectively and try
them out inside the current API - before we go ahead with any kind of
renaming of getIn() and getOut().

Maybe we could even keep getIn() / getOut() around for 2.0 (and zap in
2.1) and leave deprecated and make them just delegate to the new
method names (say getInputMessage() and getMessage()). The deprecated
message could then describe how folks typically should switch
getIn()/getOut() to using getMessage() unless they really really want
the immutable input message getInputMessage().

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Claus Ibsen <cl...@gmail.com>]

> I think we're kinda mostly on the same page (though saying it in
> different ways). BTW I'm taking off my devils advocate hat now :)...
>
> What we're agreeing on I think is that;
>
> * getIn() should be immutable (when folks try to change it we can
> throw the exception and describe how getOut() should be used to change
> the message) - to prevent folks using the old code (which will require
> code changes).
> * having the original immutable message available is handy; but mostly
> folks should concentrate on the 'out' (current name today)
> * the out should be automatically populated with a clone of the IN (to
> avoid all that pain with checking if there's an out or an in, or the
> possible loss of headers etc. Internally we can use a CopyOnWrite
> facade to reduce the cost of potentially copying a message which is
> not actually mutated.
>
> Given that; I think we're mostly agreeing. However given the confusion
> of getIn() and getOut() I'm wondering if actually your previous idea
> of changing the api of exchange to have a getMessage() which returns
> the thing a processor should be changing; then having a
> getOriginalMessage() (which can be null if you are the first in the
> chain) might reduce the amount of confusion?
>
> i.e. after sleeping on it, I'm warming to the idea of renaming getIn()
> -> getOriginalMessage() and getOut() -> getMessage(). (Or maybe
> getInputMessage() for getIn()?)
>
> Thoughts?

Very good thoughts James. I think all this will removes many obstacles
and confusing for new users to Camel.

Now even the AggregationStrategy is less tricky to use as you will
know that using getMessage gets what you are looking for.

Maybe we should create a wiki page at Camel where we can outline the
API proposal and ideas for the message facade.



>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[James Strachan <ja...@gmail.com>]

2009/7/13 Claus Ibsen <cl...@gmail.com>:
>>
>> Reading between the lines; are you really just trying to make folks
>> use (what is currently called) "getOut()" and never try mutate what is
>> currently called getIn()?
>>
>> i.e. so by default the "OUT" property is defaulted to a copy of IN
>> that folks can change/mutate.
>>
>> (what we call these 2 methods is a separate discussion, whether its
>> "in,out" or "originalMessage,message" or whatever
>>
>
> Hadrian and I had a chat today and we are clearing up some bits.
> I am more on line with him now on the IN and OUT thing.
>
> So lets keep them.
>
> And use the time to fix the tiny bits such as the getOut() doing its
> lazy creating a new empty message.
> And whether its possible to let IN be immutable.

I think we're kinda mostly on the same page (though saying it in
different ways). BTW I'm taking off my devils advocate hat now :)...

What we're agreeing on I think is that;

* getIn() should be immutable (when folks try to change it we can
throw the exception and describe how getOut() should be used to change
the message) - to prevent folks using the old code (which will require
code changes).
* having the original immutable message available is handy; but mostly
folks should concentrate on the 'out' (current name today)
* the out should be automatically populated with a clone of the IN (to
avoid all that pain with checking if there's an out or an in, or the
possible loss of headers etc. Internally we can use a CopyOnWrite
facade to reduce the cost of potentially copying a message which is
not actually mutated.

Given that; I think we're mostly agreeing. However given the confusion
of getIn() and getOut() I'm wondering if actually your previous idea
of changing the api of exchange to have a getMessage() which returns
the thing a processor should be changing; then having a
getOriginalMessage() (which can be null if you are the first in the
chain) might reduce the amount of confusion?

i.e. after sleeping on it, I'm warming to the idea of renaming getIn()
-> getOriginalMessage() and getOut() -> getMessage(). (Or maybe
getInputMessage() for getIn()?)

Thoughts?

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Claus Ibsen <cl...@gmail.com>]

>
> Reading between the lines; are you really just trying to make folks
> use (what is currently called) "getOut()" and never try mutate what is
> currently called getIn()?
>
> i.e. so by default the "OUT" property is defaulted to a copy of IN
> that folks can change/mutate.
>
> (what we call these 2 methods is a separate discussion, whether its
> "in,out" or "originalMessage,message" or whatever
>

Hadrian and I had a chat today and we are clearing up some bits.
I am more on line with him now on the IN and OUT thing.

So lets keep them.

And use the time to fix the tiny bits such as the getOut() doing its
lazy creating a new empty message.
And whether its possible to let IN be immutable.


/Claus

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[James Strachan <ja...@gmail.com>]

2009/7/13 James Strachan <ja...@gmail.com>:
> 2009/7/13 Claus Ibsen <cl...@gmail.com>:
>> On Mon, Jul 13, 2009 at 5:50 PM, James Strachan<ja...@gmail.com> wrote:
>>> I'm confused. So rather than calling Exchange.getIn() to get the IN,
>>> you'd have to do Exchange.getUnitOfWork().somethingOrOther() to get
>>> the last IN? I don't see that as any simpler; making one of the 2
>>> messages harder to find, just moves the confusion somewhere else?
>>
>> Having access to the original input in a route path is very rarely
>> requested by end users.
>> In fact I think its even more confused if getIN always returned the
>> original input. (what should that be if a message is routed over (to
>> seda/from seda) a seda endpoint?)
>>
>> So what I am saying is that we should have a model similar to Mule and
>> Spring Integration = simpler and easier to understand and use.
>> And also simpler and easier to use within the Camel framework itself -
>> eg all the code that copies from IN to OUT and OUT to IN and whatnot.
>>
>> The last IN = the message itself.
>>
>> Given this route:
>> from("direct:start").to("bean:foo").processRef("bar").transform(constant("Bye
>> World")).processRef("me");
>>
>> And we send a "Hello World" message to direct:start. The message would be
>>
>> bean:foo
>> getMessage = "Hello World
>>
>> bar
>> getMessage = "Hello World
>>
>> me
>> getMessage = "Bye World
>>
>>
>> Now imagine we mutate the messages in bean foo
>> public String reverse(String s) {
>>   // return the reversed string
>> }
>
> am confused; a bean never even sees an Exchange/Message typically?
>
>> bean:foo
>> getMessage = "Hello World"
>>
>> bar
>> getMessage = "dlroW olleH"
>>
>> me
>> getMessage = "Bye World"
>>
>>
>> No more IN vs OUT and other confusing bits. getMessage gets you the
>> payload as it is.
>
> I dunnot what 'it' is? You mean we always copy the IN to the OUT
> regardless of what the code does just in case; so you mutate the
> current message - then if you zap headers/body you have to keep a
> track of them yourself in case you want to look at them again?
>
>
>> If you have looked as much Camel code as I have you would also flag
>> that the IN OUT is leading to confusing and broken code.
>
> Note you were the one that pushed back when I said lets make getIn()
> immutable which would break some code :). Now you're saying lets just
> break *all* camel code.
>
>
>> And trust me its broken, when you use the Exchange, but the BODY
>> extractor of ProducerTemplate is fairly good at guessing and returning
>> the right one from OUT or IN :)
>>
>>
>> If its really that inportant to get hold of the original input we can
>> add a getter to it on the Exchange.
>> - getInputMessage()
>> - getOriginalMessage()
>> or what a good name would be.
>
> but this is exactly why we were talking about having getIn() and
> getOut(). Out is the output, the thing the processor/component
> mutates/generates. IN is just the immutable view of what was sent in.
>
>
>> And on top of this we do not lose headers as we do now when people use
>> a processor and do
>> exchange.getOut().setBody(myBody);
>>
>> And then all the headers from IN is lost.
>
> On this point aren't you confusing bugs in current implementation
> methods (like getOut()) with letting folks access the IN message while
> creating an OUT?
>
> As an experiment; could you mock up what the cxf and JBi components
> would look like having no getIn() and just having a getMessage() on
> Exchange? e.g. how would we map our code to CXF/JBI which require a
> separate IN and OUT?
>
> Or rather; if getOut() => getMessage() and we let folks see the
> immutable version of the original message - on a method called, say,
> getIn() - then isn't this what I was proposing last week; make getIn()
> immutable as the record of what was sent in then all
> mutation/modifications occur on getOut().
>
> Note I'm not 100% against any radical API change - I just want to make
> sure we're doing it for the right reasons.

Reading between the lines; are you really just trying to make folks
use (what is currently called) "getOut()" and never try mutate what is
currently called getIn()?

i.e. so by default the "OUT" property is defaulted to a copy of IN
that folks can change/mutate.

(what we call these 2 methods is a separate discussion, whether its
"in,out" or "originalMessage,message" or whatever


-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Hadrian Zbarcea <hz...@gmail.com>]

Roman,

Is there any api change you are proposing?  If yes, could you please  
outline the code changes you are suggesting?

Thanks,
Hadrian


On Jul 13, 2009, at 3:01 PM, Roman Kalukiewicz wrote:

> 2009/7/13 Claus Ibsen <cl...@gmail.com>:
>> On Mon, Jul 13, 2009 at 5:50 PM, James Strachan<james.strachan@gmail.com 
>> > wrote:
>>> I'm confused. So rather than calling Exchange.getIn() to get the IN,
>>> you'd have to do Exchange.getUnitOfWork().somethingOrOther() to get
>>> the last IN? I don't see that as any simpler; making one of the 2
>>> messages harder to find, just moves the confusion somewhere else?
>>>
>> James you are not the only confused, many Camel new users are - IN vs
>> OUT vs FAULT and exchange patterns and why the hell they lose they
>> message or headers.
>> eg a simple route then need the assistance from the tracer to help  
>> pin
>> point what goes wrong and where/what causes it to loose their
>> message/headers.
>>
>> No I do not want that, you need to do:
>> Exchange.getUnitOfWork().somethingOrOther()  to get IN.
>> IN is simply = getMessage.
>>
>> IN + OUT is combined into a single Message. getMessage/setMessage.
>
> It looks like Claus would like to implement my original idea of
> removing IN/OUT/FAULT as separate messages, but put them into single
> message. Then there is always CURRENT message in the exchange) or
> however we call it then.
>
>> If you really need hold of the original input (as written before that
>> is hardly ever ever needed) its currently avail on the UoW.
>> But we can expose this on the Exchange as well in case its makes a  
>> point.
>
> What I would propose would be to have an original input message kept
> somewhere, but we shouldn't mutate it. I would even prefer to have it
> in its original form like JBI Exchange, ServletRequest or whatever...
> for reference purposes, or if you want to do something very
> technology-specific. On the other hand I would prefer to have a
> generic exchange - without cusom subclasses at all.
>
> Roman
>
>>> 2009/7/12 Claus Ibsen <cl...@gmail.com>:
>>>> Hi
>>>>
>>>> Starting the kettle to brew me a grok of hot coffee, as its sunday
>>>> morning and I guess I got the energy to toss in my 2$ in the  
>>>> bucket of
>>>> ideas
>>>> Now that we are into the open talks and that it seems we can open  
>>>> the
>>>> box with the IN, OUT, FAULT again.
>>>>
>>>> I advocate for only one message on the implementation side. On  
>>>> the API
>>>> side we could still have some notion of IN / OUT to keep the API
>>>> migration / impact simpler. But at the end having one message API
>>>> makes it simpler. There could be a boolean to test whether an  
>>>> output
>>>> has been set or not. Just as if we have a test for fault.
>>>>
>>>>
>>>> Proposed Exchange API
>>>> ==================
>>>>
>>>>  Message getMessage()
>>>>  void setMessage(Message msg);
>>>>
>>>>  boolean messageChanged();
>>>>   -- false = message have newer been mutated, eg its still the  
>>>> same input
>>>>   -- true = the message or message body have been mutated in some  
>>>> way
>>>> (setMessage or setBody called)
>>>>   -- need to find a good name for this method, eg testing whether  
>>>> the
>>>> message have been changed/updated or not
>>>>
>>>> That leaves us with a simple API.
>>>>
>>>> Then we do not need to worry about all the message management
>>>> internally in Camel, where we need to figure out whether to set  
>>>> data
>>>> on IN or OUT and what else we kinda do, and it gets a bit complex  
>>>> over
>>>> time how to do this correctly.
>>>>
>>>>
>>>> MEP: InOnly
>>>> =========
>>>> Now we can do as Hadrian want. Newer return something in OUT. And
>>>> leave the original input on the Exchange.
>>>> Here we need to use James CopyOnWrite technique so we can  
>>>> preserve the
>>>> original message in case its mutated during routing.
>>>>
>>>>
>>>> MEP: InOut
>>>> =========
>>>> If an OUT message have been set (eg there is a reply), can be
>>>> determined if the messageChanged() == true. (then its not the  
>>>> original
>>>> message anymore). And yes the current code does this. It will just
>>>> copy whatever there is in IN and use it as OUT if no OUT was set.
>>>> With this proposal this will be improved as its easier to  
>>>> determine if
>>>> there is an OUT message or not.
>>>>
>>>>
>>>> Original message
>>>> =============
>>>> Currently there is an API to get the original message from the
>>>> UnitOfWork as its needed when doing redeliveries and a message was
>>>> doomed
>>>> and had to be moved to the dead letter channel. Then it makes much
>>>> more sense to move the original message than the current mutated
>>>> message.
>>>> ( i wonder if it should be default behavior )
>>>>
>>>>
>>>> AggregationStrategy
>>>> ===============
>>>> Another benefit with a single getMessage() is that end users using
>>>> AggregationStrategy will not be confused how to use it.
>>>> Should I look in IN or OUT. And the current code can actually be a
>>>> "lucky draw" as whether the data is in IN or OUT depending on facts
>>>> such as how the route path is.
>>>> We can even change it to pass in Message object instead of bare  
>>>> bone
>>>> Exchange. You can always go from Message to exchange using
>>>> getExchange().
>>>>
>>>>
>>>> All the processors / components / data formats
>>>> ===================================
>>>> Logic will be easier as well as they do not need to cater for  
>>>> IN / OUT
>>>> and where and how to set a result. Just work on the Message.
>>>>
>>>>
>>>> Use @deprecated for steady migration
>>>> ============================
>>>> Hadrian suggested that for the API migration to a single getMessage
>>>> you could let the getIn/setIn getOut/setOut delegate to
>>>> getMessage/setMessage.
>>>> And then mark them as @deprecated and then gradually change the  
>>>> camel
>>>> code as we goes. So I do not think the hold up / change would  
>>>> take a
>>>> lot of time and energy to get done.
>>>>
>>>>
>>>> Final words
>>>> ========
>>>> So if it was possible to simplify the API and reduce the IN OUT  
>>>> FAULT
>>>> to a simpler API, even go as far as I would like with a single
>>>> Message.
>>>> Then that would be really great and worth a hold up for a  
>>>> imminent 2.0 release.
>>>>
>>>>
>>>> Other frameworks
>>>> =============
>>>> I guess it wasn't final words after all :) Just wanted to say that
>>>> Mule, Spring Integration also just have a single message for the
>>>> message.
>>>> And yes I think their projects are also successful so it is not a  
>>>> loss
>>>> that they do not have IN OUT. In fact I think their API is easier  
>>>> to
>>>> work with than Camel.
>>>>
>>>> We are fortunate that most people with Camel do not work directly  
>>>> with
>>>> Exchange but work more with Camel returning the message as an  
>>>> expected
>>>> body type using its type converters. That helps a lot.
>>>>
>>>> But we have stories form Camel 1.x where people get confused whey
>>>> Camel return their original input in some situations where it was  
>>>> not
>>>> expected in the OUT message. Or the fact producer template  
>>>> sendBody is
>>>> not void in 1.x. Then people think it can be used for InOut.
>>>>
>>>> Okay end of mail.
>>>>
>>>>
>>>> On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<james.strachan@gmail.com 
>>>> > wrote:
>>>>> 2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
>>>>>> Moved this slightly different topic to a separate thread.
>>>>>>
>>>>>> ++1 from me.
>>>>>>
>>>>>> Do we need a vote on this one or it's a consensus?  Claus, it  
>>>>>> looks like you
>>>>>> agree, but pointed out that there is some work involved, correct?
>>>>>>
>>>>>> Hadrian
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>>>>>
>>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>>
>>>>>>>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<james.strachan@gmail.com 
>>>>>>>> >
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>>>>
>>>>>>>>>> Hi
>>>>>>>>>>
>>>>>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>>>>>
>>>>>>>>>> The ProducerCache that is doing the actual sending when  
>>>>>>>>>> using template
>>>>>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>>>>>
>>>>>>>>>> Well I am playing with to let it adhere to the principle  
>>>>>>>>>> Hadrian
>>>>>>>>>> pointed out. He wanted the IN to be more static.
>>>>>>>>>> And we cannot get there yet when you do routing as all the  
>>>>>>>>>> processors
>>>>>>>>>> rely on IN being able to mutate during routing.
>>>>>>>>>>
>>>>>>>>>> Anyway my grief is that when you send in an Exchange the  
>>>>>>>>>> result would
>>>>>>>>>> sometimes be stored on IN and not OUT.
>>>>>>>>>> What I want it to do always is to store the result in OUT  
>>>>>>>>>> and keep IN
>>>>>>>>>> as the original input.
>>>>>>>>>>
>>>>>>>>>> The code to do this is now a bit more complex than just  
>>>>>>>>>> before
>>>>>>>>>>
>>>>>>>>>>               // copy the original input
>>>>>>>>>>               Message original = exchange.getIn().copy();
>>>>>>>>>>
>>>>>>>>>>               producer.process(exchange);
>>>>>>>>>>
>>>>>>>>>>               // if no OUT then set current IN as result  
>>>>>>>>>> (except for
>>>>>>>>>> optional out)
>>>>>>>>>>               if (!exchange.hasOut() &&  
>>>>>>>>>> exchange.getPattern() !=
>>>>>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>>>>>                   // but only if its not the same as  
>>>>>>>>>> original IN to
>>>>>>>>>> avoid duplicating it
>>>>>>>>>>                   // and to adhere to the fact that there  
>>>>>>>>>> was no OUT
>>>>>>>>>> result at all
>>>>>>>>>>                   if (original.getBody() != null &&
>>>>>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>>>>>                       exchange.setOut(exchange.getIn());
>>>>>>>>>>                   }
>>>>>>>>>>               }
>>>>>>>>>>               // and restore original in
>>>>>>>>>>               exchange.setIn(original);
>>>>>>>>>>
>>>>>>>>>>               return exchange;
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> What I need to do is to copy the original IN message as it  
>>>>>>>>>> can be
>>>>>>>>>> mutated during routing.
>>>>>>>>>
>>>>>>>>> How about we prevent mutation of the IN message? Create a  
>>>>>>>>> Message
>>>>>>>>> facade which throws UnsupportedOperationException if you try  
>>>>>>>>> to mutate
>>>>>>>>> it in any way. Then we can pass the same read-only Message  
>>>>>>>>> around as
>>>>>>>>> the IN within retry loops or from step to step if no new  
>>>>>>>>> output is
>>>>>>>>> created (e.g. in a content based router where you just move  
>>>>>>>>> a Message
>>>>>>>>> to the right endpoint without changing it)
>>>>>>>>>
>>>>>>>>
>>>>>>>> A good idea but will break a lot of logic in Camel.
>>>>>>>
>>>>>>> Agreed. But with the benefit that we'd be able to get rid of  
>>>>>>> all the
>>>>>>> defensive copies in our code; plus we'd be able to pass the same
>>>>>>> Message from step to step. The API would be a bit more clean; to
>>>>>>> change the output, you create an OUT message (maybe by copying  
>>>>>>> the
>>>>>>> IN).
>>>>>>>
>>>>>>>
>>>>>>>> Most of the Camel
>>>>>>>> processors work on the IN message and set the result on  
>>>>>>>> either IN or
>>>>>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>>>>>> original input? Or am I mistaking?
>>>>>>>
>>>>>>> Yeah, we'd have to patch code to no longer mutate IN
>>>>>>>
>>>>>>>
>>>>>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>>>>>> immutable and always the original input?
>>>>>>>
>>>>>>> If no OUT, then no output was created, so pass the IN along...
>>>>>>>
>>>>>>>
>>>>>>> OK how about this; a CopyOnWriteMessageFacade which does not  
>>>>>>> mutate
>>>>>>> the original message at all ever; if a message is used in a  
>>>>>>> purely
>>>>>>> read only way, it does nothing but delegate to the original  
>>>>>>> message -
>>>>>>> but then as soon as someone mutates it, it creates a copy and  
>>>>>>> uses
>>>>>>> that from that point on?
>>>>>>>
>>>>>>> i.e. make the copy of the message lazy - and only make a copy  
>>>>>>> when a
>>>>>>> Processor really does try to mutate the Message?
>>>>>>>
>>>>>>> Then we'd get the best of both worlds; avoid breaking old code  
>>>>>>> but
>>>>>>> avoid tons of unnecessary copies?
>>>>>
>>>>> BTW then using the current API you could have a Message and then  
>>>>> call
>>>>>
>>>>> Message origin = ...;
>>>>> Message newMsg = origin.copy().copy().copy().copy();
>>>>>
>>>>> and the message would not actually be copied at all; new would  
>>>>> just be
>>>>> a CopyOnWriteMessageFacade which would hold a reference to  
>>>>> 'origin' as
>>>>> the readOnlyMessage (which it never mutates).
>>>>>
>>>>> The copy would only take place if you did
>>>>>
>>>>> newMsg.setBody("foo")
>>>>>
>>>>> --
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>>
>>>>> Open Source Integration
>>>>> http://fusesource.com/
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>
>>>
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://fusesource.com/
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Roman Kalukiewicz <ro...@gmail.com>]

2009/7/13 Claus Ibsen <cl...@gmail.com>:
> On Mon, Jul 13, 2009 at 5:50 PM, James Strachan<ja...@gmail.com> wrote:
>> I'm confused. So rather than calling Exchange.getIn() to get the IN,
>> you'd have to do Exchange.getUnitOfWork().somethingOrOther() to get
>> the last IN? I don't see that as any simpler; making one of the 2
>> messages harder to find, just moves the confusion somewhere else?
>>
> James you are not the only confused, many Camel new users are - IN vs
> OUT vs FAULT and exchange patterns and why the hell they lose they
> message or headers.
> eg a simple route then need the assistance from the tracer to help pin
> point what goes wrong and where/what causes it to loose their
> message/headers.
>
> No I do not want that, you need to do:
> Exchange.getUnitOfWork().somethingOrOther()  to get IN.
> IN is simply = getMessage.
>
> IN + OUT is combined into a single Message. getMessage/setMessage.

It looks like Claus would like to implement my original idea of
removing IN/OUT/FAULT as separate messages, but put them into single
message. Then there is always CURRENT message in the exchange) or
however we call it then.

> If you really need hold of the original input (as written before that
> is hardly ever ever needed) its currently avail on the UoW.
> But we can expose this on the Exchange as well in case its makes a point.

What I would propose would be to have an original input message kept
somewhere, but we shouldn't mutate it. I would even prefer to have it
in its original form like JBI Exchange, ServletRequest or whatever...
for reference purposes, or if you want to do something very
technology-specific. On the other hand I would prefer to have a
generic exchange - without cusom subclasses at all.

Roman

>> 2009/7/12 Claus Ibsen <cl...@gmail.com>:
>>> Hi
>>>
>>> Starting the kettle to brew me a grok of hot coffee, as its sunday
>>> morning and I guess I got the energy to toss in my 2$ in the bucket of
>>> ideas
>>> Now that we are into the open talks and that it seems we can open the
>>> box with the IN, OUT, FAULT again.
>>>
>>> I advocate for only one message on the implementation side. On the API
>>> side we could still have some notion of IN / OUT to keep the API
>>> migration / impact simpler. But at the end having one message API
>>> makes it simpler. There could be a boolean to test whether an output
>>> has been set or not. Just as if we have a test for fault.
>>>
>>>
>>> Proposed Exchange API
>>> ==================
>>>
>>>  Message getMessage()
>>>  void setMessage(Message msg);
>>>
>>>  boolean messageChanged();
>>>   -- false = message have newer been mutated, eg its still the same input
>>>   -- true = the message or message body have been mutated in some way
>>> (setMessage or setBody called)
>>>   -- need to find a good name for this method, eg testing whether the
>>> message have been changed/updated or not
>>>
>>> That leaves us with a simple API.
>>>
>>> Then we do not need to worry about all the message management
>>> internally in Camel, where we need to figure out whether to set data
>>> on IN or OUT and what else we kinda do, and it gets a bit complex over
>>> time how to do this correctly.
>>>
>>>
>>> MEP: InOnly
>>> =========
>>> Now we can do as Hadrian want. Newer return something in OUT. And
>>> leave the original input on the Exchange.
>>> Here we need to use James CopyOnWrite technique so we can preserve the
>>> original message in case its mutated during routing.
>>>
>>>
>>> MEP: InOut
>>> =========
>>> If an OUT message have been set (eg there is a reply), can be
>>> determined if the messageChanged() == true. (then its not the original
>>> message anymore). And yes the current code does this. It will just
>>> copy whatever there is in IN and use it as OUT if no OUT was set.
>>> With this proposal this will be improved as its easier to determine if
>>> there is an OUT message or not.
>>>
>>>
>>> Original message
>>> =============
>>> Currently there is an API to get the original message from the
>>> UnitOfWork as its needed when doing redeliveries and a message was
>>> doomed
>>> and had to be moved to the dead letter channel. Then it makes much
>>> more sense to move the original message than the current mutated
>>> message.
>>> ( i wonder if it should be default behavior )
>>>
>>>
>>> AggregationStrategy
>>> ===============
>>> Another benefit with a single getMessage() is that end users using
>>> AggregationStrategy will not be confused how to use it.
>>> Should I look in IN or OUT. And the current code can actually be a
>>> "lucky draw" as whether the data is in IN or OUT depending on facts
>>> such as how the route path is.
>>> We can even change it to pass in Message object instead of bare bone
>>> Exchange. You can always go from Message to exchange using
>>> getExchange().
>>>
>>>
>>> All the processors / components / data formats
>>> ===================================
>>> Logic will be easier as well as they do not need to cater for IN / OUT
>>> and where and how to set a result. Just work on the Message.
>>>
>>>
>>> Use @deprecated for steady migration
>>> ============================
>>> Hadrian suggested that for the API migration to a single getMessage
>>> you could let the getIn/setIn getOut/setOut delegate to
>>> getMessage/setMessage.
>>> And then mark them as @deprecated and then gradually change the camel
>>> code as we goes. So I do not think the hold up / change would take a
>>> lot of time and energy to get done.
>>>
>>>
>>> Final words
>>> ========
>>> So if it was possible to simplify the API and reduce the IN OUT FAULT
>>> to a simpler API, even go as far as I would like with a single
>>> Message.
>>> Then that would be really great and worth a hold up for a imminent 2.0 release.
>>>
>>>
>>> Other frameworks
>>> =============
>>> I guess it wasn't final words after all :) Just wanted to say that
>>> Mule, Spring Integration also just have a single message for the
>>> message.
>>> And yes I think their projects are also successful so it is not a loss
>>> that they do not have IN OUT. In fact I think their API is easier to
>>> work with than Camel.
>>>
>>> We are fortunate that most people with Camel do not work directly with
>>> Exchange but work more with Camel returning the message as an expected
>>> body type using its type converters. That helps a lot.
>>>
>>> But we have stories form Camel 1.x where people get confused whey
>>> Camel return their original input in some situations where it was not
>>> expected in the OUT message. Or the fact producer template sendBody is
>>> not void in 1.x. Then people think it can be used for InOut.
>>>
>>> Okay end of mail.
>>>
>>>
>>> On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<ja...@gmail.com> wrote:
>>>> 2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
>>>>> Moved this slightly different topic to a separate thread.
>>>>>
>>>>> ++1 from me.
>>>>>
>>>>> Do we need a vote on this one or it's a consensus?  Claus, it looks like you
>>>>> agree, but pointed out that there is some work involved, correct?
>>>>>
>>>>> Hadrian
>>>>>
>>>>>
>>>>>
>>>>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>>>>
>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>
>>>>>>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<ja...@gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>>>>
>>>>>>>>> The ProducerCache that is doing the actual sending when using template
>>>>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>>>>
>>>>>>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>>>>>>> pointed out. He wanted the IN to be more static.
>>>>>>>>> And we cannot get there yet when you do routing as all the processors
>>>>>>>>> rely on IN being able to mutate during routing.
>>>>>>>>>
>>>>>>>>> Anyway my grief is that when you send in an Exchange the result would
>>>>>>>>> sometimes be stored on IN and not OUT.
>>>>>>>>> What I want it to do always is to store the result in OUT and keep IN
>>>>>>>>> as the original input.
>>>>>>>>>
>>>>>>>>> The code to do this is now a bit more complex than just before
>>>>>>>>>
>>>>>>>>>               // copy the original input
>>>>>>>>>               Message original = exchange.getIn().copy();
>>>>>>>>>
>>>>>>>>>               producer.process(exchange);
>>>>>>>>>
>>>>>>>>>               // if no OUT then set current IN as result (except for
>>>>>>>>> optional out)
>>>>>>>>>               if (!exchange.hasOut() && exchange.getPattern() !=
>>>>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>>>>                   // but only if its not the same as original IN to
>>>>>>>>> avoid duplicating it
>>>>>>>>>                   // and to adhere to the fact that there was no OUT
>>>>>>>>> result at all
>>>>>>>>>                   if (original.getBody() != null &&
>>>>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>>>>                       exchange.setOut(exchange.getIn());
>>>>>>>>>                   }
>>>>>>>>>               }
>>>>>>>>>               // and restore original in
>>>>>>>>>               exchange.setIn(original);
>>>>>>>>>
>>>>>>>>>               return exchange;
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> What I need to do is to copy the original IN message as it can be
>>>>>>>>> mutated during routing.
>>>>>>>>
>>>>>>>> How about we prevent mutation of the IN message? Create a Message
>>>>>>>> facade which throws UnsupportedOperationException if you try to mutate
>>>>>>>> it in any way. Then we can pass the same read-only Message around as
>>>>>>>> the IN within retry loops or from step to step if no new output is
>>>>>>>> created (e.g. in a content based router where you just move a Message
>>>>>>>> to the right endpoint without changing it)
>>>>>>>>
>>>>>>>
>>>>>>> A good idea but will break a lot of logic in Camel.
>>>>>>
>>>>>> Agreed. But with the benefit that we'd be able to get rid of all the
>>>>>> defensive copies in our code; plus we'd be able to pass the same
>>>>>> Message from step to step. The API would be a bit more clean; to
>>>>>> change the output, you create an OUT message (maybe by copying the
>>>>>> IN).
>>>>>>
>>>>>>
>>>>>>> Most of the Camel
>>>>>>> processors work on the IN message and set the result on either IN or
>>>>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>>>>> original input? Or am I mistaking?
>>>>>>
>>>>>> Yeah, we'd have to patch code to no longer mutate IN
>>>>>>
>>>>>>
>>>>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>>>>> immutable and always the original input?
>>>>>>
>>>>>> If no OUT, then no output was created, so pass the IN along...
>>>>>>
>>>>>>
>>>>>> OK how about this; a CopyOnWriteMessageFacade which does not mutate
>>>>>> the original message at all ever; if a message is used in a purely
>>>>>> read only way, it does nothing but delegate to the original message -
>>>>>> but then as soon as someone mutates it, it creates a copy and uses
>>>>>> that from that point on?
>>>>>>
>>>>>> i.e. make the copy of the message lazy - and only make a copy when a
>>>>>> Processor really does try to mutate the Message?
>>>>>>
>>>>>> Then we'd get the best of both worlds; avoid breaking old code but
>>>>>> avoid tons of unnecessary copies?
>>>>
>>>> BTW then using the current API you could have a Message and then call
>>>>
>>>> Message origin = ...;
>>>> Message newMsg = origin.copy().copy().copy().copy();
>>>>
>>>> and the message would not actually be copied at all; new would just be
>>>> a CopyOnWriteMessageFacade which would hold a reference to 'origin' as
>>>> the readOnlyMessage (which it never mutates).
>>>>
>>>> The copy would only take place if you did
>>>>
>>>> newMsg.setBody("foo")
>>>>
>>>> --
>>>> James
>>>> -------
>>>> http://macstrac.blogspot.com/
>>>>
>>>> Open Source Integration
>>>> http://fusesource.com/
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>
>>
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://fusesource.com/
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Claus Ibsen <cl...@gmail.com>]

On Mon, Jul 13, 2009 at 5:50 PM, James Strachan<ja...@gmail.com> wrote:
> I'm confused. So rather than calling Exchange.getIn() to get the IN,
> you'd have to do Exchange.getUnitOfWork().somethingOrOther() to get
> the last IN? I don't see that as any simpler; making one of the 2
> messages harder to find, just moves the confusion somewhere else?
>
James you are not the only confused, many Camel new users are - IN vs
OUT vs FAULT and exchange patterns and why the hell they lose they
message or headers.
eg a simple route then need the assistance from the tracer to help pin
point what goes wrong and where/what causes it to loose their
message/headers.

No I do not want that, you need to do:
Exchange.getUnitOfWork().somethingOrOther()  to get IN.
IN is simply = getMessage.

IN + OUT is combined into a single Message. getMessage/setMessage.

If you really need hold of the original input (as written before that
is hardly ever ever needed) its currently avail on the UoW.
But we can expose this on the Exchange as well in case its makes a point.



> 2009/7/12 Claus Ibsen <cl...@gmail.com>:
>> Hi
>>
>> Starting the kettle to brew me a grok of hot coffee, as its sunday
>> morning and I guess I got the energy to toss in my 2$ in the bucket of
>> ideas
>> Now that we are into the open talks and that it seems we can open the
>> box with the IN, OUT, FAULT again.
>>
>> I advocate for only one message on the implementation side. On the API
>> side we could still have some notion of IN / OUT to keep the API
>> migration / impact simpler. But at the end having one message API
>> makes it simpler. There could be a boolean to test whether an output
>> has been set or not. Just as if we have a test for fault.
>>
>>
>> Proposed Exchange API
>> ==================
>>
>>  Message getMessage()
>>  void setMessage(Message msg);
>>
>>  boolean messageChanged();
>>   -- false = message have newer been mutated, eg its still the same input
>>   -- true = the message or message body have been mutated in some way
>> (setMessage or setBody called)
>>   -- need to find a good name for this method, eg testing whether the
>> message have been changed/updated or not
>>
>> That leaves us with a simple API.
>>
>> Then we do not need to worry about all the message management
>> internally in Camel, where we need to figure out whether to set data
>> on IN or OUT and what else we kinda do, and it gets a bit complex over
>> time how to do this correctly.
>>
>>
>> MEP: InOnly
>> =========
>> Now we can do as Hadrian want. Newer return something in OUT. And
>> leave the original input on the Exchange.
>> Here we need to use James CopyOnWrite technique so we can preserve the
>> original message in case its mutated during routing.
>>
>>
>> MEP: InOut
>> =========
>> If an OUT message have been set (eg there is a reply), can be
>> determined if the messageChanged() == true. (then its not the original
>> message anymore). And yes the current code does this. It will just
>> copy whatever there is in IN and use it as OUT if no OUT was set.
>> With this proposal this will be improved as its easier to determine if
>> there is an OUT message or not.
>>
>>
>> Original message
>> =============
>> Currently there is an API to get the original message from the
>> UnitOfWork as its needed when doing redeliveries and a message was
>> doomed
>> and had to be moved to the dead letter channel. Then it makes much
>> more sense to move the original message than the current mutated
>> message.
>> ( i wonder if it should be default behavior )
>>
>>
>> AggregationStrategy
>> ===============
>> Another benefit with a single getMessage() is that end users using
>> AggregationStrategy will not be confused how to use it.
>> Should I look in IN or OUT. And the current code can actually be a
>> "lucky draw" as whether the data is in IN or OUT depending on facts
>> such as how the route path is.
>> We can even change it to pass in Message object instead of bare bone
>> Exchange. You can always go from Message to exchange using
>> getExchange().
>>
>>
>> All the processors / components / data formats
>> ===================================
>> Logic will be easier as well as they do not need to cater for IN / OUT
>> and where and how to set a result. Just work on the Message.
>>
>>
>> Use @deprecated for steady migration
>> ============================
>> Hadrian suggested that for the API migration to a single getMessage
>> you could let the getIn/setIn getOut/setOut delegate to
>> getMessage/setMessage.
>> And then mark them as @deprecated and then gradually change the camel
>> code as we goes. So I do not think the hold up / change would take a
>> lot of time and energy to get done.
>>
>>
>> Final words
>> ========
>> So if it was possible to simplify the API and reduce the IN OUT FAULT
>> to a simpler API, even go as far as I would like with a single
>> Message.
>> Then that would be really great and worth a hold up for a imminent 2.0 release.
>>
>>
>> Other frameworks
>> =============
>> I guess it wasn't final words after all :) Just wanted to say that
>> Mule, Spring Integration also just have a single message for the
>> message.
>> And yes I think their projects are also successful so it is not a loss
>> that they do not have IN OUT. In fact I think their API is easier to
>> work with than Camel.
>>
>> We are fortunate that most people with Camel do not work directly with
>> Exchange but work more with Camel returning the message as an expected
>> body type using its type converters. That helps a lot.
>>
>> But we have stories form Camel 1.x where people get confused whey
>> Camel return their original input in some situations where it was not
>> expected in the OUT message. Or the fact producer template sendBody is
>> not void in 1.x. Then people think it can be used for InOut.
>>
>> Okay end of mail.
>>
>>
>> On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<ja...@gmail.com> wrote:
>>> 2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
>>>> Moved this slightly different topic to a separate thread.
>>>>
>>>> ++1 from me.
>>>>
>>>> Do we need a vote on this one or it's a consensus?  Claus, it looks like you
>>>> agree, but pointed out that there is some work involved, correct?
>>>>
>>>> Hadrian
>>>>
>>>>
>>>>
>>>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>>>
>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>
>>>>>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<ja...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>>>
>>>>>>>> The ProducerCache that is doing the actual sending when using template
>>>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>>>
>>>>>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>>>>>> pointed out. He wanted the IN to be more static.
>>>>>>>> And we cannot get there yet when you do routing as all the processors
>>>>>>>> rely on IN being able to mutate during routing.
>>>>>>>>
>>>>>>>> Anyway my grief is that when you send in an Exchange the result would
>>>>>>>> sometimes be stored on IN and not OUT.
>>>>>>>> What I want it to do always is to store the result in OUT and keep IN
>>>>>>>> as the original input.
>>>>>>>>
>>>>>>>> The code to do this is now a bit more complex than just before
>>>>>>>>
>>>>>>>>               // copy the original input
>>>>>>>>               Message original = exchange.getIn().copy();
>>>>>>>>
>>>>>>>>               producer.process(exchange);
>>>>>>>>
>>>>>>>>               // if no OUT then set current IN as result (except for
>>>>>>>> optional out)
>>>>>>>>               if (!exchange.hasOut() && exchange.getPattern() !=
>>>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>>>                   // but only if its not the same as original IN to
>>>>>>>> avoid duplicating it
>>>>>>>>                   // and to adhere to the fact that there was no OUT
>>>>>>>> result at all
>>>>>>>>                   if (original.getBody() != null &&
>>>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>>>                       exchange.setOut(exchange.getIn());
>>>>>>>>                   }
>>>>>>>>               }
>>>>>>>>               // and restore original in
>>>>>>>>               exchange.setIn(original);
>>>>>>>>
>>>>>>>>               return exchange;
>>>>>>>>
>>>>>>>>
>>>>>>>> What I need to do is to copy the original IN message as it can be
>>>>>>>> mutated during routing.
>>>>>>>
>>>>>>> How about we prevent mutation of the IN message? Create a Message
>>>>>>> facade which throws UnsupportedOperationException if you try to mutate
>>>>>>> it in any way. Then we can pass the same read-only Message around as
>>>>>>> the IN within retry loops or from step to step if no new output is
>>>>>>> created (e.g. in a content based router where you just move a Message
>>>>>>> to the right endpoint without changing it)
>>>>>>>
>>>>>>
>>>>>> A good idea but will break a lot of logic in Camel.
>>>>>
>>>>> Agreed. But with the benefit that we'd be able to get rid of all the
>>>>> defensive copies in our code; plus we'd be able to pass the same
>>>>> Message from step to step. The API would be a bit more clean; to
>>>>> change the output, you create an OUT message (maybe by copying the
>>>>> IN).
>>>>>
>>>>>
>>>>>> Most of the Camel
>>>>>> processors work on the IN message and set the result on either IN or
>>>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>>>> original input? Or am I mistaking?
>>>>>
>>>>> Yeah, we'd have to patch code to no longer mutate IN
>>>>>
>>>>>
>>>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>>>> immutable and always the original input?
>>>>>
>>>>> If no OUT, then no output was created, so pass the IN along...
>>>>>
>>>>>
>>>>> OK how about this; a CopyOnWriteMessageFacade which does not mutate
>>>>> the original message at all ever; if a message is used in a purely
>>>>> read only way, it does nothing but delegate to the original message -
>>>>> but then as soon as someone mutates it, it creates a copy and uses
>>>>> that from that point on?
>>>>>
>>>>> i.e. make the copy of the message lazy - and only make a copy when a
>>>>> Processor really does try to mutate the Message?
>>>>>
>>>>> Then we'd get the best of both worlds; avoid breaking old code but
>>>>> avoid tons of unnecessary copies?
>>>
>>> BTW then using the current API you could have a Message and then call
>>>
>>> Message origin = ...;
>>> Message newMsg = origin.copy().copy().copy().copy();
>>>
>>> and the message would not actually be copied at all; new would just be
>>> a CopyOnWriteMessageFacade which would hold a reference to 'origin' as
>>> the readOnlyMessage (which it never mutates).
>>>
>>> The copy would only take place if you did
>>>
>>> newMsg.setBody("foo")
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://fusesource.com/
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Claus Ibsen <cl...@gmail.com>]

On Mon, Jul 13, 2009 at 6:33 PM, James Strachan<ja...@gmail.com> wrote:
> 2009/7/13 Claus Ibsen <cl...@gmail.com>:
>> On Mon, Jul 13, 2009 at 5:50 PM, James Strachan<ja...@gmail.com> wrote:
>>> I'm confused. So rather than calling Exchange.getIn() to get the IN,
>>> you'd have to do Exchange.getUnitOfWork().somethingOrOther() to get
>>> the last IN? I don't see that as any simpler; making one of the 2
>>> messages harder to find, just moves the confusion somewhere else?
>>
>> Having access to the original input in a route path is very rarely
>> requested by end users.
>> In fact I think its even more confused if getIN always returned the
>> original input. (what should that be if a message is routed over (to
>> seda/from seda) a seda endpoint?)
>>
>> So what I am saying is that we should have a model similar to Mule and
>> Spring Integration = simpler and easier to understand and use.
>> And also simpler and easier to use within the Camel framework itself -
>> eg all the code that copies from IN to OUT and OUT to IN and whatnot.
>>
>> The last IN = the message itself.
>>
>> Given this route:
>> from("direct:start").to("bean:foo").processRef("bar").transform(constant("Bye
>> World")).processRef("me");
>>
>> And we send a "Hello World" message to direct:start. The message would be
>>
>> bean:foo
>> getMessage = "Hello World
>>
>> bar
>> getMessage = "Hello World
>>
>> me
>> getMessage = "Bye World
>>
>>
>> Now imagine we mutate the messages in bean foo
>> public String reverse(String s) {
>>   // return the reversed string
>> }
>
> am confused; a bean never even sees an Exchange/Message typically?
>
>> bean:foo
>> getMessage = "Hello World"
>>
>> bar
>> getMessage = "dlroW olleH"
>>
>> me
>> getMessage = "Bye World"
>>
>>
>> No more IN vs OUT and other confusing bits. getMessage gets you the
>> payload as it is.
>
> I dunnot what 'it' is? You mean we always copy the IN to the OUT
> regardless of what the code does just in case; so you mutate the
> current message - then if you zap headers/body you have to keep a
> track of them yourself in case you want to look at them again?
>

The current code in Camel makes it very easy for Camel to zap your
headers unintended.

Lets look at how it is today.

1:
IN = At the very first consumer its the original input (= good)

IN Body = Hello World
OUT = null

2:
Then the pipeline kicks in and invokes the next processor with the Exchange

Lets say this processor does some work on the exchange, it want to set
a new payload
Today you can set this payload in either IN or OUT, eg your choice.

Lets start with IN (see 3a)
exchange.getIn().setBody("I did this")

And try OUT instead (see 3b)
exchange.getOut().setBody("I also did this")


3a:
The pipeline still routes it and lets send it to a final destination,
a file producer that writes the content as a file
IN Body = I did this
OUT = null

This is lucky as the file producer always uses IN body to as source
for file content so the file is: I did this


3b
The pipeline still routes it and lets send it to a final destination,
a file producer that writes the content as a file
IN Body = I also did this
OUT = I also did this

We set an OUT body but the pipeline copied it to IN so the message is
what we expect
and the file producer can use IN to write the expected file content.

But now the Exchange is a bit confusing as we got both IN and OUT = the same.
And the original IN is lost.





>
>> If you have looked as much Camel code as I have you would also flag
>> that the IN OUT is leading to confusing and broken code.
>
> Note you were the one that pushed back when I said lets make getIn()
> immutable which would break some code :). Now you're saying lets just
> break *all* camel code.
>
>
>> And trust me its broken, when you use the Exchange, but the BODY
>> extractor of ProducerTemplate is fairly good at guessing and returning
>> the right one from OUT or IN :)
>>
>>
>> If its really that inportant to get hold of the original input we can
>> add a getter to it on the Exchange.
>> - getInputMessage()
>> - getOriginalMessage()
>> or what a good name would be.
>
> but this is exactly why we were talking about having getIn() and
> getOut(). Out is the output, the thing the processor/component
> mutates/generates. IN is just the immutable view of what was sent in.
>

The current model in Camel is IN not a immutable view. It can be
mutated as well, and this is actually essential that its done
as all the Camel processors rely on this that IN contains = the
current message. (and not what the original input is when the message
was consumed)

If we go by your suggestions.
1) All Camel code must be adapter to use OUT instead of IN as = the
current message. And eg fallback to IN if no OUT have been set
2) End users from Camel 1.x that are familiar with the (odd) IN and
OUT would have their code breaked and cannot understand why

3) So to make all aware that IN and OUT have changed we simply do not
have them anymore, or have them @deprecated.


>
>> And on top of this we do not lose headers as we do now when people use
>> a processor and do
>> exchange.getOut().setBody(myBody);
>>
>> And then all the headers from IN is lost.
>
> On this point aren't you confusing bugs in current implementation
> methods (like getOut()) with letting folks access the IN message while
> creating an OUT?
>
> As an experiment; could you mock up what the cxf and JBi components
> would look like having no getIn() and just having a getMessage() on
> Exchange? e.g. how would we map our code to CXF/JBI which require a
> separate IN and OUT?
>

Yeah these frameworks could require a bit more work. However we got a
camel-cxf and ca camel-jbi component where such logic can be
implemented.
If CXF can work with Mule or Spring Integration (they do not have
IN/OUT) why can it not work with Camel also?



> Or rather; if getOut() => getMessage() and we let folks see the
> immutable version of the original message - on a method called, say,
> getIn() - then isn't this what I was proposing last week; make getIn()
> immutable as the record of what was sent in then all
> mutation/modifications occur on getOut().
>
> Note I'm not 100% against any radical API change - I just want to make
> sure we're doing it for the right reasons.
>

It was in fact Hadrian that got the ball rolling. We where close to
close the lid and get 2.0 out.
However his proposal with FAULT -> OUT is good, but FAULT is not often
used anyway so the impact is not big.



> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[James Strachan <ja...@gmail.com>]

2009/7/13 Claus Ibsen <cl...@gmail.com>:
> On Mon, Jul 13, 2009 at 5:50 PM, James Strachan<ja...@gmail.com> wrote:
>> I'm confused. So rather than calling Exchange.getIn() to get the IN,
>> you'd have to do Exchange.getUnitOfWork().somethingOrOther() to get
>> the last IN? I don't see that as any simpler; making one of the 2
>> messages harder to find, just moves the confusion somewhere else?
>
> Having access to the original input in a route path is very rarely
> requested by end users.
> In fact I think its even more confused if getIN always returned the
> original input. (what should that be if a message is routed over (to
> seda/from seda) a seda endpoint?)
>
> So what I am saying is that we should have a model similar to Mule and
> Spring Integration = simpler and easier to understand and use.
> And also simpler and easier to use within the Camel framework itself -
> eg all the code that copies from IN to OUT and OUT to IN and whatnot.
>
> The last IN = the message itself.
>
> Given this route:
> from("direct:start").to("bean:foo").processRef("bar").transform(constant("Bye
> World")).processRef("me");
>
> And we send a "Hello World" message to direct:start. The message would be
>
> bean:foo
> getMessage = "Hello World
>
> bar
> getMessage = "Hello World
>
> me
> getMessage = "Bye World
>
>
> Now imagine we mutate the messages in bean foo
> public String reverse(String s) {
>   // return the reversed string
> }

am confused; a bean never even sees an Exchange/Message typically?

> bean:foo
> getMessage = "Hello World"
>
> bar
> getMessage = "dlroW olleH"
>
> me
> getMessage = "Bye World"
>
>
> No more IN vs OUT and other confusing bits. getMessage gets you the
> payload as it is.

I dunnot what 'it' is? You mean we always copy the IN to the OUT
regardless of what the code does just in case; so you mutate the
current message - then if you zap headers/body you have to keep a
track of them yourself in case you want to look at them again?


> If you have looked as much Camel code as I have you would also flag
> that the IN OUT is leading to confusing and broken code.

Note you were the one that pushed back when I said lets make getIn()
immutable which would break some code :). Now you're saying lets just
break *all* camel code.


> And trust me its broken, when you use the Exchange, but the BODY
> extractor of ProducerTemplate is fairly good at guessing and returning
> the right one from OUT or IN :)
>
>
> If its really that inportant to get hold of the original input we can
> add a getter to it on the Exchange.
> - getInputMessage()
> - getOriginalMessage()
> or what a good name would be.

but this is exactly why we were talking about having getIn() and
getOut(). Out is the output, the thing the processor/component
mutates/generates. IN is just the immutable view of what was sent in.


> And on top of this we do not lose headers as we do now when people use
> a processor and do
> exchange.getOut().setBody(myBody);
>
> And then all the headers from IN is lost.

On this point aren't you confusing bugs in current implementation
methods (like getOut()) with letting folks access the IN message while
creating an OUT?

As an experiment; could you mock up what the cxf and JBi components
would look like having no getIn() and just having a getMessage() on
Exchange? e.g. how would we map our code to CXF/JBI which require a
separate IN and OUT?

Or rather; if getOut() => getMessage() and we let folks see the
immutable version of the original message - on a method called, say,
getIn() - then isn't this what I was proposing last week; make getIn()
immutable as the record of what was sent in then all
mutation/modifications occur on getOut().

Note I'm not 100% against any radical API change - I just want to make
sure we're doing it for the right reasons.

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Claus Ibsen <cl...@gmail.com>]

On Mon, Jul 13, 2009 at 5:50 PM, James Strachan<ja...@gmail.com> wrote:
> I'm confused. So rather than calling Exchange.getIn() to get the IN,
> you'd have to do Exchange.getUnitOfWork().somethingOrOther() to get
> the last IN? I don't see that as any simpler; making one of the 2
> messages harder to find, just moves the confusion somewhere else?

Having access to the original input in a route path is very rarely
requested by end users.
In fact I think its even more confused if getIN always returned the
original input. (what should that be if a message is routed over (to
seda/from seda) a seda endpoint?)

So what I am saying is that we should have a model similar to Mule and
Spring Integration = simpler and easier to understand and use.
And also simpler and easier to use within the Camel framework itself -
eg all the code that copies from IN to OUT and OUT to IN and whatnot.

The last IN = the message itself.

Given this route:
from("direct:start").to("bean:foo").processRef("bar").transform(constant("Bye
World")).processRef("me");

And we send a "Hello World" message to direct:start. The message would be

bean:foo
getMessage = "Hello World

bar
getMessage = "Hello World

me
getMessage = "Bye World


Now imagine we mutate the messages in bean foo
public String reverse(String s) {
   // return the reversed string
}


bean:foo
getMessage = "Hello World"

bar
getMessage = "dlroW olleH"

me
getMessage = "Bye World"


No more IN vs OUT and other confusing bits. getMessage gets you the
payload as it is.

If you have looked as much Camel code as I have you would also flag
that the IN OUT is leading to confusing and broken code.
And trust me its broken, when you use the Exchange, but the BODY
extractor of ProducerTemplate is fairly good at guessing and returning
the right one from OUT or IN :)


If its really that inportant to get hold of the original input we can
add a getter to it on the Exchange.
- getInputMessage()
- getOriginalMessage()
or what a good name would be.


And on top of this we do not lose headers as we do now when people use
a processor and do
exchange.getOut().setBody(myBody);

And then all the headers from IN is lost.



>
> 2009/7/12 Claus Ibsen <cl...@gmail.com>:
>> Hi
>>
>> Starting the kettle to brew me a grok of hot coffee, as its sunday
>> morning and I guess I got the energy to toss in my 2$ in the bucket of
>> ideas
>> Now that we are into the open talks and that it seems we can open the
>> box with the IN, OUT, FAULT again.
>>
>> I advocate for only one message on the implementation side. On the API
>> side we could still have some notion of IN / OUT to keep the API
>> migration / impact simpler. But at the end having one message API
>> makes it simpler. There could be a boolean to test whether an output
>> has been set or not. Just as if we have a test for fault.
>>
>>
>> Proposed Exchange API
>> ==================
>>
>>  Message getMessage()
>>  void setMessage(Message msg);
>>
>>  boolean messageChanged();
>>   -- false = message have newer been mutated, eg its still the same input
>>   -- true = the message or message body have been mutated in some way
>> (setMessage or setBody called)
>>   -- need to find a good name for this method, eg testing whether the
>> message have been changed/updated or not
>>
>> That leaves us with a simple API.
>>
>> Then we do not need to worry about all the message management
>> internally in Camel, where we need to figure out whether to set data
>> on IN or OUT and what else we kinda do, and it gets a bit complex over
>> time how to do this correctly.
>>
>>
>> MEP: InOnly
>> =========
>> Now we can do as Hadrian want. Newer return something in OUT. And
>> leave the original input on the Exchange.
>> Here we need to use James CopyOnWrite technique so we can preserve the
>> original message in case its mutated during routing.
>>
>>
>> MEP: InOut
>> =========
>> If an OUT message have been set (eg there is a reply), can be
>> determined if the messageChanged() == true. (then its not the original
>> message anymore). And yes the current code does this. It will just
>> copy whatever there is in IN and use it as OUT if no OUT was set.
>> With this proposal this will be improved as its easier to determine if
>> there is an OUT message or not.
>>
>>
>> Original message
>> =============
>> Currently there is an API to get the original message from the
>> UnitOfWork as its needed when doing redeliveries and a message was
>> doomed
>> and had to be moved to the dead letter channel. Then it makes much
>> more sense to move the original message than the current mutated
>> message.
>> ( i wonder if it should be default behavior )
>>
>>
>> AggregationStrategy
>> ===============
>> Another benefit with a single getMessage() is that end users using
>> AggregationStrategy will not be confused how to use it.
>> Should I look in IN or OUT. And the current code can actually be a
>> "lucky draw" as whether the data is in IN or OUT depending on facts
>> such as how the route path is.
>> We can even change it to pass in Message object instead of bare bone
>> Exchange. You can always go from Message to exchange using
>> getExchange().
>>
>>
>> All the processors / components / data formats
>> ===================================
>> Logic will be easier as well as they do not need to cater for IN / OUT
>> and where and how to set a result. Just work on the Message.
>>
>>
>> Use @deprecated for steady migration
>> ============================
>> Hadrian suggested that for the API migration to a single getMessage
>> you could let the getIn/setIn getOut/setOut delegate to
>> getMessage/setMessage.
>> And then mark them as @deprecated and then gradually change the camel
>> code as we goes. So I do not think the hold up / change would take a
>> lot of time and energy to get done.
>>
>>
>> Final words
>> ========
>> So if it was possible to simplify the API and reduce the IN OUT FAULT
>> to a simpler API, even go as far as I would like with a single
>> Message.
>> Then that would be really great and worth a hold up for a imminent 2.0 release.
>>
>>
>> Other frameworks
>> =============
>> I guess it wasn't final words after all :) Just wanted to say that
>> Mule, Spring Integration also just have a single message for the
>> message.
>> And yes I think their projects are also successful so it is not a loss
>> that they do not have IN OUT. In fact I think their API is easier to
>> work with than Camel.
>>
>> We are fortunate that most people with Camel do not work directly with
>> Exchange but work more with Camel returning the message as an expected
>> body type using its type converters. That helps a lot.
>>
>> But we have stories form Camel 1.x where people get confused whey
>> Camel return their original input in some situations where it was not
>> expected in the OUT message. Or the fact producer template sendBody is
>> not void in 1.x. Then people think it can be used for InOut.
>>
>> Okay end of mail.
>>
>>
>> On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<ja...@gmail.com> wrote:
>>> 2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
>>>> Moved this slightly different topic to a separate thread.
>>>>
>>>> ++1 from me.
>>>>
>>>> Do we need a vote on this one or it's a consensus?  Claus, it looks like you
>>>> agree, but pointed out that there is some work involved, correct?
>>>>
>>>> Hadrian
>>>>
>>>>
>>>>
>>>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>>>
>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>
>>>>>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<ja...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>>>
>>>>>>>> The ProducerCache that is doing the actual sending when using template
>>>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>>>
>>>>>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>>>>>> pointed out. He wanted the IN to be more static.
>>>>>>>> And we cannot get there yet when you do routing as all the processors
>>>>>>>> rely on IN being able to mutate during routing.
>>>>>>>>
>>>>>>>> Anyway my grief is that when you send in an Exchange the result would
>>>>>>>> sometimes be stored on IN and not OUT.
>>>>>>>> What I want it to do always is to store the result in OUT and keep IN
>>>>>>>> as the original input.
>>>>>>>>
>>>>>>>> The code to do this is now a bit more complex than just before
>>>>>>>>
>>>>>>>>               // copy the original input
>>>>>>>>               Message original = exchange.getIn().copy();
>>>>>>>>
>>>>>>>>               producer.process(exchange);
>>>>>>>>
>>>>>>>>               // if no OUT then set current IN as result (except for
>>>>>>>> optional out)
>>>>>>>>               if (!exchange.hasOut() && exchange.getPattern() !=
>>>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>>>                   // but only if its not the same as original IN to
>>>>>>>> avoid duplicating it
>>>>>>>>                   // and to adhere to the fact that there was no OUT
>>>>>>>> result at all
>>>>>>>>                   if (original.getBody() != null &&
>>>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>>>                       exchange.setOut(exchange.getIn());
>>>>>>>>                   }
>>>>>>>>               }
>>>>>>>>               // and restore original in
>>>>>>>>               exchange.setIn(original);
>>>>>>>>
>>>>>>>>               return exchange;
>>>>>>>>
>>>>>>>>
>>>>>>>> What I need to do is to copy the original IN message as it can be
>>>>>>>> mutated during routing.
>>>>>>>
>>>>>>> How about we prevent mutation of the IN message? Create a Message
>>>>>>> facade which throws UnsupportedOperationException if you try to mutate
>>>>>>> it in any way. Then we can pass the same read-only Message around as
>>>>>>> the IN within retry loops or from step to step if no new output is
>>>>>>> created (e.g. in a content based router where you just move a Message
>>>>>>> to the right endpoint without changing it)
>>>>>>>
>>>>>>
>>>>>> A good idea but will break a lot of logic in Camel.
>>>>>
>>>>> Agreed. But with the benefit that we'd be able to get rid of all the
>>>>> defensive copies in our code; plus we'd be able to pass the same
>>>>> Message from step to step. The API would be a bit more clean; to
>>>>> change the output, you create an OUT message (maybe by copying the
>>>>> IN).
>>>>>
>>>>>
>>>>>> Most of the Camel
>>>>>> processors work on the IN message and set the result on either IN or
>>>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>>>> original input? Or am I mistaking?
>>>>>
>>>>> Yeah, we'd have to patch code to no longer mutate IN
>>>>>
>>>>>
>>>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>>>> immutable and always the original input?
>>>>>
>>>>> If no OUT, then no output was created, so pass the IN along...
>>>>>
>>>>>
>>>>> OK how about this; a CopyOnWriteMessageFacade which does not mutate
>>>>> the original message at all ever; if a message is used in a purely
>>>>> read only way, it does nothing but delegate to the original message -
>>>>> but then as soon as someone mutates it, it creates a copy and uses
>>>>> that from that point on?
>>>>>
>>>>> i.e. make the copy of the message lazy - and only make a copy when a
>>>>> Processor really does try to mutate the Message?
>>>>>
>>>>> Then we'd get the best of both worlds; avoid breaking old code but
>>>>> avoid tons of unnecessary copies?
>>>
>>> BTW then using the current API you could have a Message and then call
>>>
>>> Message origin = ...;
>>> Message newMsg = origin.copy().copy().copy().copy();
>>>
>>> and the message would not actually be copied at all; new would just be
>>> a CopyOnWriteMessageFacade which would hold a reference to 'origin' as
>>> the readOnlyMessage (which it never mutates).
>>>
>>> The copy would only take place if you did
>>>
>>> newMsg.setBody("foo")
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://fusesource.com/
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[James Strachan <ja...@gmail.com>]

I'm confused. So rather than calling Exchange.getIn() to get the IN,
you'd have to do Exchange.getUnitOfWork().somethingOrOther() to get
the last IN? I don't see that as any simpler; making one of the 2
messages harder to find, just moves the confusion somewhere else?

2009/7/12 Claus Ibsen <cl...@gmail.com>:
> Hi
>
> Starting the kettle to brew me a grok of hot coffee, as its sunday
> morning and I guess I got the energy to toss in my 2$ in the bucket of
> ideas
> Now that we are into the open talks and that it seems we can open the
> box with the IN, OUT, FAULT again.
>
> I advocate for only one message on the implementation side. On the API
> side we could still have some notion of IN / OUT to keep the API
> migration / impact simpler. But at the end having one message API
> makes it simpler. There could be a boolean to test whether an output
> has been set or not. Just as if we have a test for fault.
>
>
> Proposed Exchange API
> ==================
>
>  Message getMessage()
>  void setMessage(Message msg);
>
>  boolean messageChanged();
>   -- false = message have newer been mutated, eg its still the same input
>   -- true = the message or message body have been mutated in some way
> (setMessage or setBody called)
>   -- need to find a good name for this method, eg testing whether the
> message have been changed/updated or not
>
> That leaves us with a simple API.
>
> Then we do not need to worry about all the message management
> internally in Camel, where we need to figure out whether to set data
> on IN or OUT and what else we kinda do, and it gets a bit complex over
> time how to do this correctly.
>
>
> MEP: InOnly
> =========
> Now we can do as Hadrian want. Newer return something in OUT. And
> leave the original input on the Exchange.
> Here we need to use James CopyOnWrite technique so we can preserve the
> original message in case its mutated during routing.
>
>
> MEP: InOut
> =========
> If an OUT message have been set (eg there is a reply), can be
> determined if the messageChanged() == true. (then its not the original
> message anymore). And yes the current code does this. It will just
> copy whatever there is in IN and use it as OUT if no OUT was set.
> With this proposal this will be improved as its easier to determine if
> there is an OUT message or not.
>
>
> Original message
> =============
> Currently there is an API to get the original message from the
> UnitOfWork as its needed when doing redeliveries and a message was
> doomed
> and had to be moved to the dead letter channel. Then it makes much
> more sense to move the original message than the current mutated
> message.
> ( i wonder if it should be default behavior )
>
>
> AggregationStrategy
> ===============
> Another benefit with a single getMessage() is that end users using
> AggregationStrategy will not be confused how to use it.
> Should I look in IN or OUT. And the current code can actually be a
> "lucky draw" as whether the data is in IN or OUT depending on facts
> such as how the route path is.
> We can even change it to pass in Message object instead of bare bone
> Exchange. You can always go from Message to exchange using
> getExchange().
>
>
> All the processors / components / data formats
> ===================================
> Logic will be easier as well as they do not need to cater for IN / OUT
> and where and how to set a result. Just work on the Message.
>
>
> Use @deprecated for steady migration
> ============================
> Hadrian suggested that for the API migration to a single getMessage
> you could let the getIn/setIn getOut/setOut delegate to
> getMessage/setMessage.
> And then mark them as @deprecated and then gradually change the camel
> code as we goes. So I do not think the hold up / change would take a
> lot of time and energy to get done.
>
>
> Final words
> ========
> So if it was possible to simplify the API and reduce the IN OUT FAULT
> to a simpler API, even go as far as I would like with a single
> Message.
> Then that would be really great and worth a hold up for a imminent 2.0 release.
>
>
> Other frameworks
> =============
> I guess it wasn't final words after all :) Just wanted to say that
> Mule, Spring Integration also just have a single message for the
> message.
> And yes I think their projects are also successful so it is not a loss
> that they do not have IN OUT. In fact I think their API is easier to
> work with than Camel.
>
> We are fortunate that most people with Camel do not work directly with
> Exchange but work more with Camel returning the message as an expected
> body type using its type converters. That helps a lot.
>
> But we have stories form Camel 1.x where people get confused whey
> Camel return their original input in some situations where it was not
> expected in the OUT message. Or the fact producer template sendBody is
> not void in 1.x. Then people think it can be used for InOut.
>
> Okay end of mail.
>
>
> On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<ja...@gmail.com> wrote:
>> 2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
>>> Moved this slightly different topic to a separate thread.
>>>
>>> ++1 from me.
>>>
>>> Do we need a vote on this one or it's a consensus?  Claus, it looks like you
>>> agree, but pointed out that there is some work involved, correct?
>>>
>>> Hadrian
>>>
>>>
>>>
>>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>>
>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>
>>>>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<ja...@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>>
>>>>>>> The ProducerCache that is doing the actual sending when using template
>>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>>
>>>>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>>>>> pointed out. He wanted the IN to be more static.
>>>>>>> And we cannot get there yet when you do routing as all the processors
>>>>>>> rely on IN being able to mutate during routing.
>>>>>>>
>>>>>>> Anyway my grief is that when you send in an Exchange the result would
>>>>>>> sometimes be stored on IN and not OUT.
>>>>>>> What I want it to do always is to store the result in OUT and keep IN
>>>>>>> as the original input.
>>>>>>>
>>>>>>> The code to do this is now a bit more complex than just before
>>>>>>>
>>>>>>>               // copy the original input
>>>>>>>               Message original = exchange.getIn().copy();
>>>>>>>
>>>>>>>               producer.process(exchange);
>>>>>>>
>>>>>>>               // if no OUT then set current IN as result (except for
>>>>>>> optional out)
>>>>>>>               if (!exchange.hasOut() && exchange.getPattern() !=
>>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>>                   // but only if its not the same as original IN to
>>>>>>> avoid duplicating it
>>>>>>>                   // and to adhere to the fact that there was no OUT
>>>>>>> result at all
>>>>>>>                   if (original.getBody() != null &&
>>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>>                       exchange.setOut(exchange.getIn());
>>>>>>>                   }
>>>>>>>               }
>>>>>>>               // and restore original in
>>>>>>>               exchange.setIn(original);
>>>>>>>
>>>>>>>               return exchange;
>>>>>>>
>>>>>>>
>>>>>>> What I need to do is to copy the original IN message as it can be
>>>>>>> mutated during routing.
>>>>>>
>>>>>> How about we prevent mutation of the IN message? Create a Message
>>>>>> facade which throws UnsupportedOperationException if you try to mutate
>>>>>> it in any way. Then we can pass the same read-only Message around as
>>>>>> the IN within retry loops or from step to step if no new output is
>>>>>> created (e.g. in a content based router where you just move a Message
>>>>>> to the right endpoint without changing it)
>>>>>>
>>>>>
>>>>> A good idea but will break a lot of logic in Camel.
>>>>
>>>> Agreed. But with the benefit that we'd be able to get rid of all the
>>>> defensive copies in our code; plus we'd be able to pass the same
>>>> Message from step to step. The API would be a bit more clean; to
>>>> change the output, you create an OUT message (maybe by copying the
>>>> IN).
>>>>
>>>>
>>>>> Most of the Camel
>>>>> processors work on the IN message and set the result on either IN or
>>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>>> original input? Or am I mistaking?
>>>>
>>>> Yeah, we'd have to patch code to no longer mutate IN
>>>>
>>>>
>>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>>> immutable and always the original input?
>>>>
>>>> If no OUT, then no output was created, so pass the IN along...
>>>>
>>>>
>>>> OK how about this; a CopyOnWriteMessageFacade which does not mutate
>>>> the original message at all ever; if a message is used in a purely
>>>> read only way, it does nothing but delegate to the original message -
>>>> but then as soon as someone mutates it, it creates a copy and uses
>>>> that from that point on?
>>>>
>>>> i.e. make the copy of the message lazy - and only make a copy when a
>>>> Processor really does try to mutate the Message?
>>>>
>>>> Then we'd get the best of both worlds; avoid breaking old code but
>>>> avoid tons of unnecessary copies?
>>
>> BTW then using the current API you could have a Message and then call
>>
>> Message origin = ...;
>> Message newMsg = origin.copy().copy().copy().copy();
>>
>> and the message would not actually be copied at all; new would just be
>> a CopyOnWriteMessageFacade which would hold a reference to 'origin' as
>> the readOnlyMessage (which it never mutates).
>>
>> The copy would only take place if you did
>>
>> newMsg.setBody("foo")
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://fusesource.com/
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>



-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS][IMPORTANT] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Claus Ibsen <cl...@gmail.com>]

On Mon, Jul 13, 2009 at 5:44 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
> Hi,
>
> Are you in fact proposing to rename get/setOut() to get/setMessage()?  And
> hasOut() to be renamed as messageChanged(), kinda?  This is pretty much how
> it sounds to me.
>

Yes,

However I would rename getIn/setIn instead as a lot of Camel
processors etc rely on getIn() instead of getOut when they get access
to the payload/message.



> I like the semantic cleanup.  Not sure about passing the message instead of
> the exchange.  That would have a significant impact on projects using camel
> and I don't think the benefit would justify it.
>
> I would love for others to pitch in and express their thoughts.
> Hadrian
>
>
>
> On Jul 12, 2009, at 5:23 AM, Claus Ibsen wrote:
>
>> Hi
>>
>> Starting the kettle to brew me a grok of hot coffee, as its sunday
>> morning and I guess I got the energy to toss in my 2$ in the bucket of
>> ideas
>> Now that we are into the open talks and that it seems we can open the
>> box with the IN, OUT, FAULT again.
>>
>> I advocate for only one message on the implementation side. On the API
>> side we could still have some notion of IN / OUT to keep the API
>> migration / impact simpler. But at the end having one message API
>> makes it simpler. There could be a boolean to test whether an output
>> has been set or not. Just as if we have a test for fault.
>>
>>
>> Proposed Exchange API
>> ==================
>>
>>  Message getMessage()
>>  void setMessage(Message msg);
>>
>>  boolean messageChanged();
>>  -- false = message have newer been mutated, eg its still the same input
>>  -- true = the message or message body have been mutated in some way
>> (setMessage or setBody called)
>>  -- need to find a good name for this method, eg testing whether the
>> message have been changed/updated or not
>>
>> That leaves us with a simple API.
>>
>> Then we do not need to worry about all the message management
>> internally in Camel, where we need to figure out whether to set data
>> on IN or OUT and what else we kinda do, and it gets a bit complex over
>> time how to do this correctly.
>>
>>
>> MEP: InOnly
>> =========
>> Now we can do as Hadrian want. Newer return something in OUT. And
>> leave the original input on the Exchange.
>> Here we need to use James CopyOnWrite technique so we can preserve the
>> original message in case its mutated during routing.
>>
>>
>> MEP: InOut
>> =========
>> If an OUT message have been set (eg there is a reply), can be
>> determined if the messageChanged() == true. (then its not the original
>> message anymore). And yes the current code does this. It will just
>> copy whatever there is in IN and use it as OUT if no OUT was set.
>> With this proposal this will be improved as its easier to determine if
>> there is an OUT message or not.
>>
>>
>> Original message
>> =============
>> Currently there is an API to get the original message from the
>> UnitOfWork as its needed when doing redeliveries and a message was
>> doomed
>> and had to be moved to the dead letter channel. Then it makes much
>> more sense to move the original message than the current mutated
>> message.
>> ( i wonder if it should be default behavior )
>>
>>
>> AggregationStrategy
>> ===============
>> Another benefit with a single getMessage() is that end users using
>> AggregationStrategy will not be confused how to use it.
>> Should I look in IN or OUT. And the current code can actually be a
>> "lucky draw" as whether the data is in IN or OUT depending on facts
>> such as how the route path is.
>> We can even change it to pass in Message object instead of bare bone
>> Exchange. You can always go from Message to exchange using
>> getExchange().
>>
>>
>> All the processors / components / data formats
>> ===================================
>> Logic will be easier as well as they do not need to cater for IN / OUT
>> and where and how to set a result. Just work on the Message.
>>
>>
>> Use @deprecated for steady migration
>> ============================
>> Hadrian suggested that for the API migration to a single getMessage
>> you could let the getIn/setIn getOut/setOut delegate to
>> getMessage/setMessage.
>> And then mark them as @deprecated and then gradually change the camel
>> code as we goes. So I do not think the hold up / change would take a
>> lot of time and energy to get done.
>>
>>
>> Final words
>> ========
>> So if it was possible to simplify the API and reduce the IN OUT FAULT
>> to a simpler API, even go as far as I would like with a single
>> Message.
>> Then that would be really great and worth a hold up for a imminent 2.0
>> release.
>>
>>
>> Other frameworks
>> =============
>> I guess it wasn't final words after all :) Just wanted to say that
>> Mule, Spring Integration also just have a single message for the
>> message.
>> And yes I think their projects are also successful so it is not a loss
>> that they do not have IN OUT. In fact I think their API is easier to
>> work with than Camel.
>>
>> We are fortunate that most people with Camel do not work directly with
>> Exchange but work more with Camel returning the message as an expected
>> body type using its type converters. That helps a lot.
>>
>> But we have stories form Camel 1.x where people get confused whey
>> Camel return their original input in some situations where it was not
>> expected in the OUT message. Or the fact producer template sendBody is
>> not void in 1.x. Then people think it can be used for InOut.
>>
>> Okay end of mail.
>>
>>
>> On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<ja...@gmail.com>
>> wrote:
>>>
>>> 2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
>>>>
>>>> Moved this slightly different topic to a separate thread.
>>>>
>>>> ++1 from me.
>>>>
>>>> Do we need a vote on this one or it's a consensus?  Claus, it looks like
>>>> you
>>>> agree, but pointed out that there is some work involved, correct?
>>>>
>>>> Hadrian
>>>>
>>>>
>>>>
>>>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>>>
>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>
>>>>>> On Fri, Jul 10, 2009 at 1:20 PM, James
>>>>>> Strachan<ja...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>>>
>>>>>>>> The ProducerCache that is doing the actual sending when using
>>>>>>>> template
>>>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>>>
>>>>>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>>>>>> pointed out. He wanted the IN to be more static.
>>>>>>>> And we cannot get there yet when you do routing as all the
>>>>>>>> processors
>>>>>>>> rely on IN being able to mutate during routing.
>>>>>>>>
>>>>>>>> Anyway my grief is that when you send in an Exchange the result
>>>>>>>> would
>>>>>>>> sometimes be stored on IN and not OUT.
>>>>>>>> What I want it to do always is to store the result in OUT and keep
>>>>>>>> IN
>>>>>>>> as the original input.
>>>>>>>>
>>>>>>>> The code to do this is now a bit more complex than just before
>>>>>>>>
>>>>>>>>              // copy the original input
>>>>>>>>              Message original = exchange.getIn().copy();
>>>>>>>>
>>>>>>>>              producer.process(exchange);
>>>>>>>>
>>>>>>>>              // if no OUT then set current IN as result (except for
>>>>>>>> optional out)
>>>>>>>>              if (!exchange.hasOut() && exchange.getPattern() !=
>>>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>>>                  // but only if its not the same as original IN to
>>>>>>>> avoid duplicating it
>>>>>>>>                  // and to adhere to the fact that there was no OUT
>>>>>>>> result at all
>>>>>>>>                  if (original.getBody() != null &&
>>>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>>>                      exchange.setOut(exchange.getIn());
>>>>>>>>                  }
>>>>>>>>              }
>>>>>>>>              // and restore original in
>>>>>>>>              exchange.setIn(original);
>>>>>>>>
>>>>>>>>              return exchange;
>>>>>>>>
>>>>>>>>
>>>>>>>> What I need to do is to copy the original IN message as it can be
>>>>>>>> mutated during routing.
>>>>>>>
>>>>>>> How about we prevent mutation of the IN message? Create a Message
>>>>>>> facade which throws UnsupportedOperationException if you try to
>>>>>>> mutate
>>>>>>> it in any way. Then we can pass the same read-only Message around as
>>>>>>> the IN within retry loops or from step to step if no new output is
>>>>>>> created (e.g. in a content based router where you just move a Message
>>>>>>> to the right endpoint without changing it)
>>>>>>>
>>>>>>
>>>>>> A good idea but will break a lot of logic in Camel.
>>>>>
>>>>> Agreed. But with the benefit that we'd be able to get rid of all the
>>>>> defensive copies in our code; plus we'd be able to pass the same
>>>>> Message from step to step. The API would be a bit more clean; to
>>>>> change the output, you create an OUT message (maybe by copying the
>>>>> IN).
>>>>>
>>>>>
>>>>>> Most of the Camel
>>>>>> processors work on the IN message and set the result on either IN or
>>>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>>>> original input? Or am I mistaking?
>>>>>
>>>>> Yeah, we'd have to patch code to no longer mutate IN
>>>>>
>>>>>
>>>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>>>> immutable and always the original input?
>>>>>
>>>>> If no OUT, then no output was created, so pass the IN along...
>>>>>
>>>>>
>>>>> OK how about this; a CopyOnWriteMessageFacade which does not mutate
>>>>> the original message at all ever; if a message is used in a purely
>>>>> read only way, it does nothing but delegate to the original message -
>>>>> but then as soon as someone mutates it, it creates a copy and uses
>>>>> that from that point on?
>>>>>
>>>>> i.e. make the copy of the message lazy - and only make a copy when a
>>>>> Processor really does try to mutate the Message?
>>>>>
>>>>> Then we'd get the best of both worlds; avoid breaking old code but
>>>>> avoid tons of unnecessary copies?
>>>
>>> BTW then using the current API you could have a Message and then call
>>>
>>> Message origin = ...;
>>> Message newMsg = origin.copy().copy().copy().copy();
>>>
>>> and the message would not actually be copied at all; new would just be
>>> a CopyOnWriteMessageFacade which would hold a reference to 'origin' as
>>> the readOnlyMessage (which it never mutates).
>>>
>>> The copy would only take place if you did
>>>
>>> newMsg.setBody("foo")
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://fusesource.com/
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS][IMPORTANT] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Claus Ibsen <cl...@gmail.com>]

On Mon, Jul 13, 2009 at 5:44 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
> Hi,
>
> Are you in fact proposing to rename get/setOut() to get/setMessage()?  And
> hasOut() to be renamed as messageChanged(), kinda?  This is pretty much how
> it sounds to me.
>
> I like the semantic cleanup.  Not sure about passing the message instead of
> the exchange.  That would have a significant impact on projects using camel
> and I don't think the benefit would justify it.
Its hardly a significant impact. Removing the generics is a bigger
impact that changing a single interface

And what they used to do in the AggregationStrategy (with much
confusion, may I stress out - see the user mailinglist)

public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
  // hmmm is my payload in IN or OUT today?
 String oldBody = oldExchange.getIn().getBody(String.class);
String newBody = newExchange.getIn().getBody(String.class);
 ... do more stuff
 ... jesus should I store the aggregated body on IN or OUT
 .. if I store it in the wrong one I lose all my headers ... . help
Camel riders!!!
... for instance as:
   oldExchange.getOut().setBody(aggregatedBody);
   return oldExchange;
}

And this simpler API

public Message aggregate(Message oldMessage, Exchange newMessage) {
  // yeah I have the message directly no more IN or OUT confusion
 String oldBody = oldMessage.getBody(String.class);
String newBody = newMessage.getBody(String.class);
 ... do more stuff
... great no more lost headers
}


The API migration is super simple. Before they had to go from exchange
-> IN or OUT (???) -> Message
Now they have it directly.


>
> I would love for others to pitch in and express their thoughts.
> Hadrian
>
>
>
> On Jul 12, 2009, at 5:23 AM, Claus Ibsen wrote:
>
>> Hi
>>
>> Starting the kettle to brew me a grok of hot coffee, as its sunday
>> morning and I guess I got the energy to toss in my 2$ in the bucket of
>> ideas
>> Now that we are into the open talks and that it seems we can open the
>> box with the IN, OUT, FAULT again.
>>
>> I advocate for only one message on the implementation side. On the API
>> side we could still have some notion of IN / OUT to keep the API
>> migration / impact simpler. But at the end having one message API
>> makes it simpler. There could be a boolean to test whether an output
>> has been set or not. Just as if we have a test for fault.
>>
>>
>> Proposed Exchange API
>> ==================
>>
>>  Message getMessage()
>>  void setMessage(Message msg);
>>
>>  boolean messageChanged();
>>  -- false = message have newer been mutated, eg its still the same input
>>  -- true = the message or message body have been mutated in some way
>> (setMessage or setBody called)
>>  -- need to find a good name for this method, eg testing whether the
>> message have been changed/updated or not
>>
>> That leaves us with a simple API.
>>
>> Then we do not need to worry about all the message management
>> internally in Camel, where we need to figure out whether to set data
>> on IN or OUT and what else we kinda do, and it gets a bit complex over
>> time how to do this correctly.
>>
>>
>> MEP: InOnly
>> =========
>> Now we can do as Hadrian want. Newer return something in OUT. And
>> leave the original input on the Exchange.
>> Here we need to use James CopyOnWrite technique so we can preserve the
>> original message in case its mutated during routing.
>>
>>
>> MEP: InOut
>> =========
>> If an OUT message have been set (eg there is a reply), can be
>> determined if the messageChanged() == true. (then its not the original
>> message anymore). And yes the current code does this. It will just
>> copy whatever there is in IN and use it as OUT if no OUT was set.
>> With this proposal this will be improved as its easier to determine if
>> there is an OUT message or not.
>>
>>
>> Original message
>> =============
>> Currently there is an API to get the original message from the
>> UnitOfWork as its needed when doing redeliveries and a message was
>> doomed
>> and had to be moved to the dead letter channel. Then it makes much
>> more sense to move the original message than the current mutated
>> message.
>> ( i wonder if it should be default behavior )
>>
>>
>> AggregationStrategy
>> ===============
>> Another benefit with a single getMessage() is that end users using
>> AggregationStrategy will not be confused how to use it.
>> Should I look in IN or OUT. And the current code can actually be a
>> "lucky draw" as whether the data is in IN or OUT depending on facts
>> such as how the route path is.
>> We can even change it to pass in Message object instead of bare bone
>> Exchange. You can always go from Message to exchange using
>> getExchange().
>>
>>
>> All the processors / components / data formats
>> ===================================
>> Logic will be easier as well as they do not need to cater for IN / OUT
>> and where and how to set a result. Just work on the Message.
>>
>>
>> Use @deprecated for steady migration
>> ============================
>> Hadrian suggested that for the API migration to a single getMessage
>> you could let the getIn/setIn getOut/setOut delegate to
>> getMessage/setMessage.
>> And then mark them as @deprecated and then gradually change the camel
>> code as we goes. So I do not think the hold up / change would take a
>> lot of time and energy to get done.
>>
>>
>> Final words
>> ========
>> So if it was possible to simplify the API and reduce the IN OUT FAULT
>> to a simpler API, even go as far as I would like with a single
>> Message.
>> Then that would be really great and worth a hold up for a imminent 2.0
>> release.
>>
>>
>> Other frameworks
>> =============
>> I guess it wasn't final words after all :) Just wanted to say that
>> Mule, Spring Integration also just have a single message for the
>> message.
>> And yes I think their projects are also successful so it is not a loss
>> that they do not have IN OUT. In fact I think their API is easier to
>> work with than Camel.
>>
>> We are fortunate that most people with Camel do not work directly with
>> Exchange but work more with Camel returning the message as an expected
>> body type using its type converters. That helps a lot.
>>
>> But we have stories form Camel 1.x where people get confused whey
>> Camel return their original input in some situations where it was not
>> expected in the OUT message. Or the fact producer template sendBody is
>> not void in 1.x. Then people think it can be used for InOut.
>>
>> Okay end of mail.
>>
>>
>> On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<ja...@gmail.com>
>> wrote:
>>>
>>> 2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
>>>>
>>>> Moved this slightly different topic to a separate thread.
>>>>
>>>> ++1 from me.
>>>>
>>>> Do we need a vote on this one or it's a consensus?  Claus, it looks like
>>>> you
>>>> agree, but pointed out that there is some work involved, correct?
>>>>
>>>> Hadrian
>>>>
>>>>
>>>>
>>>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>>>
>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>
>>>>>> On Fri, Jul 10, 2009 at 1:20 PM, James
>>>>>> Strachan<ja...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>>>
>>>>>>>> The ProducerCache that is doing the actual sending when using
>>>>>>>> template
>>>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>>>
>>>>>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>>>>>> pointed out. He wanted the IN to be more static.
>>>>>>>> And we cannot get there yet when you do routing as all the
>>>>>>>> processors
>>>>>>>> rely on IN being able to mutate during routing.
>>>>>>>>
>>>>>>>> Anyway my grief is that when you send in an Exchange the result
>>>>>>>> would
>>>>>>>> sometimes be stored on IN and not OUT.
>>>>>>>> What I want it to do always is to store the result in OUT and keep
>>>>>>>> IN
>>>>>>>> as the original input.
>>>>>>>>
>>>>>>>> The code to do this is now a bit more complex than just before
>>>>>>>>
>>>>>>>>              // copy the original input
>>>>>>>>              Message original = exchange.getIn().copy();
>>>>>>>>
>>>>>>>>              producer.process(exchange);
>>>>>>>>
>>>>>>>>              // if no OUT then set current IN as result (except for
>>>>>>>> optional out)
>>>>>>>>              if (!exchange.hasOut() && exchange.getPattern() !=
>>>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>>>                  // but only if its not the same as original IN to
>>>>>>>> avoid duplicating it
>>>>>>>>                  // and to adhere to the fact that there was no OUT
>>>>>>>> result at all
>>>>>>>>                  if (original.getBody() != null &&
>>>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>>>                      exchange.setOut(exchange.getIn());
>>>>>>>>                  }
>>>>>>>>              }
>>>>>>>>              // and restore original in
>>>>>>>>              exchange.setIn(original);
>>>>>>>>
>>>>>>>>              return exchange;
>>>>>>>>
>>>>>>>>
>>>>>>>> What I need to do is to copy the original IN message as it can be
>>>>>>>> mutated during routing.
>>>>>>>
>>>>>>> How about we prevent mutation of the IN message? Create a Message
>>>>>>> facade which throws UnsupportedOperationException if you try to
>>>>>>> mutate
>>>>>>> it in any way. Then we can pass the same read-only Message around as
>>>>>>> the IN within retry loops or from step to step if no new output is
>>>>>>> created (e.g. in a content based router where you just move a Message
>>>>>>> to the right endpoint without changing it)
>>>>>>>
>>>>>>
>>>>>> A good idea but will break a lot of logic in Camel.
>>>>>
>>>>> Agreed. But with the benefit that we'd be able to get rid of all the
>>>>> defensive copies in our code; plus we'd be able to pass the same
>>>>> Message from step to step. The API would be a bit more clean; to
>>>>> change the output, you create an OUT message (maybe by copying the
>>>>> IN).
>>>>>
>>>>>
>>>>>> Most of the Camel
>>>>>> processors work on the IN message and set the result on either IN or
>>>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>>>> original input? Or am I mistaking?
>>>>>
>>>>> Yeah, we'd have to patch code to no longer mutate IN
>>>>>
>>>>>
>>>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>>>> immutable and always the original input?
>>>>>
>>>>> If no OUT, then no output was created, so pass the IN along...
>>>>>
>>>>>
>>>>> OK how about this; a CopyOnWriteMessageFacade which does not mutate
>>>>> the original message at all ever; if a message is used in a purely
>>>>> read only way, it does nothing but delegate to the original message -
>>>>> but then as soon as someone mutates it, it creates a copy and uses
>>>>> that from that point on?
>>>>>
>>>>> i.e. make the copy of the message lazy - and only make a copy when a
>>>>> Processor really does try to mutate the Message?
>>>>>
>>>>> Then we'd get the best of both worlds; avoid breaking old code but
>>>>> avoid tons of unnecessary copies?
>>>
>>> BTW then using the current API you could have a Message and then call
>>>
>>> Message origin = ...;
>>> Message newMsg = origin.copy().copy().copy().copy();
>>>
>>> and the message would not actually be copied at all; new would just be
>>> a CopyOnWriteMessageFacade which would hold a reference to 'origin' as
>>> the readOnlyMessage (which it never mutates).
>>>
>>> The copy would only take place if you did
>>>
>>> newMsg.setBody("foo")
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://fusesource.com/
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS][IMPORTANT] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Hadrian Zbarcea <hz...@gmail.com>]

Hi,

Are you in fact proposing to rename get/setOut() to get/setMessage()?   
And hasOut() to be renamed as messageChanged(), kinda?  This is pretty  
much how it sounds to me.

I like the semantic cleanup.  Not sure about passing the message  
instead of the exchange.  That would have a significant impact on  
projects using camel and I don't think the benefit would justify it.

I would love for others to pitch in and express their thoughts.
Hadrian



On Jul 12, 2009, at 5:23 AM, Claus Ibsen wrote:

> Hi
>
> Starting the kettle to brew me a grok of hot coffee, as its sunday
> morning and I guess I got the energy to toss in my 2$ in the bucket of
> ideas
> Now that we are into the open talks and that it seems we can open the
> box with the IN, OUT, FAULT again.
>
> I advocate for only one message on the implementation side. On the API
> side we could still have some notion of IN / OUT to keep the API
> migration / impact simpler. But at the end having one message API
> makes it simpler. There could be a boolean to test whether an output
> has been set or not. Just as if we have a test for fault.
>
>
> Proposed Exchange API
> ==================
>
>  Message getMessage()
>  void setMessage(Message msg);
>
>  boolean messageChanged();
>   -- false = message have newer been mutated, eg its still the same  
> input
>   -- true = the message or message body have been mutated in some way
> (setMessage or setBody called)
>   -- need to find a good name for this method, eg testing whether the
> message have been changed/updated or not
>
> That leaves us with a simple API.
>
> Then we do not need to worry about all the message management
> internally in Camel, where we need to figure out whether to set data
> on IN or OUT and what else we kinda do, and it gets a bit complex over
> time how to do this correctly.
>
>
> MEP: InOnly
> =========
> Now we can do as Hadrian want. Newer return something in OUT. And
> leave the original input on the Exchange.
> Here we need to use James CopyOnWrite technique so we can preserve the
> original message in case its mutated during routing.
>
>
> MEP: InOut
> =========
> If an OUT message have been set (eg there is a reply), can be
> determined if the messageChanged() == true. (then its not the original
> message anymore). And yes the current code does this. It will just
> copy whatever there is in IN and use it as OUT if no OUT was set.
> With this proposal this will be improved as its easier to determine if
> there is an OUT message or not.
>
>
> Original message
> =============
> Currently there is an API to get the original message from the
> UnitOfWork as its needed when doing redeliveries and a message was
> doomed
> and had to be moved to the dead letter channel. Then it makes much
> more sense to move the original message than the current mutated
> message.
> ( i wonder if it should be default behavior )
>
>
> AggregationStrategy
> ===============
> Another benefit with a single getMessage() is that end users using
> AggregationStrategy will not be confused how to use it.
> Should I look in IN or OUT. And the current code can actually be a
> "lucky draw" as whether the data is in IN or OUT depending on facts
> such as how the route path is.
> We can even change it to pass in Message object instead of bare bone
> Exchange. You can always go from Message to exchange using
> getExchange().
>
>
> All the processors / components / data formats
> ===================================
> Logic will be easier as well as they do not need to cater for IN / OUT
> and where and how to set a result. Just work on the Message.
>
>
> Use @deprecated for steady migration
> ============================
> Hadrian suggested that for the API migration to a single getMessage
> you could let the getIn/setIn getOut/setOut delegate to
> getMessage/setMessage.
> And then mark them as @deprecated and then gradually change the camel
> code as we goes. So I do not think the hold up / change would take a
> lot of time and energy to get done.
>
>
> Final words
> ========
> So if it was possible to simplify the API and reduce the IN OUT FAULT
> to a simpler API, even go as far as I would like with a single
> Message.
> Then that would be really great and worth a hold up for a imminent  
> 2.0 release.
>
>
> Other frameworks
> =============
> I guess it wasn't final words after all :) Just wanted to say that
> Mule, Spring Integration also just have a single message for the
> message.
> And yes I think their projects are also successful so it is not a loss
> that they do not have IN OUT. In fact I think their API is easier to
> work with than Camel.
>
> We are fortunate that most people with Camel do not work directly with
> Exchange but work more with Camel returning the message as an expected
> body type using its type converters. That helps a lot.
>
> But we have stories form Camel 1.x where people get confused whey
> Camel return their original input in some situations where it was not
> expected in the OUT message. Or the fact producer template sendBody is
> not void in 1.x. Then people think it can be used for InOut.
>
> Okay end of mail.
>
>
> On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<james.strachan@gmail.com 
> > wrote:
>> 2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
>>> Moved this slightly different topic to a separate thread.
>>>
>>> ++1 from me.
>>>
>>> Do we need a vote on this one or it's a consensus?  Claus, it  
>>> looks like you
>>> agree, but pointed out that there is some work involved, correct?
>>>
>>> Hadrian
>>>
>>>
>>>
>>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>>
>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>
>>>>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<james.strachan@gmail.com 
>>>>> >
>>>>> wrote:
>>>>>>
>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>>
>>>>>>> The ProducerCache that is doing the actual sending when using  
>>>>>>> template
>>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>>
>>>>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>>>>> pointed out. He wanted the IN to be more static.
>>>>>>> And we cannot get there yet when you do routing as all the  
>>>>>>> processors
>>>>>>> rely on IN being able to mutate during routing.
>>>>>>>
>>>>>>> Anyway my grief is that when you send in an Exchange the  
>>>>>>> result would
>>>>>>> sometimes be stored on IN and not OUT.
>>>>>>> What I want it to do always is to store the result in OUT and  
>>>>>>> keep IN
>>>>>>> as the original input.
>>>>>>>
>>>>>>> The code to do this is now a bit more complex than just before
>>>>>>>
>>>>>>>               // copy the original input
>>>>>>>               Message original = exchange.getIn().copy();
>>>>>>>
>>>>>>>               producer.process(exchange);
>>>>>>>
>>>>>>>               // if no OUT then set current IN as result  
>>>>>>> (except for
>>>>>>> optional out)
>>>>>>>               if (!exchange.hasOut() && exchange.getPattern() !=
>>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>>                   // but only if its not the same as original  
>>>>>>> IN to
>>>>>>> avoid duplicating it
>>>>>>>                   // and to adhere to the fact that there was  
>>>>>>> no OUT
>>>>>>> result at all
>>>>>>>                   if (original.getBody() != null &&
>>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>>                       exchange.setOut(exchange.getIn());
>>>>>>>                   }
>>>>>>>               }
>>>>>>>               // and restore original in
>>>>>>>               exchange.setIn(original);
>>>>>>>
>>>>>>>               return exchange;
>>>>>>>
>>>>>>>
>>>>>>> What I need to do is to copy the original IN message as it can  
>>>>>>> be
>>>>>>> mutated during routing.
>>>>>>
>>>>>> How about we prevent mutation of the IN message? Create a Message
>>>>>> facade which throws UnsupportedOperationException if you try to  
>>>>>> mutate
>>>>>> it in any way. Then we can pass the same read-only Message  
>>>>>> around as
>>>>>> the IN within retry loops or from step to step if no new output  
>>>>>> is
>>>>>> created (e.g. in a content based router where you just move a  
>>>>>> Message
>>>>>> to the right endpoint without changing it)
>>>>>>
>>>>>
>>>>> A good idea but will break a lot of logic in Camel.
>>>>
>>>> Agreed. But with the benefit that we'd be able to get rid of all  
>>>> the
>>>> defensive copies in our code; plus we'd be able to pass the same
>>>> Message from step to step. The API would be a bit more clean; to
>>>> change the output, you create an OUT message (maybe by copying the
>>>> IN).
>>>>
>>>>
>>>>> Most of the Camel
>>>>> processors work on the IN message and set the result on either  
>>>>> IN or
>>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>>> original input? Or am I mistaking?
>>>>
>>>> Yeah, we'd have to patch code to no longer mutate IN
>>>>
>>>>
>>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>>> immutable and always the original input?
>>>>
>>>> If no OUT, then no output was created, so pass the IN along...
>>>>
>>>>
>>>> OK how about this; a CopyOnWriteMessageFacade which does not mutate
>>>> the original message at all ever; if a message is used in a purely
>>>> read only way, it does nothing but delegate to the original  
>>>> message -
>>>> but then as soon as someone mutates it, it creates a copy and uses
>>>> that from that point on?
>>>>
>>>> i.e. make the copy of the message lazy - and only make a copy  
>>>> when a
>>>> Processor really does try to mutate the Message?
>>>>
>>>> Then we'd get the best of both worlds; avoid breaking old code but
>>>> avoid tons of unnecessary copies?
>>
>> BTW then using the current API you could have a Message and then call
>>
>> Message origin = ...;
>> Message newMsg = origin.copy().copy().copy().copy();
>>
>> and the message would not actually be copied at all; new would just  
>> be
>> a CopyOnWriteMessageFacade which would hold a reference to 'origin'  
>> as
>> the readOnlyMessage (which it never mutates).
>>
>> The copy would only take place if you did
>>
>> newMsg.setBody("foo")
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://fusesource.com/
>>
>
>
>
> -- 
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Claus Ibsen <cl...@gmail.com>]

Hi

Starting the kettle to brew me a grok of hot coffee, as its sunday
morning and I guess I got the energy to toss in my 2$ in the bucket of
ideas
Now that we are into the open talks and that it seems we can open the
box with the IN, OUT, FAULT again.

I advocate for only one message on the implementation side. On the API
side we could still have some notion of IN / OUT to keep the API
migration / impact simpler. But at the end having one message API
makes it simpler. There could be a boolean to test whether an output
has been set or not. Just as if we have a test for fault.


Proposed Exchange API
==================

  Message getMessage()
  void setMessage(Message msg);

  boolean messageChanged();
   -- false = message have newer been mutated, eg its still the same input
   -- true = the message or message body have been mutated in some way
(setMessage or setBody called)
   -- need to find a good name for this method, eg testing whether the
message have been changed/updated or not

That leaves us with a simple API.

Then we do not need to worry about all the message management
internally in Camel, where we need to figure out whether to set data
on IN or OUT and what else we kinda do, and it gets a bit complex over
time how to do this correctly.


MEP: InOnly
=========
Now we can do as Hadrian want. Newer return something in OUT. And
leave the original input on the Exchange.
Here we need to use James CopyOnWrite technique so we can preserve the
original message in case its mutated during routing.


MEP: InOut
=========
If an OUT message have been set (eg there is a reply), can be
determined if the messageChanged() == true. (then its not the original
message anymore). And yes the current code does this. It will just
copy whatever there is in IN and use it as OUT if no OUT was set.
With this proposal this will be improved as its easier to determine if
there is an OUT message or not.


Original message
=============
Currently there is an API to get the original message from the
UnitOfWork as its needed when doing redeliveries and a message was
doomed
and had to be moved to the dead letter channel. Then it makes much
more sense to move the original message than the current mutated
message.
( i wonder if it should be default behavior )


AggregationStrategy
===============
Another benefit with a single getMessage() is that end users using
AggregationStrategy will not be confused how to use it.
Should I look in IN or OUT. And the current code can actually be a
"lucky draw" as whether the data is in IN or OUT depending on facts
such as how the route path is.
We can even change it to pass in Message object instead of bare bone
Exchange. You can always go from Message to exchange using
getExchange().


All the processors / components / data formats
===================================
Logic will be easier as well as they do not need to cater for IN / OUT
and where and how to set a result. Just work on the Message.


Use @deprecated for steady migration
============================
Hadrian suggested that for the API migration to a single getMessage
you could let the getIn/setIn getOut/setOut delegate to
getMessage/setMessage.
And then mark them as @deprecated and then gradually change the camel
code as we goes. So I do not think the hold up / change would take a
lot of time and energy to get done.


Final words
========
So if it was possible to simplify the API and reduce the IN OUT FAULT
to a simpler API, even go as far as I would like with a single
Message.
Then that would be really great and worth a hold up for a imminent 2.0 release.


Other frameworks
=============
I guess it wasn't final words after all :) Just wanted to say that
Mule, Spring Integration also just have a single message for the
message.
And yes I think their projects are also successful so it is not a loss
that they do not have IN OUT. In fact I think their API is easier to
work with than Camel.

We are fortunate that most people with Camel do not work directly with
Exchange but work more with Camel returning the message as an expected
body type using its type converters. That helps a lot.

But we have stories form Camel 1.x where people get confused whey
Camel return their original input in some situations where it was not
expected in the OUT message. Or the fact producer template sendBody is
not void in 1.x. Then people think it can be used for InOut.

Okay end of mail.


On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<ja...@gmail.com> wrote:
> 2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
>> Moved this slightly different topic to a separate thread.
>>
>> ++1 from me.
>>
>> Do we need a vote on this one or it's a consensus?  Claus, it looks like you
>> agree, but pointed out that there is some work involved, correct?
>>
>> Hadrian
>>
>>
>>
>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>
>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>
>>>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<ja...@gmail.com>
>>>> wrote:
>>>>>
>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>
>>>>>> Hi
>>>>>>
>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>
>>>>>> The ProducerCache that is doing the actual sending when using template
>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>
>>>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>>>> pointed out. He wanted the IN to be more static.
>>>>>> And we cannot get there yet when you do routing as all the processors
>>>>>> rely on IN being able to mutate during routing.
>>>>>>
>>>>>> Anyway my grief is that when you send in an Exchange the result would
>>>>>> sometimes be stored on IN and not OUT.
>>>>>> What I want it to do always is to store the result in OUT and keep IN
>>>>>> as the original input.
>>>>>>
>>>>>> The code to do this is now a bit more complex than just before
>>>>>>
>>>>>>               // copy the original input
>>>>>>               Message original = exchange.getIn().copy();
>>>>>>
>>>>>>               producer.process(exchange);
>>>>>>
>>>>>>               // if no OUT then set current IN as result (except for
>>>>>> optional out)
>>>>>>               if (!exchange.hasOut() && exchange.getPattern() !=
>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>                   // but only if its not the same as original IN to
>>>>>> avoid duplicating it
>>>>>>                   // and to adhere to the fact that there was no OUT
>>>>>> result at all
>>>>>>                   if (original.getBody() != null &&
>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>                       exchange.setOut(exchange.getIn());
>>>>>>                   }
>>>>>>               }
>>>>>>               // and restore original in
>>>>>>               exchange.setIn(original);
>>>>>>
>>>>>>               return exchange;
>>>>>>
>>>>>>
>>>>>> What I need to do is to copy the original IN message as it can be
>>>>>> mutated during routing.
>>>>>
>>>>> How about we prevent mutation of the IN message? Create a Message
>>>>> facade which throws UnsupportedOperationException if you try to mutate
>>>>> it in any way. Then we can pass the same read-only Message around as
>>>>> the IN within retry loops or from step to step if no new output is
>>>>> created (e.g. in a content based router where you just move a Message
>>>>> to the right endpoint without changing it)
>>>>>
>>>>
>>>> A good idea but will break a lot of logic in Camel.
>>>
>>> Agreed. But with the benefit that we'd be able to get rid of all the
>>> defensive copies in our code; plus we'd be able to pass the same
>>> Message from step to step. The API would be a bit more clean; to
>>> change the output, you create an OUT message (maybe by copying the
>>> IN).
>>>
>>>
>>>> Most of the Camel
>>>> processors work on the IN message and set the result on either IN or
>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>> original input? Or am I mistaking?
>>>
>>> Yeah, we'd have to patch code to no longer mutate IN
>>>
>>>
>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>> immutable and always the original input?
>>>
>>> If no OUT, then no output was created, so pass the IN along...
>>>
>>>
>>> OK how about this; a CopyOnWriteMessageFacade which does not mutate
>>> the original message at all ever; if a message is used in a purely
>>> read only way, it does nothing but delegate to the original message -
>>> but then as soon as someone mutates it, it creates a copy and uses
>>> that from that point on?
>>>
>>> i.e. make the copy of the message lazy - and only make a copy when a
>>> Processor really does try to mutate the Message?
>>>
>>> Then we'd get the best of both worlds; avoid breaking old code but
>>> avoid tons of unnecessary copies?
>
> BTW then using the current API you could have a Message and then call
>
> Message origin = ...;
> Message newMsg = origin.copy().copy().copy().copy();
>
> and the message would not actually be copied at all; new would just be
> a CopyOnWriteMessageFacade which would hold a reference to 'origin' as
> the readOnlyMessage (which it never mutates).
>
> The copy would only take place if you did
>
> newMsg.setBody("foo")
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[James Strachan <ja...@gmail.com>]

2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
> Moved this slightly different topic to a separate thread.
>
> ++1 from me.
>
> Do we need a vote on this one or it's a consensus?  Claus, it looks like you
> agree, but pointed out that there is some work involved, correct?
>
> Hadrian
>
>
>
> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>
>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>
>>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<ja...@gmail.com>
>>> wrote:
>>>>
>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>
>>>>> Hi
>>>>>
>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>
>>>>> The ProducerCache that is doing the actual sending when using template
>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>
>>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>>> pointed out. He wanted the IN to be more static.
>>>>> And we cannot get there yet when you do routing as all the processors
>>>>> rely on IN being able to mutate during routing.
>>>>>
>>>>> Anyway my grief is that when you send in an Exchange the result would
>>>>> sometimes be stored on IN and not OUT.
>>>>> What I want it to do always is to store the result in OUT and keep IN
>>>>> as the original input.
>>>>>
>>>>> The code to do this is now a bit more complex than just before
>>>>>
>>>>>               // copy the original input
>>>>>               Message original = exchange.getIn().copy();
>>>>>
>>>>>               producer.process(exchange);
>>>>>
>>>>>               // if no OUT then set current IN as result (except for
>>>>> optional out)
>>>>>               if (!exchange.hasOut() && exchange.getPattern() !=
>>>>> ExchangePattern.InOptionalOut) {
>>>>>                   // but only if its not the same as original IN to
>>>>> avoid duplicating it
>>>>>                   // and to adhere to the fact that there was no OUT
>>>>> result at all
>>>>>                   if (original.getBody() != null &&
>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>                       exchange.setOut(exchange.getIn());
>>>>>                   }
>>>>>               }
>>>>>               // and restore original in
>>>>>               exchange.setIn(original);
>>>>>
>>>>>               return exchange;
>>>>>
>>>>>
>>>>> What I need to do is to copy the original IN message as it can be
>>>>> mutated during routing.
>>>>
>>>> How about we prevent mutation of the IN message? Create a Message
>>>> facade which throws UnsupportedOperationException if you try to mutate
>>>> it in any way. Then we can pass the same read-only Message around as
>>>> the IN within retry loops or from step to step if no new output is
>>>> created (e.g. in a content based router where you just move a Message
>>>> to the right endpoint without changing it)
>>>>
>>>
>>> A good idea but will break a lot of logic in Camel.
>>
>> Agreed. But with the benefit that we'd be able to get rid of all the
>> defensive copies in our code; plus we'd be able to pass the same
>> Message from step to step. The API would be a bit more clean; to
>> change the output, you create an OUT message (maybe by copying the
>> IN).
>>
>>
>>> Most of the Camel
>>> processors work on the IN message and set the result on either IN or
>>> OUT. At best they set it on OUT. But then the IN is always the
>>> original input? Or am I mistaking?
>>
>> Yeah, we'd have to patch code to no longer mutate IN
>>
>>
>>> How will this work with the Pipes And Filters EIP if the IN is
>>> immutable and always the original input?
>>
>> If no OUT, then no output was created, so pass the IN along...
>>
>>
>> OK how about this; a CopyOnWriteMessageFacade which does not mutate
>> the original message at all ever; if a message is used in a purely
>> read only way, it does nothing but delegate to the original message -
>> but then as soon as someone mutates it, it creates a copy and uses
>> that from that point on?
>>
>> i.e. make the copy of the message lazy - and only make a copy when a
>> Processor really does try to mutate the Message?
>>
>> Then we'd get the best of both worlds; avoid breaking old code but
>> avoid tons of unnecessary copies?

BTW then using the current API you could have a Message and then call

Message origin = ...;
Message newMsg = origin.copy().copy().copy().copy();

and the message would not actually be copied at all; new would just be
a CopyOnWriteMessageFacade which would hold a reference to 'origin' as
the readOnlyMessage (which it never mutates).

The copy would only take place if you did

newMsg.setBody("foo")

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

[DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Hadrian Zbarcea <hz...@gmail.com>]

Moved this slightly different topic to a separate thread.

++1 from me.

Do we need a vote on this one or it's a consensus?  Claus, it looks  
like you agree, but pointed out that there is some work involved,  
correct?

Hadrian



On Jul 10, 2009, at 8:40 AM, James Strachan wrote:

> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<james.strachan@gmail.com 
>> > wrote:
>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>> Hi
>>>>
>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>
>>>> The ProducerCache that is doing the actual sending when using  
>>>> template
>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>
>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>> pointed out. He wanted the IN to be more static.
>>>> And we cannot get there yet when you do routing as all the  
>>>> processors
>>>> rely on IN being able to mutate during routing.
>>>>
>>>> Anyway my grief is that when you send in an Exchange the result  
>>>> would
>>>> sometimes be stored on IN and not OUT.
>>>> What I want it to do always is to store the result in OUT and  
>>>> keep IN
>>>> as the original input.
>>>>
>>>> The code to do this is now a bit more complex than just before
>>>>
>>>>                // copy the original input
>>>>                Message original = exchange.getIn().copy();
>>>>
>>>>                producer.process(exchange);
>>>>
>>>>                // if no OUT then set current IN as result (except  
>>>> for
>>>> optional out)
>>>>                if (!exchange.hasOut() && exchange.getPattern() !=
>>>> ExchangePattern.InOptionalOut) {
>>>>                    // but only if its not the same as original IN  
>>>> to
>>>> avoid duplicating it
>>>>                    // and to adhere to the fact that there was no  
>>>> OUT
>>>> result at all
>>>>                    if (original.getBody() != null &&
>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>                        exchange.setOut(exchange.getIn());
>>>>                    }
>>>>                }
>>>>                // and restore original in
>>>>                exchange.setIn(original);
>>>>
>>>>                return exchange;
>>>>
>>>>
>>>> What I need to do is to copy the original IN message as it can be
>>>> mutated during routing.
>>>
>>> How about we prevent mutation of the IN message? Create a Message
>>> facade which throws UnsupportedOperationException if you try to  
>>> mutate
>>> it in any way. Then we can pass the same read-only Message around as
>>> the IN within retry loops or from step to step if no new output is
>>> created (e.g. in a content based router where you just move a  
>>> Message
>>> to the right endpoint without changing it)
>>>
>>
>> A good idea but will break a lot of logic in Camel.
>
> Agreed. But with the benefit that we'd be able to get rid of all the
> defensive copies in our code; plus we'd be able to pass the same
> Message from step to step. The API would be a bit more clean; to
> change the output, you create an OUT message (maybe by copying the
> IN).
>
>
>> Most of the Camel
>> processors work on the IN message and set the result on either IN or
>> OUT. At best they set it on OUT. But then the IN is always the
>> original input? Or am I mistaking?
>
> Yeah, we'd have to patch code to no longer mutate IN
>
>
>> How will this work with the Pipes And Filters EIP if the IN is
>> immutable and always the original input?
>
> If no OUT, then no output was created, so pass the IN along...
>
>
> OK how about this; a CopyOnWriteMessageFacade which does not mutate
> the original message at all ever; if a message is used in a purely
> read only way, it does nothing but delegate to the original message -
> but then as soon as someone mutates it, it creates a copy and uses
> that from that point on?
>
> i.e. make the copy of the message lazy - and only make a copy when a
> Processor really does try to mutate the Message?
>
> Then we'd get the best of both worlds; avoid breaking old code but
> avoid tons of unnecessary copies?
>
> -- 
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/

Re: [DISCUSS] Faults and Exceptions in Camel

[James Strachan <ja...@gmail.com>]

2009/7/10 Claus Ibsen <cl...@gmail.com>:
> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<ja...@gmail.com> wrote:
>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>> Hi
>>>
>>> Another update on the IN vs OUT when you send in an Exchange.
>>>
>>> The ProducerCache that is doing the actual sending when using template
>>> or sendTo etc, its basically doing all send X to endpoint.
>>>
>>> Well I am playing with to let it adhere to the principle Hadrian
>>> pointed out. He wanted the IN to be more static.
>>> And we cannot get there yet when you do routing as all the processors
>>> rely on IN being able to mutate during routing.
>>>
>>> Anyway my grief is that when you send in an Exchange the result would
>>> sometimes be stored on IN and not OUT.
>>> What I want it to do always is to store the result in OUT and keep IN
>>> as the original input.
>>>
>>> The code to do this is now a bit more complex than just before
>>>
>>>                // copy the original input
>>>                Message original = exchange.getIn().copy();
>>>
>>>                producer.process(exchange);
>>>
>>>                // if no OUT then set current IN as result (except for
>>> optional out)
>>>                if (!exchange.hasOut() && exchange.getPattern() !=
>>> ExchangePattern.InOptionalOut) {
>>>                    // but only if its not the same as original IN to
>>> avoid duplicating it
>>>                    // and to adhere to the fact that there was no OUT
>>> result at all
>>>                    if (original.getBody() != null &&
>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>                        exchange.setOut(exchange.getIn());
>>>                    }
>>>                }
>>>                // and restore original in
>>>                exchange.setIn(original);
>>>
>>>                return exchange;
>>>
>>>
>>> What I need to do is to copy the original IN message as it can be
>>> mutated during routing.
>>
>> How about we prevent mutation of the IN message? Create a Message
>> facade which throws UnsupportedOperationException if you try to mutate
>> it in any way. Then we can pass the same read-only Message around as
>> the IN within retry loops or from step to step if no new output is
>> created (e.g. in a content based router where you just move a Message
>> to the right endpoint without changing it)
>>
>
> A good idea but will break a lot of logic in Camel.

Agreed. But with the benefit that we'd be able to get rid of all the
defensive copies in our code; plus we'd be able to pass the same
Message from step to step. The API would be a bit more clean; to
change the output, you create an OUT message (maybe by copying the
IN).


> Most of the Camel
> processors work on the IN message and set the result on either IN or
> OUT. At best they set it on OUT. But then the IN is always the
> original input? Or am I mistaking?

Yeah, we'd have to patch code to no longer mutate IN


> How will this work with the Pipes And Filters EIP if the IN is
> immutable and always the original input?

If no OUT, then no output was created, so pass the IN along...


OK how about this; a CopyOnWriteMessageFacade which does not mutate
the original message at all ever; if a message is used in a purely
read only way, it does nothing but delegate to the original message -
but then as soon as someone mutates it, it creates a copy and uses
that from that point on?

i.e. make the copy of the message lazy - and only make a copy when a
Processor really does try to mutate the Message?

Then we'd get the best of both worlds; avoid breaking old code but
avoid tons of unnecessary copies?

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Faults and Exceptions in Camel

[Hadrian Zbarcea <hz...@gmail.com>]

Roman,

Freedom can be a dangerous thing :).

With overwhelming consensus I'll crate a jira and implement this  
before Monday using James' proposal #3 with
setFault(boolean);

Thanks everybody for pitching in
Hadrian



On Jul 10, 2009, at 8:26 AM, Roman Kalukiewicz wrote:

> 2009/7/9 Claus Ibsen <cl...@gmail.com>:
>> I think the names should be
>>
>> boolean isFault()
>> void setFault()
>>
>> Yeah I do not think people should call setFault(false)
>> or later want to change an existing OUT message from fault to out  
>> or vice versa.
>>
>> But is it confusing with a setter that dont accept parameters, does  
>> it
>> violate the bean spec?
>
> I would propose to leave setFault(boolean) anyway. Does it hurt anyone
> if we give more freedom to our users? I can easily imagine situation
> when I call CXF endpoint, receive a fault and then want to simply send
> it to JMS queue whatever the response was. Then I just clear the fault
> flag and I'm done.
>
> Another thing is how faults (if they exist) should be handled. I
> believe we should have our error handling extended so we can write
> something like:
> .doTry()
> .to("cxf:bean:faultThrowingService")
> .doCatch(body().instanceOf(MyFault.class))
> ...
> .end()
>
> It can handle faults almost like exceptions but it has more
> flexibility (but it is core API discussion anymore).
>
> Roman

Re: [DISCUSS] Faults and Exceptions in Camel

[Roman Kalukiewicz <ro...@gmail.com>]

2009/7/9 Claus Ibsen <cl...@gmail.com>:
> I think the names should be
>
> boolean isFault()
> void setFault()
>
> Yeah I do not think people should call setFault(false)
> or later want to change an existing OUT message from fault to out or vice versa.
>
> But is it confusing with a setter that dont accept parameters, does it
> violate the bean spec?

I would propose to leave setFault(boolean) anyway. Does it hurt anyone
if we give more freedom to our users? I can easily imagine situation
when I call CXF endpoint, receive a fault and then want to simply send
it to JMS queue whatever the response was. Then I just clear the fault
flag and I'm done.

Another thing is how faults (if they exist) should be handled. I
believe we should have our error handling extended so we can write
something like:
.doTry()
.to("cxf:bean:faultThrowingService")
.doCatch(body().instanceOf(MyFault.class))
...
.end()

It can handle faults almost like exceptions but it has more
flexibility (but it is core API discussion anymore).

Roman

Re: [DISCUSS] Faults and Exceptions in Camel

[Roman Kalukiewicz <ro...@gmail.com>]

2009/7/10 James Strachan <ja...@gmail.com>:
> 2009/7/10 Roman Kalukiewicz <ro...@gmail.com>:
>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>> Now that we have opened the box with IN OUT FAULT api changes I would
>>> like to point out issues related to OUT
>>>
>>> Given this code:
>>>
>>>        Exchange out = template.send("direct:start", new Processor() {
>>>            public void process(Exchange exchange) throws Exception {
>>>                exchange.getIn().setBody("Hello World");
>>>                exchange.setPattern(ExchangePattern.InOnly);
>>>            }
>>>        });
>>>
>>> And this route:
>>>
>>>        from("direct:start").transform(constant("Bye World"));
>>>
>>> What would the expected output of Exchange be?
>>>
>>> The current code asserts this:
>>>
>>>        assertEquals("Hello World", out.getIn().getBody());
>>>        assertEquals("Bye World", out.getOut().getBody());
>>>
>>> That looks fair. The route transforms (= set an OUT body) and we
>>> preserve the original IN.
>>> But the exchange pattern was InOnly but we get data in OUT also? Camel
>>> does not adhere strictly to the patterns.
>>
>> This is another point I would like to see changed - exchange patterns.
>>
>> My personal impression is that exchange shouldn't have pattern at all.
>> When I create a flow I (usually) know at design time if I want given
>> endpoint to operate as InOnly or InOut. I really cannot imagine a
>> situation, when I design a flow and I don't really know if it should
>> operate in InOnly or in InOut mode, especially that sometimes it
>> changes a lot in the behavior of the flow.
>>
>> I would like to see it as an endpoint property rather than exchange
>> property especially, that in majority of endpoints the pattern is
>> ignored anyway.
>>
>> My proposal: Remove MEP concept at all from Camel API. Leave it for
>> specific components.
>
> So what about consuming a JMS message which could be an InOnly or
> InOut based on the presence of a JMSReplyTo header?
>
> Or invoking a (say) JMS endpoint from application code using either
> InOnly or InOut where the client decides based (say) an @InOnly
> annotation on a Java method when doing Spring Remoting or a different
> method on the ProducerTemplate in application code?
>
> The endpoint cannot always decide by itself the MEP; sometimes the
> client or the message it receives can decide it. So I think the MEP
> needs to be associated with the Exchange.

OK - My fault (don't confuse it with FAULT discussion ;)). I would
rather propose MEP to be specified on Producer/Consumer. In fact Camel
2.0 somehow did it by introducing to(Pattern, Endpoint) construct, so
MEPs are specified on a producer - not on an exchange (at least from
logical point of view).

Moreover the fact, that you sent InOut JMS message to an endpoint,
doesn't really mean, that it should be processed this way in Camel -
we even have "disableReplyTo" option on the endpoint, so we already
have a kind of MEP control on the endpoint level.

So this is the client or server, who should decide how MEPs are used
(if they make sense for given protocol). But there is no need to
PROPAGATE it with the exchange, because then the decision made by
client in one endpoint influences the behavior of totally different
endpoint.

So two facts:
1) MEPs are used in just few components, so I don't think they should
be in Camel API - they could be customized at producer/consumer level
2) We already have some options that affect MEPs on endpoint level
like "disableReplyTo".

Because of this we could have:
from("direct:start").
to("jms:in-queue?mep=InOut")
as it makes sense in case of JMS - then we know what will happen here.

Now when I see:
from("direct:start").
to("jms:in-queue")
I don't really know if my message is affected or not. Will I wait for
some response, or not. It simply depends on MEP I receive from
"direct:start".

>> My impression is that there are more common things that could be
>> included into API than MEP, like timeout concept. I don't propose
>> timeout on API level, but definitely it is more useful and common than
>> MEP.
>
> Timeouts are really useful for InOuts! Something we might want to add
> - or at least have standard header/properties on the exchange?

Definitely agree.

>> Then (if we agree that OUT is needed) everything creates OUT always -
>> this way we are consistent and no guesses are needed. Moreover if we
>> agree that everything creates OUT then everything can operate on IN
>> the same way. What you can always do in your code is
>>
>> Object in = exchange.getMessage().getBody();
>> exchange.getMessage().setBody(out);
>>
>> This way IF YOU NEED, you can have your IN message in the code, but it
>> doesn't spoil an API.
>
> Not sure how you'd access the in and out together after the out had
> been modified in some way?

I have a reference to the IN message here (or the old message if we
decide to have only one). I can do whatever I want with it as it is a
variable - i don't really have to have it referenced from the exchange
itself.

Roman

Re: [DISCUSS] Faults and Exceptions in Camel

[Hadrian Zbarcea <hz...@gmail.com>]

Thanks James :),

-1 from on the same counts
Hadrian

On Jul 10, 2009, at 9:00 AM, James Strachan wrote:

> 2009/7/10 Roman Kalukiewicz <ro...@gmail.com>:
>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>> Now that we have opened the box with IN OUT FAULT api changes I  
>>> would
>>> like to point out issues related to OUT
>>>
>>> Given this code:
>>>
>>>        Exchange out = template.send("direct:start", new  
>>> Processor() {
>>>            public void process(Exchange exchange) throws Exception {
>>>                exchange.getIn().setBody("Hello World");
>>>                exchange.setPattern(ExchangePattern.InOnly);
>>>            }
>>>        });
>>>
>>> And this route:
>>>
>>>        from("direct:start").transform(constant("Bye World"));
>>>
>>> What would the expected output of Exchange be?
>>>
>>> The current code asserts this:
>>>
>>>        assertEquals("Hello World", out.getIn().getBody());
>>>        assertEquals("Bye World", out.getOut().getBody());
>>>
>>> That looks fair. The route transforms (= set an OUT body) and we
>>> preserve the original IN.
>>> But the exchange pattern was InOnly but we get data in OUT also?  
>>> Camel
>>> does not adhere strictly to the patterns.
>>
>> This is another point I would like to see changed - exchange  
>> patterns.
>>
>> My personal impression is that exchange shouldn't have pattern at  
>> all.
>> When I create a flow I (usually) know at design time if I want given
>> endpoint to operate as InOnly or InOut. I really cannot imagine a
>> situation, when I design a flow and I don't really know if it should
>> operate in InOnly or in InOut mode, especially that sometimes it
>> changes a lot in the behavior of the flow.
>>
>> I would like to see it as an endpoint property rather than exchange
>> property especially, that in majority of endpoints the pattern is
>> ignored anyway.
>>
>> My proposal: Remove MEP concept at all from Camel API. Leave it for
>> specific components.
>
> So what about consuming a JMS message which could be an InOnly or
> InOut based on the presence of a JMSReplyTo header?
>
> Or invoking a (say) JMS endpoint from application code using either
> InOnly or InOut where the client decides based (say) an @InOnly
> annotation on a Java method when doing Spring Remoting or a different
> method on the ProducerTemplate in application code?
>
> The endpoint cannot always decide by itself the MEP; sometimes the
> client or the message it receives can decide it. So I think the MEP
> needs to be associated with the Exchange.
>
>
>> My impression is that there are more common things that could be
>> included into API than MEP, like timeout concept. I don't propose
>> timeout on API level, but definitely it is more useful and common  
>> than
>> MEP.
>
> Timeouts are really useful for InOuts! Something we might want to add
> - or at least have standard header/properties on the exchange?
>
>
>> Then (if we agree that OUT is needed) everything creates OUT always -
>> this way we are consistent and no guesses are needed. Moreover if we
>> agree that everything creates OUT then everything can operate on IN
>> the same way. What you can always do in your code is
>>
>> Object in = exchange.getMessage().getBody();
>> exchange.getMessage().setBody(out);
>>
>> This way IF YOU NEED, you can have your IN message in the code, but  
>> it
>> doesn't spoil an API.
>
> Not sure how you'd access the in and out together after the out had
> been modified in some way?
> -- 
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/

Re: [DISCUSS] Faults and Exceptions in Camel

[James Strachan <ja...@gmail.com>]

2009/7/10 Roman Kalukiewicz <ro...@gmail.com>:
> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>> Now that we have opened the box with IN OUT FAULT api changes I would
>> like to point out issues related to OUT
>>
>> Given this code:
>>
>>        Exchange out = template.send("direct:start", new Processor() {
>>            public void process(Exchange exchange) throws Exception {
>>                exchange.getIn().setBody("Hello World");
>>                exchange.setPattern(ExchangePattern.InOnly);
>>            }
>>        });
>>
>> And this route:
>>
>>        from("direct:start").transform(constant("Bye World"));
>>
>> What would the expected output of Exchange be?
>>
>> The current code asserts this:
>>
>>        assertEquals("Hello World", out.getIn().getBody());
>>        assertEquals("Bye World", out.getOut().getBody());
>>
>> That looks fair. The route transforms (= set an OUT body) and we
>> preserve the original IN.
>> But the exchange pattern was InOnly but we get data in OUT also? Camel
>> does not adhere strictly to the patterns.
>
> This is another point I would like to see changed - exchange patterns.
>
> My personal impression is that exchange shouldn't have pattern at all.
> When I create a flow I (usually) know at design time if I want given
> endpoint to operate as InOnly or InOut. I really cannot imagine a
> situation, when I design a flow and I don't really know if it should
> operate in InOnly or in InOut mode, especially that sometimes it
> changes a lot in the behavior of the flow.
>
> I would like to see it as an endpoint property rather than exchange
> property especially, that in majority of endpoints the pattern is
> ignored anyway.
>
> My proposal: Remove MEP concept at all from Camel API. Leave it for
> specific components.

So what about consuming a JMS message which could be an InOnly or
InOut based on the presence of a JMSReplyTo header?

Or invoking a (say) JMS endpoint from application code using either
InOnly or InOut where the client decides based (say) an @InOnly
annotation on a Java method when doing Spring Remoting or a different
method on the ProducerTemplate in application code?

The endpoint cannot always decide by itself the MEP; sometimes the
client or the message it receives can decide it. So I think the MEP
needs to be associated with the Exchange.


> My impression is that there are more common things that could be
> included into API than MEP, like timeout concept. I don't propose
> timeout on API level, but definitely it is more useful and common than
> MEP.

Timeouts are really useful for InOuts! Something we might want to add
- or at least have standard header/properties on the exchange?


> Then (if we agree that OUT is needed) everything creates OUT always -
> this way we are consistent and no guesses are needed. Moreover if we
> agree that everything creates OUT then everything can operate on IN
> the same way. What you can always do in your code is
>
> Object in = exchange.getMessage().getBody();
> exchange.getMessage().setBody(out);
>
> This way IF YOU NEED, you can have your IN message in the code, but it
> doesn't spoil an API.

Not sure how you'd access the in and out together after the out had
been modified in some way?
-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Faults and Exceptions in Camel

[Claus Ibsen <cl...@gmail.com>]

On Thu, Jul 9, 2009 at 7:25 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
> Ah, your proposal is a variation of the first one, where isFault and
> setFault() are Message methods, not Exchange.  I am fine with that too.  Not
> sure about the boolean argument for setFault, but that's a detail.  Why
> would somebody invoke:
> message.setFault(false);
>
> 3.
> Exchange interface
>   Message getOut();
>   void setOut(Message out);
>
> Message interface
>   boolean hasFault();
>   void setFault();      // or
>   void setFault(boolean value);
>
> I am ok with all three api versions proposed so far.  I guess [3] is clearer
> more intuitive.
Yeah #3 is nice, makes Exchange has less methods.

I think the names should be

boolean isFault()
void setFault()

Yeah I do not think people should call setFault(false)
or later want to change an existing OUT message from fault to out or vice versa.

But is it confusing with a setter that dont accept parameters, does it
violate the bean spec?


>
> Thanks
> Hadrian
>
>
> On Jul 9, 2009, at 1:11 PM, James Strachan wrote:
>
>> 2009/7/9 Hadrian Zbarcea <hz...@gmail.com>:
>>>
>>> That is *only one* of getOut() and getFault() would return a non null
>>> object
>>> depending on the CAMEL_FAULT header being present.
>>> setOut() and setFault() would (re)set the CAMEL_FAULT header as needed.
>>>  OUT
>>> and FAULT are mutually exclusive.
>>
>> Which is why I'm thinking we just have one propery, "out" which the
>> message has an isFault() / setFault(boolean) property on it.
>>
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://fusesource.com/
>
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Faults and Exceptions in Camel

[Claus Ibsen <cl...@gmail.com>]

On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<ja...@gmail.com> wrote:
> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>> Hi
>>
>> Another update on the IN vs OUT when you send in an Exchange.
>>
>> The ProducerCache that is doing the actual sending when using template
>> or sendTo etc, its basically doing all send X to endpoint.
>>
>> Well I am playing with to let it adhere to the principle Hadrian
>> pointed out. He wanted the IN to be more static.
>> And we cannot get there yet when you do routing as all the processors
>> rely on IN being able to mutate during routing.
>>
>> Anyway my grief is that when you send in an Exchange the result would
>> sometimes be stored on IN and not OUT.
>> What I want it to do always is to store the result in OUT and keep IN
>> as the original input.
>>
>> The code to do this is now a bit more complex than just before
>>
>>                // copy the original input
>>                Message original = exchange.getIn().copy();
>>
>>                producer.process(exchange);
>>
>>                // if no OUT then set current IN as result (except for
>> optional out)
>>                if (!exchange.hasOut() && exchange.getPattern() !=
>> ExchangePattern.InOptionalOut) {
>>                    // but only if its not the same as original IN to
>> avoid duplicating it
>>                    // and to adhere to the fact that there was no OUT
>> result at all
>>                    if (original.getBody() != null &&
>> !original.getBody().equals(exchange.getIn().getBody())) {
>>                        exchange.setOut(exchange.getIn());
>>                    }
>>                }
>>                // and restore original in
>>                exchange.setIn(original);
>>
>>                return exchange;
>>
>>
>> What I need to do is to copy the original IN message as it can be
>> mutated during routing.
>
> How about we prevent mutation of the IN message? Create a Message
> facade which throws UnsupportedOperationException if you try to mutate
> it in any way. Then we can pass the same read-only Message around as
> the IN within retry loops or from step to step if no new output is
> created (e.g. in a content based router where you just move a Message
> to the right endpoint without changing it)
>

A good idea but will break a lot of logic in Camel. Most of the Camel
processors work on the IN message and set the result on either IN or
OUT. At best they set it on OUT. But then the IN is always the
original input? Or am I mistaking?

How will this work with the Pipes And Filters EIP if the IN is
immutable and always the original input?

And all the bean binding also grab the IN message when it binds to a POJO.

If people wanted the original IN message it can be stored as an
Exchange. In fact its already there on the Unit Of Work. We could just
expose it better.
    Message original = exchange.getUnitOfWork().getOriginalMessage();




> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Faults and Exceptions in Camel

[James Strachan <ja...@gmail.com>]

2009/7/10 Claus Ibsen <cl...@gmail.com>:
> Hi
>
> Another update on the IN vs OUT when you send in an Exchange.
>
> The ProducerCache that is doing the actual sending when using template
> or sendTo etc, its basically doing all send X to endpoint.
>
> Well I am playing with to let it adhere to the principle Hadrian
> pointed out. He wanted the IN to be more static.
> And we cannot get there yet when you do routing as all the processors
> rely on IN being able to mutate during routing.
>
> Anyway my grief is that when you send in an Exchange the result would
> sometimes be stored on IN and not OUT.
> What I want it to do always is to store the result in OUT and keep IN
> as the original input.
>
> The code to do this is now a bit more complex than just before
>
>                // copy the original input
>                Message original = exchange.getIn().copy();
>
>                producer.process(exchange);
>
>                // if no OUT then set current IN as result (except for
> optional out)
>                if (!exchange.hasOut() && exchange.getPattern() !=
> ExchangePattern.InOptionalOut) {
>                    // but only if its not the same as original IN to
> avoid duplicating it
>                    // and to adhere to the fact that there was no OUT
> result at all
>                    if (original.getBody() != null &&
> !original.getBody().equals(exchange.getIn().getBody())) {
>                        exchange.setOut(exchange.getIn());
>                    }
>                }
>                // and restore original in
>                exchange.setIn(original);
>
>                return exchange;
>
>
> What I need to do is to copy the original IN message as it can be
> mutated during routing.

How about we prevent mutation of the IN message? Create a Message
facade which throws UnsupportedOperationException if you try to mutate
it in any way. Then we can pass the same read-only Message around as
the IN within retry loops or from step to step if no new output is
created (e.g. in a content based router where you just move a Message
to the right endpoint without changing it)

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Faults and Exceptions in Camel

[Claus Ibsen <cl...@gmail.com>]

Hi

Another update on the IN vs OUT when you send in an Exchange.

The ProducerCache that is doing the actual sending when using template
or sendTo etc, its basically doing all send X to endpoint.

Well I am playing with to let it adhere to the principle Hadrian
pointed out. He wanted the IN to be more static.
And we cannot get there yet when you do routing as all the processors
rely on IN being able to mutate during routing.

Anyway my grief is that when you send in an Exchange the result would
sometimes be stored on IN and not OUT.
What I want it to do always is to store the result in OUT and keep IN
as the original input.

The code to do this is now a bit more complex than just before

                // copy the original input
                Message original = exchange.getIn().copy();

                producer.process(exchange);

                // if no OUT then set current IN as result (except for
optional out)
                if (!exchange.hasOut() && exchange.getPattern() !=
ExchangePattern.InOptionalOut) {
                    // but only if its not the same as original IN to
avoid duplicating it
                    // and to adhere to the fact that there was no OUT
result at all
                    if (original.getBody() != null &&
!original.getBody().equals(exchange.getIn().getBody())) {
                        exchange.setOut(exchange.getIn());
                    }
                }
                // and restore original in
                exchange.setIn(original);

                return exchange;


What I need to do is to copy the original IN message as it can be
mutated during routing.
Then after processing I need to set the output on the OUT (and take
care of OptionalOut)
And as well also take care of the fact that the OUT result could be
exactly the same as IN (some Camel processors copy IN to OUT).

All together this makes it more consistent to work with sending an
Exchange to Camel.
IN = original input
OUT = the result (if any)

Any thoughts on this change?


PS: I will not affect radically as people often use sendBody to send a
message to Camel and these methods returns a body response.
And thus we dont have to juggle with where should we store the
response IN or OUT and did we mutate IN and what not.








On Fri, Jul 10, 2009 at 7:05 AM, Claus Ibsen<cl...@gmail.com> wrote:
> Now that we have opened the box with IN OUT FAULT api changes I would
> like to point out issues related to OUT
>
> Given this code:
>
>        Exchange out = template.send("direct:start", new Processor() {
>            public void process(Exchange exchange) throws Exception {
>                exchange.getIn().setBody("Hello World");
>                exchange.setPattern(ExchangePattern.InOnly);
>            }
>        });
>
> And this route:
>
>        from("direct:start").transform(constant("Bye World"));
>
> What would the expected output of Exchange be?
>
> The current code asserts this:
>
>        assertEquals("Hello World", out.getIn().getBody());
>        assertEquals("Bye World", out.getOut().getBody());
>
> That looks fair. The route transforms (= set an OUT body) and we
> preserve the original IN.
> But the exchange pattern was InOnly but we get data in OUT also? Camel
> does not adhere strictly to the patterns.
>
>
> Now what if the route only changes the IN message (setBody only changes IN)
>
>        from("direct:start").setBody(constant("Bye World"));
>
> What should the expected outcome be?
>
> Should it be as before?
>
>        assertEquals("Hello World", out.getIn().getBody());
>        assertEquals("Bye World", out.getOut().getBody());
>
> Or as this:
>
>        assertEquals("Bye World", out.getIn().getBody());
>        assertEquals(null, out.getOut().getBody());
>
>
> Its actually now that easy to get a closure on this one. Either we should
> - always store "result" in OUT and copy back the original input to IN
> - OR try to adhere the exchange pattern, and store "result" in either
> IN or OUT depending on the pattern.
>
> This is often only a matter when you send an Exchange to Camel. If you
> use the sendBody then Camel will extract
> the correct result and thus its not a problem here.
>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Faults and Exceptions in Camel

[James Strachan <ja...@gmail.com>]

2009/7/9 Hadrian Zbarcea <hz...@gmail.com>:
> Ah, your proposal is a variation of the first one, where isFault and
> setFault() are Message methods, not Exchange.  I am fine with that too.  Not
> sure about the boolean argument for setFault, but that's a detail.  Why
> would somebody invoke:
> message.setFault(false);

Because you cloned a fault message, wanted to preserve the headers,
but wanted to clear the fault flag?


-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Faults and Exceptions in Camel

[Roman Kalukiewicz <ro...@gmail.com>]

2009/7/10 Claus Ibsen <cl...@gmail.com>:
> Now that we have opened the box with IN OUT FAULT api changes I would
> like to point out issues related to OUT
>
> Given this code:
>
>        Exchange out = template.send("direct:start", new Processor() {
>            public void process(Exchange exchange) throws Exception {
>                exchange.getIn().setBody("Hello World");
>                exchange.setPattern(ExchangePattern.InOnly);
>            }
>        });
>
> And this route:
>
>        from("direct:start").transform(constant("Bye World"));
>
> What would the expected output of Exchange be?
>
> The current code asserts this:
>
>        assertEquals("Hello World", out.getIn().getBody());
>        assertEquals("Bye World", out.getOut().getBody());
>
> That looks fair. The route transforms (= set an OUT body) and we
> preserve the original IN.
> But the exchange pattern was InOnly but we get data in OUT also? Camel
> does not adhere strictly to the patterns.

This is another point I would like to see changed - exchange patterns.

My personal impression is that exchange shouldn't have pattern at all.
When I create a flow I (usually) know at design time if I want given
endpoint to operate as InOnly or InOut. I really cannot imagine a
situation, when I design a flow and I don't really know if it should
operate in InOnly or in InOut mode, especially that sometimes it
changes a lot in the behavior of the flow.

I would like to see it as an endpoint property rather than exchange
property especially, that in majority of endpoints the pattern is
ignored anyway.

My proposal: Remove MEP concept at all from Camel API. Leave it for
specific components.

My impression is that there are more common things that could be
included into API than MEP, like timeout concept. I don't propose
timeout on API level, but definitely it is more useful and common than
MEP.

Then (if we agree that OUT is needed) everything creates OUT always -
this way we are consistent and no guesses are needed. Moreover if we
agree that everything creates OUT then everything can operate on IN
the same way. What you can always do in your code is

Object in = exchange.getMessage().getBody();
exchange.getMessage().setBody(out);

This way IF YOU NEED, you can have your IN message in the code, but it
doesn't spoil an API. It guarantees header propagation by default. No
more questions if something is in IN or OUT. Moreover IN/OUT from
user's point of view doesn't exist - pipeline hides this distinction
almost completely, because whatever you use, you will have IN in a
next step anyway.

Roman

> Now what if the route only changes the IN message (setBody only changes IN)
>
>        from("direct:start").setBody(constant("Bye World"));
>
> What should the expected outcome be?
>
> Should it be as before?
>
>        assertEquals("Hello World", out.getIn().getBody());
>        assertEquals("Bye World", out.getOut().getBody());
>
> Or as this:
>
>        assertEquals("Bye World", out.getIn().getBody());
>        assertEquals(null, out.getOut().getBody());
>
>
> Its actually now that easy to get a closure on this one. Either we should
> - always store "result" in OUT and copy back the original input to IN
> - OR try to adhere the exchange pattern, and store "result" in either
> IN or OUT depending on the pattern.
>
> This is often only a matter when you send an Exchange to Camel. If you
> use the sendBody then Camel will extract
> the correct result and thus its not a problem here.
>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>

Re: [DISCUSS] Faults and Exceptions in Camel

[Claus Ibsen <cl...@gmail.com>]

Hi

An update on my working on this issue.

I follow the principle outlined by James on the Pipeline restructure.

The prepare exchange for next step is reduced to this principle:

    protected Exchange prepareExchange(Exchange exchange) {
        // now lets set the input of the next exchange to the output of the
        // previous message if it is not null
        if (exchange.hasOut()) {
            exchange.setIn(exchange.getOut());
            exchange.setOut(null);
        }
        return exchange;
    }

That fits well with the Pipes And Filters EIP: output is the next input.

So the Pipelines is becoming nice and slick now.


On Fri, Jul 10, 2009 at 7:05 AM, Claus Ibsen<cl...@gmail.com> wrote:
> Now that we have opened the box with IN OUT FAULT api changes I would
> like to point out issues related to OUT
>
> Given this code:
>
>        Exchange out = template.send("direct:start", new Processor() {
>            public void process(Exchange exchange) throws Exception {
>                exchange.getIn().setBody("Hello World");
>                exchange.setPattern(ExchangePattern.InOnly);
>            }
>        });
>
> And this route:
>
>        from("direct:start").transform(constant("Bye World"));
>
> What would the expected output of Exchange be?
>
> The current code asserts this:
>
>        assertEquals("Hello World", out.getIn().getBody());
>        assertEquals("Bye World", out.getOut().getBody());
>
> That looks fair. The route transforms (= set an OUT body) and we
> preserve the original IN.
> But the exchange pattern was InOnly but we get data in OUT also? Camel
> does not adhere strictly to the patterns.
>
>
> Now what if the route only changes the IN message (setBody only changes IN)
>
>        from("direct:start").setBody(constant("Bye World"));
>
> What should the expected outcome be?
>
> Should it be as before?
>
>        assertEquals("Hello World", out.getIn().getBody());
>        assertEquals("Bye World", out.getOut().getBody());
>
> Or as this:
>
>        assertEquals("Bye World", out.getIn().getBody());
>        assertEquals(null, out.getOut().getBody());
>
>
> Its actually now that easy to get a closure on this one. Either we should
> - always store "result" in OUT and copy back the original input to IN
> - OR try to adhere the exchange pattern, and store "result" in either
> IN or OUT depending on the pattern.
>
> This is often only a matter when you send an Exchange to Camel. If you
> use the sendBody then Camel will extract
> the correct result and thus its not a problem here.
>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Faults and Exceptions in Camel

[Hadrian Zbarcea <hz...@gmail.com>]

Yeah, the caller doesn't care about an out, so he's not getting any.   
Save some bandwidth. Green Camel :)
Hadrian


On Jul 10, 2009, at 9:55 AM, Claus Ibsen wrote:

> On Fri, Jul 10, 2009 at 3:50 PM, Hadrian Zbarcea<hz...@gmail.com>  
> wrote:
>> Oops, my bad, didn't pay enough attention.  I guess I meant your  
>> third
>> version then :)
>>
>>>     assertEquals("Hello World", out.getIn().getBody());
>>>     assertEquals(null, out.getOut().getBody());
>>
>>
>> The in is preserved, the OUT is lost on it's way back to the caller.
> And its lost because the exchange pattern was set to InOnly?
>
>
>
>>
>> Sorry,
>> Hadrian
>>
>> On Jul 10, 2009, at 9:43 AM, Claus Ibsen wrote:
>>
>>> On Fri, Jul 10, 2009 at 3:27 PM, Hadrian Zbarcea<hz...@gmail.com>
>>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> My view of this is that InOnly does not mean that throughout  
>>>> processing
>>>> you
>>>> only have an IN in the exchange, but from a consumer perspective  
>>>> it does
>>>> not
>>>> need to spit out an OUT.  That is you can have OUTs during  
>>>> processing,
>>>> and
>>>> the last one at the end will be just dropped.
>>>>
>>>> Your second piece of code looks correct to me and
>>>> I would *not* store the result in an IN ever, always OUT.
>>>
>>> I assume you mean this as first and second code?
>>>
>>> First:
>>>     assertEquals("Hello World", out.getIn().getBody());
>>>     assertEquals("Bye World", out.getOut().getBody());
>>>
>>> Second:
>>>     assertEquals("Bye World", out.getIn().getBody());
>>>     assertEquals(null, out.getOut().getBody());
>>>
>>>
>>> Then the second do not comply with what you said.  "I would *not*
>>> store the result in an IN ever, always OUT."
>>> The second code stores the result in IN at the client side.
>>>
>>>
>>>
>>>
>>>>
>>>> My $0.02
>>>> Hadrian
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Jul 10, 2009, at 1:05 AM, Claus Ibsen wrote:
>>>>
>>>>> Now that we have opened the box with IN OUT FAULT api changes I  
>>>>> would
>>>>> like to point out issues related to OUT
>>>>>
>>>>> Given this code:
>>>>>
>>>>>      Exchange out = template.send("direct:start", new  
>>>>> Processor() {
>>>>>          public void process(Exchange exchange) throws Exception {
>>>>>              exchange.getIn().setBody("Hello World");
>>>>>              exchange.setPattern(ExchangePattern.InOnly);
>>>>>          }
>>>>>      });
>>>>>
>>>>> And this route:
>>>>>
>>>>>      from("direct:start").transform(constant("Bye World"));
>>>>>
>>>>> What would the expected output of Exchange be?
>>>>>
>>>>> The current code asserts this:
>>>>>
>>>>>      assertEquals("Hello World", out.getIn().getBody());
>>>>>      assertEquals("Bye World", out.getOut().getBody());
>>>>>
>>>>> That looks fair. The route transforms (= set an OUT body) and we
>>>>> preserve the original IN.
>>>>> But the exchange pattern was InOnly but we get data in OUT also?  
>>>>> Camel
>>>>> does not adhere strictly to the patterns.
>>>>>
>>>>>
>>>>> Now what if the route only changes the IN message (setBody only  
>>>>> changes
>>>>> IN)
>>>>>
>>>>>      from("direct:start").setBody(constant("Bye World"));
>>>>>
>>>>> What should the expected outcome be?
>>>>>
>>>>> Should it be as before?
>>>>>
>>>>>      assertEquals("Hello World", out.getIn().getBody());
>>>>>      assertEquals("Bye World", out.getOut().getBody());
>>>>>
>>>>> Or as this:
>>>>>
>>>>>      assertEquals("Bye World", out.getIn().getBody());
>>>>>      assertEquals(null, out.getOut().getBody());
>>>>>
>>>>>
>>>>> Its actually now that easy to get a closure on this one. Either we
>>>>> should
>>>>> - always store "result" in OUT and copy back the original input  
>>>>> to IN
>>>>> - OR try to adhere the exchange pattern, and store "result" in  
>>>>> either
>>>>> IN or OUT depending on the pattern.
>>>>>
>>>>> This is often only a matter when you send an Exchange to Camel.  
>>>>> If you
>>>>> use the sendBody then Camel will extract
>>>>> the correct result and thus its not a problem here.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>
>>
>
>
>
> -- 
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Faults and Exceptions in Camel

[Claus Ibsen <cl...@gmail.com>]

On Fri, Jul 10, 2009 at 3:50 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
> Oops, my bad, didn't pay enough attention.  I guess I meant your third
> version then :)
>
>>     assertEquals("Hello World", out.getIn().getBody());
>>     assertEquals(null, out.getOut().getBody());
>
>
> The in is preserved, the OUT is lost on it's way back to the caller.
And its lost because the exchange pattern was set to InOnly?



>
> Sorry,
> Hadrian
>
> On Jul 10, 2009, at 9:43 AM, Claus Ibsen wrote:
>
>> On Fri, Jul 10, 2009 at 3:27 PM, Hadrian Zbarcea<hz...@gmail.com>
>> wrote:
>>>
>>> Hi,
>>>
>>> My view of this is that InOnly does not mean that throughout processing
>>> you
>>> only have an IN in the exchange, but from a consumer perspective it does
>>> not
>>> need to spit out an OUT.  That is you can have OUTs during processing,
>>> and
>>> the last one at the end will be just dropped.
>>>
>>> Your second piece of code looks correct to me and
>>> I would *not* store the result in an IN ever, always OUT.
>>
>> I assume you mean this as first and second code?
>>
>> First:
>>     assertEquals("Hello World", out.getIn().getBody());
>>     assertEquals("Bye World", out.getOut().getBody());
>>
>> Second:
>>     assertEquals("Bye World", out.getIn().getBody());
>>     assertEquals(null, out.getOut().getBody());
>>
>>
>> Then the second do not comply with what you said.  "I would *not*
>> store the result in an IN ever, always OUT."
>> The second code stores the result in IN at the client side.
>>
>>
>>
>>
>>>
>>> My $0.02
>>> Hadrian
>>>
>>>
>>>
>>>
>>>
>>> On Jul 10, 2009, at 1:05 AM, Claus Ibsen wrote:
>>>
>>>> Now that we have opened the box with IN OUT FAULT api changes I would
>>>> like to point out issues related to OUT
>>>>
>>>> Given this code:
>>>>
>>>>      Exchange out = template.send("direct:start", new Processor() {
>>>>          public void process(Exchange exchange) throws Exception {
>>>>              exchange.getIn().setBody("Hello World");
>>>>              exchange.setPattern(ExchangePattern.InOnly);
>>>>          }
>>>>      });
>>>>
>>>> And this route:
>>>>
>>>>      from("direct:start").transform(constant("Bye World"));
>>>>
>>>> What would the expected output of Exchange be?
>>>>
>>>> The current code asserts this:
>>>>
>>>>      assertEquals("Hello World", out.getIn().getBody());
>>>>      assertEquals("Bye World", out.getOut().getBody());
>>>>
>>>> That looks fair. The route transforms (= set an OUT body) and we
>>>> preserve the original IN.
>>>> But the exchange pattern was InOnly but we get data in OUT also? Camel
>>>> does not adhere strictly to the patterns.
>>>>
>>>>
>>>> Now what if the route only changes the IN message (setBody only changes
>>>> IN)
>>>>
>>>>      from("direct:start").setBody(constant("Bye World"));
>>>>
>>>> What should the expected outcome be?
>>>>
>>>> Should it be as before?
>>>>
>>>>      assertEquals("Hello World", out.getIn().getBody());
>>>>      assertEquals("Bye World", out.getOut().getBody());
>>>>
>>>> Or as this:
>>>>
>>>>      assertEquals("Bye World", out.getIn().getBody());
>>>>      assertEquals(null, out.getOut().getBody());
>>>>
>>>>
>>>> Its actually now that easy to get a closure on this one. Either we
>>>> should
>>>> - always store "result" in OUT and copy back the original input to IN
>>>> - OR try to adhere the exchange pattern, and store "result" in either
>>>> IN or OUT depending on the pattern.
>>>>
>>>> This is often only a matter when you send an Exchange to Camel. If you
>>>> use the sendBody then Camel will extract
>>>> the correct result and thus its not a problem here.
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Faults and Exceptions in Camel

[Hadrian Zbarcea <hz...@gmail.com>]

Oops, my bad, didn't pay enough attention.  I guess I meant your third  
version then :)

>      assertEquals("Hello World", out.getIn().getBody());
>      assertEquals(null, out.getOut().getBody());


The in is preserved, the OUT is lost on it's way back to the caller.

Sorry,
Hadrian

On Jul 10, 2009, at 9:43 AM, Claus Ibsen wrote:

> On Fri, Jul 10, 2009 at 3:27 PM, Hadrian Zbarcea<hz...@gmail.com>  
> wrote:
>> Hi,
>>
>> My view of this is that InOnly does not mean that throughout  
>> processing you
>> only have an IN in the exchange, but from a consumer perspective it  
>> does not
>> need to spit out an OUT.  That is you can have OUTs during  
>> processing, and
>> the last one at the end will be just dropped.
>>
>> Your second piece of code looks correct to me and
>> I would *not* store the result in an IN ever, always OUT.
>
> I assume you mean this as first and second code?
>
> First:
>      assertEquals("Hello World", out.getIn().getBody());
>      assertEquals("Bye World", out.getOut().getBody());
>
> Second:
>      assertEquals("Bye World", out.getIn().getBody());
>      assertEquals(null, out.getOut().getBody());
>
>
> Then the second do not comply with what you said.  "I would *not*
> store the result in an IN ever, always OUT."
> The second code stores the result in IN at the client side.
>
>
>
>
>>
>> My $0.02
>> Hadrian
>>
>>
>>
>>
>>
>> On Jul 10, 2009, at 1:05 AM, Claus Ibsen wrote:
>>
>>> Now that we have opened the box with IN OUT FAULT api changes I  
>>> would
>>> like to point out issues related to OUT
>>>
>>> Given this code:
>>>
>>>       Exchange out = template.send("direct:start", new Processor() {
>>>           public void process(Exchange exchange) throws Exception {
>>>               exchange.getIn().setBody("Hello World");
>>>               exchange.setPattern(ExchangePattern.InOnly);
>>>           }
>>>       });
>>>
>>> And this route:
>>>
>>>       from("direct:start").transform(constant("Bye World"));
>>>
>>> What would the expected output of Exchange be?
>>>
>>> The current code asserts this:
>>>
>>>       assertEquals("Hello World", out.getIn().getBody());
>>>       assertEquals("Bye World", out.getOut().getBody());
>>>
>>> That looks fair. The route transforms (= set an OUT body) and we
>>> preserve the original IN.
>>> But the exchange pattern was InOnly but we get data in OUT also?  
>>> Camel
>>> does not adhere strictly to the patterns.
>>>
>>>
>>> Now what if the route only changes the IN message (setBody only  
>>> changes
>>> IN)
>>>
>>>       from("direct:start").setBody(constant("Bye World"));
>>>
>>> What should the expected outcome be?
>>>
>>> Should it be as before?
>>>
>>>       assertEquals("Hello World", out.getIn().getBody());
>>>       assertEquals("Bye World", out.getOut().getBody());
>>>
>>> Or as this:
>>>
>>>       assertEquals("Bye World", out.getIn().getBody());
>>>       assertEquals(null, out.getOut().getBody());
>>>
>>>
>>> Its actually now that easy to get a closure on this one. Either we  
>>> should
>>> - always store "result" in OUT and copy back the original input to  
>>> IN
>>> - OR try to adhere the exchange pattern, and store "result" in  
>>> either
>>> IN or OUT depending on the pattern.
>>>
>>> This is often only a matter when you send an Exchange to Camel. If  
>>> you
>>> use the sendBody then Camel will extract
>>> the correct result and thus its not a problem here.
>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>
>>
>
>
>
> -- 
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Faults and Exceptions in Camel

[Claus Ibsen <cl...@gmail.com>]

On Fri, Jul 10, 2009 at 3:27 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
> Hi,
>
> My view of this is that InOnly does not mean that throughout processing you
> only have an IN in the exchange, but from a consumer perspective it does not
> need to spit out an OUT.  That is you can have OUTs during processing, and
> the last one at the end will be just dropped.
>
> Your second piece of code looks correct to me and
> I would *not* store the result in an IN ever, always OUT.

I assume you mean this as first and second code?

First:
      assertEquals("Hello World", out.getIn().getBody());
      assertEquals("Bye World", out.getOut().getBody());

Second:
      assertEquals("Bye World", out.getIn().getBody());
      assertEquals(null, out.getOut().getBody());


Then the second do not comply with what you said.  "I would *not*
store the result in an IN ever, always OUT."
The second code stores the result in IN at the client side.




>
> My $0.02
> Hadrian
>
>
>
>
>
> On Jul 10, 2009, at 1:05 AM, Claus Ibsen wrote:
>
>> Now that we have opened the box with IN OUT FAULT api changes I would
>> like to point out issues related to OUT
>>
>> Given this code:
>>
>>       Exchange out = template.send("direct:start", new Processor() {
>>           public void process(Exchange exchange) throws Exception {
>>               exchange.getIn().setBody("Hello World");
>>               exchange.setPattern(ExchangePattern.InOnly);
>>           }
>>       });
>>
>> And this route:
>>
>>       from("direct:start").transform(constant("Bye World"));
>>
>> What would the expected output of Exchange be?
>>
>> The current code asserts this:
>>
>>       assertEquals("Hello World", out.getIn().getBody());
>>       assertEquals("Bye World", out.getOut().getBody());
>>
>> That looks fair. The route transforms (= set an OUT body) and we
>> preserve the original IN.
>> But the exchange pattern was InOnly but we get data in OUT also? Camel
>> does not adhere strictly to the patterns.
>>
>>
>> Now what if the route only changes the IN message (setBody only changes
>> IN)
>>
>>       from("direct:start").setBody(constant("Bye World"));
>>
>> What should the expected outcome be?
>>
>> Should it be as before?
>>
>>       assertEquals("Hello World", out.getIn().getBody());
>>       assertEquals("Bye World", out.getOut().getBody());
>>
>> Or as this:
>>
>>       assertEquals("Bye World", out.getIn().getBody());
>>       assertEquals(null, out.getOut().getBody());
>>
>>
>> Its actually now that easy to get a closure on this one. Either we should
>> - always store "result" in OUT and copy back the original input to IN
>> - OR try to adhere the exchange pattern, and store "result" in either
>> IN or OUT depending on the pattern.
>>
>> This is often only a matter when you send an Exchange to Camel. If you
>> use the sendBody then Camel will extract
>> the correct result and thus its not a problem here.
>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Faults and Exceptions in Camel

[Hadrian Zbarcea <hz...@gmail.com>]

Hi,

My view of this is that InOnly does not mean that throughout  
processing you only have an IN in the exchange, but from a consumer  
perspective it does not need to spit out an OUT.  That is you can have  
OUTs during processing, and the last one at the end will be just  
dropped.

Your second piece of code looks correct to me and
I would *not* store the result in an IN ever, always OUT.

My $0.02
Hadrian





On Jul 10, 2009, at 1:05 AM, Claus Ibsen wrote:

> Now that we have opened the box with IN OUT FAULT api changes I would
> like to point out issues related to OUT
>
> Given this code:
>
>        Exchange out = template.send("direct:start", new Processor() {
>            public void process(Exchange exchange) throws Exception {
>                exchange.getIn().setBody("Hello World");
>                exchange.setPattern(ExchangePattern.InOnly);
>            }
>        });
>
> And this route:
>
>        from("direct:start").transform(constant("Bye World"));
>
> What would the expected output of Exchange be?
>
> The current code asserts this:
>
>        assertEquals("Hello World", out.getIn().getBody());
>        assertEquals("Bye World", out.getOut().getBody());
>
> That looks fair. The route transforms (= set an OUT body) and we
> preserve the original IN.
> But the exchange pattern was InOnly but we get data in OUT also? Camel
> does not adhere strictly to the patterns.
>
>
> Now what if the route only changes the IN message (setBody only  
> changes IN)
>
>        from("direct:start").setBody(constant("Bye World"));
>
> What should the expected outcome be?
>
> Should it be as before?
>
>        assertEquals("Hello World", out.getIn().getBody());
>        assertEquals("Bye World", out.getOut().getBody());
>
> Or as this:
>
>        assertEquals("Bye World", out.getIn().getBody());
>        assertEquals(null, out.getOut().getBody());
>
>
> Its actually now that easy to get a closure on this one. Either we  
> should
> - always store "result" in OUT and copy back the original input to IN
> - OR try to adhere the exchange pattern, and store "result" in either
> IN or OUT depending on the pattern.
>
> This is often only a matter when you send an Exchange to Camel. If you
> use the sendBody then Camel will extract
> the correct result and thus its not a problem here.
>
>
>
>
> -- 
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Faults and Exceptions in Camel

[Claus Ibsen <cl...@gmail.com>]

Now that we have opened the box with IN OUT FAULT api changes I would
like to point out issues related to OUT

Given this code:

        Exchange out = template.send("direct:start", new Processor() {
            public void process(Exchange exchange) throws Exception {
                exchange.getIn().setBody("Hello World");
                exchange.setPattern(ExchangePattern.InOnly);
            }
        });

And this route:

        from("direct:start").transform(constant("Bye World"));

What would the expected output of Exchange be?

The current code asserts this:

        assertEquals("Hello World", out.getIn().getBody());
        assertEquals("Bye World", out.getOut().getBody());

That looks fair. The route transforms (= set an OUT body) and we
preserve the original IN.
But the exchange pattern was InOnly but we get data in OUT also? Camel
does not adhere strictly to the patterns.


Now what if the route only changes the IN message (setBody only changes IN)

        from("direct:start").setBody(constant("Bye World"));

What should the expected outcome be?

Should it be as before?

        assertEquals("Hello World", out.getIn().getBody());
        assertEquals("Bye World", out.getOut().getBody());

Or as this:

        assertEquals("Bye World", out.getIn().getBody());
        assertEquals(null, out.getOut().getBody());


Its actually now that easy to get a closure on this one. Either we should
- always store "result" in OUT and copy back the original input to IN
- OR try to adhere the exchange pattern, and store "result" in either
IN or OUT depending on the pattern.

This is often only a matter when you send an Exchange to Camel. If you
use the sendBody then Camel will extract
the correct result and thus its not a problem here.




-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Faults and Exceptions in Camel

[Claus Ibsen <cl...@gmail.com>]

On Thu, Jul 9, 2009 at 7:37 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
> After more coffee, I think this is indeed the very best proposal of all
> three.  Any other proposals / opinions?

+1 to number #3



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Faults and Exceptions in Camel

[Hadrian Zbarcea <hz...@gmail.com>]

After more coffee, I think this is indeed the very best proposal of  
all three.  Any other proposals / opinions?

Thanks James
Hadrian

On Jul 9, 2009, at 1:25 PM, Hadrian Zbarcea wrote:

> Ah, your proposal is a variation of the first one, where isFault and  
> setFault() are Message methods, not Exchange.  I am fine with that  
> too.  Not sure about the boolean argument for setFault, but that's a  
> detail.  Why would somebody invoke:
> message.setFault(false);
>
> 3.
> Exchange interface
>    Message getOut();
>    void setOut(Message out);
>
> Message interface
>    boolean hasFault();
>    void setFault();      // or
>    void setFault(boolean value);
>
> I am ok with all three api versions proposed so far.  I guess [3] is  
> clearer more intuitive.
>
> Thanks
> Hadrian
>
>
> On Jul 9, 2009, at 1:11 PM, James Strachan wrote:
>
>> 2009/7/9 Hadrian Zbarcea <hz...@gmail.com>:
>>> That is *only one* of getOut() and getFault() would return a non  
>>> null object
>>> depending on the CAMEL_FAULT header being present.
>>> setOut() and setFault() would (re)set the CAMEL_FAULT header as  
>>> needed.  OUT
>>> and FAULT are mutually exclusive.
>>
>> Which is why I'm thinking we just have one propery, "out" which the
>> message has an isFault() / setFault(boolean) property on it.
>>
>>
>> -- 
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://fusesource.com/
>

Re: [DISCUSS] Faults and Exceptions in Camel

[Hadrian Zbarcea <hz...@gmail.com>]

Ah, your proposal is a variation of the first one, where isFault and  
setFault() are Message methods, not Exchange.  I am fine with that  
too.  Not sure about the boolean argument for setFault, but that's a  
detail.  Why would somebody invoke:
message.setFault(false);

3.
Exchange interface
    Message getOut();
    void setOut(Message out);

Message interface
    boolean hasFault();
    void setFault();      // or
    void setFault(boolean value);

I am ok with all three api versions proposed so far.  I guess [3] is  
clearer more intuitive.

Thanks
Hadrian


On Jul 9, 2009, at 1:11 PM, James Strachan wrote:

> 2009/7/9 Hadrian Zbarcea <hz...@gmail.com>:
>> That is *only one* of getOut() and getFault() would return a non  
>> null object
>> depending on the CAMEL_FAULT header being present.
>> setOut() and setFault() would (re)set the CAMEL_FAULT header as  
>> needed.  OUT
>> and FAULT are mutually exclusive.
>
> Which is why I'm thinking we just have one propery, "out" which the
> message has an isFault() / setFault(boolean) property on it.
>
>
> -- 
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/

Re: [DISCUSS] Faults and Exceptions in Camel

[James Strachan <ja...@gmail.com>]

2009/7/9 Hadrian Zbarcea <hz...@gmail.com>:
> That is *only one* of getOut() and getFault() would return a non null object
> depending on the CAMEL_FAULT header being present.
> setOut() and setFault() would (re)set the CAMEL_FAULT header as needed.  OUT
> and FAULT are mutually exclusive.

Which is why I'm thinking we just have one propery, "out" which the
message has an isFault() / setFault(boolean) property on it.


-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Faults and Exceptions in Camel

[Hadrian Zbarcea <hz...@gmail.com>]

That is *only one* of getOut() and getFault() would return a non null  
object depending on the CAMEL_FAULT header being present.
setOut() and setFault() would (re)set the CAMEL_FAULT header as  
needed.  OUT and FAULT are mutually exclusive.

Hadrian

On Jul 9, 2009, at 1:04 PM, Hadrian Zbarcea wrote:

> @James, yes it would, but why have createFault()?  Why not have it  
> closer to what it is now with
>
> Message getFault();
> void setFault(Message fault);  // instead of create fault
> (and then we won't need the setFault(boolean);)
>
> To me it looks pretty clean and simple.
>
> if we set the OUT field with a header or not would be an  
> implementation detail.
>
> From what it looks we may have a solution soon.  Any more thoughts  
> on the IN vs OUT question?
>
> $0.02
> Hadrian
>
>
> On Jul 9, 2009, at 12:47 PM, James Strachan wrote:
>
>> 2009/7/9 Claus Ibsen <cl...@gmail.com>:
>>> Hi
>>>
>>> I think it would be confusing if there is a setFault but no  
>>> getFault?
>>> But I understand the reason when you want to use OUT for both  
>>> regular
>>> OUT and FAULT. Where a header determine the type.
>>>
>>> As its uncommon to set a fault we could as James suggested have a
>>> factory method on Exchange to create a new one:
>>>
>>> Message msg = exchange.createFault();
>>> msg.setBody("Unknown order id " + id);
>>> exchange.setOut(msg);
>>
>> I wonder if this latter line is required? i.e. should
>> exchange.createFault() internally set the OUT?
>>
>> -- 
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://fusesource.com/
>

Re: [DISCUSS] Faults and Exceptions in Camel

[Claus Ibsen <cl...@gmail.com>]

On Thu, Jul 9, 2009 at 7:18 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
> Yeah, exactly, but createFault() confused me a bit.  I don't really se a
> place for it.  I understand these two proposals, what's the third (complete)
> proposal?
>
> 1. the original one which Claus seemed to not like, dunno why.
>   Message getOut();
>   void setOut(Message out);
>   boolean hasFault();
>   void setFault();
> (setFault() does not need a boolean arg, as an fault cannot become an out, I
> think.  setOut() would reset the fault flag)
Is it intended that setFault() do not accept any parameters?



>
> 2.
>   Message getOut();
>   void setOut(Message out);
>   Message getFault();
>   void setFault(Message fault);

Yeah this is nice and simple. Not confusing as OUT and FAULT have
excactly the same set of methods,


And the 3. could be James idea

This is on Message. - This is on Message - This is on Message. (not Exchange)

void setFault(boolean fault)
boolean isFault()

Then we do not need the fault on Exchange at all.



>
> Am I missing something?
>
> Hadrian
>
> On Jul 9, 2009, at 1:07 PM, James Strachan wrote:
>
>> 2009/7/9 Hadrian Zbarcea <hz...@gmail.com>:
>>>
>>> @James, yes it would, but why have createFault()?  Why not have it closer
>>> to
>>> what it is now with
>>>
>>> Message getFault();
>>> void setFault(Message fault);  // instead of create fault
>>> (and then we won't need the setFault(boolean);)
>>
>> As we're musing about having a single property called "out" which may
>> be marked as a fault or not.
>>
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://fusesource.com/
>
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Faults and Exceptions in Camel

[Hadrian Zbarcea <hz...@gmail.com>]

Yeah, exactly, but createFault() confused me a bit.  I don't really se  
a place for it.  I understand these two proposals, what's the third  
(complete) proposal?

1. the original one which Claus seemed to not like, dunno why.
    Message getOut();
    void setOut(Message out);
    boolean hasFault();
    void setFault();
(setFault() does not need a boolean arg, as an fault cannot become an  
out, I think.  setOut() would reset the fault flag)

2.
    Message getOut();
    void setOut(Message out);
    Message getFault();
    void setFault(Message fault);

Am I missing something?

Hadrian

On Jul 9, 2009, at 1:07 PM, James Strachan wrote:

> 2009/7/9 Hadrian Zbarcea <hz...@gmail.com>:
>> @James, yes it would, but why have createFault()?  Why not have it  
>> closer to
>> what it is now with
>>
>> Message getFault();
>> void setFault(Message fault);  // instead of create fault
>> (and then we won't need the setFault(boolean);)
>
> As we're musing about having a single property called "out" which may
> be marked as a fault or not.
>
>
> -- 
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/

Re: [DISCUSS] Faults and Exceptions in Camel

[James Strachan <ja...@gmail.com>]

2009/7/9 Hadrian Zbarcea <hz...@gmail.com>:
> @James, yes it would, but why have createFault()?  Why not have it closer to
> what it is now with
>
> Message getFault();
> void setFault(Message fault);  // instead of create fault
> (and then we won't need the setFault(boolean);)

As we're musing about having a single property called "out" which may
be marked as a fault or not.


-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Faults and Exceptions in Camel

[Hadrian Zbarcea <hz...@gmail.com>]

@James, yes it would, but why have createFault()?  Why not have it  
closer to what it is now with

Message getFault();
void setFault(Message fault);  // instead of create fault
(and then we won't need the setFault(boolean);)

To me it looks pretty clean and simple.

if we set the OUT field with a header or not would be an  
implementation detail.

 From what it looks we may have a solution soon.  Any more thoughts on  
the IN vs OUT question?

$0.02
Hadrian


On Jul 9, 2009, at 12:47 PM, James Strachan wrote:

> 2009/7/9 Claus Ibsen <cl...@gmail.com>:
>> Hi
>>
>> I think it would be confusing if there is a setFault but no getFault?
>> But I understand the reason when you want to use OUT for both regular
>> OUT and FAULT. Where a header determine the type.
>>
>> As its uncommon to set a fault we could as James suggested have a
>> factory method on Exchange to create a new one:
>>
>> Message msg = exchange.createFault();
>> msg.setBody("Unknown order id " + id);
>> exchange.setOut(msg);
>
> I wonder if this latter line is required? i.e. should
> exchange.createFault() internally set the OUT?
>
> -- 
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/

Re: [DISCUSS] Faults and Exceptions in Camel

[James Strachan <ja...@gmail.com>]

2009/7/9 Claus Ibsen <cl...@gmail.com>:
> Hi
>
> I think it would be confusing if there is a setFault but no getFault?
> But I understand the reason when you want to use OUT for both regular
> OUT and FAULT. Where a header determine the type.
>
> As its uncommon to set a fault we could as James suggested have a
> factory method on Exchange to create a new one:
>
> Message msg = exchange.createFault();
> msg.setBody("Unknown order id " + id);
> exchange.setOut(msg);

I wonder if this latter line is required? i.e. should
exchange.createFault() internally set the OUT?

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Faults and Exceptions in Camel

[Claus Ibsen <cl...@gmail.com>]

Hi

I think it would be confusing if there is a setFault but no getFault?
But I understand the reason when you want to use OUT for both regular
OUT and FAULT. Where a header determine the type.

As its uncommon to set a fault we could as James suggested have a
factory method on Exchange to create a new one:

Message msg = exchange.createFault();
msg.setBody("Unknown order id " + id);
exchange.setOut(msg);

This could reduce any get/set fault etc. on the Exchange to only the
factory method.

Then what is left is probably a either to
- keep the hasFault on exchange
- or move it to Message and name it isFault()


Then we only get IN and OUT on the Exchange, and the API is reduced.



On Thu, Jul 9, 2009 at 4:35 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
> My intention with this thread is to try to cleanup the api for 2.0.
> Implementation we can fix in any 2.x.
> Although I agree with having a Fault being stored as an out with a flag,
> it's not very relevant right now for me.
>
> As api changes, if I understand correctly, you are suggesting;
>
>   Message getOut();
>   boolean hasOut();
>   void setOut(Message out);
>
>   boolean hasFault();
>   void setFault(boolean value);
>
> These methods go:
>   Message getOut(boolean lazyCreate);
>   Message getFault();
>   Message getFault(boolean lazyCreate);
>   void removeFault();
>
> The equivalency being:
>
>   Message fault = getFault();
> equivalent to:
>   if (hasFault())
>       Message fault = getOut();
>
> or to set the fault instead of the lazyCreate thing:
>   setOut(message);
>   setFault(true);
>
> removeFault() becomes:
>   setOut(null);
>
> I am perfectly fine with this.  It's the responsibility of the caller to
> check if the out is a fault or not, which may or may not be relevant
> depending on the component.  We just need to clearly document that.
>
> +1
> Hadrian
>
>
>
> On Jul 9, 2009, at 6:06 AM, James Strachan wrote:
>
>> A few random thoughts.
>>
>> So we definitely need a way to determine if the output on an exchange
>> is an OUT, fault, exception. We could have an OUT and a Fault Message;
>> or have a single OUT Message with a boolean fault property.
>>
>> We could store an Exception as a body of the OUT maybe, though I can't
>> help feel that an Exception is a bit different to an OUT/Fault (which
>> are messages). e.g. you might want to clear the exception but keep the
>> OUT?
>>
>> To process the output in a pipeline we could do something like...
>>
>>  Throwable e = exchange.getException();
>>  if (e != null) {
>>   // process the exception
>>  }
>>  else {
>>    // we should now have now valid output
>>   Message out = exchange.getOut();
>>   if (out == null) {
>>     // no output created, so reuse input?
>>     out = exchange.getIn();
>>   }
>>
>>   // we might not care if its a fault or out, we might just wanna
>> return it anyway
>>  if (out.isFault()) {
>>    // do some fault specific coding here...
>>  }
>> }
>>
>> So we could use the OUT for a fault or a regular OUT (as you could say
>> its an output message, whether a fault or real response) so code might
>> not care if its a fault or not? So maybe a flag on OUT is neater?
>>
>> Exceptions seem different though; its typically something you'd wanna
>> look at (and might wanna know what the OUT was when the exception
>> threw), so having that as a property on Exchange feels right to me.
>>
>>
>> Main problem seems to be the lazy create stuff. (Bad James!). Maybe we
>> just need OUT to be null and if someone wants to create an OUT there's
>> a factory method...
>>
>> Message out = exchange.createOut();
>>
>> (it could maybe be smart enough that if there's already an OUT defined
>> it returns that one?). Once someone calls createOut() then the OUT is
>> set and getOut() returns a non null value?
>>
>> Then if its a fault...
>>
>> out.setFault(true);
>>
>>
>> Then if folks call exchange.getOut() it will typically return null and
>> not do any lazy creation or gobble up messages etc as Claus says?
>>
>>
>> 2009/7/9 Claus Ibsen <cl...@gmail.com>:
>>>
>>> On Thu, Jul 9, 2009 at 9:03 AM, Guillaume Nodet<gn...@gmail.com> wrote:
>>>>
>>>> In web services world, faults are not exceptions, but usually an xml
>>>> payload.  In the java world, faults would be like checked exceptions
>>>> and errors runtime exceptions.  However, distinguishing a fault from
>>>> an out message by (instanceof Exception) is imho not sufficient
>>>
>>> Yeah I was about to say the same. I think the FAULT message makes
>>> sense. Fits better with the web service if you have declared faults in
>>> the wsdl.
>>> Then you can construct a XML message and set it as getFault().setBody().
>>>
>>> And I would also think it is much more confusing to set a fault
>>> message, wrapped as an exception on the OUT message as opposed to use
>>> setException on the exchange instead. That would be odd.
>>>
>>> So the API is good, my only griefs are
>>> a) the OUT creating empty messages.
>>> b) and that people might think as if they can during routing processes
>>> piece by piece assemble an OUT message.
>>>
>>> a)
>>> See previous mails about this.
>>>
>>> b)
>>> An example to illustrate this. For instance in the route below A, B, C
>>> is independent steps that enrich a message with order details.
>>> Suppose the input message is an order id. And the expected message is
>>> details about this order.
>>>
>>> from("direct:start").process(A).process(B).process(C);
>>>
>>> from
>>> IN = 123
>>> OUT = null
>>>
>>> A
>>> IN = 123
>>> OUT = Orderid: 123.
>>>
>>> B
>>> IN = 123
>>> OUT = Orderid: 123. Status = IN PROGRESS
>>>
>>> C
>>> IN = 123
>>> OUT = Orderid: 123. Status = IN PROGRESS. ETA: 2009/08/13
>>>
>>> Client receives
>>> OUT: Orderid: 123. Status = IN PROGRESS. ETA: 2009/08/13
>>>
>>>
>>> But the nature of pipes and filter, this is what happens
>>>
>>> from
>>> IN = 123
>>> OUT = null
>>>
>>> A
>>> IN = 123
>>> OUT = Orderid: 123.
>>>
>>> B
>>> IN = Orderid: 123.
>>> OUT = null
>>>
>>> Kabom!!! now we got a partly OUT message as IN instead of the original
>>> IN message.
>>>
>>> This is right according to the pipes and filters, where previous OUT
>>> should be next IN.
>>>
>>>
>>> But people should just be aware with this in relation to IN and OUT -
>>> that the OUT is not something that you can piece by piece assemble.
>>> And OUT is really not that useable.
>>>
>>>
>>>
>>>
>>>>
>>>> On Thu, Jul 9, 2009 at 07:35, Hadrian Zbarcea<hz...@gmail.com> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> Comments inline.
>>>>>
>>>>> Hadrian
>>>>>
>>>>> On Jul 9, 2009, at 12:54 AM, Claus Ibsen wrote:
>>>>>
>>>>>> On Wed, Jul 8, 2009 at 9:52 PM, Hadrian Zbarcea<hz...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> As we approach the 2.0, there is one more hanging issue I would like
>>>>>>> addressed, if possible.  It's the thorny issue of Faults and
>>>>>>> Exceptions
>>>>>>> that
>>>>>>> started in
>>>>>>> http://issues.apache.org/activemq/browse/CAMEL-316 (see also the
>>>>>>> related
>>>>>>> nabble thread linked in the issue description).
>>>>>>>
>>>>>>> I am less concerned about how the DefaultExchange is implemented and
>>>>>>> I
>>>>>>> hope
>>>>>>> to reach an agreement on what the Exchange api should be (please find
>>>>>>> the
>>>>>>> list of Exchange methods below).
>>>>>>>
>>>>>>> As far as faults/exceptions are concerned, Roman thinks that the
>>>>>>> whole
>>>>>>> concept of in/out/fault/exception is artificial, and only one payload
>>>>>>> (message) api should be enough (Roman please correct me if I
>>>>>>> misinterpret
>>>>>>> your position).  My opinion is that we *must* distinguish between
>>>>>>> persistent
>>>>>>> (fault) and transient (exception) errors for the simple reason that
>>>>>>> they
>>>>>>> have different semantics.  As Roman correctly points out, faults are
>>>>>>> more
>>>>>>> like outputs, have more of application level semantics and are
>>>>>>> normally
>>>>>>> handled by the client, where exceptions (transient errors) are
>>>>>>> something
>>>>>>> camel could try to recover from, without much knowledge about the
>>>>>>> application.  I think that the presence of fault in the camel api is
>>>>>>> not
>>>>>>> due
>>>>>>> to it being explicitly modeled by jbi and wsdl, as Roman suggests,
>>>>>>> and
>>>>>>> Camel
>>>>>>> simply copying that, but it's modeled in Camel for the same reason
>>>>>>> it's
>>>>>>> modeled in jbi and wsdl, to differentiate transient from persistent
>>>>>>> errors
>>>>>>> in a non ambiguous way.
>>>>>>
>>>>>> I am one of the persons that would love the Camel Exchange / Message
>>>>>> API to be a bit simpler. It has a fair shares of methods.
>>>>>>
>>>>>> Having listening and discussing with Hadrian on this and doing my own
>>>>>> investigations and whatnot I do belive that Hadrian is absolutely
>>>>>> right when it comes to FAULT. It has a good place in the API.  I am +1
>>>>>> on having FAULT as we do now.
>>>>>>
>>>>>> The grief I have left is that the IN and OUT. It makes sense to have
>>>>>> them and they provide a good value. However they have a big drawnback
>>>>>> in how they are routed in Camel with the Pipeline processor, that
>>>>>> mimics the pipes and filters EIP. And as a result the OUT will be used
>>>>>> as IN
>>>>>> in the next step in the route. So its not like you can steadily build
>>>>>> up an OUT message on-the-fly during many steps in the route path.
>>>>>>
>>>>>> Example
>>>>>> from("direct:start").process(new Processor()).to("log:foo");
>>>>>>
>>>>>> a) From
>>>>>> IN = Hello World
>>>>>> OUT = null
>>>>>>
>>>>>> b) Processor
>>>>>> IN Hello World
>>>>>> OUT = Bye World
>>>>>>
>>>>>> c) Log
>>>>>> IN = Bye World
>>>>>> OUT = null
>>>>>>
>>>>> Yes, from an external observer's perspective, this is precisely what
>>>>> happens.  How we decide to store it, how many fields we need, is an
>>>>> implementation detail of the DefaultExchange.  I don't think the copy
>>>>> from
>>>>> out/in is too expensive, but I would be ok with having only one field
>>>>> to
>>>>> store the current message in DefaultExchange (I assume that's what you
>>>>> propose).  However, my question is about what the api should be.
>>>>>
>>>>>
>>>>>>
>>>>>> And then the getOut() method that lazy creates a new empty OUT message
>>>>>> is also a pita, as it can lead to people loosing their messages if
>>>>>> they do some System out logging of their own
>>>>>>
>>>>>> public void process(Exchange e) {
>>>>>>  System.out.println(exchange.getIn());
>>>>>>  System.out.println(exchange.getOut());
>>>>>>  // boom you lost your message when its routed to next node in route
>>>>>> path, as getOut() created a new empty OUT message that will by used in
>>>>>> the pipes and filters EIP routed with the Pipeline
>>>>>> }
>>>>>>
>>>>>> We had this IN OUT discussion a while back and at that time we ended
>>>>>> up with a compromise of having a hasOut() method so you should do, to
>>>>>> be safe:
>>>>>>  System.out.println(exchange.getIn());
>>>>>>  if (exchange.hasOut()) {
>>>>>>     System.out.println(exchange.getOut());
>>>>>>  }
>>>>>>
>>>>>> Still a pita with the lazy creation IMHO.
>>>>>
>>>>> The lazyCreate methods are actually deprecated, and imho should be
>>>>> removed
>>>>> now.  This would eliminate the confusion.
>>>>>
>>>>>
>>>>>>>
>>>>>>> If we were to go with only get/setMessage() api, we would still need
>>>>>>> methods
>>>>>>> (or some ways) to distinguish between the kind of message we are
>>>>>>> dealing
>>>>>>> with (in/out/fault/exception) so we'd only move the problem somewhere
>>>>>>> else.
>>>>>>>
>>>>>>> So the question becomes if we leave the api the way it is, or we
>>>>>>> replace
>>>>>>> the
>>>>>>> get/setFault apis with get/setOut, in which case we'll need something
>>>>>>> like:
>>>>>>>  boolean isFault();
>>>>>>> method in the Message api or keep the hasFault() method on the
>>>>>>> Exchange.
>>>>>>
>>>>>> Good question
>>>>>>
>>>>>> If you use OUT instead then we need to add a isFault() on the
>>>>>> org.apache.camel.Message API that
>>>>>> the IN message also implements.
>>>>>>
>>>>>> It could make sense to use OUT as well for FAULT.
>>>>>> But how should the API look like to set an OUT as Fault?
>>>>>>
>>>>>> Something a like this?
>>>>>>
>>>>>> getOut().setBody("Unknown bank account number.");
>>>>>> getOut().setFault(true);
>>>>>
>>>>> Not quite, I had something like this in mind (there is no setFault()
>>>>> method):
>>>>> getOut().setBody(java-lang-Exception-derivedObject);
>>>>>
>>>>> boolean isFault() {
>>>>>   return getBody() instanceof Exception;
>>>>> }
>>>>>
>>>>> Personally I am ok with the limitation not being able to have an out of
>>>>> a
>>>>> java.lang.Exception type (that would then be a Fault).  I can't imagine
>>>>> a
>>>>> case where an Exception would be an expected out, and in such cases one
>>>>> could always serialize or wrap it.  The fact that the Fault would be an
>>>>> Exception type would be a camel convention that needs to be followed by
>>>>> all
>>>>> components.
>>>>>
>>>>> Another option would be add  header HAS_FAULT or something like that,
>>>>> in
>>>>> which case both the out and the fault could be of any type.
>>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>> Thoughts?
>>>>>>>
>>>>>>>
>>>>>>>  ExchangePattern getPattern();
>>>>>>>  void setPattern(ExchangePattern pattern);
>>>>>>>
>>>>>>>  Object getProperty(String name);
>>>>>>>  <T> T getProperty(String name, Class<T> type);
>>>>>>>  void setProperty(String name, Object value);
>>>>>>>  Object removeProperty(String name);
>>>>>>>  Map<String, Object> getProperties();
>>>>>>>
>>>>>>>  Message getIn();
>>>>>>>  void setIn(Message in);
>>>>>>>
>>>>>>>  Message getOut();
>>>>>>>  boolean hasOut();
>>>>>>>  Message getOut(boolean lazyCreate);
>>>>>>>  void setOut(Message out);
>>>>>>>
>>>>>>>  Message getFault();
>>>>>>>  boolean hasFault();
>>>>>>>  Message getFault(boolean lazyCreate);
>>>>>>>  void removeFault();
>>>>>>> // removeFault() is only used in one place
>>>>>>
>>>>>> +1 to remove it. You can just do setFault(null) instead. I will fix it
>>>>>> asap.
>>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>  Exception getException();
>>>>>>>  <T> T getException(Class<T> type);
>>>>>>>  void setException(Exception e);
>>>>>>>  boolean isFailed();
>>>>>>>
>>>>>>>  boolean isTransacted();
>>>>>>>  boolean isRollbackOnly();
>>>>>>>
>>>>>>>  CamelContext getContext();
>>>>>>>
>>>>>>>  Exchange newInstance();
>>>>>>>  Exchange copy();
>>>>>>>  Exchange newCopy(boolean handoverOnCompletion);
>>>>>>>  void copyFrom(Exchange source);
>>>>>>>
>>>>>>>  Endpoint getFromEndpoint();
>>>>>>>  void setFromEndpoint(Endpoint fromEndpoint);
>>>>>>>
>>>>>>>  UnitOfWork getUnitOfWork();
>>>>>>>  void setUnitOfWork(UnitOfWork unitOfWork);
>>>>>>>
>>>>>>>  String getExchangeId();
>>>>>>>  void setExchangeId(String id);
>>>>>>>
>>>>>>>  void addOnCompletion(Synchronization onCompletion);
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Claus Ibsen
>>>>>> Apache Camel Committer
>>>>>>
>>>>>> Open Source Integration: http://fusesource.com
>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Cheers,
>>>> Guillaume Nodet
>>>> ------------------------
>>>> Blog: http://gnodet.blogspot.com/
>>>> ------------------------
>>>> Open Source SOA
>>>> http://fusesource.com
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>
>>
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://fusesource.com/
>
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Faults and Exceptions in Camel

[Hadrian Zbarcea <hz...@gmail.com>]

My intention with this thread is to try to cleanup the api for 2.0.    
Implementation we can fix in any 2.x.
Although I agree with having a Fault being stored as an out with a  
flag, it's not very relevant right now for me.

As api changes, if I understand correctly, you are suggesting;

    Message getOut();
    boolean hasOut();
    void setOut(Message out);

    boolean hasFault();
    void setFault(boolean value);

These methods go:
    Message getOut(boolean lazyCreate);
    Message getFault();
    Message getFault(boolean lazyCreate);
    void removeFault();

The equivalency being:

    Message fault = getFault();
equivalent to:
    if (hasFault())
        Message fault = getOut();

or to set the fault instead of the lazyCreate thing:
    setOut(message);
    setFault(true);

removeFault() becomes:
    setOut(null);

I am perfectly fine with this.  It's the responsibility of the caller  
to check if the out is a fault or not, which may or may not be  
relevant depending on the component.  We just need to clearly document  
that.

+1
Hadrian



On Jul 9, 2009, at 6:06 AM, James Strachan wrote:

> A few random thoughts.
>
> So we definitely need a way to determine if the output on an exchange
> is an OUT, fault, exception. We could have an OUT and a Fault Message;
> or have a single OUT Message with a boolean fault property.
>
> We could store an Exception as a body of the OUT maybe, though I can't
> help feel that an Exception is a bit different to an OUT/Fault (which
> are messages). e.g. you might want to clear the exception but keep the
> OUT?
>
> To process the output in a pipeline we could do something like...
>
>  Throwable e = exchange.getException();
>  if (e != null) {
>    // process the exception
>  }
>  else {
>     // we should now have now valid output
>    Message out = exchange.getOut();
>    if (out == null) {
>      // no output created, so reuse input?
>      out = exchange.getIn();
>    }
>
>    // we might not care if its a fault or out, we might just wanna
> return it anyway
>   if (out.isFault()) {
>     // do some fault specific coding here...
>   }
> }
>
> So we could use the OUT for a fault or a regular OUT (as you could say
> its an output message, whether a fault or real response) so code might
> not care if its a fault or not? So maybe a flag on OUT is neater?
>
> Exceptions seem different though; its typically something you'd wanna
> look at (and might wanna know what the OUT was when the exception
> threw), so having that as a property on Exchange feels right to me.
>
>
> Main problem seems to be the lazy create stuff. (Bad James!). Maybe we
> just need OUT to be null and if someone wants to create an OUT there's
> a factory method...
>
> Message out = exchange.createOut();
>
> (it could maybe be smart enough that if there's already an OUT defined
> it returns that one?). Once someone calls createOut() then the OUT is
> set and getOut() returns a non null value?
>
> Then if its a fault...
>
> out.setFault(true);
>
>
> Then if folks call exchange.getOut() it will typically return null and
> not do any lazy creation or gobble up messages etc as Claus says?
>
>
> 2009/7/9 Claus Ibsen <cl...@gmail.com>:
>> On Thu, Jul 9, 2009 at 9:03 AM, Guillaume Nodet<gn...@gmail.com>  
>> wrote:
>>> In web services world, faults are not exceptions, but usually an xml
>>> payload.  In the java world, faults would be like checked exceptions
>>> and errors runtime exceptions.  However, distinguishing a fault from
>>> an out message by (instanceof Exception) is imho not sufficient
>>
>> Yeah I was about to say the same. I think the FAULT message makes
>> sense. Fits better with the web service if you have declared faults  
>> in
>> the wsdl.
>> Then you can construct a XML message and set it as  
>> getFault().setBody().
>>
>> And I would also think it is much more confusing to set a fault
>> message, wrapped as an exception on the OUT message as opposed to use
>> setException on the exchange instead. That would be odd.
>>
>> So the API is good, my only griefs are
>> a) the OUT creating empty messages.
>> b) and that people might think as if they can during routing  
>> processes
>> piece by piece assemble an OUT message.
>>
>> a)
>> See previous mails about this.
>>
>> b)
>> An example to illustrate this. For instance in the route below A,  
>> B, C
>> is independent steps that enrich a message with order details.
>> Suppose the input message is an order id. And the expected message is
>> details about this order.
>>
>> from("direct:start").process(A).process(B).process(C);
>>
>> from
>> IN = 123
>> OUT = null
>>
>> A
>> IN = 123
>> OUT = Orderid: 123.
>>
>> B
>> IN = 123
>> OUT = Orderid: 123. Status = IN PROGRESS
>>
>> C
>> IN = 123
>> OUT = Orderid: 123. Status = IN PROGRESS. ETA: 2009/08/13
>>
>> Client receives
>> OUT: Orderid: 123. Status = IN PROGRESS. ETA: 2009/08/13
>>
>>
>> But the nature of pipes and filter, this is what happens
>>
>> from
>> IN = 123
>> OUT = null
>>
>> A
>> IN = 123
>> OUT = Orderid: 123.
>>
>> B
>> IN = Orderid: 123.
>> OUT = null
>>
>> Kabom!!! now we got a partly OUT message as IN instead of the  
>> original
>> IN message.
>>
>> This is right according to the pipes and filters, where previous OUT
>> should be next IN.
>>
>>
>> But people should just be aware with this in relation to IN and OUT -
>> that the OUT is not something that you can piece by piece assemble.
>> And OUT is really not that useable.
>>
>>
>>
>>
>>>
>>> On Thu, Jul 9, 2009 at 07:35, Hadrian Zbarcea<hz...@gmail.com>  
>>> wrote:
>>>> Hi,
>>>>
>>>> Comments inline.
>>>>
>>>> Hadrian
>>>>
>>>> On Jul 9, 2009, at 12:54 AM, Claus Ibsen wrote:
>>>>
>>>>> On Wed, Jul 8, 2009 at 9:52 PM, Hadrian  
>>>>> Zbarcea<hz...@gmail.com> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> As we approach the 2.0, there is one more hanging issue I would  
>>>>>> like
>>>>>> addressed, if possible.  It's the thorny issue of Faults and  
>>>>>> Exceptions
>>>>>> that
>>>>>> started in
>>>>>> http://issues.apache.org/activemq/browse/CAMEL-316 (see also  
>>>>>> the related
>>>>>> nabble thread linked in the issue description).
>>>>>>
>>>>>> I am less concerned about how the DefaultExchange is  
>>>>>> implemented and I
>>>>>> hope
>>>>>> to reach an agreement on what the Exchange api should be  
>>>>>> (please find the
>>>>>> list of Exchange methods below).
>>>>>>
>>>>>> As far as faults/exceptions are concerned, Roman thinks that  
>>>>>> the whole
>>>>>> concept of in/out/fault/exception is artificial, and only one  
>>>>>> payload
>>>>>> (message) api should be enough (Roman please correct me if I  
>>>>>> misinterpret
>>>>>> your position).  My opinion is that we *must* distinguish between
>>>>>> persistent
>>>>>> (fault) and transient (exception) errors for the simple reason  
>>>>>> that they
>>>>>> have different semantics.  As Roman correctly points out,  
>>>>>> faults are more
>>>>>> like outputs, have more of application level semantics and are  
>>>>>> normally
>>>>>> handled by the client, where exceptions (transient errors) are  
>>>>>> something
>>>>>> camel could try to recover from, without much knowledge about the
>>>>>> application.  I think that the presence of fault in the camel  
>>>>>> api is not
>>>>>> due
>>>>>> to it being explicitly modeled by jbi and wsdl, as Roman  
>>>>>> suggests, and
>>>>>> Camel
>>>>>> simply copying that, but it's modeled in Camel for the same  
>>>>>> reason it's
>>>>>> modeled in jbi and wsdl, to differentiate transient from  
>>>>>> persistent
>>>>>> errors
>>>>>> in a non ambiguous way.
>>>>>
>>>>> I am one of the persons that would love the Camel Exchange /  
>>>>> Message
>>>>> API to be a bit simpler. It has a fair shares of methods.
>>>>>
>>>>> Having listening and discussing with Hadrian on this and doing  
>>>>> my own
>>>>> investigations and whatnot I do belive that Hadrian is absolutely
>>>>> right when it comes to FAULT. It has a good place in the API.  I  
>>>>> am +1
>>>>> on having FAULT as we do now.
>>>>>
>>>>> The grief I have left is that the IN and OUT. It makes sense to  
>>>>> have
>>>>> them and they provide a good value. However they have a big  
>>>>> drawnback
>>>>> in how they are routed in Camel with the Pipeline processor, that
>>>>> mimics the pipes and filters EIP. And as a result the OUT will  
>>>>> be used
>>>>> as IN
>>>>> in the next step in the route. So its not like you can steadily  
>>>>> build
>>>>> up an OUT message on-the-fly during many steps in the route path.
>>>>>
>>>>> Example
>>>>> from("direct:start").process(new Processor()).to("log:foo");
>>>>>
>>>>> a) From
>>>>> IN = Hello World
>>>>> OUT = null
>>>>>
>>>>> b) Processor
>>>>> IN Hello World
>>>>> OUT = Bye World
>>>>>
>>>>> c) Log
>>>>> IN = Bye World
>>>>> OUT = null
>>>>>
>>>> Yes, from an external observer's perspective, this is precisely  
>>>> what
>>>> happens.  How we decide to store it, how many fields we need, is an
>>>> implementation detail of the DefaultExchange.  I don't think the  
>>>> copy from
>>>> out/in is too expensive, but I would be ok with having only one  
>>>> field to
>>>> store the current message in DefaultExchange (I assume that's  
>>>> what you
>>>> propose).  However, my question is about what the api should be.
>>>>
>>>>
>>>>>
>>>>> And then the getOut() method that lazy creates a new empty OUT  
>>>>> message
>>>>> is also a pita, as it can lead to people loosing their messages if
>>>>> they do some System out logging of their own
>>>>>
>>>>> public void process(Exchange e) {
>>>>>  System.out.println(exchange.getIn());
>>>>>  System.out.println(exchange.getOut());
>>>>>  // boom you lost your message when its routed to next node in  
>>>>> route
>>>>> path, as getOut() created a new empty OUT message that will by  
>>>>> used in
>>>>> the pipes and filters EIP routed with the Pipeline
>>>>> }
>>>>>
>>>>> We had this IN OUT discussion a while back and at that time we  
>>>>> ended
>>>>> up with a compromise of having a hasOut() method so you should  
>>>>> do, to
>>>>> be safe:
>>>>>  System.out.println(exchange.getIn());
>>>>>  if (exchange.hasOut()) {
>>>>>      System.out.println(exchange.getOut());
>>>>>  }
>>>>>
>>>>> Still a pita with the lazy creation IMHO.
>>>>
>>>> The lazyCreate methods are actually deprecated, and imho should  
>>>> be removed
>>>> now.  This would eliminate the confusion.
>>>>
>>>>
>>>>>>
>>>>>> If we were to go with only get/setMessage() api, we would still  
>>>>>> need
>>>>>> methods
>>>>>> (or some ways) to distinguish between the kind of message we  
>>>>>> are dealing
>>>>>> with (in/out/fault/exception) so we'd only move the problem  
>>>>>> somewhere
>>>>>> else.
>>>>>>
>>>>>> So the question becomes if we leave the api the way it is, or  
>>>>>> we replace
>>>>>> the
>>>>>> get/setFault apis with get/setOut, in which case we'll need  
>>>>>> something
>>>>>> like:
>>>>>>   boolean isFault();
>>>>>> method in the Message api or keep the hasFault() method on the  
>>>>>> Exchange.
>>>>>
>>>>> Good question
>>>>>
>>>>> If you use OUT instead then we need to add a isFault() on the
>>>>> org.apache.camel.Message API that
>>>>> the IN message also implements.
>>>>>
>>>>> It could make sense to use OUT as well for FAULT.
>>>>> But how should the API look like to set an OUT as Fault?
>>>>>
>>>>> Something a like this?
>>>>>
>>>>> getOut().setBody("Unknown bank account number.");
>>>>> getOut().setFault(true);
>>>>
>>>> Not quite, I had something like this in mind (there is no  
>>>> setFault()
>>>> method):
>>>> getOut().setBody(java-lang-Exception-derivedObject);
>>>>
>>>> boolean isFault() {
>>>>    return getBody() instanceof Exception;
>>>> }
>>>>
>>>> Personally I am ok with the limitation not being able to have an  
>>>> out of a
>>>> java.lang.Exception type (that would then be a Fault).  I can't  
>>>> imagine a
>>>> case where an Exception would be an expected out, and in such  
>>>> cases one
>>>> could always serialize or wrap it.  The fact that the Fault would  
>>>> be an
>>>> Exception type would be a camel convention that needs to be  
>>>> followed by all
>>>> components.
>>>>
>>>> Another option would be add  header HAS_FAULT or something like  
>>>> that, in
>>>> which case both the out and the fault could be of any type.
>>>>
>>>>
>>>>>
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>>
>>>>>>   ExchangePattern getPattern();
>>>>>>   void setPattern(ExchangePattern pattern);
>>>>>>
>>>>>>   Object getProperty(String name);
>>>>>>   <T> T getProperty(String name, Class<T> type);
>>>>>>   void setProperty(String name, Object value);
>>>>>>   Object removeProperty(String name);
>>>>>>   Map<String, Object> getProperties();
>>>>>>
>>>>>>   Message getIn();
>>>>>>   void setIn(Message in);
>>>>>>
>>>>>>   Message getOut();
>>>>>>   boolean hasOut();
>>>>>>   Message getOut(boolean lazyCreate);
>>>>>>   void setOut(Message out);
>>>>>>
>>>>>>   Message getFault();
>>>>>>   boolean hasFault();
>>>>>>   Message getFault(boolean lazyCreate);
>>>>>>   void removeFault();
>>>>>> // removeFault() is only used in one place
>>>>>
>>>>> +1 to remove it. You can just do setFault(null) instead. I will  
>>>>> fix it
>>>>> asap.
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>   Exception getException();
>>>>>>   <T> T getException(Class<T> type);
>>>>>>   void setException(Exception e);
>>>>>>   boolean isFailed();
>>>>>>
>>>>>>   boolean isTransacted();
>>>>>>   boolean isRollbackOnly();
>>>>>>
>>>>>>   CamelContext getContext();
>>>>>>
>>>>>>   Exchange newInstance();
>>>>>>   Exchange copy();
>>>>>>   Exchange newCopy(boolean handoverOnCompletion);
>>>>>>   void copyFrom(Exchange source);
>>>>>>
>>>>>>   Endpoint getFromEndpoint();
>>>>>>   void setFromEndpoint(Endpoint fromEndpoint);
>>>>>>
>>>>>>   UnitOfWork getUnitOfWork();
>>>>>>   void setUnitOfWork(UnitOfWork unitOfWork);
>>>>>>
>>>>>>   String getExchangeId();
>>>>>>   void setExchangeId(String id);
>>>>>>
>>>>>>   void addOnCompletion(Synchronization onCompletion);
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Cheers,
>>> Guillaume Nodet
>>> ------------------------
>>> Blog: http://gnodet.blogspot.com/
>>> ------------------------
>>> Open Source SOA
>>> http://fusesource.com
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>
>
> -- 
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/

Re: [DISCUSS] Faults and Exceptions in Camel

[James Strachan <ja...@gmail.com>]

A few random thoughts.

So we definitely need a way to determine if the output on an exchange
is an OUT, fault, exception. We could have an OUT and a Fault Message;
or have a single OUT Message with a boolean fault property.

We could store an Exception as a body of the OUT maybe, though I can't
help feel that an Exception is a bit different to an OUT/Fault (which
are messages). e.g. you might want to clear the exception but keep the
OUT?

To process the output in a pipeline we could do something like...

  Throwable e = exchange.getException();
  if (e != null) {
    // process the exception
  }
  else {
     // we should now have now valid output
    Message out = exchange.getOut();
    if (out == null) {
      // no output created, so reuse input?
      out = exchange.getIn();
    }

    // we might not care if its a fault or out, we might just wanna
return it anyway
   if (out.isFault()) {
     // do some fault specific coding here...
   }
}

So we could use the OUT for a fault or a regular OUT (as you could say
its an output message, whether a fault or real response) so code might
not care if its a fault or not? So maybe a flag on OUT is neater?

Exceptions seem different though; its typically something you'd wanna
look at (and might wanna know what the OUT was when the exception
threw), so having that as a property on Exchange feels right to me.


Main problem seems to be the lazy create stuff. (Bad James!). Maybe we
just need OUT to be null and if someone wants to create an OUT there's
a factory method...

Message out = exchange.createOut();

(it could maybe be smart enough that if there's already an OUT defined
it returns that one?). Once someone calls createOut() then the OUT is
set and getOut() returns a non null value?

Then if its a fault...

out.setFault(true);


Then if folks call exchange.getOut() it will typically return null and
not do any lazy creation or gobble up messages etc as Claus says?


2009/7/9 Claus Ibsen <cl...@gmail.com>:
> On Thu, Jul 9, 2009 at 9:03 AM, Guillaume Nodet<gn...@gmail.com> wrote:
>> In web services world, faults are not exceptions, but usually an xml
>> payload.  In the java world, faults would be like checked exceptions
>> and errors runtime exceptions.  However, distinguishing a fault from
>> an out message by (instanceof Exception) is imho not sufficient
>
> Yeah I was about to say the same. I think the FAULT message makes
> sense. Fits better with the web service if you have declared faults in
> the wsdl.
> Then you can construct a XML message and set it as getFault().setBody().
>
> And I would also think it is much more confusing to set a fault
> message, wrapped as an exception on the OUT message as opposed to use
> setException on the exchange instead. That would be odd.
>
> So the API is good, my only griefs are
> a) the OUT creating empty messages.
> b) and that people might think as if they can during routing processes
> piece by piece assemble an OUT message.
>
> a)
> See previous mails about this.
>
> b)
> An example to illustrate this. For instance in the route below A, B, C
> is independent steps that enrich a message with order details.
> Suppose the input message is an order id. And the expected message is
> details about this order.
>
> from("direct:start").process(A).process(B).process(C);
>
> from
> IN = 123
> OUT = null
>
> A
> IN = 123
> OUT = Orderid: 123.
>
> B
> IN = 123
> OUT = Orderid: 123. Status = IN PROGRESS
>
> C
> IN = 123
> OUT = Orderid: 123. Status = IN PROGRESS. ETA: 2009/08/13
>
> Client receives
> OUT: Orderid: 123. Status = IN PROGRESS. ETA: 2009/08/13
>
>
> But the nature of pipes and filter, this is what happens
>
> from
> IN = 123
> OUT = null
>
> A
> IN = 123
> OUT = Orderid: 123.
>
> B
> IN = Orderid: 123.
> OUT = null
>
> Kabom!!! now we got a partly OUT message as IN instead of the original
> IN message.
>
> This is right according to the pipes and filters, where previous OUT
> should be next IN.
>
>
> But people should just be aware with this in relation to IN and OUT -
> that the OUT is not something that you can piece by piece assemble.
> And OUT is really not that useable.
>
>
>
>
>>
>> On Thu, Jul 9, 2009 at 07:35, Hadrian Zbarcea<hz...@gmail.com> wrote:
>>> Hi,
>>>
>>> Comments inline.
>>>
>>> Hadrian
>>>
>>> On Jul 9, 2009, at 12:54 AM, Claus Ibsen wrote:
>>>
>>>> On Wed, Jul 8, 2009 at 9:52 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> As we approach the 2.0, there is one more hanging issue I would like
>>>>> addressed, if possible.  It's the thorny issue of Faults and Exceptions
>>>>> that
>>>>> started in
>>>>> http://issues.apache.org/activemq/browse/CAMEL-316 (see also the related
>>>>> nabble thread linked in the issue description).
>>>>>
>>>>> I am less concerned about how the DefaultExchange is implemented and I
>>>>> hope
>>>>> to reach an agreement on what the Exchange api should be (please find the
>>>>> list of Exchange methods below).
>>>>>
>>>>> As far as faults/exceptions are concerned, Roman thinks that the whole
>>>>> concept of in/out/fault/exception is artificial, and only one payload
>>>>> (message) api should be enough (Roman please correct me if I misinterpret
>>>>> your position).  My opinion is that we *must* distinguish between
>>>>> persistent
>>>>> (fault) and transient (exception) errors for the simple reason that they
>>>>> have different semantics.  As Roman correctly points out, faults are more
>>>>> like outputs, have more of application level semantics and are normally
>>>>> handled by the client, where exceptions (transient errors) are something
>>>>> camel could try to recover from, without much knowledge about the
>>>>> application.  I think that the presence of fault in the camel api is not
>>>>> due
>>>>> to it being explicitly modeled by jbi and wsdl, as Roman suggests, and
>>>>> Camel
>>>>> simply copying that, but it's modeled in Camel for the same reason it's
>>>>> modeled in jbi and wsdl, to differentiate transient from persistent
>>>>> errors
>>>>> in a non ambiguous way.
>>>>
>>>> I am one of the persons that would love the Camel Exchange / Message
>>>> API to be a bit simpler. It has a fair shares of methods.
>>>>
>>>> Having listening and discussing with Hadrian on this and doing my own
>>>> investigations and whatnot I do belive that Hadrian is absolutely
>>>> right when it comes to FAULT. It has a good place in the API.  I am +1
>>>> on having FAULT as we do now.
>>>>
>>>> The grief I have left is that the IN and OUT. It makes sense to have
>>>> them and they provide a good value. However they have a big drawnback
>>>> in how they are routed in Camel with the Pipeline processor, that
>>>> mimics the pipes and filters EIP. And as a result the OUT will be used
>>>> as IN
>>>> in the next step in the route. So its not like you can steadily build
>>>> up an OUT message on-the-fly during many steps in the route path.
>>>>
>>>> Example
>>>> from("direct:start").process(new Processor()).to("log:foo");
>>>>
>>>> a) From
>>>> IN = Hello World
>>>> OUT = null
>>>>
>>>> b) Processor
>>>> IN Hello World
>>>> OUT = Bye World
>>>>
>>>> c) Log
>>>> IN = Bye World
>>>> OUT = null
>>>>
>>> Yes, from an external observer's perspective, this is precisely what
>>> happens.  How we decide to store it, how many fields we need, is an
>>> implementation detail of the DefaultExchange.  I don't think the copy from
>>> out/in is too expensive, but I would be ok with having only one field to
>>> store the current message in DefaultExchange (I assume that's what you
>>> propose).  However, my question is about what the api should be.
>>>
>>>
>>>>
>>>> And then the getOut() method that lazy creates a new empty OUT message
>>>> is also a pita, as it can lead to people loosing their messages if
>>>> they do some System out logging of their own
>>>>
>>>> public void process(Exchange e) {
>>>>  System.out.println(exchange.getIn());
>>>>  System.out.println(exchange.getOut());
>>>>  // boom you lost your message when its routed to next node in route
>>>> path, as getOut() created a new empty OUT message that will by used in
>>>> the pipes and filters EIP routed with the Pipeline
>>>> }
>>>>
>>>> We had this IN OUT discussion a while back and at that time we ended
>>>> up with a compromise of having a hasOut() method so you should do, to
>>>> be safe:
>>>>  System.out.println(exchange.getIn());
>>>>  if (exchange.hasOut()) {
>>>>      System.out.println(exchange.getOut());
>>>>  }
>>>>
>>>> Still a pita with the lazy creation IMHO.
>>>
>>> The lazyCreate methods are actually deprecated, and imho should be removed
>>> now.  This would eliminate the confusion.
>>>
>>>
>>>>>
>>>>> If we were to go with only get/setMessage() api, we would still need
>>>>> methods
>>>>> (or some ways) to distinguish between the kind of message we are dealing
>>>>> with (in/out/fault/exception) so we'd only move the problem somewhere
>>>>> else.
>>>>>
>>>>> So the question becomes if we leave the api the way it is, or we replace
>>>>> the
>>>>> get/setFault apis with get/setOut, in which case we'll need something
>>>>> like:
>>>>>   boolean isFault();
>>>>> method in the Message api or keep the hasFault() method on the Exchange.
>>>>
>>>> Good question
>>>>
>>>> If you use OUT instead then we need to add a isFault() on the
>>>> org.apache.camel.Message API that
>>>> the IN message also implements.
>>>>
>>>> It could make sense to use OUT as well for FAULT.
>>>> But how should the API look like to set an OUT as Fault?
>>>>
>>>> Something a like this?
>>>>
>>>> getOut().setBody("Unknown bank account number.");
>>>> getOut().setFault(true);
>>>
>>> Not quite, I had something like this in mind (there is no setFault()
>>> method):
>>> getOut().setBody(java-lang-Exception-derivedObject);
>>>
>>> boolean isFault() {
>>>    return getBody() instanceof Exception;
>>> }
>>>
>>> Personally I am ok with the limitation not being able to have an out of a
>>> java.lang.Exception type (that would then be a Fault).  I can't imagine a
>>> case where an Exception would be an expected out, and in such cases one
>>> could always serialize or wrap it.  The fact that the Fault would be an
>>> Exception type would be a camel convention that needs to be followed by all
>>> components.
>>>
>>> Another option would be add  header HAS_FAULT or something like that, in
>>> which case both the out and the fault could be of any type.
>>>
>>>
>>>>
>>>>>
>>>>> Thoughts?
>>>>>
>>>>>
>>>>>   ExchangePattern getPattern();
>>>>>   void setPattern(ExchangePattern pattern);
>>>>>
>>>>>   Object getProperty(String name);
>>>>>   <T> T getProperty(String name, Class<T> type);
>>>>>   void setProperty(String name, Object value);
>>>>>   Object removeProperty(String name);
>>>>>   Map<String, Object> getProperties();
>>>>>
>>>>>   Message getIn();
>>>>>   void setIn(Message in);
>>>>>
>>>>>   Message getOut();
>>>>>   boolean hasOut();
>>>>>   Message getOut(boolean lazyCreate);
>>>>>   void setOut(Message out);
>>>>>
>>>>>   Message getFault();
>>>>>   boolean hasFault();
>>>>>   Message getFault(boolean lazyCreate);
>>>>>   void removeFault();
>>>>> // removeFault() is only used in one place
>>>>
>>>> +1 to remove it. You can just do setFault(null) instead. I will fix it
>>>> asap.
>>>>
>>>>
>>>>
>>>>>
>>>>>   Exception getException();
>>>>>   <T> T getException(Class<T> type);
>>>>>   void setException(Exception e);
>>>>>   boolean isFailed();
>>>>>
>>>>>   boolean isTransacted();
>>>>>   boolean isRollbackOnly();
>>>>>
>>>>>   CamelContext getContext();
>>>>>
>>>>>   Exchange newInstance();
>>>>>   Exchange copy();
>>>>>   Exchange newCopy(boolean handoverOnCompletion);
>>>>>   void copyFrom(Exchange source);
>>>>>
>>>>>   Endpoint getFromEndpoint();
>>>>>   void setFromEndpoint(Endpoint fromEndpoint);
>>>>>
>>>>>   UnitOfWork getUnitOfWork();
>>>>>   void setUnitOfWork(UnitOfWork unitOfWork);
>>>>>
>>>>>   String getExchangeId();
>>>>>   void setExchangeId(String id);
>>>>>
>>>>>   void addOnCompletion(Synchronization onCompletion);
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>
>>>
>>
>>
>>
>> --
>> Cheers,
>> Guillaume Nodet
>> ------------------------
>> Blog: http://gnodet.blogspot.com/
>> ------------------------
>> Open Source SOA
>> http://fusesource.com
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>



-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Faults and Exceptions in Camel

[Willem Jiang <wi...@gmail.com>]

Can we store the exception into the Fault Message body?

For the Out Message, I think because we have the InOnly and InOut 
message exchange pattern endpoints, when these two kind of message 
exchange pattern endpoint meet with the pipeline , Camel have to do some 
judgment to know what is the right message which need to pass to next 
processor.

If we know that, it will be easy to understand.

Willem

Claus Ibsen wrote:
> On Thu, Jul 9, 2009 at 9:03 AM, Guillaume Nodet<gn...@gmail.com> wrote:
>> In web services world, faults are not exceptions, but usually an xml
>> payload.  In the java world, faults would be like checked exceptions
>> and errors runtime exceptions.  However, distinguishing a fault from
>> an out message by (instanceof Exception) is imho not sufficient
> 
> Yeah I was about to say the same. I think the FAULT message makes
> sense. Fits better with the web service if you have declared faults in
> the wsdl.
> Then you can construct a XML message and set it as getFault().setBody().
> 
> And I would also think it is much more confusing to set a fault
> message, wrapped as an exception on the OUT message as opposed to use
> setException on the exchange instead. That would be odd.
> 
> So the API is good, my only griefs are
> a) the OUT creating empty messages.
> b) and that people might think as if they can during routing processes
> piece by piece assemble an OUT message.
> 
> a)
> See previous mails about this.
> 
> b)
> An example to illustrate this. For instance in the route below A, B, C
> is independent steps that enrich a message with order details.
> Suppose the input message is an order id. And the expected message is
> details about this order.
> 
> from("direct:start").process(A).process(B).process(C);
> 
> from
> IN = 123
> OUT = null
> 
> A
> IN = 123
> OUT = Orderid: 123.
> 
> B
> IN = 123
> OUT = Orderid: 123. Status = IN PROGRESS
> 
> C
> IN = 123
> OUT = Orderid: 123. Status = IN PROGRESS. ETA: 2009/08/13
> 
> Client receives
> OUT: Orderid: 123. Status = IN PROGRESS. ETA: 2009/08/13
> 
> 
> But the nature of pipes and filter, this is what happens
> 
> from
> IN = 123
> OUT = null
> 
> A
> IN = 123
> OUT = Orderid: 123.
> 
> B
> IN = Orderid: 123.
> OUT = null
> 
> Kabom!!! now we got a partly OUT message as IN instead of the original
> IN message.
> 
> This is right according to the pipes and filters, where previous OUT
> should be next IN.
> 
> 
> But people should just be aware with this in relation to IN and OUT -
> that the OUT is not something that you can piece by piece assemble.
> And OUT is really not that useable.
> 
> 
> 
> 
>> On Thu, Jul 9, 2009 at 07:35, Hadrian Zbarcea<hz...@gmail.com> wrote:
>>> Hi,
>>>
>>> Comments inline.
>>>
>>> Hadrian
>>>
>>> On Jul 9, 2009, at 12:54 AM, Claus Ibsen wrote:
>>>
>>>> On Wed, Jul 8, 2009 at 9:52 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> As we approach the 2.0, there is one more hanging issue I would like
>>>>> addressed, if possible.  It's the thorny issue of Faults and Exceptions
>>>>> that
>>>>> started in
>>>>> http://issues.apache.org/activemq/browse/CAMEL-316 (see also the related
>>>>> nabble thread linked in the issue description).
>>>>>
>>>>> I am less concerned about how the DefaultExchange is implemented and I
>>>>> hope
>>>>> to reach an agreement on what the Exchange api should be (please find the
>>>>> list of Exchange methods below).
>>>>>
>>>>> As far as faults/exceptions are concerned, Roman thinks that the whole
>>>>> concept of in/out/fault/exception is artificial, and only one payload
>>>>> (message) api should be enough (Roman please correct me if I misinterpret
>>>>> your position).  My opinion is that we *must* distinguish between
>>>>> persistent
>>>>> (fault) and transient (exception) errors for the simple reason that they
>>>>> have different semantics.  As Roman correctly points out, faults are more
>>>>> like outputs, have more of application level semantics and are normally
>>>>> handled by the client, where exceptions (transient errors) are something
>>>>> camel could try to recover from, without much knowledge about the
>>>>> application.  I think that the presence of fault in the camel api is not
>>>>> due
>>>>> to it being explicitly modeled by jbi and wsdl, as Roman suggests, and
>>>>> Camel
>>>>> simply copying that, but it's modeled in Camel for the same reason it's
>>>>> modeled in jbi and wsdl, to differentiate transient from persistent
>>>>> errors
>>>>> in a non ambiguous way.
>>>> I am one of the persons that would love the Camel Exchange / Message
>>>> API to be a bit simpler. It has a fair shares of methods.
>>>>
>>>> Having listening and discussing with Hadrian on this and doing my own
>>>> investigations and whatnot I do belive that Hadrian is absolutely
>>>> right when it comes to FAULT. It has a good place in the API.  I am +1
>>>> on having FAULT as we do now.
>>>>
>>>> The grief I have left is that the IN and OUT. It makes sense to have
>>>> them and they provide a good value. However they have a big drawnback
>>>> in how they are routed in Camel with the Pipeline processor, that
>>>> mimics the pipes and filters EIP. And as a result the OUT will be used
>>>> as IN
>>>> in the next step in the route. So its not like you can steadily build
>>>> up an OUT message on-the-fly during many steps in the route path.
>>>>
>>>> Example
>>>> from("direct:start").process(new Processor()).to("log:foo");
>>>>
>>>> a) From
>>>> IN = Hello World
>>>> OUT = null
>>>>
>>>> b) Processor
>>>> IN Hello World
>>>> OUT = Bye World
>>>>
>>>> c) Log
>>>> IN = Bye World
>>>> OUT = null
>>>>
>>> Yes, from an external observer's perspective, this is precisely what
>>> happens.  How we decide to store it, how many fields we need, is an
>>> implementation detail of the DefaultExchange.  I don't think the copy from
>>> out/in is too expensive, but I would be ok with having only one field to
>>> store the current message in DefaultExchange (I assume that's what you
>>> propose).  However, my question is about what the api should be.
>>>
>>>
>>>> And then the getOut() method that lazy creates a new empty OUT message
>>>> is also a pita, as it can lead to people loosing their messages if
>>>> they do some System out logging of their own
>>>>
>>>> public void process(Exchange e) {
>>>>  System.out.println(exchange.getIn());
>>>>  System.out.println(exchange.getOut());
>>>>  // boom you lost your message when its routed to next node in route
>>>> path, as getOut() created a new empty OUT message that will by used in
>>>> the pipes and filters EIP routed with the Pipeline
>>>> }
>>>>
>>>> We had this IN OUT discussion a while back and at that time we ended
>>>> up with a compromise of having a hasOut() method so you should do, to
>>>> be safe:
>>>>  System.out.println(exchange.getIn());
>>>>  if (exchange.hasOut()) {
>>>>      System.out.println(exchange.getOut());
>>>>  }
>>>>
>>>> Still a pita with the lazy creation IMHO.
>>> The lazyCreate methods are actually deprecated, and imho should be removed
>>> now.  This would eliminate the confusion.
>>>
>>>
>>>>> If we were to go with only get/setMessage() api, we would still need
>>>>> methods
>>>>> (or some ways) to distinguish between the kind of message we are dealing
>>>>> with (in/out/fault/exception) so we'd only move the problem somewhere
>>>>> else.
>>>>>
>>>>> So the question becomes if we leave the api the way it is, or we replace
>>>>> the
>>>>> get/setFault apis with get/setOut, in which case we'll need something
>>>>> like:
>>>>>   boolean isFault();
>>>>> method in the Message api or keep the hasFault() method on the Exchange.
>>>> Good question
>>>>
>>>> If you use OUT instead then we need to add a isFault() on the
>>>> org.apache.camel.Message API that
>>>> the IN message also implements.
>>>>
>>>> It could make sense to use OUT as well for FAULT.
>>>> But how should the API look like to set an OUT as Fault?
>>>>
>>>> Something a like this?
>>>>
>>>> getOut().setBody("Unknown bank account number.");
>>>> getOut().setFault(true);
>>> Not quite, I had something like this in mind (there is no setFault()
>>> method):
>>> getOut().setBody(java-lang-Exception-derivedObject);
>>>
>>> boolean isFault() {
>>>    return getBody() instanceof Exception;
>>> }
>>>
>>> Personally I am ok with the limitation not being able to have an out of a
>>> java.lang.Exception type (that would then be a Fault).  I can't imagine a
>>> case where an Exception would be an expected out, and in such cases one
>>> could always serialize or wrap it.  The fact that the Fault would be an
>>> Exception type would be a camel convention that needs to be followed by all
>>> components.
>>>
>>> Another option would be add  header HAS_FAULT or something like that, in
>>> which case both the out and the fault could be of any type.
>>>
>>>
>>>>> Thoughts?
>>>>>
>>>>>
>>>>>   ExchangePattern getPattern();
>>>>>   void setPattern(ExchangePattern pattern);
>>>>>
>>>>>   Object getProperty(String name);
>>>>>   <T> T getProperty(String name, Class<T> type);
>>>>>   void setProperty(String name, Object value);
>>>>>   Object removeProperty(String name);
>>>>>   Map<String, Object> getProperties();
>>>>>
>>>>>   Message getIn();
>>>>>   void setIn(Message in);
>>>>>
>>>>>   Message getOut();
>>>>>   boolean hasOut();
>>>>>   Message getOut(boolean lazyCreate);
>>>>>   void setOut(Message out);
>>>>>
>>>>>   Message getFault();
>>>>>   boolean hasFault();
>>>>>   Message getFault(boolean lazyCreate);
>>>>>   void removeFault();
>>>>> // removeFault() is only used in one place
>>>> +1 to remove it. You can just do setFault(null) instead. I will fix it
>>>> asap.
>>>>
>>>>
>>>>
>>>>>   Exception getException();
>>>>>   <T> T getException(Class<T> type);
>>>>>   void setException(Exception e);
>>>>>   boolean isFailed();
>>>>>
>>>>>   boolean isTransacted();
>>>>>   boolean isRollbackOnly();
>>>>>
>>>>>   CamelContext getContext();
>>>>>
>>>>>   Exchange newInstance();
>>>>>   Exchange copy();
>>>>>   Exchange newCopy(boolean handoverOnCompletion);
>>>>>   void copyFrom(Exchange source);
>>>>>
>>>>>   Endpoint getFromEndpoint();
>>>>>   void setFromEndpoint(Endpoint fromEndpoint);
>>>>>
>>>>>   UnitOfWork getUnitOfWork();
>>>>>   void setUnitOfWork(UnitOfWork unitOfWork);
>>>>>
>>>>>   String getExchangeId();
>>>>>   void setExchangeId(String id);
>>>>>
>>>>>   void addOnCompletion(Synchronization onCompletion);
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>
>>
>>
>> --
>> Cheers,
>> Guillaume Nodet
>> ------------------------
>> Blog: http://gnodet.blogspot.com/
>> ------------------------
>> Open Source SOA
>> http://fusesource.com
>>
> 
> 
> 

Re: [DISCUSS] Faults and Exceptions in Camel

[Claus Ibsen <cl...@gmail.com>]

On Thu, Jul 9, 2009 at 9:03 AM, Guillaume Nodet<gn...@gmail.com> wrote:
> In web services world, faults are not exceptions, but usually an xml
> payload.  In the java world, faults would be like checked exceptions
> and errors runtime exceptions.  However, distinguishing a fault from
> an out message by (instanceof Exception) is imho not sufficient

Yeah I was about to say the same. I think the FAULT message makes
sense. Fits better with the web service if you have declared faults in
the wsdl.
Then you can construct a XML message and set it as getFault().setBody().

And I would also think it is much more confusing to set a fault
message, wrapped as an exception on the OUT message as opposed to use
setException on the exchange instead. That would be odd.

So the API is good, my only griefs are
a) the OUT creating empty messages.
b) and that people might think as if they can during routing processes
piece by piece assemble an OUT message.

a)
See previous mails about this.

b)
An example to illustrate this. For instance in the route below A, B, C
is independent steps that enrich a message with order details.
Suppose the input message is an order id. And the expected message is
details about this order.

from("direct:start").process(A).process(B).process(C);

from
IN = 123
OUT = null

A
IN = 123
OUT = Orderid: 123.

B
IN = 123
OUT = Orderid: 123. Status = IN PROGRESS

C
IN = 123
OUT = Orderid: 123. Status = IN PROGRESS. ETA: 2009/08/13

Client receives
OUT: Orderid: 123. Status = IN PROGRESS. ETA: 2009/08/13


But the nature of pipes and filter, this is what happens

from
IN = 123
OUT = null

A
IN = 123
OUT = Orderid: 123.

B
IN = Orderid: 123.
OUT = null

Kabom!!! now we got a partly OUT message as IN instead of the original
IN message.

This is right according to the pipes and filters, where previous OUT
should be next IN.


But people should just be aware with this in relation to IN and OUT -
that the OUT is not something that you can piece by piece assemble.
And OUT is really not that useable.




>
> On Thu, Jul 9, 2009 at 07:35, Hadrian Zbarcea<hz...@gmail.com> wrote:
>> Hi,
>>
>> Comments inline.
>>
>> Hadrian
>>
>> On Jul 9, 2009, at 12:54 AM, Claus Ibsen wrote:
>>
>>> On Wed, Jul 8, 2009 at 9:52 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> As we approach the 2.0, there is one more hanging issue I would like
>>>> addressed, if possible.  It's the thorny issue of Faults and Exceptions
>>>> that
>>>> started in
>>>> http://issues.apache.org/activemq/browse/CAMEL-316 (see also the related
>>>> nabble thread linked in the issue description).
>>>>
>>>> I am less concerned about how the DefaultExchange is implemented and I
>>>> hope
>>>> to reach an agreement on what the Exchange api should be (please find the
>>>> list of Exchange methods below).
>>>>
>>>> As far as faults/exceptions are concerned, Roman thinks that the whole
>>>> concept of in/out/fault/exception is artificial, and only one payload
>>>> (message) api should be enough (Roman please correct me if I misinterpret
>>>> your position).  My opinion is that we *must* distinguish between
>>>> persistent
>>>> (fault) and transient (exception) errors for the simple reason that they
>>>> have different semantics.  As Roman correctly points out, faults are more
>>>> like outputs, have more of application level semantics and are normally
>>>> handled by the client, where exceptions (transient errors) are something
>>>> camel could try to recover from, without much knowledge about the
>>>> application.  I think that the presence of fault in the camel api is not
>>>> due
>>>> to it being explicitly modeled by jbi and wsdl, as Roman suggests, and
>>>> Camel
>>>> simply copying that, but it's modeled in Camel for the same reason it's
>>>> modeled in jbi and wsdl, to differentiate transient from persistent
>>>> errors
>>>> in a non ambiguous way.
>>>
>>> I am one of the persons that would love the Camel Exchange / Message
>>> API to be a bit simpler. It has a fair shares of methods.
>>>
>>> Having listening and discussing with Hadrian on this and doing my own
>>> investigations and whatnot I do belive that Hadrian is absolutely
>>> right when it comes to FAULT. It has a good place in the API.  I am +1
>>> on having FAULT as we do now.
>>>
>>> The grief I have left is that the IN and OUT. It makes sense to have
>>> them and they provide a good value. However they have a big drawnback
>>> in how they are routed in Camel with the Pipeline processor, that
>>> mimics the pipes and filters EIP. And as a result the OUT will be used
>>> as IN
>>> in the next step in the route. So its not like you can steadily build
>>> up an OUT message on-the-fly during many steps in the route path.
>>>
>>> Example
>>> from("direct:start").process(new Processor()).to("log:foo");
>>>
>>> a) From
>>> IN = Hello World
>>> OUT = null
>>>
>>> b) Processor
>>> IN Hello World
>>> OUT = Bye World
>>>
>>> c) Log
>>> IN = Bye World
>>> OUT = null
>>>
>> Yes, from an external observer's perspective, this is precisely what
>> happens.  How we decide to store it, how many fields we need, is an
>> implementation detail of the DefaultExchange.  I don't think the copy from
>> out/in is too expensive, but I would be ok with having only one field to
>> store the current message in DefaultExchange (I assume that's what you
>> propose).  However, my question is about what the api should be.
>>
>>
>>>
>>> And then the getOut() method that lazy creates a new empty OUT message
>>> is also a pita, as it can lead to people loosing their messages if
>>> they do some System out logging of their own
>>>
>>> public void process(Exchange e) {
>>>  System.out.println(exchange.getIn());
>>>  System.out.println(exchange.getOut());
>>>  // boom you lost your message when its routed to next node in route
>>> path, as getOut() created a new empty OUT message that will by used in
>>> the pipes and filters EIP routed with the Pipeline
>>> }
>>>
>>> We had this IN OUT discussion a while back and at that time we ended
>>> up with a compromise of having a hasOut() method so you should do, to
>>> be safe:
>>>  System.out.println(exchange.getIn());
>>>  if (exchange.hasOut()) {
>>>      System.out.println(exchange.getOut());
>>>  }
>>>
>>> Still a pita with the lazy creation IMHO.
>>
>> The lazyCreate methods are actually deprecated, and imho should be removed
>> now.  This would eliminate the confusion.
>>
>>
>>>>
>>>> If we were to go with only get/setMessage() api, we would still need
>>>> methods
>>>> (or some ways) to distinguish between the kind of message we are dealing
>>>> with (in/out/fault/exception) so we'd only move the problem somewhere
>>>> else.
>>>>
>>>> So the question becomes if we leave the api the way it is, or we replace
>>>> the
>>>> get/setFault apis with get/setOut, in which case we'll need something
>>>> like:
>>>>   boolean isFault();
>>>> method in the Message api or keep the hasFault() method on the Exchange.
>>>
>>> Good question
>>>
>>> If you use OUT instead then we need to add a isFault() on the
>>> org.apache.camel.Message API that
>>> the IN message also implements.
>>>
>>> It could make sense to use OUT as well for FAULT.
>>> But how should the API look like to set an OUT as Fault?
>>>
>>> Something a like this?
>>>
>>> getOut().setBody("Unknown bank account number.");
>>> getOut().setFault(true);
>>
>> Not quite, I had something like this in mind (there is no setFault()
>> method):
>> getOut().setBody(java-lang-Exception-derivedObject);
>>
>> boolean isFault() {
>>    return getBody() instanceof Exception;
>> }
>>
>> Personally I am ok with the limitation not being able to have an out of a
>> java.lang.Exception type (that would then be a Fault).  I can't imagine a
>> case where an Exception would be an expected out, and in such cases one
>> could always serialize or wrap it.  The fact that the Fault would be an
>> Exception type would be a camel convention that needs to be followed by all
>> components.
>>
>> Another option would be add  header HAS_FAULT or something like that, in
>> which case both the out and the fault could be of any type.
>>
>>
>>>
>>>>
>>>> Thoughts?
>>>>
>>>>
>>>>   ExchangePattern getPattern();
>>>>   void setPattern(ExchangePattern pattern);
>>>>
>>>>   Object getProperty(String name);
>>>>   <T> T getProperty(String name, Class<T> type);
>>>>   void setProperty(String name, Object value);
>>>>   Object removeProperty(String name);
>>>>   Map<String, Object> getProperties();
>>>>
>>>>   Message getIn();
>>>>   void setIn(Message in);
>>>>
>>>>   Message getOut();
>>>>   boolean hasOut();
>>>>   Message getOut(boolean lazyCreate);
>>>>   void setOut(Message out);
>>>>
>>>>   Message getFault();
>>>>   boolean hasFault();
>>>>   Message getFault(boolean lazyCreate);
>>>>   void removeFault();
>>>> // removeFault() is only used in one place
>>>
>>> +1 to remove it. You can just do setFault(null) instead. I will fix it
>>> asap.
>>>
>>>
>>>
>>>>
>>>>   Exception getException();
>>>>   <T> T getException(Class<T> type);
>>>>   void setException(Exception e);
>>>>   boolean isFailed();
>>>>
>>>>   boolean isTransacted();
>>>>   boolean isRollbackOnly();
>>>>
>>>>   CamelContext getContext();
>>>>
>>>>   Exchange newInstance();
>>>>   Exchange copy();
>>>>   Exchange newCopy(boolean handoverOnCompletion);
>>>>   void copyFrom(Exchange source);
>>>>
>>>>   Endpoint getFromEndpoint();
>>>>   void setFromEndpoint(Endpoint fromEndpoint);
>>>>
>>>>   UnitOfWork getUnitOfWork();
>>>>   void setUnitOfWork(UnitOfWork unitOfWork);
>>>>
>>>>   String getExchangeId();
>>>>   void setExchangeId(String id);
>>>>
>>>>   void addOnCompletion(Synchronization onCompletion);
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>
>>
>
>
>
> --
> Cheers,
> Guillaume Nodet
> ------------------------
> Blog: http://gnodet.blogspot.com/
> ------------------------
> Open Source SOA
> http://fusesource.com
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Faults and Exceptions in Camel

[Hadrian Zbarcea <hz...@gmail.com>]

+1
Hadrian

On Jul 9, 2009, at 12:04 PM, James Strachan wrote:

> 2009/7/9 Hadrian Zbarcea <hz...@gmail.com>:
>> Guillaume, you are obviously right, with the comment that it's xml  
>> or some
>> equivalent.  Internally to a framework, it could be whatever.  Some  
>> projects
>> that use code generation from wsdl to create Exception derived  
>> types for
>> wsdl faults.  I am not really an advocate of it, but I would be ok  
>> with it,
>> uh, maybe.  Something like (message instanceof CamelFaultException).
>
> Its type could be Document, String or byte[] though. I think a flag is
> way better.
> -- 
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/

Re: [DISCUSS] Faults and Exceptions in Camel

[James Strachan <ja...@gmail.com>]

2009/7/9 Hadrian Zbarcea <hz...@gmail.com>:
> Guillaume, you are obviously right, with the comment that it's xml or some
> equivalent.  Internally to a framework, it could be whatever.  Some projects
> that use code generation from wsdl to create Exception derived types for
> wsdl faults.  I am not really an advocate of it, but I would be ok with it,
> uh, maybe.  Something like (message instanceof CamelFaultException).

Its type could be Document, String or byte[] though. I think a flag is
way better.
-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Faults and Exceptions in Camel

[Hadrian Zbarcea <hz...@gmail.com>]

Guillaume, you are obviously right, with the comment that it's xml or  
some equivalent.  Internally to a framework, it could be whatever.   
Some projects that use code generation from wsdl to create Exception  
derived types for wsdl faults.  I am not really an advocate of it, but  
I would be ok with it, uh, maybe.  Something like (message instanceof  
CamelFaultException).

But then again, I am not sure that's the best solution.  Leaving it as  
a normal Message with a CAMEL_FAULT header is probably better.

Thanks
Hadrian



On Jul 9, 2009, at 3:03 AM, Guillaume Nodet wrote:

> In web services world, faults are not exceptions, but usually an xml
> payload.  In the java world, faults would be like checked exceptions
> and errors runtime exceptions.  However, distinguishing a fault from
> an out message by (instanceof Exception) is imho not sufficient
>
> On Thu, Jul 9, 2009 at 07:35, Hadrian Zbarcea<hz...@gmail.com>  
> wrote:
>> Hi,
>>
>> Comments inline.
>>
>> Hadrian
>>
>> On Jul 9, 2009, at 12:54 AM, Claus Ibsen wrote:
>>
>>> On Wed, Jul 8, 2009 at 9:52 PM, Hadrian  
>>> Zbarcea<hz...@gmail.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> As we approach the 2.0, there is one more hanging issue I would  
>>>> like
>>>> addressed, if possible.  It's the thorny issue of Faults and  
>>>> Exceptions
>>>> that
>>>> started in
>>>> http://issues.apache.org/activemq/browse/CAMEL-316 (see also the  
>>>> related
>>>> nabble thread linked in the issue description).
>>>>
>>>> I am less concerned about how the DefaultExchange is implemented  
>>>> and I
>>>> hope
>>>> to reach an agreement on what the Exchange api should be (please  
>>>> find the
>>>> list of Exchange methods below).
>>>>
>>>> As far as faults/exceptions are concerned, Roman thinks that the  
>>>> whole
>>>> concept of in/out/fault/exception is artificial, and only one  
>>>> payload
>>>> (message) api should be enough (Roman please correct me if I  
>>>> misinterpret
>>>> your position).  My opinion is that we *must* distinguish between
>>>> persistent
>>>> (fault) and transient (exception) errors for the simple reason  
>>>> that they
>>>> have different semantics.  As Roman correctly points out, faults  
>>>> are more
>>>> like outputs, have more of application level semantics and are  
>>>> normally
>>>> handled by the client, where exceptions (transient errors) are  
>>>> something
>>>> camel could try to recover from, without much knowledge about the
>>>> application.  I think that the presence of fault in the camel api  
>>>> is not
>>>> due
>>>> to it being explicitly modeled by jbi and wsdl, as Roman  
>>>> suggests, and
>>>> Camel
>>>> simply copying that, but it's modeled in Camel for the same  
>>>> reason it's
>>>> modeled in jbi and wsdl, to differentiate transient from persistent
>>>> errors
>>>> in a non ambiguous way.
>>>
>>> I am one of the persons that would love the Camel Exchange / Message
>>> API to be a bit simpler. It has a fair shares of methods.
>>>
>>> Having listening and discussing with Hadrian on this and doing my  
>>> own
>>> investigations and whatnot I do belive that Hadrian is absolutely
>>> right when it comes to FAULT. It has a good place in the API.  I  
>>> am +1
>>> on having FAULT as we do now.
>>>
>>> The grief I have left is that the IN and OUT. It makes sense to have
>>> them and they provide a good value. However they have a big  
>>> drawnback
>>> in how they are routed in Camel with the Pipeline processor, that
>>> mimics the pipes and filters EIP. And as a result the OUT will be  
>>> used
>>> as IN
>>> in the next step in the route. So its not like you can steadily  
>>> build
>>> up an OUT message on-the-fly during many steps in the route path.
>>>
>>> Example
>>> from("direct:start").process(new Processor()).to("log:foo");
>>>
>>> a) From
>>> IN = Hello World
>>> OUT = null
>>>
>>> b) Processor
>>> IN Hello World
>>> OUT = Bye World
>>>
>>> c) Log
>>> IN = Bye World
>>> OUT = null
>>>
>> Yes, from an external observer's perspective, this is precisely what
>> happens.  How we decide to store it, how many fields we need, is an
>> implementation detail of the DefaultExchange.  I don't think the  
>> copy from
>> out/in is too expensive, but I would be ok with having only one  
>> field to
>> store the current message in DefaultExchange (I assume that's what  
>> you
>> propose).  However, my question is about what the api should be.
>>
>>
>>>
>>> And then the getOut() method that lazy creates a new empty OUT  
>>> message
>>> is also a pita, as it can lead to people loosing their messages if
>>> they do some System out logging of their own
>>>
>>> public void process(Exchange e) {
>>>  System.out.println(exchange.getIn());
>>>  System.out.println(exchange.getOut());
>>>  // boom you lost your message when its routed to next node in route
>>> path, as getOut() created a new empty OUT message that will by  
>>> used in
>>> the pipes and filters EIP routed with the Pipeline
>>> }
>>>
>>> We had this IN OUT discussion a while back and at that time we ended
>>> up with a compromise of having a hasOut() method so you should do,  
>>> to
>>> be safe:
>>>  System.out.println(exchange.getIn());
>>>  if (exchange.hasOut()) {
>>>      System.out.println(exchange.getOut());
>>>  }
>>>
>>> Still a pita with the lazy creation IMHO.
>>
>> The lazyCreate methods are actually deprecated, and imho should be  
>> removed
>> now.  This would eliminate the confusion.
>>
>>
>>>>
>>>> If we were to go with only get/setMessage() api, we would still  
>>>> need
>>>> methods
>>>> (or some ways) to distinguish between the kind of message we are  
>>>> dealing
>>>> with (in/out/fault/exception) so we'd only move the problem  
>>>> somewhere
>>>> else.
>>>>
>>>> So the question becomes if we leave the api the way it is, or we  
>>>> replace
>>>> the
>>>> get/setFault apis with get/setOut, in which case we'll need  
>>>> something
>>>> like:
>>>>   boolean isFault();
>>>> method in the Message api or keep the hasFault() method on the  
>>>> Exchange.
>>>
>>> Good question
>>>
>>> If you use OUT instead then we need to add a isFault() on the
>>> org.apache.camel.Message API that
>>> the IN message also implements.
>>>
>>> It could make sense to use OUT as well for FAULT.
>>> But how should the API look like to set an OUT as Fault?
>>>
>>> Something a like this?
>>>
>>> getOut().setBody("Unknown bank account number.");
>>> getOut().setFault(true);
>>
>> Not quite, I had something like this in mind (there is no setFault()
>> method):
>> getOut().setBody(java-lang-Exception-derivedObject);
>>
>> boolean isFault() {
>>    return getBody() instanceof Exception;
>> }
>>
>> Personally I am ok with the limitation not being able to have an  
>> out of a
>> java.lang.Exception type (that would then be a Fault).  I can't  
>> imagine a
>> case where an Exception would be an expected out, and in such cases  
>> one
>> could always serialize or wrap it.  The fact that the Fault would  
>> be an
>> Exception type would be a camel convention that needs to be  
>> followed by all
>> components.
>>
>> Another option would be add  header HAS_FAULT or something like  
>> that, in
>> which case both the out and the fault could be of any type.
>>
>>
>>>
>>>>
>>>> Thoughts?
>>>>
>>>>
>>>>   ExchangePattern getPattern();
>>>>   void setPattern(ExchangePattern pattern);
>>>>
>>>>   Object getProperty(String name);
>>>>   <T> T getProperty(String name, Class<T> type);
>>>>   void setProperty(String name, Object value);
>>>>   Object removeProperty(String name);
>>>>   Map<String, Object> getProperties();
>>>>
>>>>   Message getIn();
>>>>   void setIn(Message in);
>>>>
>>>>   Message getOut();
>>>>   boolean hasOut();
>>>>   Message getOut(boolean lazyCreate);
>>>>   void setOut(Message out);
>>>>
>>>>   Message getFault();
>>>>   boolean hasFault();
>>>>   Message getFault(boolean lazyCreate);
>>>>   void removeFault();
>>>> // removeFault() is only used in one place
>>>
>>> +1 to remove it. You can just do setFault(null) instead. I will  
>>> fix it
>>> asap.
>>>
>>>
>>>
>>>>
>>>>   Exception getException();
>>>>   <T> T getException(Class<T> type);
>>>>   void setException(Exception e);
>>>>   boolean isFailed();
>>>>
>>>>   boolean isTransacted();
>>>>   boolean isRollbackOnly();
>>>>
>>>>   CamelContext getContext();
>>>>
>>>>   Exchange newInstance();
>>>>   Exchange copy();
>>>>   Exchange newCopy(boolean handoverOnCompletion);
>>>>   void copyFrom(Exchange source);
>>>>
>>>>   Endpoint getFromEndpoint();
>>>>   void setFromEndpoint(Endpoint fromEndpoint);
>>>>
>>>>   UnitOfWork getUnitOfWork();
>>>>   void setUnitOfWork(UnitOfWork unitOfWork);
>>>>
>>>>   String getExchangeId();
>>>>   void setExchangeId(String id);
>>>>
>>>>   void addOnCompletion(Synchronization onCompletion);
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>
>>
>
>
>
> -- 
> Cheers,
> Guillaume Nodet
> ------------------------
> Blog: http://gnodet.blogspot.com/
> ------------------------
> Open Source SOA
> http://fusesource.com

Re: [DISCUSS] Faults and Exceptions in Camel

[Guillaume Nodet <gn...@gmail.com>]

In web services world, faults are not exceptions, but usually an xml
payload.  In the java world, faults would be like checked exceptions
and errors runtime exceptions.  However, distinguishing a fault from
an out message by (instanceof Exception) is imho not sufficient

On Thu, Jul 9, 2009 at 07:35, Hadrian Zbarcea<hz...@gmail.com> wrote:
> Hi,
>
> Comments inline.
>
> Hadrian
>
> On Jul 9, 2009, at 12:54 AM, Claus Ibsen wrote:
>
>> On Wed, Jul 8, 2009 at 9:52 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> As we approach the 2.0, there is one more hanging issue I would like
>>> addressed, if possible.  It's the thorny issue of Faults and Exceptions
>>> that
>>> started in
>>> http://issues.apache.org/activemq/browse/CAMEL-316 (see also the related
>>> nabble thread linked in the issue description).
>>>
>>> I am less concerned about how the DefaultExchange is implemented and I
>>> hope
>>> to reach an agreement on what the Exchange api should be (please find the
>>> list of Exchange methods below).
>>>
>>> As far as faults/exceptions are concerned, Roman thinks that the whole
>>> concept of in/out/fault/exception is artificial, and only one payload
>>> (message) api should be enough (Roman please correct me if I misinterpret
>>> your position).  My opinion is that we *must* distinguish between
>>> persistent
>>> (fault) and transient (exception) errors for the simple reason that they
>>> have different semantics.  As Roman correctly points out, faults are more
>>> like outputs, have more of application level semantics and are normally
>>> handled by the client, where exceptions (transient errors) are something
>>> camel could try to recover from, without much knowledge about the
>>> application.  I think that the presence of fault in the camel api is not
>>> due
>>> to it being explicitly modeled by jbi and wsdl, as Roman suggests, and
>>> Camel
>>> simply copying that, but it's modeled in Camel for the same reason it's
>>> modeled in jbi and wsdl, to differentiate transient from persistent
>>> errors
>>> in a non ambiguous way.
>>
>> I am one of the persons that would love the Camel Exchange / Message
>> API to be a bit simpler. It has a fair shares of methods.
>>
>> Having listening and discussing with Hadrian on this and doing my own
>> investigations and whatnot I do belive that Hadrian is absolutely
>> right when it comes to FAULT. It has a good place in the API.  I am +1
>> on having FAULT as we do now.
>>
>> The grief I have left is that the IN and OUT. It makes sense to have
>> them and they provide a good value. However they have a big drawnback
>> in how they are routed in Camel with the Pipeline processor, that
>> mimics the pipes and filters EIP. And as a result the OUT will be used
>> as IN
>> in the next step in the route. So its not like you can steadily build
>> up an OUT message on-the-fly during many steps in the route path.
>>
>> Example
>> from("direct:start").process(new Processor()).to("log:foo");
>>
>> a) From
>> IN = Hello World
>> OUT = null
>>
>> b) Processor
>> IN Hello World
>> OUT = Bye World
>>
>> c) Log
>> IN = Bye World
>> OUT = null
>>
> Yes, from an external observer's perspective, this is precisely what
> happens.  How we decide to store it, how many fields we need, is an
> implementation detail of the DefaultExchange.  I don't think the copy from
> out/in is too expensive, but I would be ok with having only one field to
> store the current message in DefaultExchange (I assume that's what you
> propose).  However, my question is about what the api should be.
>
>
>>
>> And then the getOut() method that lazy creates a new empty OUT message
>> is also a pita, as it can lead to people loosing their messages if
>> they do some System out logging of their own
>>
>> public void process(Exchange e) {
>>  System.out.println(exchange.getIn());
>>  System.out.println(exchange.getOut());
>>  // boom you lost your message when its routed to next node in route
>> path, as getOut() created a new empty OUT message that will by used in
>> the pipes and filters EIP routed with the Pipeline
>> }
>>
>> We had this IN OUT discussion a while back and at that time we ended
>> up with a compromise of having a hasOut() method so you should do, to
>> be safe:
>>  System.out.println(exchange.getIn());
>>  if (exchange.hasOut()) {
>>      System.out.println(exchange.getOut());
>>  }
>>
>> Still a pita with the lazy creation IMHO.
>
> The lazyCreate methods are actually deprecated, and imho should be removed
> now.  This would eliminate the confusion.
>
>
>>>
>>> If we were to go with only get/setMessage() api, we would still need
>>> methods
>>> (or some ways) to distinguish between the kind of message we are dealing
>>> with (in/out/fault/exception) so we'd only move the problem somewhere
>>> else.
>>>
>>> So the question becomes if we leave the api the way it is, or we replace
>>> the
>>> get/setFault apis with get/setOut, in which case we'll need something
>>> like:
>>>   boolean isFault();
>>> method in the Message api or keep the hasFault() method on the Exchange.
>>
>> Good question
>>
>> If you use OUT instead then we need to add a isFault() on the
>> org.apache.camel.Message API that
>> the IN message also implements.
>>
>> It could make sense to use OUT as well for FAULT.
>> But how should the API look like to set an OUT as Fault?
>>
>> Something a like this?
>>
>> getOut().setBody("Unknown bank account number.");
>> getOut().setFault(true);
>
> Not quite, I had something like this in mind (there is no setFault()
> method):
> getOut().setBody(java-lang-Exception-derivedObject);
>
> boolean isFault() {
>    return getBody() instanceof Exception;
> }
>
> Personally I am ok with the limitation not being able to have an out of a
> java.lang.Exception type (that would then be a Fault).  I can't imagine a
> case where an Exception would be an expected out, and in such cases one
> could always serialize or wrap it.  The fact that the Fault would be an
> Exception type would be a camel convention that needs to be followed by all
> components.
>
> Another option would be add  header HAS_FAULT or something like that, in
> which case both the out and the fault could be of any type.
>
>
>>
>>>
>>> Thoughts?
>>>
>>>
>>>   ExchangePattern getPattern();
>>>   void setPattern(ExchangePattern pattern);
>>>
>>>   Object getProperty(String name);
>>>   <T> T getProperty(String name, Class<T> type);
>>>   void setProperty(String name, Object value);
>>>   Object removeProperty(String name);
>>>   Map<String, Object> getProperties();
>>>
>>>   Message getIn();
>>>   void setIn(Message in);
>>>
>>>   Message getOut();
>>>   boolean hasOut();
>>>   Message getOut(boolean lazyCreate);
>>>   void setOut(Message out);
>>>
>>>   Message getFault();
>>>   boolean hasFault();
>>>   Message getFault(boolean lazyCreate);
>>>   void removeFault();
>>> // removeFault() is only used in one place
>>
>> +1 to remove it. You can just do setFault(null) instead. I will fix it
>> asap.
>>
>>
>>
>>>
>>>   Exception getException();
>>>   <T> T getException(Class<T> type);
>>>   void setException(Exception e);
>>>   boolean isFailed();
>>>
>>>   boolean isTransacted();
>>>   boolean isRollbackOnly();
>>>
>>>   CamelContext getContext();
>>>
>>>   Exchange newInstance();
>>>   Exchange copy();
>>>   Exchange newCopy(boolean handoverOnCompletion);
>>>   void copyFrom(Exchange source);
>>>
>>>   Endpoint getFromEndpoint();
>>>   void setFromEndpoint(Endpoint fromEndpoint);
>>>
>>>   UnitOfWork getUnitOfWork();
>>>   void setUnitOfWork(UnitOfWork unitOfWork);
>>>
>>>   String getExchangeId();
>>>   void setExchangeId(String id);
>>>
>>>   void addOnCompletion(Synchronization onCompletion);
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>
>



-- 
Cheers,
Guillaume Nodet
------------------------
Blog: http://gnodet.blogspot.com/
------------------------
Open Source SOA
http://fusesource.com

Re: [DISCUSS] Faults and Exceptions in Camel

[Hadrian Zbarcea <hz...@gmail.com>]

Hi,

Comments inline.

Hadrian

On Jul 9, 2009, at 12:54 AM, Claus Ibsen wrote:

> On Wed, Jul 8, 2009 at 9:52 PM, Hadrian Zbarcea<hz...@gmail.com>  
> wrote:
>> Hi,
>>
>> As we approach the 2.0, there is one more hanging issue I would like
>> addressed, if possible.  It's the thorny issue of Faults and  
>> Exceptions that
>> started in
>> http://issues.apache.org/activemq/browse/CAMEL-316 (see also the  
>> related
>> nabble thread linked in the issue description).
>>
>> I am less concerned about how the DefaultExchange is implemented  
>> and I hope
>> to reach an agreement on what the Exchange api should be (please  
>> find the
>> list of Exchange methods below).
>>
>> As far as faults/exceptions are concerned, Roman thinks that the  
>> whole
>> concept of in/out/fault/exception is artificial, and only one payload
>> (message) api should be enough (Roman please correct me if I  
>> misinterpret
>> your position).  My opinion is that we *must* distinguish between  
>> persistent
>> (fault) and transient (exception) errors for the simple reason that  
>> they
>> have different semantics.  As Roman correctly points out, faults  
>> are more
>> like outputs, have more of application level semantics and are  
>> normally
>> handled by the client, where exceptions (transient errors) are  
>> something
>> camel could try to recover from, without much knowledge about the
>> application.  I think that the presence of fault in the camel api  
>> is not due
>> to it being explicitly modeled by jbi and wsdl, as Roman suggests,  
>> and Camel
>> simply copying that, but it's modeled in Camel for the same reason  
>> it's
>> modeled in jbi and wsdl, to differentiate transient from persistent  
>> errors
>> in a non ambiguous way.
>
> I am one of the persons that would love the Camel Exchange / Message
> API to be a bit simpler. It has a fair shares of methods.
>
> Having listening and discussing with Hadrian on this and doing my own
> investigations and whatnot I do belive that Hadrian is absolutely
> right when it comes to FAULT. It has a good place in the API.  I am +1
> on having FAULT as we do now.
>
> The grief I have left is that the IN and OUT. It makes sense to have
> them and they provide a good value. However they have a big drawnback
> in how they are routed in Camel with the Pipeline processor, that
> mimics the pipes and filters EIP. And as a result the OUT will be used
> as IN
> in the next step in the route. So its not like you can steadily build
> up an OUT message on-the-fly during many steps in the route path.
>
> Example
> from("direct:start").process(new Processor()).to("log:foo");
>
> a) From
> IN = Hello World
> OUT = null
>
> b) Processor
> IN Hello World
> OUT = Bye World
>
> c) Log
> IN = Bye World
> OUT = null
>
Yes, from an external observer's perspective, this is precisely what  
happens.  How we decide to store it, how many fields we need, is an  
implementation detail of the DefaultExchange.  I don't think the copy  
from out/in is too expensive, but I would be ok with having only one  
field to store the current message in DefaultExchange (I assume that's  
what you propose).  However, my question is about what the api should  
be.


>
> And then the getOut() method that lazy creates a new empty OUT message
> is also a pita, as it can lead to people loosing their messages if
> they do some System out logging of their own
>
> public void process(Exchange e) {
>   System.out.println(exchange.getIn());
>   System.out.println(exchange.getOut());
>   // boom you lost your message when its routed to next node in route
> path, as getOut() created a new empty OUT message that will by used in
> the pipes and filters EIP routed with the Pipeline
> }
>
> We had this IN OUT discussion a while back and at that time we ended
> up with a compromise of having a hasOut() method so you should do, to
> be safe:
>   System.out.println(exchange.getIn());
>   if (exchange.hasOut()) {
>       System.out.println(exchange.getOut());
>   }
>
> Still a pita with the lazy creation IMHO.
The lazyCreate methods are actually deprecated, and imho should be  
removed now.  This would eliminate the confusion.


>>
>> If we were to go with only get/setMessage() api, we would still  
>> need methods
>> (or some ways) to distinguish between the kind of message we are  
>> dealing
>> with (in/out/fault/exception) so we'd only move the problem  
>> somewhere else.
>>
>> So the question becomes if we leave the api the way it is, or we  
>> replace the
>> get/setFault apis with get/setOut, in which case we'll need  
>> something like:
>>    boolean isFault();
>> method in the Message api or keep the hasFault() method on the  
>> Exchange.
> Good question
>
> If you use OUT instead then we need to add a isFault() on the
> org.apache.camel.Message API that
> the IN message also implements.
>
> It could make sense to use OUT as well for FAULT.
> But how should the API look like to set an OUT as Fault?
>
> Something a like this?
>
> getOut().setBody("Unknown bank account number.");
> getOut().setFault(true);

Not quite, I had something like this in mind (there is no setFault()  
method):
getOut().setBody(java-lang-Exception-derivedObject);

boolean isFault() {
     return getBody() instanceof Exception;
}

Personally I am ok with the limitation not being able to have an out  
of a java.lang.Exception type (that would then be a Fault).  I can't  
imagine a case where an Exception would be an expected out, and in  
such cases one could always serialize or wrap it.  The fact that the  
Fault would be an Exception type would be a camel convention that  
needs to be followed by all components.

Another option would be add  header HAS_FAULT or something like that,  
in which case both the out and the fault could be of any type.


>
>>
>> Thoughts?
>>
>>
>>    ExchangePattern getPattern();
>>    void setPattern(ExchangePattern pattern);
>>
>>    Object getProperty(String name);
>>    <T> T getProperty(String name, Class<T> type);
>>    void setProperty(String name, Object value);
>>    Object removeProperty(String name);
>>    Map<String, Object> getProperties();
>>
>>    Message getIn();
>>    void setIn(Message in);
>>
>>    Message getOut();
>>    boolean hasOut();
>>    Message getOut(boolean lazyCreate);
>>    void setOut(Message out);
>>
>>    Message getFault();
>>    boolean hasFault();
>>    Message getFault(boolean lazyCreate);
>>    void removeFault();
>> // removeFault() is only used in one place
> +1 to remove it. You can just do setFault(null) instead. I will fix  
> it asap.
>
>
>
>>
>>    Exception getException();
>>    <T> T getException(Class<T> type);
>>    void setException(Exception e);
>>    boolean isFailed();
>>
>>    boolean isTransacted();
>>    boolean isRollbackOnly();
>>
>>    CamelContext getContext();
>>
>>    Exchange newInstance();
>>    Exchange copy();
>>    Exchange newCopy(boolean handoverOnCompletion);
>>    void copyFrom(Exchange source);
>>
>>    Endpoint getFromEndpoint();
>>    void setFromEndpoint(Endpoint fromEndpoint);
>>
>>    UnitOfWork getUnitOfWork();
>>    void setUnitOfWork(UnitOfWork unitOfWork);
>>
>>    String getExchangeId();
>>    void setExchangeId(String id);
>>
>>    void addOnCompletion(Synchronization onCompletion);
>>
>>
>>
>>
>>
>
>
>
> -- 
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Faults and Exceptions in Camel

[Claus Ibsen <cl...@gmail.com>]

On Wed, Jul 8, 2009 at 9:52 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
> Hi,
>
> As we approach the 2.0, there is one more hanging issue I would like
> addressed, if possible.  It's the thorny issue of Faults and Exceptions that
> started in
> http://issues.apache.org/activemq/browse/CAMEL-316 (see also the related
> nabble thread linked in the issue description).
>
> I am less concerned about how the DefaultExchange is implemented and I hope
> to reach an agreement on what the Exchange api should be (please find the
> list of Exchange methods below).
>
> As far as faults/exceptions are concerned, Roman thinks that the whole
> concept of in/out/fault/exception is artificial, and only one payload
> (message) api should be enough (Roman please correct me if I misinterpret
> your position).  My opinion is that we *must* distinguish between persistent
> (fault) and transient (exception) errors for the simple reason that they
> have different semantics.  As Roman correctly points out, faults are more
> like outputs, have more of application level semantics and are normally
> handled by the client, where exceptions (transient errors) are something
> camel could try to recover from, without much knowledge about the
> application.  I think that the presence of fault in the camel api is not due
> to it being explicitly modeled by jbi and wsdl, as Roman suggests, and Camel
> simply copying that, but it's modeled in Camel for the same reason it's
> modeled in jbi and wsdl, to differentiate transient from persistent errors
> in a non ambiguous way.

I am one of the persons that would love the Camel Exchange / Message
API to be a bit simpler. It has a fair shares of methods.

Having listening and discussing with Hadrian on this and doing my own
investigations and whatnot I do belive that Hadrian is absolutely
right when it comes to FAULT. It has a good place in the API.  I am +1
on having FAULT as we do now.

The grief I have left is that the IN and OUT. It makes sense to have
them and they provide a good value. However they have a big drawnback
in how they are routed in Camel with the Pipeline processor, that
mimics the pipes and filters EIP. And as a result the OUT will be used
as IN
in the next step in the route. So its not like you can steadily build
up an OUT message on-the-fly during many steps in the route path.

Example
from("direct:start").process(new Processor()).to("log:foo");

a) From
IN = Hello World
OUT = null

b) Processor
IN Hello World
OUT = Bye World

c) Log
IN = Bye World
OUT = null


And then the getOut() method that lazy creates a new empty OUT message
is also a pita, as it can lead to people loosing their messages if
they do some System out logging of their own

public void process(Exchange e) {
   System.out.println(exchange.getIn());
   System.out.println(exchange.getOut());
   // boom you lost your message when its routed to next node in route
path, as getOut() created a new empty OUT message that will by used in
the pipes and filters EIP routed with the Pipeline
}

We had this IN OUT discussion a while back and at that time we ended
up with a compromise of having a hasOut() method so you should do, to
be safe:
   System.out.println(exchange.getIn());
   if (exchange.hasOut()) {
       System.out.println(exchange.getOut());
   }

Still a pita with the lazy creation IMHO.



>
> If we were to go with only get/setMessage() api, we would still need methods
> (or some ways) to distinguish between the kind of message we are dealing
> with (in/out/fault/exception) so we'd only move the problem somewhere else.
>
> So the question becomes if we leave the api the way it is, or we replace the
> get/setFault apis with get/setOut, in which case we'll need something like:
>    boolean isFault();
> method in the Message api or keep the hasFault() method on the Exchange.
Good question

If you use OUT instead then we need to add a isFault() on the
org.apache.camel.Message API that
the IN message also implements.

It could make sense to use OUT as well for FAULT.
But how should the API look like to set an OUT as Fault?

Something a like this?

getOut().setBody("Unknown bank account number.");
getOut().setFault(true);





>
> Thoughts?
>
>
>    ExchangePattern getPattern();
>    void setPattern(ExchangePattern pattern);
>
>    Object getProperty(String name);
>    <T> T getProperty(String name, Class<T> type);
>    void setProperty(String name, Object value);
>    Object removeProperty(String name);
>    Map<String, Object> getProperties();
>
>    Message getIn();
>    void setIn(Message in);
>
>    Message getOut();
>    boolean hasOut();
>    Message getOut(boolean lazyCreate);
>    void setOut(Message out);
>
>    Message getFault();
>    boolean hasFault();
>    Message getFault(boolean lazyCreate);
>    void removeFault();
> // removeFault() is only used in one place
+1 to remove it. You can just do setFault(null) instead. I will fix it asap.



>
>    Exception getException();
>    <T> T getException(Class<T> type);
>    void setException(Exception e);
>    boolean isFailed();
>
>    boolean isTransacted();
>    boolean isRollbackOnly();
>
>    CamelContext getContext();
>
>    Exchange newInstance();
>    Exchange copy();
>    Exchange newCopy(boolean handoverOnCompletion);
>    void copyFrom(Exchange source);
>
>    Endpoint getFromEndpoint();
>    void setFromEndpoint(Endpoint fromEndpoint);
>
>    UnitOfWork getUnitOfWork();
>    void setUnitOfWork(UnitOfWork unitOfWork);
>
>    String getExchangeId();
>    void setExchangeId(String id);
>
>    void addOnCompletion(Synchronization onCompletion);
>
>
>
>
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus