[DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Hadrian Zbarcea <hz...@gmail.com>]

Moved this slightly different topic to a separate thread.

++1 from me.

Do we need a vote on this one or it's a consensus?  Claus, it looks  
like you agree, but pointed out that there is some work involved,  
correct?

Hadrian



On Jul 10, 2009, at 8:40 AM, James Strachan wrote:

> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<james.strachan@gmail.com 
>> > wrote:
>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>> Hi
>>>>
>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>
>>>> The ProducerCache that is doing the actual sending when using  
>>>> template
>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>
>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>> pointed out. He wanted the IN to be more static.
>>>> And we cannot get there yet when you do routing as all the  
>>>> processors
>>>> rely on IN being able to mutate during routing.
>>>>
>>>> Anyway my grief is that when you send in an Exchange the result  
>>>> would
>>>> sometimes be stored on IN and not OUT.
>>>> What I want it to do always is to store the result in OUT and  
>>>> keep IN
>>>> as the original input.
>>>>
>>>> The code to do this is now a bit more complex than just before
>>>>
>>>>                // copy the original input
>>>>                Message original = exchange.getIn().copy();
>>>>
>>>>                producer.process(exchange);
>>>>
>>>>                // if no OUT then set current IN as result (except  
>>>> for
>>>> optional out)
>>>>                if (!exchange.hasOut() && exchange.getPattern() !=
>>>> ExchangePattern.InOptionalOut) {
>>>>                    // but only if its not the same as original IN  
>>>> to
>>>> avoid duplicating it
>>>>                    // and to adhere to the fact that there was no  
>>>> OUT
>>>> result at all
>>>>                    if (original.getBody() != null &&
>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>                        exchange.setOut(exchange.getIn());
>>>>                    }
>>>>                }
>>>>                // and restore original in
>>>>                exchange.setIn(original);
>>>>
>>>>                return exchange;
>>>>
>>>>
>>>> What I need to do is to copy the original IN message as it can be
>>>> mutated during routing.
>>>
>>> How about we prevent mutation of the IN message? Create a Message
>>> facade which throws UnsupportedOperationException if you try to  
>>> mutate
>>> it in any way. Then we can pass the same read-only Message around as
>>> the IN within retry loops or from step to step if no new output is
>>> created (e.g. in a content based router where you just move a  
>>> Message
>>> to the right endpoint without changing it)
>>>
>>
>> A good idea but will break a lot of logic in Camel.
>
> Agreed. But with the benefit that we'd be able to get rid of all the
> defensive copies in our code; plus we'd be able to pass the same
> Message from step to step. The API would be a bit more clean; to
> change the output, you create an OUT message (maybe by copying the
> IN).
>
>
>> Most of the Camel
>> processors work on the IN message and set the result on either IN or
>> OUT. At best they set it on OUT. But then the IN is always the
>> original input? Or am I mistaking?
>
> Yeah, we'd have to patch code to no longer mutate IN
>
>
>> How will this work with the Pipes And Filters EIP if the IN is
>> immutable and always the original input?
>
> If no OUT, then no output was created, so pass the IN along...
>
>
> OK how about this; a CopyOnWriteMessageFacade which does not mutate
> the original message at all ever; if a message is used in a purely
> read only way, it does nothing but delegate to the original message -
> but then as soon as someone mutates it, it creates a copy and uses
> that from that point on?
>
> i.e. make the copy of the message lazy - and only make a copy when a
> Processor really does try to mutate the Message?
>
> Then we'd get the best of both worlds; avoid breaking old code but
> avoid tons of unnecessary copies?
>
> -- 
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[James Strachan <ja...@gmail.com>]

2009/7/14 Roman Kalukiewicz <ro...@gmail.com>:
> 2009/7/14 James Strachan <ja...@gmail.com>:
>> 2009/7/13 Claus Ibsen <cl...@gmail.com>:
>>>>
>>>> Reading between the lines; are you really just trying to make folks
>>>> use (what is currently called) "getOut()" and never try mutate what is
>>>> currently called getIn()?
>>>>
>>>> i.e. so by default the "OUT" property is defaulted to a copy of IN
>>>> that folks can change/mutate.
>>>>
>>>> (what we call these 2 methods is a separate discussion, whether its
>>>> "in,out" or "originalMessage,message" or whatever
>>>>
>>>
>>> Hadrian and I had a chat today and we are clearing up some bits.
>>> I am more on line with him now on the IN and OUT thing.
>>>
>>> So lets keep them.
>>>
>>> And use the time to fix the tiny bits such as the getOut() doing its
>>> lazy creating a new empty message.
>>> And whether its possible to let IN be immutable.
>>
>> I think we're kinda mostly on the same page (though saying it in
>> different ways). BTW I'm taking off my devils advocate hat now :)...
>>
>> What we're agreeing on I think is that;
>>
>> * getIn() should be immutable (when folks try to change it we can
>> throw the exception and describe how getOut() should be used to change
>> the message) - to prevent folks using the old code (which will require
>> code changes).
>
> Definitely agree.
>
>> * having the original immutable message available is handy; but mostly
>> folks should concentrate on the 'out' (current name today)
>
> It is handy.But the question is if we should add another message while
> both client and processor implementor can store original message (body
> or headers he wants) in a variable if they only want to store them. Do
> we have to store those things for them no mater if they need it or
> not? This comment applies only if we want to have IN message on
> processor level (every processor receives its new IN message).
>
> I personally don't see a huge difference between:
> Object oldBody = exchange.getMessage().getBody();
> template.send(exchange);
> Object newBody = exchange.getMessage.getBody();
>
> and
> template.send(exchange);
> Object oldBody = exchange.getOriginalMessage().getBody();
> Object newBody = exchange.getMessage().getBody();
>
> while the second one complicates an API by having additional method on
> the exchange.
>
>> * the out should be automatically populated with a clone of the IN (to
>> avoid all that pain with checking if there's an out or an in, or the
>> possible loss of headers etc. Internally we can use a CopyOnWrite
>> facade to reduce the cost of potentially copying a message which is
>> not actually mutated.
>
> True
>
>> Given that; I think we're mostly agreeing. However given the confusion
>> of getIn() and getOut() I'm wondering if actually your previous idea
>> of changing the api of exchange to have a getMessage() which returns
>> the thing a processor should be changing; then having a
>> getOriginalMessage() (which can be null if you are the first in the
>> chain) might reduce the amount of confusion?
>
> Because of previous comment about IN messages I believe that maybe we
> should have getOriginalMessage() that returns an original message as
> JMS Message, JBI Exchange and so on. This assures that users can
> always reach full information they received. It would look like
> from("jms:q1")  // Original message contains JMS message received
> .transform(expression) // Still original JMS message - no endpoint involved
> .setHeader("foo", expression) // Still original JMS message - still I
> can reach some JMS specific things
> .to("jbi:service") // Now I have JBI exchange in original message
> .bean(myBean) // My Bean reaches exchange directly to reach some stuff
> from ServiceMix specific JBI exchange
> .to("jms:q2") // JMS response message in original message
> .to("direct:end") // Camel Exchange in original message as it is now
> endpoint specific type
> .to("seda:foo") // InOnly operation. I'm not sure if it should be null
> in original message, or maybe nothing modified.
>
> I propose it because I don't really like our custom Message
> subclasses. They tend to mirror Camel message model and some
> underlying technology's model. But sometimes they are not compatible
> really and strange things happen. But maybe this is the discussion for
> another thread?
>
>> i.e. after sleeping on it, I'm warming to the idea of renaming getIn()
>> -> getOriginalMessage() and getOut() -> getMessage(). (Or maybe
>> getInputMessage() for getIn()?)
>>
>> Thoughts?
>
> I definitely agree that for the moment we can add those methods,
> deprecate getIn()/getOut() and let them simply return getMessage().
>
> To summarize my ideal model is (helpers like getBody(Class) omitted):
> CamelMessage
> Object get/setBody() //obvious
> Map<String, Object> getHeaders() // for technology specific headers
> received/sent
> Map<String, Object> getProperties() // for business specific
> information that is generally not touched by endpoints - user can
> store there any information he wants to be sent through the flow and
> not be lost at the endpoint
> Object get/setOriginalMessage() // For technology specific message
> representation like JMS Message/JBI Exchange
> Exception get/setException() // For exception handling
> boolean is/setFault() // For business fault detection
>
> In a model where only one message exists, exchange is not even needed.
>
> Heh - I feel guilty for enlisting all those things now while it looks
> that an agreement on API changes exists ;)
> Maybe a proposal for 2.1? ;)

FWIW I still like the Exchange being there as a holder of the Message;
as folks can then, if they want, create a totally new, empty Message
and set it on the Exchange (rather than just mutating the incoming
one).

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Semantics of IN and OUT

[Willem Jiang <wi...@gmail.com>]

Roman Kalukiewicz wrote:
> 2009/7/14 James Strachan <ja...@gmail.com>:
>> 2009/7/13 Claus Ibsen <cl...@gmail.com>:
>>>> Reading between the lines; are you really just trying to make folks
>>>> use (what is currently called) "getOut()" and never try mutate what is
>>>> currently called getIn()?
>>>>
>>>> i.e. so by default the "OUT" property is defaulted to a copy of IN
>>>> that folks can change/mutate.
>>>>
>>>> (what we call these 2 methods is a separate discussion, whether its
>>>> "in,out" or "originalMessage,message" or whatever
>>>>
>>> Hadrian and I had a chat today and we are clearing up some bits.
>>> I am more on line with him now on the IN and OUT thing.
>>>
>>> So lets keep them.
>>>
>>> And use the time to fix the tiny bits such as the getOut() doing its
>>> lazy creating a new empty message.
>>> And whether its possible to let IN be immutable.
>> I think we're kinda mostly on the same page (though saying it in
>> different ways). BTW I'm taking off my devils advocate hat now :)...
>>
>> What we're agreeing on I think is that;
>>
>> * getIn() should be immutable (when folks try to change it we can
>> throw the exception and describe how getOut() should be used to change
>> the message) - to prevent folks using the old code (which will require
>> code changes).
> 
> Definitely agree.
> 
>> * having the original immutable message available is handy; but mostly
>> folks should concentrate on the 'out' (current name today)
> 
> It is handy.But the question is if we should add another message while
> both client and processor implementor can store original message (body
> or headers he wants) in a variable if they only want to store them. Do
> we have to store those things for them no mater if they need it or
> not? This comment applies only if we want to have IN message on
> processor level (every processor receives its new IN message).
> 
> I personally don't see a huge difference between:
> Object oldBody = exchange.getMessage().getBody();
> template.send(exchange);
> Object newBody = exchange.getMessage.getBody();
> 
> and
> template.send(exchange);
> Object oldBody = exchange.getOriginalMessage().getBody();
> Object newBody = exchange.getMessage().getBody();
> 
> while the second one complicates an API by having additional method on
> the exchange.
> 
>> * the out should be automatically populated with a clone of the IN (to
>> avoid all that pain with checking if there's an out or an in, or the
>> possible loss of headers etc. Internally we can use a CopyOnWrite
>> facade to reduce the cost of potentially copying a message which is
>> not actually mutated.
> 
> True
> 
>> Given that; I think we're mostly agreeing. However given the confusion
>> of getIn() and getOut() I'm wondering if actually your previous idea
>> of changing the api of exchange to have a getMessage() which returns
>> the thing a processor should be changing; then having a
>> getOriginalMessage() (which can be null if you are the first in the
>> chain) might reduce the amount of confusion?
> 
> Because of previous comment about IN messages I believe that maybe we
> should have getOriginalMessage() that returns an original message as
> JMS Message, JBI Exchange and so on. This assures that users can
> always reach full information they received. It would look like
> from("jms:q1")  // Original message contains JMS message received
> .transform(expression) // Still original JMS message - no endpoint involved
> .setHeader("foo", expression) // Still original JMS message - still I
> can reach some JMS specific things
> .to("jbi:service") // Now I have JBI exchange in original message
> .bean(myBean) // My Bean reaches exchange directly to reach some stuff
> from ServiceMix specific JBI exchange
> .to("jms:q2") // JMS response message in original message
> .to("direct:end") // Camel Exchange in original message as it is now
> endpoint specific type
> .to("seda:foo") // InOnly operation. I'm not sure if it should be null
> in original message, or maybe nothing modified.
> 
> I propose it because I don't really like our custom Message
> subclasses. They tend to mirror Camel message model and some
> underlying technology's model. But sometimes they are not compatible
> really and strange things happen. But maybe this is the discussion for
> another thread?
In most of Camel 2.0's components , we don't create the custom Message 
subclasses anymore, we just copy the component specify message context 
into the DefaultMessage header. In this way we could avoid the some 
component specify message copy work which will cause some trouble when 
routing the message between different components.

And we also did some exchange clean up work on Camel 2.0 recently  :).

> 
>> i.e. after sleeping on it, I'm warming to the idea of renaming getIn()
>> -> getOriginalMessage() and getOut() -> getMessage(). (Or maybe
>> getInputMessage() for getIn()?)
>>
>> Thoughts?
> 
> I definitely agree that for the moment we can add those methods,
> deprecate getIn()/getOut() and let them simply return getMessage().
> 
> To summarize my ideal model is (helpers like getBody(Class) omitted):
> CamelMessage
> Object get/setBody() //obvious
> Map<String, Object> getHeaders() // for technology specific headers
> received/sent
> Map<String, Object> getProperties() // for business specific
> information that is generally not touched by endpoints - user can
> store there any information he wants to be sent through the flow and
> not be lost at the endpoint
> Object get/setOriginalMessage() // For technology specific message
> representation like JMS Message/JBI Exchange
> Exception get/setException() // For exception handling
> boolean is/setFault() // For business fault detection
> 
> In a model where only one message exists, exchange is not even needed.
> 
> Heh - I feel guilty for enlisting all those things now while it looks
> that an agreement on API changes exists ;)
> Maybe a proposal for 2.1? ;)
> 
> Roman
> 

Willem

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Roman Kalukiewicz <ro...@gmail.com>]

2009/7/14 James Strachan <ja...@gmail.com>:
> 2009/7/13 Claus Ibsen <cl...@gmail.com>:
>>>
>>> Reading between the lines; are you really just trying to make folks
>>> use (what is currently called) "getOut()" and never try mutate what is
>>> currently called getIn()?
>>>
>>> i.e. so by default the "OUT" property is defaulted to a copy of IN
>>> that folks can change/mutate.
>>>
>>> (what we call these 2 methods is a separate discussion, whether its
>>> "in,out" or "originalMessage,message" or whatever
>>>
>>
>> Hadrian and I had a chat today and we are clearing up some bits.
>> I am more on line with him now on the IN and OUT thing.
>>
>> So lets keep them.
>>
>> And use the time to fix the tiny bits such as the getOut() doing its
>> lazy creating a new empty message.
>> And whether its possible to let IN be immutable.
>
> I think we're kinda mostly on the same page (though saying it in
> different ways). BTW I'm taking off my devils advocate hat now :)...
>
> What we're agreeing on I think is that;
>
> * getIn() should be immutable (when folks try to change it we can
> throw the exception and describe how getOut() should be used to change
> the message) - to prevent folks using the old code (which will require
> code changes).

Definitely agree.

> * having the original immutable message available is handy; but mostly
> folks should concentrate on the 'out' (current name today)

It is handy.But the question is if we should add another message while
both client and processor implementor can store original message (body
or headers he wants) in a variable if they only want to store them. Do
we have to store those things for them no mater if they need it or
not? This comment applies only if we want to have IN message on
processor level (every processor receives its new IN message).

I personally don't see a huge difference between:
Object oldBody = exchange.getMessage().getBody();
template.send(exchange);
Object newBody = exchange.getMessage.getBody();

and
template.send(exchange);
Object oldBody = exchange.getOriginalMessage().getBody();
Object newBody = exchange.getMessage().getBody();

while the second one complicates an API by having additional method on
the exchange.

> * the out should be automatically populated with a clone of the IN (to
> avoid all that pain with checking if there's an out or an in, or the
> possible loss of headers etc. Internally we can use a CopyOnWrite
> facade to reduce the cost of potentially copying a message which is
> not actually mutated.

True

> Given that; I think we're mostly agreeing. However given the confusion
> of getIn() and getOut() I'm wondering if actually your previous idea
> of changing the api of exchange to have a getMessage() which returns
> the thing a processor should be changing; then having a
> getOriginalMessage() (which can be null if you are the first in the
> chain) might reduce the amount of confusion?

Because of previous comment about IN messages I believe that maybe we
should have getOriginalMessage() that returns an original message as
JMS Message, JBI Exchange and so on. This assures that users can
always reach full information they received. It would look like
from("jms:q1")  // Original message contains JMS message received
.transform(expression) // Still original JMS message - no endpoint involved
.setHeader("foo", expression) // Still original JMS message - still I
can reach some JMS specific things
.to("jbi:service") // Now I have JBI exchange in original message
.bean(myBean) // My Bean reaches exchange directly to reach some stuff
from ServiceMix specific JBI exchange
.to("jms:q2") // JMS response message in original message
.to("direct:end") // Camel Exchange in original message as it is now
endpoint specific type
.to("seda:foo") // InOnly operation. I'm not sure if it should be null
in original message, or maybe nothing modified.

I propose it because I don't really like our custom Message
subclasses. They tend to mirror Camel message model and some
underlying technology's model. But sometimes they are not compatible
really and strange things happen. But maybe this is the discussion for
another thread?

> i.e. after sleeping on it, I'm warming to the idea of renaming getIn()
> -> getOriginalMessage() and getOut() -> getMessage(). (Or maybe
> getInputMessage() for getIn()?)
>
> Thoughts?

I definitely agree that for the moment we can add those methods,
deprecate getIn()/getOut() and let them simply return getMessage().

To summarize my ideal model is (helpers like getBody(Class) omitted):
CamelMessage
Object get/setBody() //obvious
Map<String, Object> getHeaders() // for technology specific headers
received/sent
Map<String, Object> getProperties() // for business specific
information that is generally not touched by endpoints - user can
store there any information he wants to be sent through the flow and
not be lost at the endpoint
Object get/setOriginalMessage() // For technology specific message
representation like JMS Message/JBI Exchange
Exception get/setException() // For exception handling
boolean is/setFault() // For business fault detection

In a model where only one message exists, exchange is not even needed.

Heh - I feel guilty for enlisting all those things now while it looks
that an agreement on API changes exists ;)
Maybe a proposal for 2.1? ;)

Roman

Re: [DISCUSS] Semantics of IN and OUT

[James Strachan <ja...@gmail.com>]

2009/7/14 Willem Jiang <wi...@gmail.com>:
> James Strachan wrote:
>>
>> 2009/7/13 Claus Ibsen <cl...@gmail.com>:
>>>>
>>>> Reading between the lines; are you really just trying to make folks
>>>> use (what is currently called) "getOut()" and never try mutate what is
>>>> currently called getIn()?
>>>>
>>>> i.e. so by default the "OUT" property is defaulted to a copy of IN
>>>> that folks can change/mutate.
>>>>
>>>> (what we call these 2 methods is a separate discussion, whether its
>>>> "in,out" or "originalMessage,message" or whatever
>>>>
>>> Hadrian and I had a chat today and we are clearing up some bits.
>>> I am more on line with him now on the IN and OUT thing.
>>>
>>> So lets keep them.
>>>
>>> And use the time to fix the tiny bits such as the getOut() doing its
>>> lazy creating a new empty message.
>>> And whether its possible to let IN be immutable.
>>
>> I think we're kinda mostly on the same page (though saying it in
>> different ways). BTW I'm taking off my devils advocate hat now :)...
>>
>> What we're agreeing on I think is that;
>>
>> * getIn() should be immutable (when folks try to change it we can
>> throw the exception and describe how getOut() should be used to change
>> the message) - to prevent folks using the old code (which will require
>> code changes).
>> * having the original immutable message available is handy; but mostly
>> folks should concentrate on the 'out' (current name today)
>> * the out should be automatically populated with a clone of the IN (to
>> avoid all that pain with checking if there's an out or an in, or the
>> possible loss of headers etc. Internally we can use a CopyOnWrite
>> facade to reduce the cost of potentially copying a message which is
>> not actually mutated.
>>
>> Given that; I think we're mostly agreeing. However given the confusion
>> of getIn() and getOut() I'm wondering if actually your previous idea
>> of changing the api of exchange to have a getMessage() which returns
>> the thing a processor should be changing; then having a
>> getOriginalMessage() (which can be null if you are the first in the
>> chain) might reduce the amount of confusion?
>>
>> i.e. after sleeping on it, I'm warming to the idea of renaming getIn()
>> -> getOriginalMessage() and getOut() -> getMessage(). (Or maybe
>> getInputMessage() for getIn()?)
>>
>> Thoughts?
>>
> +1 for this change.
>
> For the InOnly Message Exchange Pattern, when we want to change the message
> , we just call the exchange's getMessage() and don't not care about if it's
> a in message or an out message.
>
> For the ProduceTemplate sendMessage() method we can get the response message
> back from the exchange, by using exchange.getMessage() method.

Good point.

When sending a message, the getOriginalMessage() will be null (we
should document that its an optional property that might be null etc).


-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Semantics of IN and OUT

[Willem Jiang <wi...@gmail.com>]

James Strachan wrote:
> 2009/7/13 Claus Ibsen <cl...@gmail.com>:
>>> Reading between the lines; are you really just trying to make folks
>>> use (what is currently called) "getOut()" and never try mutate what is
>>> currently called getIn()?
>>>
>>> i.e. so by default the "OUT" property is defaulted to a copy of IN
>>> that folks can change/mutate.
>>>
>>> (what we call these 2 methods is a separate discussion, whether its
>>> "in,out" or "originalMessage,message" or whatever
>>>
>> Hadrian and I had a chat today and we are clearing up some bits.
>> I am more on line with him now on the IN and OUT thing.
>>
>> So lets keep them.
>>
>> And use the time to fix the tiny bits such as the getOut() doing its
>> lazy creating a new empty message.
>> And whether its possible to let IN be immutable.
> 
> I think we're kinda mostly on the same page (though saying it in
> different ways). BTW I'm taking off my devils advocate hat now :)...
> 
> What we're agreeing on I think is that;
> 
> * getIn() should be immutable (when folks try to change it we can
> throw the exception and describe how getOut() should be used to change
> the message) - to prevent folks using the old code (which will require
> code changes).
> * having the original immutable message available is handy; but mostly
> folks should concentrate on the 'out' (current name today)
> * the out should be automatically populated with a clone of the IN (to
> avoid all that pain with checking if there's an out or an in, or the
> possible loss of headers etc. Internally we can use a CopyOnWrite
> facade to reduce the cost of potentially copying a message which is
> not actually mutated.
> 
> Given that; I think we're mostly agreeing. However given the confusion
> of getIn() and getOut() I'm wondering if actually your previous idea
> of changing the api of exchange to have a getMessage() which returns
> the thing a processor should be changing; then having a
> getOriginalMessage() (which can be null if you are the first in the
> chain) might reduce the amount of confusion?
> 
> i.e. after sleeping on it, I'm warming to the idea of renaming getIn()
> -> getOriginalMessage() and getOut() -> getMessage(). (Or maybe
> getInputMessage() for getIn()?)
> 
> Thoughts?
> 
+1 for this change.

For the InOnly Message Exchange Pattern, when we want to change the 
message , we just call the exchange's getMessage() and don't not care 
about if it's a in message or an out message.

For the ProduceTemplate sendMessage() method we can get the response 
message back from the exchange, by using exchange.getMessage() method.


Willem

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Claus Ibsen <cl...@gmail.com>]

> Yeah.
>
> BTW we could leave getIn() and getOut() & try implement the
> ImmutableWrapper and CopyOnWriteFacade for In/Out respectively and try
> them out inside the current API - before we go ahead with any kind of
> renaming of getIn() and getOut().
>

> Maybe we could even keep getIn() / getOut() around for 2.0 (and zap in
> 2.1) and leave deprecated and make them just delegate to the new
> method names (say getInputMessage() and getMessage()). The deprecated
> message could then describe how folks typically should switch
> getIn()/getOut() to using getMessage() unless they really really want
> the immutable input message getInputMessage().
>

Brilliant idea and a good migration path as well. And it wont delay
Camel 2.0 as much as a full blown migration would have done.

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Hadrian Zbarcea <hz...@gmail.com>]

Awesome.  This is very similar to what we agreed upon yesterday with  
Claus.

I personally much prefer the in/out terminology as it has pretty  
clearly defined meaning in a few specs.
The 'original' term is a bit confusing and I would avoid it.  It could  
mean either that the original coming into the consumer, or original  
coming into a processor, and I think we mean the latter.  I am not  
totally opposed to changing the method names, but this seems to me  
just a renaming and since it would incur extra work for the users, I'd  
rather keep it as is.

With cleaning up the implementation and the documentation, which could  
be done now or post 2.0, I am pretty happy.

Cheers,
Hadrian


On Jul 14, 2009, at 5:06 AM, James Strachan wrote:

> 2009/7/14 Claus Ibsen <cl...@gmail.com>:
>>> I think we're kinda mostly on the same page (though saying it in
>>> different ways). BTW I'm taking off my devils advocate hat now :)...
>>>
>>> What we're agreeing on I think is that;
>>>
>>> * getIn() should be immutable (when folks try to change it we can
>>> throw the exception and describe how getOut() should be used to  
>>> change
>>> the message) - to prevent folks using the old code (which will  
>>> require
>>> code changes).
>>> * having the original immutable message available is handy; but  
>>> mostly
>>> folks should concentrate on the 'out' (current name today)
>>> * the out should be automatically populated with a clone of the IN  
>>> (to
>>> avoid all that pain with checking if there's an out or an in, or the
>>> possible loss of headers etc. Internally we can use a CopyOnWrite
>>> facade to reduce the cost of potentially copying a message which is
>>> not actually mutated.
>>>
>>> Given that; I think we're mostly agreeing. However given the  
>>> confusion
>>> of getIn() and getOut() I'm wondering if actually your previous idea
>>> of changing the api of exchange to have a getMessage() which returns
>>> the thing a processor should be changing; then having a
>>> getOriginalMessage() (which can be null if you are the first in the
>>> chain) might reduce the amount of confusion?
>>>
>>> i.e. after sleeping on it, I'm warming to the idea of renaming  
>>> getIn()
>>> -> getOriginalMessage() and getOut() -> getMessage(). (Or maybe
>>> getInputMessage() for getIn()?)
>>>
>>> Thoughts?
>>
>> Very good thoughts James. I think all this will removes many  
>> obstacles
>> and confusing for new users to Camel.
>>
>> Now even the AggregationStrategy is less tricky to use as you will
>> know that using getMessage gets what you are looking for.
>>
>> Maybe we should create a wiki page at Camel where we can outline the
>> API proposal and ideas for the message facade.
>
> Yeah.
>
> BTW we could leave getIn() and getOut() & try implement the
> ImmutableWrapper and CopyOnWriteFacade for In/Out respectively and try
> them out inside the current API - before we go ahead with any kind of
> renaming of getIn() and getOut().
>
> Maybe we could even keep getIn() / getOut() around for 2.0 (and zap in
> 2.1) and leave deprecated and make them just delegate to the new
> method names (say getInputMessage() and getMessage()). The deprecated
> message could then describe how folks typically should switch
> getIn()/getOut() to using getMessage() unless they really really want
> the immutable input message getInputMessage().
>
> -- 
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[James Strachan <ja...@gmail.com>]

2009/7/14 Claus Ibsen <cl...@gmail.com>:
>> I think we're kinda mostly on the same page (though saying it in
>> different ways). BTW I'm taking off my devils advocate hat now :)...
>>
>> What we're agreeing on I think is that;
>>
>> * getIn() should be immutable (when folks try to change it we can
>> throw the exception and describe how getOut() should be used to change
>> the message) - to prevent folks using the old code (which will require
>> code changes).
>> * having the original immutable message available is handy; but mostly
>> folks should concentrate on the 'out' (current name today)
>> * the out should be automatically populated with a clone of the IN (to
>> avoid all that pain with checking if there's an out or an in, or the
>> possible loss of headers etc. Internally we can use a CopyOnWrite
>> facade to reduce the cost of potentially copying a message which is
>> not actually mutated.
>>
>> Given that; I think we're mostly agreeing. However given the confusion
>> of getIn() and getOut() I'm wondering if actually your previous idea
>> of changing the api of exchange to have a getMessage() which returns
>> the thing a processor should be changing; then having a
>> getOriginalMessage() (which can be null if you are the first in the
>> chain) might reduce the amount of confusion?
>>
>> i.e. after sleeping on it, I'm warming to the idea of renaming getIn()
>> -> getOriginalMessage() and getOut() -> getMessage(). (Or maybe
>> getInputMessage() for getIn()?)
>>
>> Thoughts?
>
> Very good thoughts James. I think all this will removes many obstacles
> and confusing for new users to Camel.
>
> Now even the AggregationStrategy is less tricky to use as you will
> know that using getMessage gets what you are looking for.
>
> Maybe we should create a wiki page at Camel where we can outline the
> API proposal and ideas for the message facade.

Yeah.

BTW we could leave getIn() and getOut() & try implement the
ImmutableWrapper and CopyOnWriteFacade for In/Out respectively and try
them out inside the current API - before we go ahead with any kind of
renaming of getIn() and getOut().

Maybe we could even keep getIn() / getOut() around for 2.0 (and zap in
2.1) and leave deprecated and make them just delegate to the new
method names (say getInputMessage() and getMessage()). The deprecated
message could then describe how folks typically should switch
getIn()/getOut() to using getMessage() unless they really really want
the immutable input message getInputMessage().

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Claus Ibsen <cl...@gmail.com>]

> I think we're kinda mostly on the same page (though saying it in
> different ways). BTW I'm taking off my devils advocate hat now :)...
>
> What we're agreeing on I think is that;
>
> * getIn() should be immutable (when folks try to change it we can
> throw the exception and describe how getOut() should be used to change
> the message) - to prevent folks using the old code (which will require
> code changes).
> * having the original immutable message available is handy; but mostly
> folks should concentrate on the 'out' (current name today)
> * the out should be automatically populated with a clone of the IN (to
> avoid all that pain with checking if there's an out or an in, or the
> possible loss of headers etc. Internally we can use a CopyOnWrite
> facade to reduce the cost of potentially copying a message which is
> not actually mutated.
>
> Given that; I think we're mostly agreeing. However given the confusion
> of getIn() and getOut() I'm wondering if actually your previous idea
> of changing the api of exchange to have a getMessage() which returns
> the thing a processor should be changing; then having a
> getOriginalMessage() (which can be null if you are the first in the
> chain) might reduce the amount of confusion?
>
> i.e. after sleeping on it, I'm warming to the idea of renaming getIn()
> -> getOriginalMessage() and getOut() -> getMessage(). (Or maybe
> getInputMessage() for getIn()?)
>
> Thoughts?

Very good thoughts James. I think all this will removes many obstacles
and confusing for new users to Camel.

Now even the AggregationStrategy is less tricky to use as you will
know that using getMessage gets what you are looking for.

Maybe we should create a wiki page at Camel where we can outline the
API proposal and ideas for the message facade.



>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[James Strachan <ja...@gmail.com>]

2009/7/13 Claus Ibsen <cl...@gmail.com>:
>>
>> Reading between the lines; are you really just trying to make folks
>> use (what is currently called) "getOut()" and never try mutate what is
>> currently called getIn()?
>>
>> i.e. so by default the "OUT" property is defaulted to a copy of IN
>> that folks can change/mutate.
>>
>> (what we call these 2 methods is a separate discussion, whether its
>> "in,out" or "originalMessage,message" or whatever
>>
>
> Hadrian and I had a chat today and we are clearing up some bits.
> I am more on line with him now on the IN and OUT thing.
>
> So lets keep them.
>
> And use the time to fix the tiny bits such as the getOut() doing its
> lazy creating a new empty message.
> And whether its possible to let IN be immutable.

I think we're kinda mostly on the same page (though saying it in
different ways). BTW I'm taking off my devils advocate hat now :)...

What we're agreeing on I think is that;

* getIn() should be immutable (when folks try to change it we can
throw the exception and describe how getOut() should be used to change
the message) - to prevent folks using the old code (which will require
code changes).
* having the original immutable message available is handy; but mostly
folks should concentrate on the 'out' (current name today)
* the out should be automatically populated with a clone of the IN (to
avoid all that pain with checking if there's an out or an in, or the
possible loss of headers etc. Internally we can use a CopyOnWrite
facade to reduce the cost of potentially copying a message which is
not actually mutated.

Given that; I think we're mostly agreeing. However given the confusion
of getIn() and getOut() I'm wondering if actually your previous idea
of changing the api of exchange to have a getMessage() which returns
the thing a processor should be changing; then having a
getOriginalMessage() (which can be null if you are the first in the
chain) might reduce the amount of confusion?

i.e. after sleeping on it, I'm warming to the idea of renaming getIn()
-> getOriginalMessage() and getOut() -> getMessage(). (Or maybe
getInputMessage() for getIn()?)

Thoughts?

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Claus Ibsen <cl...@gmail.com>]

>
> Reading between the lines; are you really just trying to make folks
> use (what is currently called) "getOut()" and never try mutate what is
> currently called getIn()?
>
> i.e. so by default the "OUT" property is defaulted to a copy of IN
> that folks can change/mutate.
>
> (what we call these 2 methods is a separate discussion, whether its
> "in,out" or "originalMessage,message" or whatever
>

Hadrian and I had a chat today and we are clearing up some bits.
I am more on line with him now on the IN and OUT thing.

So lets keep them.

And use the time to fix the tiny bits such as the getOut() doing its
lazy creating a new empty message.
And whether its possible to let IN be immutable.


/Claus

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[James Strachan <ja...@gmail.com>]

2009/7/13 James Strachan <ja...@gmail.com>:
> 2009/7/13 Claus Ibsen <cl...@gmail.com>:
>> On Mon, Jul 13, 2009 at 5:50 PM, James Strachan<ja...@gmail.com> wrote:
>>> I'm confused. So rather than calling Exchange.getIn() to get the IN,
>>> you'd have to do Exchange.getUnitOfWork().somethingOrOther() to get
>>> the last IN? I don't see that as any simpler; making one of the 2
>>> messages harder to find, just moves the confusion somewhere else?
>>
>> Having access to the original input in a route path is very rarely
>> requested by end users.
>> In fact I think its even more confused if getIN always returned the
>> original input. (what should that be if a message is routed over (to
>> seda/from seda) a seda endpoint?)
>>
>> So what I am saying is that we should have a model similar to Mule and
>> Spring Integration = simpler and easier to understand and use.
>> And also simpler and easier to use within the Camel framework itself -
>> eg all the code that copies from IN to OUT and OUT to IN and whatnot.
>>
>> The last IN = the message itself.
>>
>> Given this route:
>> from("direct:start").to("bean:foo").processRef("bar").transform(constant("Bye
>> World")).processRef("me");
>>
>> And we send a "Hello World" message to direct:start. The message would be
>>
>> bean:foo
>> getMessage = "Hello World
>>
>> bar
>> getMessage = "Hello World
>>
>> me
>> getMessage = "Bye World
>>
>>
>> Now imagine we mutate the messages in bean foo
>> public String reverse(String s) {
>>   // return the reversed string
>> }
>
> am confused; a bean never even sees an Exchange/Message typically?
>
>> bean:foo
>> getMessage = "Hello World"
>>
>> bar
>> getMessage = "dlroW olleH"
>>
>> me
>> getMessage = "Bye World"
>>
>>
>> No more IN vs OUT and other confusing bits. getMessage gets you the
>> payload as it is.
>
> I dunnot what 'it' is? You mean we always copy the IN to the OUT
> regardless of what the code does just in case; so you mutate the
> current message - then if you zap headers/body you have to keep a
> track of them yourself in case you want to look at them again?
>
>
>> If you have looked as much Camel code as I have you would also flag
>> that the IN OUT is leading to confusing and broken code.
>
> Note you were the one that pushed back when I said lets make getIn()
> immutable which would break some code :). Now you're saying lets just
> break *all* camel code.
>
>
>> And trust me its broken, when you use the Exchange, but the BODY
>> extractor of ProducerTemplate is fairly good at guessing and returning
>> the right one from OUT or IN :)
>>
>>
>> If its really that inportant to get hold of the original input we can
>> add a getter to it on the Exchange.
>> - getInputMessage()
>> - getOriginalMessage()
>> or what a good name would be.
>
> but this is exactly why we were talking about having getIn() and
> getOut(). Out is the output, the thing the processor/component
> mutates/generates. IN is just the immutable view of what was sent in.
>
>
>> And on top of this we do not lose headers as we do now when people use
>> a processor and do
>> exchange.getOut().setBody(myBody);
>>
>> And then all the headers from IN is lost.
>
> On this point aren't you confusing bugs in current implementation
> methods (like getOut()) with letting folks access the IN message while
> creating an OUT?
>
> As an experiment; could you mock up what the cxf and JBi components
> would look like having no getIn() and just having a getMessage() on
> Exchange? e.g. how would we map our code to CXF/JBI which require a
> separate IN and OUT?
>
> Or rather; if getOut() => getMessage() and we let folks see the
> immutable version of the original message - on a method called, say,
> getIn() - then isn't this what I was proposing last week; make getIn()
> immutable as the record of what was sent in then all
> mutation/modifications occur on getOut().
>
> Note I'm not 100% against any radical API change - I just want to make
> sure we're doing it for the right reasons.

Reading between the lines; are you really just trying to make folks
use (what is currently called) "getOut()" and never try mutate what is
currently called getIn()?

i.e. so by default the "OUT" property is defaulted to a copy of IN
that folks can change/mutate.

(what we call these 2 methods is a separate discussion, whether its
"in,out" or "originalMessage,message" or whatever


-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Hadrian Zbarcea <hz...@gmail.com>]

Roman,

Is there any api change you are proposing?  If yes, could you please  
outline the code changes you are suggesting?

Thanks,
Hadrian


On Jul 13, 2009, at 3:01 PM, Roman Kalukiewicz wrote:

> 2009/7/13 Claus Ibsen <cl...@gmail.com>:
>> On Mon, Jul 13, 2009 at 5:50 PM, James Strachan<james.strachan@gmail.com 
>> > wrote:
>>> I'm confused. So rather than calling Exchange.getIn() to get the IN,
>>> you'd have to do Exchange.getUnitOfWork().somethingOrOther() to get
>>> the last IN? I don't see that as any simpler; making one of the 2
>>> messages harder to find, just moves the confusion somewhere else?
>>>
>> James you are not the only confused, many Camel new users are - IN vs
>> OUT vs FAULT and exchange patterns and why the hell they lose they
>> message or headers.
>> eg a simple route then need the assistance from the tracer to help  
>> pin
>> point what goes wrong and where/what causes it to loose their
>> message/headers.
>>
>> No I do not want that, you need to do:
>> Exchange.getUnitOfWork().somethingOrOther()  to get IN.
>> IN is simply = getMessage.
>>
>> IN + OUT is combined into a single Message. getMessage/setMessage.
>
> It looks like Claus would like to implement my original idea of
> removing IN/OUT/FAULT as separate messages, but put them into single
> message. Then there is always CURRENT message in the exchange) or
> however we call it then.
>
>> If you really need hold of the original input (as written before that
>> is hardly ever ever needed) its currently avail on the UoW.
>> But we can expose this on the Exchange as well in case its makes a  
>> point.
>
> What I would propose would be to have an original input message kept
> somewhere, but we shouldn't mutate it. I would even prefer to have it
> in its original form like JBI Exchange, ServletRequest or whatever...
> for reference purposes, or if you want to do something very
> technology-specific. On the other hand I would prefer to have a
> generic exchange - without cusom subclasses at all.
>
> Roman
>
>>> 2009/7/12 Claus Ibsen <cl...@gmail.com>:
>>>> Hi
>>>>
>>>> Starting the kettle to brew me a grok of hot coffee, as its sunday
>>>> morning and I guess I got the energy to toss in my 2$ in the  
>>>> bucket of
>>>> ideas
>>>> Now that we are into the open talks and that it seems we can open  
>>>> the
>>>> box with the IN, OUT, FAULT again.
>>>>
>>>> I advocate for only one message on the implementation side. On  
>>>> the API
>>>> side we could still have some notion of IN / OUT to keep the API
>>>> migration / impact simpler. But at the end having one message API
>>>> makes it simpler. There could be a boolean to test whether an  
>>>> output
>>>> has been set or not. Just as if we have a test for fault.
>>>>
>>>>
>>>> Proposed Exchange API
>>>> ==================
>>>>
>>>>  Message getMessage()
>>>>  void setMessage(Message msg);
>>>>
>>>>  boolean messageChanged();
>>>>   -- false = message have newer been mutated, eg its still the  
>>>> same input
>>>>   -- true = the message or message body have been mutated in some  
>>>> way
>>>> (setMessage or setBody called)
>>>>   -- need to find a good name for this method, eg testing whether  
>>>> the
>>>> message have been changed/updated or not
>>>>
>>>> That leaves us with a simple API.
>>>>
>>>> Then we do not need to worry about all the message management
>>>> internally in Camel, where we need to figure out whether to set  
>>>> data
>>>> on IN or OUT and what else we kinda do, and it gets a bit complex  
>>>> over
>>>> time how to do this correctly.
>>>>
>>>>
>>>> MEP: InOnly
>>>> =========
>>>> Now we can do as Hadrian want. Newer return something in OUT. And
>>>> leave the original input on the Exchange.
>>>> Here we need to use James CopyOnWrite technique so we can  
>>>> preserve the
>>>> original message in case its mutated during routing.
>>>>
>>>>
>>>> MEP: InOut
>>>> =========
>>>> If an OUT message have been set (eg there is a reply), can be
>>>> determined if the messageChanged() == true. (then its not the  
>>>> original
>>>> message anymore). And yes the current code does this. It will just
>>>> copy whatever there is in IN and use it as OUT if no OUT was set.
>>>> With this proposal this will be improved as its easier to  
>>>> determine if
>>>> there is an OUT message or not.
>>>>
>>>>
>>>> Original message
>>>> =============
>>>> Currently there is an API to get the original message from the
>>>> UnitOfWork as its needed when doing redeliveries and a message was
>>>> doomed
>>>> and had to be moved to the dead letter channel. Then it makes much
>>>> more sense to move the original message than the current mutated
>>>> message.
>>>> ( i wonder if it should be default behavior )
>>>>
>>>>
>>>> AggregationStrategy
>>>> ===============
>>>> Another benefit with a single getMessage() is that end users using
>>>> AggregationStrategy will not be confused how to use it.
>>>> Should I look in IN or OUT. And the current code can actually be a
>>>> "lucky draw" as whether the data is in IN or OUT depending on facts
>>>> such as how the route path is.
>>>> We can even change it to pass in Message object instead of bare  
>>>> bone
>>>> Exchange. You can always go from Message to exchange using
>>>> getExchange().
>>>>
>>>>
>>>> All the processors / components / data formats
>>>> ===================================
>>>> Logic will be easier as well as they do not need to cater for  
>>>> IN / OUT
>>>> and where and how to set a result. Just work on the Message.
>>>>
>>>>
>>>> Use @deprecated for steady migration
>>>> ============================
>>>> Hadrian suggested that for the API migration to a single getMessage
>>>> you could let the getIn/setIn getOut/setOut delegate to
>>>> getMessage/setMessage.
>>>> And then mark them as @deprecated and then gradually change the  
>>>> camel
>>>> code as we goes. So I do not think the hold up / change would  
>>>> take a
>>>> lot of time and energy to get done.
>>>>
>>>>
>>>> Final words
>>>> ========
>>>> So if it was possible to simplify the API and reduce the IN OUT  
>>>> FAULT
>>>> to a simpler API, even go as far as I would like with a single
>>>> Message.
>>>> Then that would be really great and worth a hold up for a  
>>>> imminent 2.0 release.
>>>>
>>>>
>>>> Other frameworks
>>>> =============
>>>> I guess it wasn't final words after all :) Just wanted to say that
>>>> Mule, Spring Integration also just have a single message for the
>>>> message.
>>>> And yes I think their projects are also successful so it is not a  
>>>> loss
>>>> that they do not have IN OUT. In fact I think their API is easier  
>>>> to
>>>> work with than Camel.
>>>>
>>>> We are fortunate that most people with Camel do not work directly  
>>>> with
>>>> Exchange but work more with Camel returning the message as an  
>>>> expected
>>>> body type using its type converters. That helps a lot.
>>>>
>>>> But we have stories form Camel 1.x where people get confused whey
>>>> Camel return their original input in some situations where it was  
>>>> not
>>>> expected in the OUT message. Or the fact producer template  
>>>> sendBody is
>>>> not void in 1.x. Then people think it can be used for InOut.
>>>>
>>>> Okay end of mail.
>>>>
>>>>
>>>> On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<james.strachan@gmail.com 
>>>> > wrote:
>>>>> 2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
>>>>>> Moved this slightly different topic to a separate thread.
>>>>>>
>>>>>> ++1 from me.
>>>>>>
>>>>>> Do we need a vote on this one or it's a consensus?  Claus, it  
>>>>>> looks like you
>>>>>> agree, but pointed out that there is some work involved, correct?
>>>>>>
>>>>>> Hadrian
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>>>>>
>>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>>
>>>>>>>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<james.strachan@gmail.com 
>>>>>>>> >
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>>>>
>>>>>>>>>> Hi
>>>>>>>>>>
>>>>>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>>>>>
>>>>>>>>>> The ProducerCache that is doing the actual sending when  
>>>>>>>>>> using template
>>>>>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>>>>>
>>>>>>>>>> Well I am playing with to let it adhere to the principle  
>>>>>>>>>> Hadrian
>>>>>>>>>> pointed out. He wanted the IN to be more static.
>>>>>>>>>> And we cannot get there yet when you do routing as all the  
>>>>>>>>>> processors
>>>>>>>>>> rely on IN being able to mutate during routing.
>>>>>>>>>>
>>>>>>>>>> Anyway my grief is that when you send in an Exchange the  
>>>>>>>>>> result would
>>>>>>>>>> sometimes be stored on IN and not OUT.
>>>>>>>>>> What I want it to do always is to store the result in OUT  
>>>>>>>>>> and keep IN
>>>>>>>>>> as the original input.
>>>>>>>>>>
>>>>>>>>>> The code to do this is now a bit more complex than just  
>>>>>>>>>> before
>>>>>>>>>>
>>>>>>>>>>               // copy the original input
>>>>>>>>>>               Message original = exchange.getIn().copy();
>>>>>>>>>>
>>>>>>>>>>               producer.process(exchange);
>>>>>>>>>>
>>>>>>>>>>               // if no OUT then set current IN as result  
>>>>>>>>>> (except for
>>>>>>>>>> optional out)
>>>>>>>>>>               if (!exchange.hasOut() &&  
>>>>>>>>>> exchange.getPattern() !=
>>>>>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>>>>>                   // but only if its not the same as  
>>>>>>>>>> original IN to
>>>>>>>>>> avoid duplicating it
>>>>>>>>>>                   // and to adhere to the fact that there  
>>>>>>>>>> was no OUT
>>>>>>>>>> result at all
>>>>>>>>>>                   if (original.getBody() != null &&
>>>>>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>>>>>                       exchange.setOut(exchange.getIn());
>>>>>>>>>>                   }
>>>>>>>>>>               }
>>>>>>>>>>               // and restore original in
>>>>>>>>>>               exchange.setIn(original);
>>>>>>>>>>
>>>>>>>>>>               return exchange;
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> What I need to do is to copy the original IN message as it  
>>>>>>>>>> can be
>>>>>>>>>> mutated during routing.
>>>>>>>>>
>>>>>>>>> How about we prevent mutation of the IN message? Create a  
>>>>>>>>> Message
>>>>>>>>> facade which throws UnsupportedOperationException if you try  
>>>>>>>>> to mutate
>>>>>>>>> it in any way. Then we can pass the same read-only Message  
>>>>>>>>> around as
>>>>>>>>> the IN within retry loops or from step to step if no new  
>>>>>>>>> output is
>>>>>>>>> created (e.g. in a content based router where you just move  
>>>>>>>>> a Message
>>>>>>>>> to the right endpoint without changing it)
>>>>>>>>>
>>>>>>>>
>>>>>>>> A good idea but will break a lot of logic in Camel.
>>>>>>>
>>>>>>> Agreed. But with the benefit that we'd be able to get rid of  
>>>>>>> all the
>>>>>>> defensive copies in our code; plus we'd be able to pass the same
>>>>>>> Message from step to step. The API would be a bit more clean; to
>>>>>>> change the output, you create an OUT message (maybe by copying  
>>>>>>> the
>>>>>>> IN).
>>>>>>>
>>>>>>>
>>>>>>>> Most of the Camel
>>>>>>>> processors work on the IN message and set the result on  
>>>>>>>> either IN or
>>>>>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>>>>>> original input? Or am I mistaking?
>>>>>>>
>>>>>>> Yeah, we'd have to patch code to no longer mutate IN
>>>>>>>
>>>>>>>
>>>>>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>>>>>> immutable and always the original input?
>>>>>>>
>>>>>>> If no OUT, then no output was created, so pass the IN along...
>>>>>>>
>>>>>>>
>>>>>>> OK how about this; a CopyOnWriteMessageFacade which does not  
>>>>>>> mutate
>>>>>>> the original message at all ever; if a message is used in a  
>>>>>>> purely
>>>>>>> read only way, it does nothing but delegate to the original  
>>>>>>> message -
>>>>>>> but then as soon as someone mutates it, it creates a copy and  
>>>>>>> uses
>>>>>>> that from that point on?
>>>>>>>
>>>>>>> i.e. make the copy of the message lazy - and only make a copy  
>>>>>>> when a
>>>>>>> Processor really does try to mutate the Message?
>>>>>>>
>>>>>>> Then we'd get the best of both worlds; avoid breaking old code  
>>>>>>> but
>>>>>>> avoid tons of unnecessary copies?
>>>>>
>>>>> BTW then using the current API you could have a Message and then  
>>>>> call
>>>>>
>>>>> Message origin = ...;
>>>>> Message newMsg = origin.copy().copy().copy().copy();
>>>>>
>>>>> and the message would not actually be copied at all; new would  
>>>>> just be
>>>>> a CopyOnWriteMessageFacade which would hold a reference to  
>>>>> 'origin' as
>>>>> the readOnlyMessage (which it never mutates).
>>>>>
>>>>> The copy would only take place if you did
>>>>>
>>>>> newMsg.setBody("foo")
>>>>>
>>>>> --
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>>
>>>>> Open Source Integration
>>>>> http://fusesource.com/
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>
>>>
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://fusesource.com/
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Roman Kalukiewicz <ro...@gmail.com>]

2009/7/13 Claus Ibsen <cl...@gmail.com>:
> On Mon, Jul 13, 2009 at 5:50 PM, James Strachan<ja...@gmail.com> wrote:
>> I'm confused. So rather than calling Exchange.getIn() to get the IN,
>> you'd have to do Exchange.getUnitOfWork().somethingOrOther() to get
>> the last IN? I don't see that as any simpler; making one of the 2
>> messages harder to find, just moves the confusion somewhere else?
>>
> James you are not the only confused, many Camel new users are - IN vs
> OUT vs FAULT and exchange patterns and why the hell they lose they
> message or headers.
> eg a simple route then need the assistance from the tracer to help pin
> point what goes wrong and where/what causes it to loose their
> message/headers.
>
> No I do not want that, you need to do:
> Exchange.getUnitOfWork().somethingOrOther()  to get IN.
> IN is simply = getMessage.
>
> IN + OUT is combined into a single Message. getMessage/setMessage.

It looks like Claus would like to implement my original idea of
removing IN/OUT/FAULT as separate messages, but put them into single
message. Then there is always CURRENT message in the exchange) or
however we call it then.

> If you really need hold of the original input (as written before that
> is hardly ever ever needed) its currently avail on the UoW.
> But we can expose this on the Exchange as well in case its makes a point.

What I would propose would be to have an original input message kept
somewhere, but we shouldn't mutate it. I would even prefer to have it
in its original form like JBI Exchange, ServletRequest or whatever...
for reference purposes, or if you want to do something very
technology-specific. On the other hand I would prefer to have a
generic exchange - without cusom subclasses at all.

Roman

>> 2009/7/12 Claus Ibsen <cl...@gmail.com>:
>>> Hi
>>>
>>> Starting the kettle to brew me a grok of hot coffee, as its sunday
>>> morning and I guess I got the energy to toss in my 2$ in the bucket of
>>> ideas
>>> Now that we are into the open talks and that it seems we can open the
>>> box with the IN, OUT, FAULT again.
>>>
>>> I advocate for only one message on the implementation side. On the API
>>> side we could still have some notion of IN / OUT to keep the API
>>> migration / impact simpler. But at the end having one message API
>>> makes it simpler. There could be a boolean to test whether an output
>>> has been set or not. Just as if we have a test for fault.
>>>
>>>
>>> Proposed Exchange API
>>> ==================
>>>
>>>  Message getMessage()
>>>  void setMessage(Message msg);
>>>
>>>  boolean messageChanged();
>>>   -- false = message have newer been mutated, eg its still the same input
>>>   -- true = the message or message body have been mutated in some way
>>> (setMessage or setBody called)
>>>   -- need to find a good name for this method, eg testing whether the
>>> message have been changed/updated or not
>>>
>>> That leaves us with a simple API.
>>>
>>> Then we do not need to worry about all the message management
>>> internally in Camel, where we need to figure out whether to set data
>>> on IN or OUT and what else we kinda do, and it gets a bit complex over
>>> time how to do this correctly.
>>>
>>>
>>> MEP: InOnly
>>> =========
>>> Now we can do as Hadrian want. Newer return something in OUT. And
>>> leave the original input on the Exchange.
>>> Here we need to use James CopyOnWrite technique so we can preserve the
>>> original message in case its mutated during routing.
>>>
>>>
>>> MEP: InOut
>>> =========
>>> If an OUT message have been set (eg there is a reply), can be
>>> determined if the messageChanged() == true. (then its not the original
>>> message anymore). And yes the current code does this. It will just
>>> copy whatever there is in IN and use it as OUT if no OUT was set.
>>> With this proposal this will be improved as its easier to determine if
>>> there is an OUT message or not.
>>>
>>>
>>> Original message
>>> =============
>>> Currently there is an API to get the original message from the
>>> UnitOfWork as its needed when doing redeliveries and a message was
>>> doomed
>>> and had to be moved to the dead letter channel. Then it makes much
>>> more sense to move the original message than the current mutated
>>> message.
>>> ( i wonder if it should be default behavior )
>>>
>>>
>>> AggregationStrategy
>>> ===============
>>> Another benefit with a single getMessage() is that end users using
>>> AggregationStrategy will not be confused how to use it.
>>> Should I look in IN or OUT. And the current code can actually be a
>>> "lucky draw" as whether the data is in IN or OUT depending on facts
>>> such as how the route path is.
>>> We can even change it to pass in Message object instead of bare bone
>>> Exchange. You can always go from Message to exchange using
>>> getExchange().
>>>
>>>
>>> All the processors / components / data formats
>>> ===================================
>>> Logic will be easier as well as they do not need to cater for IN / OUT
>>> and where and how to set a result. Just work on the Message.
>>>
>>>
>>> Use @deprecated for steady migration
>>> ============================
>>> Hadrian suggested that for the API migration to a single getMessage
>>> you could let the getIn/setIn getOut/setOut delegate to
>>> getMessage/setMessage.
>>> And then mark them as @deprecated and then gradually change the camel
>>> code as we goes. So I do not think the hold up / change would take a
>>> lot of time and energy to get done.
>>>
>>>
>>> Final words
>>> ========
>>> So if it was possible to simplify the API and reduce the IN OUT FAULT
>>> to a simpler API, even go as far as I would like with a single
>>> Message.
>>> Then that would be really great and worth a hold up for a imminent 2.0 release.
>>>
>>>
>>> Other frameworks
>>> =============
>>> I guess it wasn't final words after all :) Just wanted to say that
>>> Mule, Spring Integration also just have a single message for the
>>> message.
>>> And yes I think their projects are also successful so it is not a loss
>>> that they do not have IN OUT. In fact I think their API is easier to
>>> work with than Camel.
>>>
>>> We are fortunate that most people with Camel do not work directly with
>>> Exchange but work more with Camel returning the message as an expected
>>> body type using its type converters. That helps a lot.
>>>
>>> But we have stories form Camel 1.x where people get confused whey
>>> Camel return their original input in some situations where it was not
>>> expected in the OUT message. Or the fact producer template sendBody is
>>> not void in 1.x. Then people think it can be used for InOut.
>>>
>>> Okay end of mail.
>>>
>>>
>>> On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<ja...@gmail.com> wrote:
>>>> 2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
>>>>> Moved this slightly different topic to a separate thread.
>>>>>
>>>>> ++1 from me.
>>>>>
>>>>> Do we need a vote on this one or it's a consensus?  Claus, it looks like you
>>>>> agree, but pointed out that there is some work involved, correct?
>>>>>
>>>>> Hadrian
>>>>>
>>>>>
>>>>>
>>>>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>>>>
>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>
>>>>>>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<ja...@gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>>>>
>>>>>>>>> The ProducerCache that is doing the actual sending when using template
>>>>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>>>>
>>>>>>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>>>>>>> pointed out. He wanted the IN to be more static.
>>>>>>>>> And we cannot get there yet when you do routing as all the processors
>>>>>>>>> rely on IN being able to mutate during routing.
>>>>>>>>>
>>>>>>>>> Anyway my grief is that when you send in an Exchange the result would
>>>>>>>>> sometimes be stored on IN and not OUT.
>>>>>>>>> What I want it to do always is to store the result in OUT and keep IN
>>>>>>>>> as the original input.
>>>>>>>>>
>>>>>>>>> The code to do this is now a bit more complex than just before
>>>>>>>>>
>>>>>>>>>               // copy the original input
>>>>>>>>>               Message original = exchange.getIn().copy();
>>>>>>>>>
>>>>>>>>>               producer.process(exchange);
>>>>>>>>>
>>>>>>>>>               // if no OUT then set current IN as result (except for
>>>>>>>>> optional out)
>>>>>>>>>               if (!exchange.hasOut() && exchange.getPattern() !=
>>>>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>>>>                   // but only if its not the same as original IN to
>>>>>>>>> avoid duplicating it
>>>>>>>>>                   // and to adhere to the fact that there was no OUT
>>>>>>>>> result at all
>>>>>>>>>                   if (original.getBody() != null &&
>>>>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>>>>                       exchange.setOut(exchange.getIn());
>>>>>>>>>                   }
>>>>>>>>>               }
>>>>>>>>>               // and restore original in
>>>>>>>>>               exchange.setIn(original);
>>>>>>>>>
>>>>>>>>>               return exchange;
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> What I need to do is to copy the original IN message as it can be
>>>>>>>>> mutated during routing.
>>>>>>>>
>>>>>>>> How about we prevent mutation of the IN message? Create a Message
>>>>>>>> facade which throws UnsupportedOperationException if you try to mutate
>>>>>>>> it in any way. Then we can pass the same read-only Message around as
>>>>>>>> the IN within retry loops or from step to step if no new output is
>>>>>>>> created (e.g. in a content based router where you just move a Message
>>>>>>>> to the right endpoint without changing it)
>>>>>>>>
>>>>>>>
>>>>>>> A good idea but will break a lot of logic in Camel.
>>>>>>
>>>>>> Agreed. But with the benefit that we'd be able to get rid of all the
>>>>>> defensive copies in our code; plus we'd be able to pass the same
>>>>>> Message from step to step. The API would be a bit more clean; to
>>>>>> change the output, you create an OUT message (maybe by copying the
>>>>>> IN).
>>>>>>
>>>>>>
>>>>>>> Most of the Camel
>>>>>>> processors work on the IN message and set the result on either IN or
>>>>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>>>>> original input? Or am I mistaking?
>>>>>>
>>>>>> Yeah, we'd have to patch code to no longer mutate IN
>>>>>>
>>>>>>
>>>>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>>>>> immutable and always the original input?
>>>>>>
>>>>>> If no OUT, then no output was created, so pass the IN along...
>>>>>>
>>>>>>
>>>>>> OK how about this; a CopyOnWriteMessageFacade which does not mutate
>>>>>> the original message at all ever; if a message is used in a purely
>>>>>> read only way, it does nothing but delegate to the original message -
>>>>>> but then as soon as someone mutates it, it creates a copy and uses
>>>>>> that from that point on?
>>>>>>
>>>>>> i.e. make the copy of the message lazy - and only make a copy when a
>>>>>> Processor really does try to mutate the Message?
>>>>>>
>>>>>> Then we'd get the best of both worlds; avoid breaking old code but
>>>>>> avoid tons of unnecessary copies?
>>>>
>>>> BTW then using the current API you could have a Message and then call
>>>>
>>>> Message origin = ...;
>>>> Message newMsg = origin.copy().copy().copy().copy();
>>>>
>>>> and the message would not actually be copied at all; new would just be
>>>> a CopyOnWriteMessageFacade which would hold a reference to 'origin' as
>>>> the readOnlyMessage (which it never mutates).
>>>>
>>>> The copy would only take place if you did
>>>>
>>>> newMsg.setBody("foo")
>>>>
>>>> --
>>>> James
>>>> -------
>>>> http://macstrac.blogspot.com/
>>>>
>>>> Open Source Integration
>>>> http://fusesource.com/
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>
>>
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://fusesource.com/
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Claus Ibsen <cl...@gmail.com>]

On Mon, Jul 13, 2009 at 5:50 PM, James Strachan<ja...@gmail.com> wrote:
> I'm confused. So rather than calling Exchange.getIn() to get the IN,
> you'd have to do Exchange.getUnitOfWork().somethingOrOther() to get
> the last IN? I don't see that as any simpler; making one of the 2
> messages harder to find, just moves the confusion somewhere else?
>
James you are not the only confused, many Camel new users are - IN vs
OUT vs FAULT and exchange patterns and why the hell they lose they
message or headers.
eg a simple route then need the assistance from the tracer to help pin
point what goes wrong and where/what causes it to loose their
message/headers.

No I do not want that, you need to do:
Exchange.getUnitOfWork().somethingOrOther()  to get IN.
IN is simply = getMessage.

IN + OUT is combined into a single Message. getMessage/setMessage.

If you really need hold of the original input (as written before that
is hardly ever ever needed) its currently avail on the UoW.
But we can expose this on the Exchange as well in case its makes a point.



> 2009/7/12 Claus Ibsen <cl...@gmail.com>:
>> Hi
>>
>> Starting the kettle to brew me a grok of hot coffee, as its sunday
>> morning and I guess I got the energy to toss in my 2$ in the bucket of
>> ideas
>> Now that we are into the open talks and that it seems we can open the
>> box with the IN, OUT, FAULT again.
>>
>> I advocate for only one message on the implementation side. On the API
>> side we could still have some notion of IN / OUT to keep the API
>> migration / impact simpler. But at the end having one message API
>> makes it simpler. There could be a boolean to test whether an output
>> has been set or not. Just as if we have a test for fault.
>>
>>
>> Proposed Exchange API
>> ==================
>>
>>  Message getMessage()
>>  void setMessage(Message msg);
>>
>>  boolean messageChanged();
>>   -- false = message have newer been mutated, eg its still the same input
>>   -- true = the message or message body have been mutated in some way
>> (setMessage or setBody called)
>>   -- need to find a good name for this method, eg testing whether the
>> message have been changed/updated or not
>>
>> That leaves us with a simple API.
>>
>> Then we do not need to worry about all the message management
>> internally in Camel, where we need to figure out whether to set data
>> on IN or OUT and what else we kinda do, and it gets a bit complex over
>> time how to do this correctly.
>>
>>
>> MEP: InOnly
>> =========
>> Now we can do as Hadrian want. Newer return something in OUT. And
>> leave the original input on the Exchange.
>> Here we need to use James CopyOnWrite technique so we can preserve the
>> original message in case its mutated during routing.
>>
>>
>> MEP: InOut
>> =========
>> If an OUT message have been set (eg there is a reply), can be
>> determined if the messageChanged() == true. (then its not the original
>> message anymore). And yes the current code does this. It will just
>> copy whatever there is in IN and use it as OUT if no OUT was set.
>> With this proposal this will be improved as its easier to determine if
>> there is an OUT message or not.
>>
>>
>> Original message
>> =============
>> Currently there is an API to get the original message from the
>> UnitOfWork as its needed when doing redeliveries and a message was
>> doomed
>> and had to be moved to the dead letter channel. Then it makes much
>> more sense to move the original message than the current mutated
>> message.
>> ( i wonder if it should be default behavior )
>>
>>
>> AggregationStrategy
>> ===============
>> Another benefit with a single getMessage() is that end users using
>> AggregationStrategy will not be confused how to use it.
>> Should I look in IN or OUT. And the current code can actually be a
>> "lucky draw" as whether the data is in IN or OUT depending on facts
>> such as how the route path is.
>> We can even change it to pass in Message object instead of bare bone
>> Exchange. You can always go from Message to exchange using
>> getExchange().
>>
>>
>> All the processors / components / data formats
>> ===================================
>> Logic will be easier as well as they do not need to cater for IN / OUT
>> and where and how to set a result. Just work on the Message.
>>
>>
>> Use @deprecated for steady migration
>> ============================
>> Hadrian suggested that for the API migration to a single getMessage
>> you could let the getIn/setIn getOut/setOut delegate to
>> getMessage/setMessage.
>> And then mark them as @deprecated and then gradually change the camel
>> code as we goes. So I do not think the hold up / change would take a
>> lot of time and energy to get done.
>>
>>
>> Final words
>> ========
>> So if it was possible to simplify the API and reduce the IN OUT FAULT
>> to a simpler API, even go as far as I would like with a single
>> Message.
>> Then that would be really great and worth a hold up for a imminent 2.0 release.
>>
>>
>> Other frameworks
>> =============
>> I guess it wasn't final words after all :) Just wanted to say that
>> Mule, Spring Integration also just have a single message for the
>> message.
>> And yes I think their projects are also successful so it is not a loss
>> that they do not have IN OUT. In fact I think their API is easier to
>> work with than Camel.
>>
>> We are fortunate that most people with Camel do not work directly with
>> Exchange but work more with Camel returning the message as an expected
>> body type using its type converters. That helps a lot.
>>
>> But we have stories form Camel 1.x where people get confused whey
>> Camel return their original input in some situations where it was not
>> expected in the OUT message. Or the fact producer template sendBody is
>> not void in 1.x. Then people think it can be used for InOut.
>>
>> Okay end of mail.
>>
>>
>> On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<ja...@gmail.com> wrote:
>>> 2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
>>>> Moved this slightly different topic to a separate thread.
>>>>
>>>> ++1 from me.
>>>>
>>>> Do we need a vote on this one or it's a consensus?  Claus, it looks like you
>>>> agree, but pointed out that there is some work involved, correct?
>>>>
>>>> Hadrian
>>>>
>>>>
>>>>
>>>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>>>
>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>
>>>>>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<ja...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>>>
>>>>>>>> The ProducerCache that is doing the actual sending when using template
>>>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>>>
>>>>>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>>>>>> pointed out. He wanted the IN to be more static.
>>>>>>>> And we cannot get there yet when you do routing as all the processors
>>>>>>>> rely on IN being able to mutate during routing.
>>>>>>>>
>>>>>>>> Anyway my grief is that when you send in an Exchange the result would
>>>>>>>> sometimes be stored on IN and not OUT.
>>>>>>>> What I want it to do always is to store the result in OUT and keep IN
>>>>>>>> as the original input.
>>>>>>>>
>>>>>>>> The code to do this is now a bit more complex than just before
>>>>>>>>
>>>>>>>>               // copy the original input
>>>>>>>>               Message original = exchange.getIn().copy();
>>>>>>>>
>>>>>>>>               producer.process(exchange);
>>>>>>>>
>>>>>>>>               // if no OUT then set current IN as result (except for
>>>>>>>> optional out)
>>>>>>>>               if (!exchange.hasOut() && exchange.getPattern() !=
>>>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>>>                   // but only if its not the same as original IN to
>>>>>>>> avoid duplicating it
>>>>>>>>                   // and to adhere to the fact that there was no OUT
>>>>>>>> result at all
>>>>>>>>                   if (original.getBody() != null &&
>>>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>>>                       exchange.setOut(exchange.getIn());
>>>>>>>>                   }
>>>>>>>>               }
>>>>>>>>               // and restore original in
>>>>>>>>               exchange.setIn(original);
>>>>>>>>
>>>>>>>>               return exchange;
>>>>>>>>
>>>>>>>>
>>>>>>>> What I need to do is to copy the original IN message as it can be
>>>>>>>> mutated during routing.
>>>>>>>
>>>>>>> How about we prevent mutation of the IN message? Create a Message
>>>>>>> facade which throws UnsupportedOperationException if you try to mutate
>>>>>>> it in any way. Then we can pass the same read-only Message around as
>>>>>>> the IN within retry loops or from step to step if no new output is
>>>>>>> created (e.g. in a content based router where you just move a Message
>>>>>>> to the right endpoint without changing it)
>>>>>>>
>>>>>>
>>>>>> A good idea but will break a lot of logic in Camel.
>>>>>
>>>>> Agreed. But with the benefit that we'd be able to get rid of all the
>>>>> defensive copies in our code; plus we'd be able to pass the same
>>>>> Message from step to step. The API would be a bit more clean; to
>>>>> change the output, you create an OUT message (maybe by copying the
>>>>> IN).
>>>>>
>>>>>
>>>>>> Most of the Camel
>>>>>> processors work on the IN message and set the result on either IN or
>>>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>>>> original input? Or am I mistaking?
>>>>>
>>>>> Yeah, we'd have to patch code to no longer mutate IN
>>>>>
>>>>>
>>>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>>>> immutable and always the original input?
>>>>>
>>>>> If no OUT, then no output was created, so pass the IN along...
>>>>>
>>>>>
>>>>> OK how about this; a CopyOnWriteMessageFacade which does not mutate
>>>>> the original message at all ever; if a message is used in a purely
>>>>> read only way, it does nothing but delegate to the original message -
>>>>> but then as soon as someone mutates it, it creates a copy and uses
>>>>> that from that point on?
>>>>>
>>>>> i.e. make the copy of the message lazy - and only make a copy when a
>>>>> Processor really does try to mutate the Message?
>>>>>
>>>>> Then we'd get the best of both worlds; avoid breaking old code but
>>>>> avoid tons of unnecessary copies?
>>>
>>> BTW then using the current API you could have a Message and then call
>>>
>>> Message origin = ...;
>>> Message newMsg = origin.copy().copy().copy().copy();
>>>
>>> and the message would not actually be copied at all; new would just be
>>> a CopyOnWriteMessageFacade which would hold a reference to 'origin' as
>>> the readOnlyMessage (which it never mutates).
>>>
>>> The copy would only take place if you did
>>>
>>> newMsg.setBody("foo")
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://fusesource.com/
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Claus Ibsen <cl...@gmail.com>]

On Mon, Jul 13, 2009 at 6:33 PM, James Strachan<ja...@gmail.com> wrote:
> 2009/7/13 Claus Ibsen <cl...@gmail.com>:
>> On Mon, Jul 13, 2009 at 5:50 PM, James Strachan<ja...@gmail.com> wrote:
>>> I'm confused. So rather than calling Exchange.getIn() to get the IN,
>>> you'd have to do Exchange.getUnitOfWork().somethingOrOther() to get
>>> the last IN? I don't see that as any simpler; making one of the 2
>>> messages harder to find, just moves the confusion somewhere else?
>>
>> Having access to the original input in a route path is very rarely
>> requested by end users.
>> In fact I think its even more confused if getIN always returned the
>> original input. (what should that be if a message is routed over (to
>> seda/from seda) a seda endpoint?)
>>
>> So what I am saying is that we should have a model similar to Mule and
>> Spring Integration = simpler and easier to understand and use.
>> And also simpler and easier to use within the Camel framework itself -
>> eg all the code that copies from IN to OUT and OUT to IN and whatnot.
>>
>> The last IN = the message itself.
>>
>> Given this route:
>> from("direct:start").to("bean:foo").processRef("bar").transform(constant("Bye
>> World")).processRef("me");
>>
>> And we send a "Hello World" message to direct:start. The message would be
>>
>> bean:foo
>> getMessage = "Hello World
>>
>> bar
>> getMessage = "Hello World
>>
>> me
>> getMessage = "Bye World
>>
>>
>> Now imagine we mutate the messages in bean foo
>> public String reverse(String s) {
>>   // return the reversed string
>> }
>
> am confused; a bean never even sees an Exchange/Message typically?
>
>> bean:foo
>> getMessage = "Hello World"
>>
>> bar
>> getMessage = "dlroW olleH"
>>
>> me
>> getMessage = "Bye World"
>>
>>
>> No more IN vs OUT and other confusing bits. getMessage gets you the
>> payload as it is.
>
> I dunnot what 'it' is? You mean we always copy the IN to the OUT
> regardless of what the code does just in case; so you mutate the
> current message - then if you zap headers/body you have to keep a
> track of them yourself in case you want to look at them again?
>

The current code in Camel makes it very easy for Camel to zap your
headers unintended.

Lets look at how it is today.

1:
IN = At the very first consumer its the original input (= good)

IN Body = Hello World
OUT = null

2:
Then the pipeline kicks in and invokes the next processor with the Exchange

Lets say this processor does some work on the exchange, it want to set
a new payload
Today you can set this payload in either IN or OUT, eg your choice.

Lets start with IN (see 3a)
exchange.getIn().setBody("I did this")

And try OUT instead (see 3b)
exchange.getOut().setBody("I also did this")


3a:
The pipeline still routes it and lets send it to a final destination,
a file producer that writes the content as a file
IN Body = I did this
OUT = null

This is lucky as the file producer always uses IN body to as source
for file content so the file is: I did this


3b
The pipeline still routes it and lets send it to a final destination,
a file producer that writes the content as a file
IN Body = I also did this
OUT = I also did this

We set an OUT body but the pipeline copied it to IN so the message is
what we expect
and the file producer can use IN to write the expected file content.

But now the Exchange is a bit confusing as we got both IN and OUT = the same.
And the original IN is lost.





>
>> If you have looked as much Camel code as I have you would also flag
>> that the IN OUT is leading to confusing and broken code.
>
> Note you were the one that pushed back when I said lets make getIn()
> immutable which would break some code :). Now you're saying lets just
> break *all* camel code.
>
>
>> And trust me its broken, when you use the Exchange, but the BODY
>> extractor of ProducerTemplate is fairly good at guessing and returning
>> the right one from OUT or IN :)
>>
>>
>> If its really that inportant to get hold of the original input we can
>> add a getter to it on the Exchange.
>> - getInputMessage()
>> - getOriginalMessage()
>> or what a good name would be.
>
> but this is exactly why we were talking about having getIn() and
> getOut(). Out is the output, the thing the processor/component
> mutates/generates. IN is just the immutable view of what was sent in.
>

The current model in Camel is IN not a immutable view. It can be
mutated as well, and this is actually essential that its done
as all the Camel processors rely on this that IN contains = the
current message. (and not what the original input is when the message
was consumed)

If we go by your suggestions.
1) All Camel code must be adapter to use OUT instead of IN as = the
current message. And eg fallback to IN if no OUT have been set
2) End users from Camel 1.x that are familiar with the (odd) IN and
OUT would have their code breaked and cannot understand why

3) So to make all aware that IN and OUT have changed we simply do not
have them anymore, or have them @deprecated.


>
>> And on top of this we do not lose headers as we do now when people use
>> a processor and do
>> exchange.getOut().setBody(myBody);
>>
>> And then all the headers from IN is lost.
>
> On this point aren't you confusing bugs in current implementation
> methods (like getOut()) with letting folks access the IN message while
> creating an OUT?
>
> As an experiment; could you mock up what the cxf and JBi components
> would look like having no getIn() and just having a getMessage() on
> Exchange? e.g. how would we map our code to CXF/JBI which require a
> separate IN and OUT?
>

Yeah these frameworks could require a bit more work. However we got a
camel-cxf and ca camel-jbi component where such logic can be
implemented.
If CXF can work with Mule or Spring Integration (they do not have
IN/OUT) why can it not work with Camel also?



> Or rather; if getOut() => getMessage() and we let folks see the
> immutable version of the original message - on a method called, say,
> getIn() - then isn't this what I was proposing last week; make getIn()
> immutable as the record of what was sent in then all
> mutation/modifications occur on getOut().
>
> Note I'm not 100% against any radical API change - I just want to make
> sure we're doing it for the right reasons.
>

It was in fact Hadrian that got the ball rolling. We where close to
close the lid and get 2.0 out.
However his proposal with FAULT -> OUT is good, but FAULT is not often
used anyway so the impact is not big.



> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[James Strachan <ja...@gmail.com>]

2009/7/13 Claus Ibsen <cl...@gmail.com>:
> On Mon, Jul 13, 2009 at 5:50 PM, James Strachan<ja...@gmail.com> wrote:
>> I'm confused. So rather than calling Exchange.getIn() to get the IN,
>> you'd have to do Exchange.getUnitOfWork().somethingOrOther() to get
>> the last IN? I don't see that as any simpler; making one of the 2
>> messages harder to find, just moves the confusion somewhere else?
>
> Having access to the original input in a route path is very rarely
> requested by end users.
> In fact I think its even more confused if getIN always returned the
> original input. (what should that be if a message is routed over (to
> seda/from seda) a seda endpoint?)
>
> So what I am saying is that we should have a model similar to Mule and
> Spring Integration = simpler and easier to understand and use.
> And also simpler and easier to use within the Camel framework itself -
> eg all the code that copies from IN to OUT and OUT to IN and whatnot.
>
> The last IN = the message itself.
>
> Given this route:
> from("direct:start").to("bean:foo").processRef("bar").transform(constant("Bye
> World")).processRef("me");
>
> And we send a "Hello World" message to direct:start. The message would be
>
> bean:foo
> getMessage = "Hello World
>
> bar
> getMessage = "Hello World
>
> me
> getMessage = "Bye World
>
>
> Now imagine we mutate the messages in bean foo
> public String reverse(String s) {
>   // return the reversed string
> }

am confused; a bean never even sees an Exchange/Message typically?

> bean:foo
> getMessage = "Hello World"
>
> bar
> getMessage = "dlroW olleH"
>
> me
> getMessage = "Bye World"
>
>
> No more IN vs OUT and other confusing bits. getMessage gets you the
> payload as it is.

I dunnot what 'it' is? You mean we always copy the IN to the OUT
regardless of what the code does just in case; so you mutate the
current message - then if you zap headers/body you have to keep a
track of them yourself in case you want to look at them again?


> If you have looked as much Camel code as I have you would also flag
> that the IN OUT is leading to confusing and broken code.

Note you were the one that pushed back when I said lets make getIn()
immutable which would break some code :). Now you're saying lets just
break *all* camel code.


> And trust me its broken, when you use the Exchange, but the BODY
> extractor of ProducerTemplate is fairly good at guessing and returning
> the right one from OUT or IN :)
>
>
> If its really that inportant to get hold of the original input we can
> add a getter to it on the Exchange.
> - getInputMessage()
> - getOriginalMessage()
> or what a good name would be.

but this is exactly why we were talking about having getIn() and
getOut(). Out is the output, the thing the processor/component
mutates/generates. IN is just the immutable view of what was sent in.


> And on top of this we do not lose headers as we do now when people use
> a processor and do
> exchange.getOut().setBody(myBody);
>
> And then all the headers from IN is lost.

On this point aren't you confusing bugs in current implementation
methods (like getOut()) with letting folks access the IN message while
creating an OUT?

As an experiment; could you mock up what the cxf and JBi components
would look like having no getIn() and just having a getMessage() on
Exchange? e.g. how would we map our code to CXF/JBI which require a
separate IN and OUT?

Or rather; if getOut() => getMessage() and we let folks see the
immutable version of the original message - on a method called, say,
getIn() - then isn't this what I was proposing last week; make getIn()
immutable as the record of what was sent in then all
mutation/modifications occur on getOut().

Note I'm not 100% against any radical API change - I just want to make
sure we're doing it for the right reasons.

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Claus Ibsen <cl...@gmail.com>]

On Mon, Jul 13, 2009 at 5:50 PM, James Strachan<ja...@gmail.com> wrote:
> I'm confused. So rather than calling Exchange.getIn() to get the IN,
> you'd have to do Exchange.getUnitOfWork().somethingOrOther() to get
> the last IN? I don't see that as any simpler; making one of the 2
> messages harder to find, just moves the confusion somewhere else?

Having access to the original input in a route path is very rarely
requested by end users.
In fact I think its even more confused if getIN always returned the
original input. (what should that be if a message is routed over (to
seda/from seda) a seda endpoint?)

So what I am saying is that we should have a model similar to Mule and
Spring Integration = simpler and easier to understand and use.
And also simpler and easier to use within the Camel framework itself -
eg all the code that copies from IN to OUT and OUT to IN and whatnot.

The last IN = the message itself.

Given this route:
from("direct:start").to("bean:foo").processRef("bar").transform(constant("Bye
World")).processRef("me");

And we send a "Hello World" message to direct:start. The message would be

bean:foo
getMessage = "Hello World

bar
getMessage = "Hello World

me
getMessage = "Bye World


Now imagine we mutate the messages in bean foo
public String reverse(String s) {
   // return the reversed string
}


bean:foo
getMessage = "Hello World"

bar
getMessage = "dlroW olleH"

me
getMessage = "Bye World"


No more IN vs OUT and other confusing bits. getMessage gets you the
payload as it is.

If you have looked as much Camel code as I have you would also flag
that the IN OUT is leading to confusing and broken code.
And trust me its broken, when you use the Exchange, but the BODY
extractor of ProducerTemplate is fairly good at guessing and returning
the right one from OUT or IN :)


If its really that inportant to get hold of the original input we can
add a getter to it on the Exchange.
- getInputMessage()
- getOriginalMessage()
or what a good name would be.


And on top of this we do not lose headers as we do now when people use
a processor and do
exchange.getOut().setBody(myBody);

And then all the headers from IN is lost.



>
> 2009/7/12 Claus Ibsen <cl...@gmail.com>:
>> Hi
>>
>> Starting the kettle to brew me a grok of hot coffee, as its sunday
>> morning and I guess I got the energy to toss in my 2$ in the bucket of
>> ideas
>> Now that we are into the open talks and that it seems we can open the
>> box with the IN, OUT, FAULT again.
>>
>> I advocate for only one message on the implementation side. On the API
>> side we could still have some notion of IN / OUT to keep the API
>> migration / impact simpler. But at the end having one message API
>> makes it simpler. There could be a boolean to test whether an output
>> has been set or not. Just as if we have a test for fault.
>>
>>
>> Proposed Exchange API
>> ==================
>>
>>  Message getMessage()
>>  void setMessage(Message msg);
>>
>>  boolean messageChanged();
>>   -- false = message have newer been mutated, eg its still the same input
>>   -- true = the message or message body have been mutated in some way
>> (setMessage or setBody called)
>>   -- need to find a good name for this method, eg testing whether the
>> message have been changed/updated or not
>>
>> That leaves us with a simple API.
>>
>> Then we do not need to worry about all the message management
>> internally in Camel, where we need to figure out whether to set data
>> on IN or OUT and what else we kinda do, and it gets a bit complex over
>> time how to do this correctly.
>>
>>
>> MEP: InOnly
>> =========
>> Now we can do as Hadrian want. Newer return something in OUT. And
>> leave the original input on the Exchange.
>> Here we need to use James CopyOnWrite technique so we can preserve the
>> original message in case its mutated during routing.
>>
>>
>> MEP: InOut
>> =========
>> If an OUT message have been set (eg there is a reply), can be
>> determined if the messageChanged() == true. (then its not the original
>> message anymore). And yes the current code does this. It will just
>> copy whatever there is in IN and use it as OUT if no OUT was set.
>> With this proposal this will be improved as its easier to determine if
>> there is an OUT message or not.
>>
>>
>> Original message
>> =============
>> Currently there is an API to get the original message from the
>> UnitOfWork as its needed when doing redeliveries and a message was
>> doomed
>> and had to be moved to the dead letter channel. Then it makes much
>> more sense to move the original message than the current mutated
>> message.
>> ( i wonder if it should be default behavior )
>>
>>
>> AggregationStrategy
>> ===============
>> Another benefit with a single getMessage() is that end users using
>> AggregationStrategy will not be confused how to use it.
>> Should I look in IN or OUT. And the current code can actually be a
>> "lucky draw" as whether the data is in IN or OUT depending on facts
>> such as how the route path is.
>> We can even change it to pass in Message object instead of bare bone
>> Exchange. You can always go from Message to exchange using
>> getExchange().
>>
>>
>> All the processors / components / data formats
>> ===================================
>> Logic will be easier as well as they do not need to cater for IN / OUT
>> and where and how to set a result. Just work on the Message.
>>
>>
>> Use @deprecated for steady migration
>> ============================
>> Hadrian suggested that for the API migration to a single getMessage
>> you could let the getIn/setIn getOut/setOut delegate to
>> getMessage/setMessage.
>> And then mark them as @deprecated and then gradually change the camel
>> code as we goes. So I do not think the hold up / change would take a
>> lot of time and energy to get done.
>>
>>
>> Final words
>> ========
>> So if it was possible to simplify the API and reduce the IN OUT FAULT
>> to a simpler API, even go as far as I would like with a single
>> Message.
>> Then that would be really great and worth a hold up for a imminent 2.0 release.
>>
>>
>> Other frameworks
>> =============
>> I guess it wasn't final words after all :) Just wanted to say that
>> Mule, Spring Integration also just have a single message for the
>> message.
>> And yes I think their projects are also successful so it is not a loss
>> that they do not have IN OUT. In fact I think their API is easier to
>> work with than Camel.
>>
>> We are fortunate that most people with Camel do not work directly with
>> Exchange but work more with Camel returning the message as an expected
>> body type using its type converters. That helps a lot.
>>
>> But we have stories form Camel 1.x where people get confused whey
>> Camel return their original input in some situations where it was not
>> expected in the OUT message. Or the fact producer template sendBody is
>> not void in 1.x. Then people think it can be used for InOut.
>>
>> Okay end of mail.
>>
>>
>> On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<ja...@gmail.com> wrote:
>>> 2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
>>>> Moved this slightly different topic to a separate thread.
>>>>
>>>> ++1 from me.
>>>>
>>>> Do we need a vote on this one or it's a consensus?  Claus, it looks like you
>>>> agree, but pointed out that there is some work involved, correct?
>>>>
>>>> Hadrian
>>>>
>>>>
>>>>
>>>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>>>
>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>
>>>>>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<ja...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>>>
>>>>>>>> The ProducerCache that is doing the actual sending when using template
>>>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>>>
>>>>>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>>>>>> pointed out. He wanted the IN to be more static.
>>>>>>>> And we cannot get there yet when you do routing as all the processors
>>>>>>>> rely on IN being able to mutate during routing.
>>>>>>>>
>>>>>>>> Anyway my grief is that when you send in an Exchange the result would
>>>>>>>> sometimes be stored on IN and not OUT.
>>>>>>>> What I want it to do always is to store the result in OUT and keep IN
>>>>>>>> as the original input.
>>>>>>>>
>>>>>>>> The code to do this is now a bit more complex than just before
>>>>>>>>
>>>>>>>>               // copy the original input
>>>>>>>>               Message original = exchange.getIn().copy();
>>>>>>>>
>>>>>>>>               producer.process(exchange);
>>>>>>>>
>>>>>>>>               // if no OUT then set current IN as result (except for
>>>>>>>> optional out)
>>>>>>>>               if (!exchange.hasOut() && exchange.getPattern() !=
>>>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>>>                   // but only if its not the same as original IN to
>>>>>>>> avoid duplicating it
>>>>>>>>                   // and to adhere to the fact that there was no OUT
>>>>>>>> result at all
>>>>>>>>                   if (original.getBody() != null &&
>>>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>>>                       exchange.setOut(exchange.getIn());
>>>>>>>>                   }
>>>>>>>>               }
>>>>>>>>               // and restore original in
>>>>>>>>               exchange.setIn(original);
>>>>>>>>
>>>>>>>>               return exchange;
>>>>>>>>
>>>>>>>>
>>>>>>>> What I need to do is to copy the original IN message as it can be
>>>>>>>> mutated during routing.
>>>>>>>
>>>>>>> How about we prevent mutation of the IN message? Create a Message
>>>>>>> facade which throws UnsupportedOperationException if you try to mutate
>>>>>>> it in any way. Then we can pass the same read-only Message around as
>>>>>>> the IN within retry loops or from step to step if no new output is
>>>>>>> created (e.g. in a content based router where you just move a Message
>>>>>>> to the right endpoint without changing it)
>>>>>>>
>>>>>>
>>>>>> A good idea but will break a lot of logic in Camel.
>>>>>
>>>>> Agreed. But with the benefit that we'd be able to get rid of all the
>>>>> defensive copies in our code; plus we'd be able to pass the same
>>>>> Message from step to step. The API would be a bit more clean; to
>>>>> change the output, you create an OUT message (maybe by copying the
>>>>> IN).
>>>>>
>>>>>
>>>>>> Most of the Camel
>>>>>> processors work on the IN message and set the result on either IN or
>>>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>>>> original input? Or am I mistaking?
>>>>>
>>>>> Yeah, we'd have to patch code to no longer mutate IN
>>>>>
>>>>>
>>>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>>>> immutable and always the original input?
>>>>>
>>>>> If no OUT, then no output was created, so pass the IN along...
>>>>>
>>>>>
>>>>> OK how about this; a CopyOnWriteMessageFacade which does not mutate
>>>>> the original message at all ever; if a message is used in a purely
>>>>> read only way, it does nothing but delegate to the original message -
>>>>> but then as soon as someone mutates it, it creates a copy and uses
>>>>> that from that point on?
>>>>>
>>>>> i.e. make the copy of the message lazy - and only make a copy when a
>>>>> Processor really does try to mutate the Message?
>>>>>
>>>>> Then we'd get the best of both worlds; avoid breaking old code but
>>>>> avoid tons of unnecessary copies?
>>>
>>> BTW then using the current API you could have a Message and then call
>>>
>>> Message origin = ...;
>>> Message newMsg = origin.copy().copy().copy().copy();
>>>
>>> and the message would not actually be copied at all; new would just be
>>> a CopyOnWriteMessageFacade which would hold a reference to 'origin' as
>>> the readOnlyMessage (which it never mutates).
>>>
>>> The copy would only take place if you did
>>>
>>> newMsg.setBody("foo")
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://fusesource.com/
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[James Strachan <ja...@gmail.com>]

I'm confused. So rather than calling Exchange.getIn() to get the IN,
you'd have to do Exchange.getUnitOfWork().somethingOrOther() to get
the last IN? I don't see that as any simpler; making one of the 2
messages harder to find, just moves the confusion somewhere else?

2009/7/12 Claus Ibsen <cl...@gmail.com>:
> Hi
>
> Starting the kettle to brew me a grok of hot coffee, as its sunday
> morning and I guess I got the energy to toss in my 2$ in the bucket of
> ideas
> Now that we are into the open talks and that it seems we can open the
> box with the IN, OUT, FAULT again.
>
> I advocate for only one message on the implementation side. On the API
> side we could still have some notion of IN / OUT to keep the API
> migration / impact simpler. But at the end having one message API
> makes it simpler. There could be a boolean to test whether an output
> has been set or not. Just as if we have a test for fault.
>
>
> Proposed Exchange API
> ==================
>
>  Message getMessage()
>  void setMessage(Message msg);
>
>  boolean messageChanged();
>   -- false = message have newer been mutated, eg its still the same input
>   -- true = the message or message body have been mutated in some way
> (setMessage or setBody called)
>   -- need to find a good name for this method, eg testing whether the
> message have been changed/updated or not
>
> That leaves us with a simple API.
>
> Then we do not need to worry about all the message management
> internally in Camel, where we need to figure out whether to set data
> on IN or OUT and what else we kinda do, and it gets a bit complex over
> time how to do this correctly.
>
>
> MEP: InOnly
> =========
> Now we can do as Hadrian want. Newer return something in OUT. And
> leave the original input on the Exchange.
> Here we need to use James CopyOnWrite technique so we can preserve the
> original message in case its mutated during routing.
>
>
> MEP: InOut
> =========
> If an OUT message have been set (eg there is a reply), can be
> determined if the messageChanged() == true. (then its not the original
> message anymore). And yes the current code does this. It will just
> copy whatever there is in IN and use it as OUT if no OUT was set.
> With this proposal this will be improved as its easier to determine if
> there is an OUT message or not.
>
>
> Original message
> =============
> Currently there is an API to get the original message from the
> UnitOfWork as its needed when doing redeliveries and a message was
> doomed
> and had to be moved to the dead letter channel. Then it makes much
> more sense to move the original message than the current mutated
> message.
> ( i wonder if it should be default behavior )
>
>
> AggregationStrategy
> ===============
> Another benefit with a single getMessage() is that end users using
> AggregationStrategy will not be confused how to use it.
> Should I look in IN or OUT. And the current code can actually be a
> "lucky draw" as whether the data is in IN or OUT depending on facts
> such as how the route path is.
> We can even change it to pass in Message object instead of bare bone
> Exchange. You can always go from Message to exchange using
> getExchange().
>
>
> All the processors / components / data formats
> ===================================
> Logic will be easier as well as they do not need to cater for IN / OUT
> and where and how to set a result. Just work on the Message.
>
>
> Use @deprecated for steady migration
> ============================
> Hadrian suggested that for the API migration to a single getMessage
> you could let the getIn/setIn getOut/setOut delegate to
> getMessage/setMessage.
> And then mark them as @deprecated and then gradually change the camel
> code as we goes. So I do not think the hold up / change would take a
> lot of time and energy to get done.
>
>
> Final words
> ========
> So if it was possible to simplify the API and reduce the IN OUT FAULT
> to a simpler API, even go as far as I would like with a single
> Message.
> Then that would be really great and worth a hold up for a imminent 2.0 release.
>
>
> Other frameworks
> =============
> I guess it wasn't final words after all :) Just wanted to say that
> Mule, Spring Integration also just have a single message for the
> message.
> And yes I think their projects are also successful so it is not a loss
> that they do not have IN OUT. In fact I think their API is easier to
> work with than Camel.
>
> We are fortunate that most people with Camel do not work directly with
> Exchange but work more with Camel returning the message as an expected
> body type using its type converters. That helps a lot.
>
> But we have stories form Camel 1.x where people get confused whey
> Camel return their original input in some situations where it was not
> expected in the OUT message. Or the fact producer template sendBody is
> not void in 1.x. Then people think it can be used for InOut.
>
> Okay end of mail.
>
>
> On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<ja...@gmail.com> wrote:
>> 2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
>>> Moved this slightly different topic to a separate thread.
>>>
>>> ++1 from me.
>>>
>>> Do we need a vote on this one or it's a consensus?  Claus, it looks like you
>>> agree, but pointed out that there is some work involved, correct?
>>>
>>> Hadrian
>>>
>>>
>>>
>>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>>
>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>
>>>>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<ja...@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>>
>>>>>>> The ProducerCache that is doing the actual sending when using template
>>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>>
>>>>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>>>>> pointed out. He wanted the IN to be more static.
>>>>>>> And we cannot get there yet when you do routing as all the processors
>>>>>>> rely on IN being able to mutate during routing.
>>>>>>>
>>>>>>> Anyway my grief is that when you send in an Exchange the result would
>>>>>>> sometimes be stored on IN and not OUT.
>>>>>>> What I want it to do always is to store the result in OUT and keep IN
>>>>>>> as the original input.
>>>>>>>
>>>>>>> The code to do this is now a bit more complex than just before
>>>>>>>
>>>>>>>               // copy the original input
>>>>>>>               Message original = exchange.getIn().copy();
>>>>>>>
>>>>>>>               producer.process(exchange);
>>>>>>>
>>>>>>>               // if no OUT then set current IN as result (except for
>>>>>>> optional out)
>>>>>>>               if (!exchange.hasOut() && exchange.getPattern() !=
>>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>>                   // but only if its not the same as original IN to
>>>>>>> avoid duplicating it
>>>>>>>                   // and to adhere to the fact that there was no OUT
>>>>>>> result at all
>>>>>>>                   if (original.getBody() != null &&
>>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>>                       exchange.setOut(exchange.getIn());
>>>>>>>                   }
>>>>>>>               }
>>>>>>>               // and restore original in
>>>>>>>               exchange.setIn(original);
>>>>>>>
>>>>>>>               return exchange;
>>>>>>>
>>>>>>>
>>>>>>> What I need to do is to copy the original IN message as it can be
>>>>>>> mutated during routing.
>>>>>>
>>>>>> How about we prevent mutation of the IN message? Create a Message
>>>>>> facade which throws UnsupportedOperationException if you try to mutate
>>>>>> it in any way. Then we can pass the same read-only Message around as
>>>>>> the IN within retry loops or from step to step if no new output is
>>>>>> created (e.g. in a content based router where you just move a Message
>>>>>> to the right endpoint without changing it)
>>>>>>
>>>>>
>>>>> A good idea but will break a lot of logic in Camel.
>>>>
>>>> Agreed. But with the benefit that we'd be able to get rid of all the
>>>> defensive copies in our code; plus we'd be able to pass the same
>>>> Message from step to step. The API would be a bit more clean; to
>>>> change the output, you create an OUT message (maybe by copying the
>>>> IN).
>>>>
>>>>
>>>>> Most of the Camel
>>>>> processors work on the IN message and set the result on either IN or
>>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>>> original input? Or am I mistaking?
>>>>
>>>> Yeah, we'd have to patch code to no longer mutate IN
>>>>
>>>>
>>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>>> immutable and always the original input?
>>>>
>>>> If no OUT, then no output was created, so pass the IN along...
>>>>
>>>>
>>>> OK how about this; a CopyOnWriteMessageFacade which does not mutate
>>>> the original message at all ever; if a message is used in a purely
>>>> read only way, it does nothing but delegate to the original message -
>>>> but then as soon as someone mutates it, it creates a copy and uses
>>>> that from that point on?
>>>>
>>>> i.e. make the copy of the message lazy - and only make a copy when a
>>>> Processor really does try to mutate the Message?
>>>>
>>>> Then we'd get the best of both worlds; avoid breaking old code but
>>>> avoid tons of unnecessary copies?
>>
>> BTW then using the current API you could have a Message and then call
>>
>> Message origin = ...;
>> Message newMsg = origin.copy().copy().copy().copy();
>>
>> and the message would not actually be copied at all; new would just be
>> a CopyOnWriteMessageFacade which would hold a reference to 'origin' as
>> the readOnlyMessage (which it never mutates).
>>
>> The copy would only take place if you did
>>
>> newMsg.setBody("foo")
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://fusesource.com/
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>



-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: [DISCUSS][IMPORTANT] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Claus Ibsen <cl...@gmail.com>]

On Mon, Jul 13, 2009 at 5:44 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
> Hi,
>
> Are you in fact proposing to rename get/setOut() to get/setMessage()?  And
> hasOut() to be renamed as messageChanged(), kinda?  This is pretty much how
> it sounds to me.
>

Yes,

However I would rename getIn/setIn instead as a lot of Camel
processors etc rely on getIn() instead of getOut when they get access
to the payload/message.



> I like the semantic cleanup.  Not sure about passing the message instead of
> the exchange.  That would have a significant impact on projects using camel
> and I don't think the benefit would justify it.
>
> I would love for others to pitch in and express their thoughts.
> Hadrian
>
>
>
> On Jul 12, 2009, at 5:23 AM, Claus Ibsen wrote:
>
>> Hi
>>
>> Starting the kettle to brew me a grok of hot coffee, as its sunday
>> morning and I guess I got the energy to toss in my 2$ in the bucket of
>> ideas
>> Now that we are into the open talks and that it seems we can open the
>> box with the IN, OUT, FAULT again.
>>
>> I advocate for only one message on the implementation side. On the API
>> side we could still have some notion of IN / OUT to keep the API
>> migration / impact simpler. But at the end having one message API
>> makes it simpler. There could be a boolean to test whether an output
>> has been set or not. Just as if we have a test for fault.
>>
>>
>> Proposed Exchange API
>> ==================
>>
>>  Message getMessage()
>>  void setMessage(Message msg);
>>
>>  boolean messageChanged();
>>  -- false = message have newer been mutated, eg its still the same input
>>  -- true = the message or message body have been mutated in some way
>> (setMessage or setBody called)
>>  -- need to find a good name for this method, eg testing whether the
>> message have been changed/updated or not
>>
>> That leaves us with a simple API.
>>
>> Then we do not need to worry about all the message management
>> internally in Camel, where we need to figure out whether to set data
>> on IN or OUT and what else we kinda do, and it gets a bit complex over
>> time how to do this correctly.
>>
>>
>> MEP: InOnly
>> =========
>> Now we can do as Hadrian want. Newer return something in OUT. And
>> leave the original input on the Exchange.
>> Here we need to use James CopyOnWrite technique so we can preserve the
>> original message in case its mutated during routing.
>>
>>
>> MEP: InOut
>> =========
>> If an OUT message have been set (eg there is a reply), can be
>> determined if the messageChanged() == true. (then its not the original
>> message anymore). And yes the current code does this. It will just
>> copy whatever there is in IN and use it as OUT if no OUT was set.
>> With this proposal this will be improved as its easier to determine if
>> there is an OUT message or not.
>>
>>
>> Original message
>> =============
>> Currently there is an API to get the original message from the
>> UnitOfWork as its needed when doing redeliveries and a message was
>> doomed
>> and had to be moved to the dead letter channel. Then it makes much
>> more sense to move the original message than the current mutated
>> message.
>> ( i wonder if it should be default behavior )
>>
>>
>> AggregationStrategy
>> ===============
>> Another benefit with a single getMessage() is that end users using
>> AggregationStrategy will not be confused how to use it.
>> Should I look in IN or OUT. And the current code can actually be a
>> "lucky draw" as whether the data is in IN or OUT depending on facts
>> such as how the route path is.
>> We can even change it to pass in Message object instead of bare bone
>> Exchange. You can always go from Message to exchange using
>> getExchange().
>>
>>
>> All the processors / components / data formats
>> ===================================
>> Logic will be easier as well as they do not need to cater for IN / OUT
>> and where and how to set a result. Just work on the Message.
>>
>>
>> Use @deprecated for steady migration
>> ============================
>> Hadrian suggested that for the API migration to a single getMessage
>> you could let the getIn/setIn getOut/setOut delegate to
>> getMessage/setMessage.
>> And then mark them as @deprecated and then gradually change the camel
>> code as we goes. So I do not think the hold up / change would take a
>> lot of time and energy to get done.
>>
>>
>> Final words
>> ========
>> So if it was possible to simplify the API and reduce the IN OUT FAULT
>> to a simpler API, even go as far as I would like with a single
>> Message.
>> Then that would be really great and worth a hold up for a imminent 2.0
>> release.
>>
>>
>> Other frameworks
>> =============
>> I guess it wasn't final words after all :) Just wanted to say that
>> Mule, Spring Integration also just have a single message for the
>> message.
>> And yes I think their projects are also successful so it is not a loss
>> that they do not have IN OUT. In fact I think their API is easier to
>> work with than Camel.
>>
>> We are fortunate that most people with Camel do not work directly with
>> Exchange but work more with Camel returning the message as an expected
>> body type using its type converters. That helps a lot.
>>
>> But we have stories form Camel 1.x where people get confused whey
>> Camel return their original input in some situations where it was not
>> expected in the OUT message. Or the fact producer template sendBody is
>> not void in 1.x. Then people think it can be used for InOut.
>>
>> Okay end of mail.
>>
>>
>> On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<ja...@gmail.com>
>> wrote:
>>>
>>> 2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
>>>>
>>>> Moved this slightly different topic to a separate thread.
>>>>
>>>> ++1 from me.
>>>>
>>>> Do we need a vote on this one or it's a consensus?  Claus, it looks like
>>>> you
>>>> agree, but pointed out that there is some work involved, correct?
>>>>
>>>> Hadrian
>>>>
>>>>
>>>>
>>>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>>>
>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>
>>>>>> On Fri, Jul 10, 2009 at 1:20 PM, James
>>>>>> Strachan<ja...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>>>
>>>>>>>> The ProducerCache that is doing the actual sending when using
>>>>>>>> template
>>>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>>>
>>>>>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>>>>>> pointed out. He wanted the IN to be more static.
>>>>>>>> And we cannot get there yet when you do routing as all the
>>>>>>>> processors
>>>>>>>> rely on IN being able to mutate during routing.
>>>>>>>>
>>>>>>>> Anyway my grief is that when you send in an Exchange the result
>>>>>>>> would
>>>>>>>> sometimes be stored on IN and not OUT.
>>>>>>>> What I want it to do always is to store the result in OUT and keep
>>>>>>>> IN
>>>>>>>> as the original input.
>>>>>>>>
>>>>>>>> The code to do this is now a bit more complex than just before
>>>>>>>>
>>>>>>>>              // copy the original input
>>>>>>>>              Message original = exchange.getIn().copy();
>>>>>>>>
>>>>>>>>              producer.process(exchange);
>>>>>>>>
>>>>>>>>              // if no OUT then set current IN as result (except for
>>>>>>>> optional out)
>>>>>>>>              if (!exchange.hasOut() && exchange.getPattern() !=
>>>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>>>                  // but only if its not the same as original IN to
>>>>>>>> avoid duplicating it
>>>>>>>>                  // and to adhere to the fact that there was no OUT
>>>>>>>> result at all
>>>>>>>>                  if (original.getBody() != null &&
>>>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>>>                      exchange.setOut(exchange.getIn());
>>>>>>>>                  }
>>>>>>>>              }
>>>>>>>>              // and restore original in
>>>>>>>>              exchange.setIn(original);
>>>>>>>>
>>>>>>>>              return exchange;
>>>>>>>>
>>>>>>>>
>>>>>>>> What I need to do is to copy the original IN message as it can be
>>>>>>>> mutated during routing.
>>>>>>>
>>>>>>> How about we prevent mutation of the IN message? Create a Message
>>>>>>> facade which throws UnsupportedOperationException if you try to
>>>>>>> mutate
>>>>>>> it in any way. Then we can pass the same read-only Message around as
>>>>>>> the IN within retry loops or from step to step if no new output is
>>>>>>> created (e.g. in a content based router where you just move a Message
>>>>>>> to the right endpoint without changing it)
>>>>>>>
>>>>>>
>>>>>> A good idea but will break a lot of logic in Camel.
>>>>>
>>>>> Agreed. But with the benefit that we'd be able to get rid of all the
>>>>> defensive copies in our code; plus we'd be able to pass the same
>>>>> Message from step to step. The API would be a bit more clean; to
>>>>> change the output, you create an OUT message (maybe by copying the
>>>>> IN).
>>>>>
>>>>>
>>>>>> Most of the Camel
>>>>>> processors work on the IN message and set the result on either IN or
>>>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>>>> original input? Or am I mistaking?
>>>>>
>>>>> Yeah, we'd have to patch code to no longer mutate IN
>>>>>
>>>>>
>>>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>>>> immutable and always the original input?
>>>>>
>>>>> If no OUT, then no output was created, so pass the IN along...
>>>>>
>>>>>
>>>>> OK how about this; a CopyOnWriteMessageFacade which does not mutate
>>>>> the original message at all ever; if a message is used in a purely
>>>>> read only way, it does nothing but delegate to the original message -
>>>>> but then as soon as someone mutates it, it creates a copy and uses
>>>>> that from that point on?
>>>>>
>>>>> i.e. make the copy of the message lazy - and only make a copy when a
>>>>> Processor really does try to mutate the Message?
>>>>>
>>>>> Then we'd get the best of both worlds; avoid breaking old code but
>>>>> avoid tons of unnecessary copies?
>>>
>>> BTW then using the current API you could have a Message and then call
>>>
>>> Message origin = ...;
>>> Message newMsg = origin.copy().copy().copy().copy();
>>>
>>> and the message would not actually be copied at all; new would just be
>>> a CopyOnWriteMessageFacade which would hold a reference to 'origin' as
>>> the readOnlyMessage (which it never mutates).
>>>
>>> The copy would only take place if you did
>>>
>>> newMsg.setBody("foo")
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://fusesource.com/
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS][IMPORTANT] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Claus Ibsen <cl...@gmail.com>]

On Mon, Jul 13, 2009 at 5:44 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
> Hi,
>
> Are you in fact proposing to rename get/setOut() to get/setMessage()?  And
> hasOut() to be renamed as messageChanged(), kinda?  This is pretty much how
> it sounds to me.
>
> I like the semantic cleanup.  Not sure about passing the message instead of
> the exchange.  That would have a significant impact on projects using camel
> and I don't think the benefit would justify it.
Its hardly a significant impact. Removing the generics is a bigger
impact that changing a single interface

And what they used to do in the AggregationStrategy (with much
confusion, may I stress out - see the user mailinglist)

public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
  // hmmm is my payload in IN or OUT today?
 String oldBody = oldExchange.getIn().getBody(String.class);
String newBody = newExchange.getIn().getBody(String.class);
 ... do more stuff
 ... jesus should I store the aggregated body on IN or OUT
 .. if I store it in the wrong one I lose all my headers ... . help
Camel riders!!!
... for instance as:
   oldExchange.getOut().setBody(aggregatedBody);
   return oldExchange;
}

And this simpler API

public Message aggregate(Message oldMessage, Exchange newMessage) {
  // yeah I have the message directly no more IN or OUT confusion
 String oldBody = oldMessage.getBody(String.class);
String newBody = newMessage.getBody(String.class);
 ... do more stuff
... great no more lost headers
}


The API migration is super simple. Before they had to go from exchange
-> IN or OUT (???) -> Message
Now they have it directly.


>
> I would love for others to pitch in and express their thoughts.
> Hadrian
>
>
>
> On Jul 12, 2009, at 5:23 AM, Claus Ibsen wrote:
>
>> Hi
>>
>> Starting the kettle to brew me a grok of hot coffee, as its sunday
>> morning and I guess I got the energy to toss in my 2$ in the bucket of
>> ideas
>> Now that we are into the open talks and that it seems we can open the
>> box with the IN, OUT, FAULT again.
>>
>> I advocate for only one message on the implementation side. On the API
>> side we could still have some notion of IN / OUT to keep the API
>> migration / impact simpler. But at the end having one message API
>> makes it simpler. There could be a boolean to test whether an output
>> has been set or not. Just as if we have a test for fault.
>>
>>
>> Proposed Exchange API
>> ==================
>>
>>  Message getMessage()
>>  void setMessage(Message msg);
>>
>>  boolean messageChanged();
>>  -- false = message have newer been mutated, eg its still the same input
>>  -- true = the message or message body have been mutated in some way
>> (setMessage or setBody called)
>>  -- need to find a good name for this method, eg testing whether the
>> message have been changed/updated or not
>>
>> That leaves us with a simple API.
>>
>> Then we do not need to worry about all the message management
>> internally in Camel, where we need to figure out whether to set data
>> on IN or OUT and what else we kinda do, and it gets a bit complex over
>> time how to do this correctly.
>>
>>
>> MEP: InOnly
>> =========
>> Now we can do as Hadrian want. Newer return something in OUT. And
>> leave the original input on the Exchange.
>> Here we need to use James CopyOnWrite technique so we can preserve the
>> original message in case its mutated during routing.
>>
>>
>> MEP: InOut
>> =========
>> If an OUT message have been set (eg there is a reply), can be
>> determined if the messageChanged() == true. (then its not the original
>> message anymore). And yes the current code does this. It will just
>> copy whatever there is in IN and use it as OUT if no OUT was set.
>> With this proposal this will be improved as its easier to determine if
>> there is an OUT message or not.
>>
>>
>> Original message
>> =============
>> Currently there is an API to get the original message from the
>> UnitOfWork as its needed when doing redeliveries and a message was
>> doomed
>> and had to be moved to the dead letter channel. Then it makes much
>> more sense to move the original message than the current mutated
>> message.
>> ( i wonder if it should be default behavior )
>>
>>
>> AggregationStrategy
>> ===============
>> Another benefit with a single getMessage() is that end users using
>> AggregationStrategy will not be confused how to use it.
>> Should I look in IN or OUT. And the current code can actually be a
>> "lucky draw" as whether the data is in IN or OUT depending on facts
>> such as how the route path is.
>> We can even change it to pass in Message object instead of bare bone
>> Exchange. You can always go from Message to exchange using
>> getExchange().
>>
>>
>> All the processors / components / data formats
>> ===================================
>> Logic will be easier as well as they do not need to cater for IN / OUT
>> and where and how to set a result. Just work on the Message.
>>
>>
>> Use @deprecated for steady migration
>> ============================
>> Hadrian suggested that for the API migration to a single getMessage
>> you could let the getIn/setIn getOut/setOut delegate to
>> getMessage/setMessage.
>> And then mark them as @deprecated and then gradually change the camel
>> code as we goes. So I do not think the hold up / change would take a
>> lot of time and energy to get done.
>>
>>
>> Final words
>> ========
>> So if it was possible to simplify the API and reduce the IN OUT FAULT
>> to a simpler API, even go as far as I would like with a single
>> Message.
>> Then that would be really great and worth a hold up for a imminent 2.0
>> release.
>>
>>
>> Other frameworks
>> =============
>> I guess it wasn't final words after all :) Just wanted to say that
>> Mule, Spring Integration also just have a single message for the
>> message.
>> And yes I think their projects are also successful so it is not a loss
>> that they do not have IN OUT. In fact I think their API is easier to
>> work with than Camel.
>>
>> We are fortunate that most people with Camel do not work directly with
>> Exchange but work more with Camel returning the message as an expected
>> body type using its type converters. That helps a lot.
>>
>> But we have stories form Camel 1.x where people get confused whey
>> Camel return their original input in some situations where it was not
>> expected in the OUT message. Or the fact producer template sendBody is
>> not void in 1.x. Then people think it can be used for InOut.
>>
>> Okay end of mail.
>>
>>
>> On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<ja...@gmail.com>
>> wrote:
>>>
>>> 2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
>>>>
>>>> Moved this slightly different topic to a separate thread.
>>>>
>>>> ++1 from me.
>>>>
>>>> Do we need a vote on this one or it's a consensus?  Claus, it looks like
>>>> you
>>>> agree, but pointed out that there is some work involved, correct?
>>>>
>>>> Hadrian
>>>>
>>>>
>>>>
>>>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>>>
>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>
>>>>>> On Fri, Jul 10, 2009 at 1:20 PM, James
>>>>>> Strachan<ja...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>>>
>>>>>>>> The ProducerCache that is doing the actual sending when using
>>>>>>>> template
>>>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>>>
>>>>>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>>>>>> pointed out. He wanted the IN to be more static.
>>>>>>>> And we cannot get there yet when you do routing as all the
>>>>>>>> processors
>>>>>>>> rely on IN being able to mutate during routing.
>>>>>>>>
>>>>>>>> Anyway my grief is that when you send in an Exchange the result
>>>>>>>> would
>>>>>>>> sometimes be stored on IN and not OUT.
>>>>>>>> What I want it to do always is to store the result in OUT and keep
>>>>>>>> IN
>>>>>>>> as the original input.
>>>>>>>>
>>>>>>>> The code to do this is now a bit more complex than just before
>>>>>>>>
>>>>>>>>              // copy the original input
>>>>>>>>              Message original = exchange.getIn().copy();
>>>>>>>>
>>>>>>>>              producer.process(exchange);
>>>>>>>>
>>>>>>>>              // if no OUT then set current IN as result (except for
>>>>>>>> optional out)
>>>>>>>>              if (!exchange.hasOut() && exchange.getPattern() !=
>>>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>>>                  // but only if its not the same as original IN to
>>>>>>>> avoid duplicating it
>>>>>>>>                  // and to adhere to the fact that there was no OUT
>>>>>>>> result at all
>>>>>>>>                  if (original.getBody() != null &&
>>>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>>>                      exchange.setOut(exchange.getIn());
>>>>>>>>                  }
>>>>>>>>              }
>>>>>>>>              // and restore original in
>>>>>>>>              exchange.setIn(original);
>>>>>>>>
>>>>>>>>              return exchange;
>>>>>>>>
>>>>>>>>
>>>>>>>> What I need to do is to copy the original IN message as it can be
>>>>>>>> mutated during routing.
>>>>>>>
>>>>>>> How about we prevent mutation of the IN message? Create a Message
>>>>>>> facade which throws UnsupportedOperationException if you try to
>>>>>>> mutate
>>>>>>> it in any way. Then we can pass the same read-only Message around as
>>>>>>> the IN within retry loops or from step to step if no new output is
>>>>>>> created (e.g. in a content based router where you just move a Message
>>>>>>> to the right endpoint without changing it)
>>>>>>>
>>>>>>
>>>>>> A good idea but will break a lot of logic in Camel.
>>>>>
>>>>> Agreed. But with the benefit that we'd be able to get rid of all the
>>>>> defensive copies in our code; plus we'd be able to pass the same
>>>>> Message from step to step. The API would be a bit more clean; to
>>>>> change the output, you create an OUT message (maybe by copying the
>>>>> IN).
>>>>>
>>>>>
>>>>>> Most of the Camel
>>>>>> processors work on the IN message and set the result on either IN or
>>>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>>>> original input? Or am I mistaking?
>>>>>
>>>>> Yeah, we'd have to patch code to no longer mutate IN
>>>>>
>>>>>
>>>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>>>> immutable and always the original input?
>>>>>
>>>>> If no OUT, then no output was created, so pass the IN along...
>>>>>
>>>>>
>>>>> OK how about this; a CopyOnWriteMessageFacade which does not mutate
>>>>> the original message at all ever; if a message is used in a purely
>>>>> read only way, it does nothing but delegate to the original message -
>>>>> but then as soon as someone mutates it, it creates a copy and uses
>>>>> that from that point on?
>>>>>
>>>>> i.e. make the copy of the message lazy - and only make a copy when a
>>>>> Processor really does try to mutate the Message?
>>>>>
>>>>> Then we'd get the best of both worlds; avoid breaking old code but
>>>>> avoid tons of unnecessary copies?
>>>
>>> BTW then using the current API you could have a Message and then call
>>>
>>> Message origin = ...;
>>> Message newMsg = origin.copy().copy().copy().copy();
>>>
>>> and the message would not actually be copied at all; new would just be
>>> a CopyOnWriteMessageFacade which would hold a reference to 'origin' as
>>> the readOnlyMessage (which it never mutates).
>>>
>>> The copy would only take place if you did
>>>
>>> newMsg.setBody("foo")
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://fusesource.com/
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS][IMPORTANT] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Hadrian Zbarcea <hz...@gmail.com>]

Hi,

Are you in fact proposing to rename get/setOut() to get/setMessage()?   
And hasOut() to be renamed as messageChanged(), kinda?  This is pretty  
much how it sounds to me.

I like the semantic cleanup.  Not sure about passing the message  
instead of the exchange.  That would have a significant impact on  
projects using camel and I don't think the benefit would justify it.

I would love for others to pitch in and express their thoughts.
Hadrian



On Jul 12, 2009, at 5:23 AM, Claus Ibsen wrote:

> Hi
>
> Starting the kettle to brew me a grok of hot coffee, as its sunday
> morning and I guess I got the energy to toss in my 2$ in the bucket of
> ideas
> Now that we are into the open talks and that it seems we can open the
> box with the IN, OUT, FAULT again.
>
> I advocate for only one message on the implementation side. On the API
> side we could still have some notion of IN / OUT to keep the API
> migration / impact simpler. But at the end having one message API
> makes it simpler. There could be a boolean to test whether an output
> has been set or not. Just as if we have a test for fault.
>
>
> Proposed Exchange API
> ==================
>
>  Message getMessage()
>  void setMessage(Message msg);
>
>  boolean messageChanged();
>   -- false = message have newer been mutated, eg its still the same  
> input
>   -- true = the message or message body have been mutated in some way
> (setMessage or setBody called)
>   -- need to find a good name for this method, eg testing whether the
> message have been changed/updated or not
>
> That leaves us with a simple API.
>
> Then we do not need to worry about all the message management
> internally in Camel, where we need to figure out whether to set data
> on IN or OUT and what else we kinda do, and it gets a bit complex over
> time how to do this correctly.
>
>
> MEP: InOnly
> =========
> Now we can do as Hadrian want. Newer return something in OUT. And
> leave the original input on the Exchange.
> Here we need to use James CopyOnWrite technique so we can preserve the
> original message in case its mutated during routing.
>
>
> MEP: InOut
> =========
> If an OUT message have been set (eg there is a reply), can be
> determined if the messageChanged() == true. (then its not the original
> message anymore). And yes the current code does this. It will just
> copy whatever there is in IN and use it as OUT if no OUT was set.
> With this proposal this will be improved as its easier to determine if
> there is an OUT message or not.
>
>
> Original message
> =============
> Currently there is an API to get the original message from the
> UnitOfWork as its needed when doing redeliveries and a message was
> doomed
> and had to be moved to the dead letter channel. Then it makes much
> more sense to move the original message than the current mutated
> message.
> ( i wonder if it should be default behavior )
>
>
> AggregationStrategy
> ===============
> Another benefit with a single getMessage() is that end users using
> AggregationStrategy will not be confused how to use it.
> Should I look in IN or OUT. And the current code can actually be a
> "lucky draw" as whether the data is in IN or OUT depending on facts
> such as how the route path is.
> We can even change it to pass in Message object instead of bare bone
> Exchange. You can always go from Message to exchange using
> getExchange().
>
>
> All the processors / components / data formats
> ===================================
> Logic will be easier as well as they do not need to cater for IN / OUT
> and where and how to set a result. Just work on the Message.
>
>
> Use @deprecated for steady migration
> ============================
> Hadrian suggested that for the API migration to a single getMessage
> you could let the getIn/setIn getOut/setOut delegate to
> getMessage/setMessage.
> And then mark them as @deprecated and then gradually change the camel
> code as we goes. So I do not think the hold up / change would take a
> lot of time and energy to get done.
>
>
> Final words
> ========
> So if it was possible to simplify the API and reduce the IN OUT FAULT
> to a simpler API, even go as far as I would like with a single
> Message.
> Then that would be really great and worth a hold up for a imminent  
> 2.0 release.
>
>
> Other frameworks
> =============
> I guess it wasn't final words after all :) Just wanted to say that
> Mule, Spring Integration also just have a single message for the
> message.
> And yes I think their projects are also successful so it is not a loss
> that they do not have IN OUT. In fact I think their API is easier to
> work with than Camel.
>
> We are fortunate that most people with Camel do not work directly with
> Exchange but work more with Camel returning the message as an expected
> body type using its type converters. That helps a lot.
>
> But we have stories form Camel 1.x where people get confused whey
> Camel return their original input in some situations where it was not
> expected in the OUT message. Or the fact producer template sendBody is
> not void in 1.x. Then people think it can be used for InOut.
>
> Okay end of mail.
>
>
> On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<james.strachan@gmail.com 
> > wrote:
>> 2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
>>> Moved this slightly different topic to a separate thread.
>>>
>>> ++1 from me.
>>>
>>> Do we need a vote on this one or it's a consensus?  Claus, it  
>>> looks like you
>>> agree, but pointed out that there is some work involved, correct?
>>>
>>> Hadrian
>>>
>>>
>>>
>>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>>
>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>
>>>>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<james.strachan@gmail.com 
>>>>> >
>>>>> wrote:
>>>>>>
>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>>
>>>>>>> The ProducerCache that is doing the actual sending when using  
>>>>>>> template
>>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>>
>>>>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>>>>> pointed out. He wanted the IN to be more static.
>>>>>>> And we cannot get there yet when you do routing as all the  
>>>>>>> processors
>>>>>>> rely on IN being able to mutate during routing.
>>>>>>>
>>>>>>> Anyway my grief is that when you send in an Exchange the  
>>>>>>> result would
>>>>>>> sometimes be stored on IN and not OUT.
>>>>>>> What I want it to do always is to store the result in OUT and  
>>>>>>> keep IN
>>>>>>> as the original input.
>>>>>>>
>>>>>>> The code to do this is now a bit more complex than just before
>>>>>>>
>>>>>>>               // copy the original input
>>>>>>>               Message original = exchange.getIn().copy();
>>>>>>>
>>>>>>>               producer.process(exchange);
>>>>>>>
>>>>>>>               // if no OUT then set current IN as result  
>>>>>>> (except for
>>>>>>> optional out)
>>>>>>>               if (!exchange.hasOut() && exchange.getPattern() !=
>>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>>                   // but only if its not the same as original  
>>>>>>> IN to
>>>>>>> avoid duplicating it
>>>>>>>                   // and to adhere to the fact that there was  
>>>>>>> no OUT
>>>>>>> result at all
>>>>>>>                   if (original.getBody() != null &&
>>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>>                       exchange.setOut(exchange.getIn());
>>>>>>>                   }
>>>>>>>               }
>>>>>>>               // and restore original in
>>>>>>>               exchange.setIn(original);
>>>>>>>
>>>>>>>               return exchange;
>>>>>>>
>>>>>>>
>>>>>>> What I need to do is to copy the original IN message as it can  
>>>>>>> be
>>>>>>> mutated during routing.
>>>>>>
>>>>>> How about we prevent mutation of the IN message? Create a Message
>>>>>> facade which throws UnsupportedOperationException if you try to  
>>>>>> mutate
>>>>>> it in any way. Then we can pass the same read-only Message  
>>>>>> around as
>>>>>> the IN within retry loops or from step to step if no new output  
>>>>>> is
>>>>>> created (e.g. in a content based router where you just move a  
>>>>>> Message
>>>>>> to the right endpoint without changing it)
>>>>>>
>>>>>
>>>>> A good idea but will break a lot of logic in Camel.
>>>>
>>>> Agreed. But with the benefit that we'd be able to get rid of all  
>>>> the
>>>> defensive copies in our code; plus we'd be able to pass the same
>>>> Message from step to step. The API would be a bit more clean; to
>>>> change the output, you create an OUT message (maybe by copying the
>>>> IN).
>>>>
>>>>
>>>>> Most of the Camel
>>>>> processors work on the IN message and set the result on either  
>>>>> IN or
>>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>>> original input? Or am I mistaking?
>>>>
>>>> Yeah, we'd have to patch code to no longer mutate IN
>>>>
>>>>
>>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>>> immutable and always the original input?
>>>>
>>>> If no OUT, then no output was created, so pass the IN along...
>>>>
>>>>
>>>> OK how about this; a CopyOnWriteMessageFacade which does not mutate
>>>> the original message at all ever; if a message is used in a purely
>>>> read only way, it does nothing but delegate to the original  
>>>> message -
>>>> but then as soon as someone mutates it, it creates a copy and uses
>>>> that from that point on?
>>>>
>>>> i.e. make the copy of the message lazy - and only make a copy  
>>>> when a
>>>> Processor really does try to mutate the Message?
>>>>
>>>> Then we'd get the best of both worlds; avoid breaking old code but
>>>> avoid tons of unnecessary copies?
>>
>> BTW then using the current API you could have a Message and then call
>>
>> Message origin = ...;
>> Message newMsg = origin.copy().copy().copy().copy();
>>
>> and the message would not actually be copied at all; new would just  
>> be
>> a CopyOnWriteMessageFacade which would hold a reference to 'origin'  
>> as
>> the readOnlyMessage (which it never mutates).
>>
>> The copy would only take place if you did
>>
>> newMsg.setBody("foo")
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://fusesource.com/
>>
>
>
>
> -- 
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[Claus Ibsen <cl...@gmail.com>]

Hi

Starting the kettle to brew me a grok of hot coffee, as its sunday
morning and I guess I got the energy to toss in my 2$ in the bucket of
ideas
Now that we are into the open talks and that it seems we can open the
box with the IN, OUT, FAULT again.

I advocate for only one message on the implementation side. On the API
side we could still have some notion of IN / OUT to keep the API
migration / impact simpler. But at the end having one message API
makes it simpler. There could be a boolean to test whether an output
has been set or not. Just as if we have a test for fault.


Proposed Exchange API
==================

  Message getMessage()
  void setMessage(Message msg);

  boolean messageChanged();
   -- false = message have newer been mutated, eg its still the same input
   -- true = the message or message body have been mutated in some way
(setMessage or setBody called)
   -- need to find a good name for this method, eg testing whether the
message have been changed/updated or not

That leaves us with a simple API.

Then we do not need to worry about all the message management
internally in Camel, where we need to figure out whether to set data
on IN or OUT and what else we kinda do, and it gets a bit complex over
time how to do this correctly.


MEP: InOnly
=========
Now we can do as Hadrian want. Newer return something in OUT. And
leave the original input on the Exchange.
Here we need to use James CopyOnWrite technique so we can preserve the
original message in case its mutated during routing.


MEP: InOut
=========
If an OUT message have been set (eg there is a reply), can be
determined if the messageChanged() == true. (then its not the original
message anymore). And yes the current code does this. It will just
copy whatever there is in IN and use it as OUT if no OUT was set.
With this proposal this will be improved as its easier to determine if
there is an OUT message or not.


Original message
=============
Currently there is an API to get the original message from the
UnitOfWork as its needed when doing redeliveries and a message was
doomed
and had to be moved to the dead letter channel. Then it makes much
more sense to move the original message than the current mutated
message.
( i wonder if it should be default behavior )


AggregationStrategy
===============
Another benefit with a single getMessage() is that end users using
AggregationStrategy will not be confused how to use it.
Should I look in IN or OUT. And the current code can actually be a
"lucky draw" as whether the data is in IN or OUT depending on facts
such as how the route path is.
We can even change it to pass in Message object instead of bare bone
Exchange. You can always go from Message to exchange using
getExchange().


All the processors / components / data formats
===================================
Logic will be easier as well as they do not need to cater for IN / OUT
and where and how to set a result. Just work on the Message.


Use @deprecated for steady migration
============================
Hadrian suggested that for the API migration to a single getMessage
you could let the getIn/setIn getOut/setOut delegate to
getMessage/setMessage.
And then mark them as @deprecated and then gradually change the camel
code as we goes. So I do not think the hold up / change would take a
lot of time and energy to get done.


Final words
========
So if it was possible to simplify the API and reduce the IN OUT FAULT
to a simpler API, even go as far as I would like with a single
Message.
Then that would be really great and worth a hold up for a imminent 2.0 release.


Other frameworks
=============
I guess it wasn't final words after all :) Just wanted to say that
Mule, Spring Integration also just have a single message for the
message.
And yes I think their projects are also successful so it is not a loss
that they do not have IN OUT. In fact I think their API is easier to
work with than Camel.

We are fortunate that most people with Camel do not work directly with
Exchange but work more with Camel returning the message as an expected
body type using its type converters. That helps a lot.

But we have stories form Camel 1.x where people get confused whey
Camel return their original input in some situations where it was not
expected in the OUT message. Or the fact producer template sendBody is
not void in 1.x. Then people think it can be used for InOut.

Okay end of mail.


On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<ja...@gmail.com> wrote:
> 2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
>> Moved this slightly different topic to a separate thread.
>>
>> ++1 from me.
>>
>> Do we need a vote on this one or it's a consensus?  Claus, it looks like you
>> agree, but pointed out that there is some work involved, correct?
>>
>> Hadrian
>>
>>
>>
>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>
>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>
>>>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<ja...@gmail.com>
>>>> wrote:
>>>>>
>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>
>>>>>> Hi
>>>>>>
>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>
>>>>>> The ProducerCache that is doing the actual sending when using template
>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>
>>>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>>>> pointed out. He wanted the IN to be more static.
>>>>>> And we cannot get there yet when you do routing as all the processors
>>>>>> rely on IN being able to mutate during routing.
>>>>>>
>>>>>> Anyway my grief is that when you send in an Exchange the result would
>>>>>> sometimes be stored on IN and not OUT.
>>>>>> What I want it to do always is to store the result in OUT and keep IN
>>>>>> as the original input.
>>>>>>
>>>>>> The code to do this is now a bit more complex than just before
>>>>>>
>>>>>>               // copy the original input
>>>>>>               Message original = exchange.getIn().copy();
>>>>>>
>>>>>>               producer.process(exchange);
>>>>>>
>>>>>>               // if no OUT then set current IN as result (except for
>>>>>> optional out)
>>>>>>               if (!exchange.hasOut() && exchange.getPattern() !=
>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>                   // but only if its not the same as original IN to
>>>>>> avoid duplicating it
>>>>>>                   // and to adhere to the fact that there was no OUT
>>>>>> result at all
>>>>>>                   if (original.getBody() != null &&
>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>                       exchange.setOut(exchange.getIn());
>>>>>>                   }
>>>>>>               }
>>>>>>               // and restore original in
>>>>>>               exchange.setIn(original);
>>>>>>
>>>>>>               return exchange;
>>>>>>
>>>>>>
>>>>>> What I need to do is to copy the original IN message as it can be
>>>>>> mutated during routing.
>>>>>
>>>>> How about we prevent mutation of the IN message? Create a Message
>>>>> facade which throws UnsupportedOperationException if you try to mutate
>>>>> it in any way. Then we can pass the same read-only Message around as
>>>>> the IN within retry loops or from step to step if no new output is
>>>>> created (e.g. in a content based router where you just move a Message
>>>>> to the right endpoint without changing it)
>>>>>
>>>>
>>>> A good idea but will break a lot of logic in Camel.
>>>
>>> Agreed. But with the benefit that we'd be able to get rid of all the
>>> defensive copies in our code; plus we'd be able to pass the same
>>> Message from step to step. The API would be a bit more clean; to
>>> change the output, you create an OUT message (maybe by copying the
>>> IN).
>>>
>>>
>>>> Most of the Camel
>>>> processors work on the IN message and set the result on either IN or
>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>> original input? Or am I mistaking?
>>>
>>> Yeah, we'd have to patch code to no longer mutate IN
>>>
>>>
>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>> immutable and always the original input?
>>>
>>> If no OUT, then no output was created, so pass the IN along...
>>>
>>>
>>> OK how about this; a CopyOnWriteMessageFacade which does not mutate
>>> the original message at all ever; if a message is used in a purely
>>> read only way, it does nothing but delegate to the original message -
>>> but then as soon as someone mutates it, it creates a copy and uses
>>> that from that point on?
>>>
>>> i.e. make the copy of the message lazy - and only make a copy when a
>>> Processor really does try to mutate the Message?
>>>
>>> Then we'd get the best of both worlds; avoid breaking old code but
>>> avoid tons of unnecessary copies?
>
> BTW then using the current API you could have a Message and then call
>
> Message origin = ...;
> Message newMsg = origin.copy().copy().copy().copy();
>
> and the message would not actually be copied at all; new would just be
> a CopyOnWriteMessageFacade which would hold a reference to 'origin' as
> the readOnlyMessage (which it never mutates).
>
> The copy would only take place if you did
>
> newMsg.setBody("foo")
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)

[James Strachan <ja...@gmail.com>]

2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
> Moved this slightly different topic to a separate thread.
>
> ++1 from me.
>
> Do we need a vote on this one or it's a consensus?  Claus, it looks like you
> agree, but pointed out that there is some work involved, correct?
>
> Hadrian
>
>
>
> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>
>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>
>>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<ja...@gmail.com>
>>> wrote:
>>>>
>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>
>>>>> Hi
>>>>>
>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>
>>>>> The ProducerCache that is doing the actual sending when using template
>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>
>>>>> Well I am playing with to let it adhere to the principle Hadrian
>>>>> pointed out. He wanted the IN to be more static.
>>>>> And we cannot get there yet when you do routing as all the processors
>>>>> rely on IN being able to mutate during routing.
>>>>>
>>>>> Anyway my grief is that when you send in an Exchange the result would
>>>>> sometimes be stored on IN and not OUT.
>>>>> What I want it to do always is to store the result in OUT and keep IN
>>>>> as the original input.
>>>>>
>>>>> The code to do this is now a bit more complex than just before
>>>>>
>>>>>               // copy the original input
>>>>>               Message original = exchange.getIn().copy();
>>>>>
>>>>>               producer.process(exchange);
>>>>>
>>>>>               // if no OUT then set current IN as result (except for
>>>>> optional out)
>>>>>               if (!exchange.hasOut() && exchange.getPattern() !=
>>>>> ExchangePattern.InOptionalOut) {
>>>>>                   // but only if its not the same as original IN to
>>>>> avoid duplicating it
>>>>>                   // and to adhere to the fact that there was no OUT
>>>>> result at all
>>>>>                   if (original.getBody() != null &&
>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>                       exchange.setOut(exchange.getIn());
>>>>>                   }
>>>>>               }
>>>>>               // and restore original in
>>>>>               exchange.setIn(original);
>>>>>
>>>>>               return exchange;
>>>>>
>>>>>
>>>>> What I need to do is to copy the original IN message as it can be
>>>>> mutated during routing.
>>>>
>>>> How about we prevent mutation of the IN message? Create a Message
>>>> facade which throws UnsupportedOperationException if you try to mutate
>>>> it in any way. Then we can pass the same read-only Message around as
>>>> the IN within retry loops or from step to step if no new output is
>>>> created (e.g. in a content based router where you just move a Message
>>>> to the right endpoint without changing it)
>>>>
>>>
>>> A good idea but will break a lot of logic in Camel.
>>
>> Agreed. But with the benefit that we'd be able to get rid of all the
>> defensive copies in our code; plus we'd be able to pass the same
>> Message from step to step. The API would be a bit more clean; to
>> change the output, you create an OUT message (maybe by copying the
>> IN).
>>
>>
>>> Most of the Camel
>>> processors work on the IN message and set the result on either IN or
>>> OUT. At best they set it on OUT. But then the IN is always the
>>> original input? Or am I mistaking?
>>
>> Yeah, we'd have to patch code to no longer mutate IN
>>
>>
>>> How will this work with the Pipes And Filters EIP if the IN is
>>> immutable and always the original input?
>>
>> If no OUT, then no output was created, so pass the IN along...
>>
>>
>> OK how about this; a CopyOnWriteMessageFacade which does not mutate
>> the original message at all ever; if a message is used in a purely
>> read only way, it does nothing but delegate to the original message -
>> but then as soon as someone mutates it, it creates a copy and uses
>> that from that point on?
>>
>> i.e. make the copy of the message lazy - and only make a copy when a
>> Processor really does try to mutate the Message?
>>
>> Then we'd get the best of both worlds; avoid breaking old code but
>> avoid tons of unnecessary copies?

BTW then using the current API you could have a Message and then call

Message origin = ...;
Message newMsg = origin.copy().copy().copy().copy();

and the message would not actually be copied at all; new would just be
a CopyOnWriteMessageFacade which would hold a reference to 'origin' as
the readOnlyMessage (which it never mutates).

The copy would only take place if you did

newMsg.setBody("foo")

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/