You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by Hadrian Zbarcea <hz...@gmail.com> on 2009/07/13 22:55:29 UTC
Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)
Roman,
Is there any api change you are proposing? If yes, could you please
outline the code changes you are suggesting?
Thanks,
Hadrian
On Jul 13, 2009, at 3:01 PM, Roman Kalukiewicz wrote:
> 2009/7/13 Claus Ibsen <cl...@gmail.com>:
>> On Mon, Jul 13, 2009 at 5:50 PM, James Strachan<james.strachan@gmail.com
>> > wrote:
>>> I'm confused. So rather than calling Exchange.getIn() to get the IN,
>>> you'd have to do Exchange.getUnitOfWork().somethingOrOther() to get
>>> the last IN? I don't see that as any simpler; making one of the 2
>>> messages harder to find, just moves the confusion somewhere else?
>>>
>> James you are not the only confused, many Camel new users are - IN vs
>> OUT vs FAULT and exchange patterns and why the hell they lose they
>> message or headers.
>> eg a simple route then need the assistance from the tracer to help
>> pin
>> point what goes wrong and where/what causes it to loose their
>> message/headers.
>>
>> No I do not want that, you need to do:
>> Exchange.getUnitOfWork().somethingOrOther() to get IN.
>> IN is simply = getMessage.
>>
>> IN + OUT is combined into a single Message. getMessage/setMessage.
>
> It looks like Claus would like to implement my original idea of
> removing IN/OUT/FAULT as separate messages, but put them into single
> message. Then there is always CURRENT message in the exchange) or
> however we call it then.
>
>> If you really need hold of the original input (as written before that
>> is hardly ever ever needed) its currently avail on the UoW.
>> But we can expose this on the Exchange as well in case its makes a
>> point.
>
> What I would propose would be to have an original input message kept
> somewhere, but we shouldn't mutate it. I would even prefer to have it
> in its original form like JBI Exchange, ServletRequest or whatever...
> for reference purposes, or if you want to do something very
> technology-specific. On the other hand I would prefer to have a
> generic exchange - without cusom subclasses at all.
>
> Roman
>
>>> 2009/7/12 Claus Ibsen <cl...@gmail.com>:
>>>> Hi
>>>>
>>>> Starting the kettle to brew me a grok of hot coffee, as its sunday
>>>> morning and I guess I got the energy to toss in my 2$ in the
>>>> bucket of
>>>> ideas
>>>> Now that we are into the open talks and that it seems we can open
>>>> the
>>>> box with the IN, OUT, FAULT again.
>>>>
>>>> I advocate for only one message on the implementation side. On
>>>> the API
>>>> side we could still have some notion of IN / OUT to keep the API
>>>> migration / impact simpler. But at the end having one message API
>>>> makes it simpler. There could be a boolean to test whether an
>>>> output
>>>> has been set or not. Just as if we have a test for fault.
>>>>
>>>>
>>>> Proposed Exchange API
>>>> ==================
>>>>
>>>> Message getMessage()
>>>> void setMessage(Message msg);
>>>>
>>>> boolean messageChanged();
>>>> -- false = message have newer been mutated, eg its still the
>>>> same input
>>>> -- true = the message or message body have been mutated in some
>>>> way
>>>> (setMessage or setBody called)
>>>> -- need to find a good name for this method, eg testing whether
>>>> the
>>>> message have been changed/updated or not
>>>>
>>>> That leaves us with a simple API.
>>>>
>>>> Then we do not need to worry about all the message management
>>>> internally in Camel, where we need to figure out whether to set
>>>> data
>>>> on IN or OUT and what else we kinda do, and it gets a bit complex
>>>> over
>>>> time how to do this correctly.
>>>>
>>>>
>>>> MEP: InOnly
>>>> =========
>>>> Now we can do as Hadrian want. Newer return something in OUT. And
>>>> leave the original input on the Exchange.
>>>> Here we need to use James CopyOnWrite technique so we can
>>>> preserve the
>>>> original message in case its mutated during routing.
>>>>
>>>>
>>>> MEP: InOut
>>>> =========
>>>> If an OUT message have been set (eg there is a reply), can be
>>>> determined if the messageChanged() == true. (then its not the
>>>> original
>>>> message anymore). And yes the current code does this. It will just
>>>> copy whatever there is in IN and use it as OUT if no OUT was set.
>>>> With this proposal this will be improved as its easier to
>>>> determine if
>>>> there is an OUT message or not.
>>>>
>>>>
>>>> Original message
>>>> =============
>>>> Currently there is an API to get the original message from the
>>>> UnitOfWork as its needed when doing redeliveries and a message was
>>>> doomed
>>>> and had to be moved to the dead letter channel. Then it makes much
>>>> more sense to move the original message than the current mutated
>>>> message.
>>>> ( i wonder if it should be default behavior )
>>>>
>>>>
>>>> AggregationStrategy
>>>> ===============
>>>> Another benefit with a single getMessage() is that end users using
>>>> AggregationStrategy will not be confused how to use it.
>>>> Should I look in IN or OUT. And the current code can actually be a
>>>> "lucky draw" as whether the data is in IN or OUT depending on facts
>>>> such as how the route path is.
>>>> We can even change it to pass in Message object instead of bare
>>>> bone
>>>> Exchange. You can always go from Message to exchange using
>>>> getExchange().
>>>>
>>>>
>>>> All the processors / components / data formats
>>>> ===================================
>>>> Logic will be easier as well as they do not need to cater for
>>>> IN / OUT
>>>> and where and how to set a result. Just work on the Message.
>>>>
>>>>
>>>> Use @deprecated for steady migration
>>>> ============================
>>>> Hadrian suggested that for the API migration to a single getMessage
>>>> you could let the getIn/setIn getOut/setOut delegate to
>>>> getMessage/setMessage.
>>>> And then mark them as @deprecated and then gradually change the
>>>> camel
>>>> code as we goes. So I do not think the hold up / change would
>>>> take a
>>>> lot of time and energy to get done.
>>>>
>>>>
>>>> Final words
>>>> ========
>>>> So if it was possible to simplify the API and reduce the IN OUT
>>>> FAULT
>>>> to a simpler API, even go as far as I would like with a single
>>>> Message.
>>>> Then that would be really great and worth a hold up for a
>>>> imminent 2.0 release.
>>>>
>>>>
>>>> Other frameworks
>>>> =============
>>>> I guess it wasn't final words after all :) Just wanted to say that
>>>> Mule, Spring Integration also just have a single message for the
>>>> message.
>>>> And yes I think their projects are also successful so it is not a
>>>> loss
>>>> that they do not have IN OUT. In fact I think their API is easier
>>>> to
>>>> work with than Camel.
>>>>
>>>> We are fortunate that most people with Camel do not work directly
>>>> with
>>>> Exchange but work more with Camel returning the message as an
>>>> expected
>>>> body type using its type converters. That helps a lot.
>>>>
>>>> But we have stories form Camel 1.x where people get confused whey
>>>> Camel return their original input in some situations where it was
>>>> not
>>>> expected in the OUT message. Or the fact producer template
>>>> sendBody is
>>>> not void in 1.x. Then people think it can be used for InOut.
>>>>
>>>> Okay end of mail.
>>>>
>>>>
>>>> On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<james.strachan@gmail.com
>>>> > wrote:
>>>>> 2009/7/10 Hadrian Zbarcea <hz...@gmail.com>:
>>>>>> Moved this slightly different topic to a separate thread.
>>>>>>
>>>>>> ++1 from me.
>>>>>>
>>>>>> Do we need a vote on this one or it's a consensus? Claus, it
>>>>>> looks like you
>>>>>> agree, but pointed out that there is some work involved, correct?
>>>>>>
>>>>>> Hadrian
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>>>>>
>>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>>
>>>>>>>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<james.strachan@gmail.com
>>>>>>>> >
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> 2009/7/10 Claus Ibsen <cl...@gmail.com>:
>>>>>>>>>>
>>>>>>>>>> Hi
>>>>>>>>>>
>>>>>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>>>>>
>>>>>>>>>> The ProducerCache that is doing the actual sending when
>>>>>>>>>> using template
>>>>>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>>>>>
>>>>>>>>>> Well I am playing with to let it adhere to the principle
>>>>>>>>>> Hadrian
>>>>>>>>>> pointed out. He wanted the IN to be more static.
>>>>>>>>>> And we cannot get there yet when you do routing as all the
>>>>>>>>>> processors
>>>>>>>>>> rely on IN being able to mutate during routing.
>>>>>>>>>>
>>>>>>>>>> Anyway my grief is that when you send in an Exchange the
>>>>>>>>>> result would
>>>>>>>>>> sometimes be stored on IN and not OUT.
>>>>>>>>>> What I want it to do always is to store the result in OUT
>>>>>>>>>> and keep IN
>>>>>>>>>> as the original input.
>>>>>>>>>>
>>>>>>>>>> The code to do this is now a bit more complex than just
>>>>>>>>>> before
>>>>>>>>>>
>>>>>>>>>> // copy the original input
>>>>>>>>>> Message original = exchange.getIn().copy();
>>>>>>>>>>
>>>>>>>>>> producer.process(exchange);
>>>>>>>>>>
>>>>>>>>>> // if no OUT then set current IN as result
>>>>>>>>>> (except for
>>>>>>>>>> optional out)
>>>>>>>>>> if (!exchange.hasOut() &&
>>>>>>>>>> exchange.getPattern() !=
>>>>>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>>>>> // but only if its not the same as
>>>>>>>>>> original IN to
>>>>>>>>>> avoid duplicating it
>>>>>>>>>> // and to adhere to the fact that there
>>>>>>>>>> was no OUT
>>>>>>>>>> result at all
>>>>>>>>>> if (original.getBody() != null &&
>>>>>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>>>>> exchange.setOut(exchange.getIn());
>>>>>>>>>> }
>>>>>>>>>> }
>>>>>>>>>> // and restore original in
>>>>>>>>>> exchange.setIn(original);
>>>>>>>>>>
>>>>>>>>>> return exchange;
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> What I need to do is to copy the original IN message as it
>>>>>>>>>> can be
>>>>>>>>>> mutated during routing.
>>>>>>>>>
>>>>>>>>> How about we prevent mutation of the IN message? Create a
>>>>>>>>> Message
>>>>>>>>> facade which throws UnsupportedOperationException if you try
>>>>>>>>> to mutate
>>>>>>>>> it in any way. Then we can pass the same read-only Message
>>>>>>>>> around as
>>>>>>>>> the IN within retry loops or from step to step if no new
>>>>>>>>> output is
>>>>>>>>> created (e.g. in a content based router where you just move
>>>>>>>>> a Message
>>>>>>>>> to the right endpoint without changing it)
>>>>>>>>>
>>>>>>>>
>>>>>>>> A good idea but will break a lot of logic in Camel.
>>>>>>>
>>>>>>> Agreed. But with the benefit that we'd be able to get rid of
>>>>>>> all the
>>>>>>> defensive copies in our code; plus we'd be able to pass the same
>>>>>>> Message from step to step. The API would be a bit more clean; to
>>>>>>> change the output, you create an OUT message (maybe by copying
>>>>>>> the
>>>>>>> IN).
>>>>>>>
>>>>>>>
>>>>>>>> Most of the Camel
>>>>>>>> processors work on the IN message and set the result on
>>>>>>>> either IN or
>>>>>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>>>>>> original input? Or am I mistaking?
>>>>>>>
>>>>>>> Yeah, we'd have to patch code to no longer mutate IN
>>>>>>>
>>>>>>>
>>>>>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>>>>>> immutable and always the original input?
>>>>>>>
>>>>>>> If no OUT, then no output was created, so pass the IN along...
>>>>>>>
>>>>>>>
>>>>>>> OK how about this; a CopyOnWriteMessageFacade which does not
>>>>>>> mutate
>>>>>>> the original message at all ever; if a message is used in a
>>>>>>> purely
>>>>>>> read only way, it does nothing but delegate to the original
>>>>>>> message -
>>>>>>> but then as soon as someone mutates it, it creates a copy and
>>>>>>> uses
>>>>>>> that from that point on?
>>>>>>>
>>>>>>> i.e. make the copy of the message lazy - and only make a copy
>>>>>>> when a
>>>>>>> Processor really does try to mutate the Message?
>>>>>>>
>>>>>>> Then we'd get the best of both worlds; avoid breaking old code
>>>>>>> but
>>>>>>> avoid tons of unnecessary copies?
>>>>>
>>>>> BTW then using the current API you could have a Message and then
>>>>> call
>>>>>
>>>>> Message origin = ...;
>>>>> Message newMsg = origin.copy().copy().copy().copy();
>>>>>
>>>>> and the message would not actually be copied at all; new would
>>>>> just be
>>>>> a CopyOnWriteMessageFacade which would hold a reference to
>>>>> 'origin' as
>>>>> the readOnlyMessage (which it never mutates).
>>>>>
>>>>> The copy would only take place if you did
>>>>>
>>>>> newMsg.setBody("foo")
>>>>>
>>>>> --
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>>
>>>>> Open Source Integration
>>>>> http://fusesource.com/
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>
>>>
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://fusesource.com/
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>