You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Knut Aksnes-NOR <Kn...@jeppesen.com> on 2012/04/03 16:39:05 UTC
Weird and unexpected behavior.
In one of my routes I have seen a for me at least extremely strange behavior. It is either a bug or a potentially very dangerous feature.
First some background:
1. I am running Camel 2.9.1 from using war packaging and Geronimo 2.2.1 with Jetty.
2. The source of my problem route is a servlet.
3. The requests arrive via http post and are binary data.
4. I have written a small helper processor shown here:
public static final class BodyDebugger implements Processor {
final String text;
public BodyDebugger(String arg) {
text = arg;
}
@Override
public void process(Exchange exchange) throws Exception {
byte[] body = exchange.getIn().getBody(byte[].class);
LogFactory.getLog(getClass()).warn(
text
+ ": "
+ (body == null ? "(NULL)" : Integer
.toString(body.length)));
}
}
It's only purpose is to convert the data to byte[] and show the array length. I do this conversion as I need the data block as byte array at a later stage (As input to another route, this time using ActiveMQ.)
Now to the strange behavior: Invoking this processor twice in a row this way .process(new BodyDebugger("Foo")).process(new BodyDebugger("Bar") results in two different results the first one might be correct, the second time around the body length is 0, that is a body consisting of an empty array, not the null pointer.
Re: Weird and unexpected behavior.
Posted by Raul Kripalani <ra...@fusesource.com>.
Big congrats to the entire team!
Nice of you to post your success story in the forum. Most guys only
remember us when they have a problem ;)
Thanks.
On 4 Apr 2012, at 22:14, Knut Aksnes-NOR <Kn...@jeppesen.com> wrote:
> Unfortunately I doesn't have it in its orginal form, it was small, have a working version now using a small processor converting and assigning. I will dig it out later and posting it. Basically the route went straight into a choice picking out the POST's processing only those, returning an error string for GET- etc.
> POSTS were logged to a file (via a wiretap) then sent to another route (a direct route).
>
> This other route is essentially ActiveMQ->Composed Message Processor->ActiveMQ
> Everything InOut.
>
> At the end we got everything worked and within far too tight time constraints, so far a big success for Camel and ActiveMQ. At the time being we are quite pleased with Camel, it is not perfect but for this kind of project better than the alternatives we know about. This have been the kind of project where we normally would have said no to. Meeting the deadlines with the resources available would be very difficult using traditional approaches, consultants would be difficult due to the massive domain knowledge needed. With Camel and ActiveMQ we said that it can be done, risky, but can be done. We have had lots of problems unrelated to this technology choice and still made it.
>
> -----Original Message-----
> From: Raul Kripalani [mailto:raul@fusesource.com]
> Sent: 4. april 2012 11:14
> To: users@camel.apache.org
> Subject: Re: Weird and unexpected behavior.
>
> Hi,
>
> Can we see the route where you use convertBodyTo?
>
> Thanks,
> Raul.
>
> On 4 Apr 2012, at 07:24, Knut Aksnes-NOR <Kn...@jeppesen.com> wrote:
>
>> Isn't this more or less what convertBodyTo(byte[].class) would do?
>>
>> -----Original Message-----
>> From: Raul Kripalani [mailto:raul@fusesource.com]
>> Sent: 4. april 2012 03:56
>> To: users@camel.apache.org
>> Subject: Re: Weird and unexpected behavior.
>>
>> Sorry to say this behaviour is *not* weird at all ;)
>>
>> You are converting the Stream to a byte[] the first time around, but you are not storing the result on the Exchange.
>>
>> Therefore, the Exchange still contains a Stream, which happens to already be exhausted. Hence the byte[] size 0 after the second processor.
>>
>> Just set the IN message to the byte[] resulting from the conversion, by using Exchange.getIn().setBody(byte[]). And you should be good to go (unless I have failed to spot something else!).
>>
>> Regards,
>> Raul.
>>
>> On 3 Apr 2012, at 22:31, "Knut-Håvard Aksnes" <kh...@gmail.com> wrote:
>>
>>> Shouldn't .convertBodyTo(byte[].class) convert the stream content to
>>> a re-readable byte array?
>>>
>>> (I am the same person as Knut Aksnes-NOR, different accounts due to
>>> recent changes in mail setup at job.)
>>>
>>> Are there an easy way of tracing type conversions and/or list which
>>> type converters are available, when debugging routes in camel? Both
>>> options would ease debugging a lot. Such options might be there, but
>>> I am not aware of how to use them. So I have had to fall back to
>>> processors and tracer. I have been searching for such tools on the web pages and in Camel in Action.
>>>
>>> --
>>> View this message in context:
>>> http://camel.465427.n5.nabble.com/Weird-and-unexpected-behavior-tp561
>>> 5 358p5616461.html Sent from the Camel - Users mailing list archive
>>> at Nabble.com.
RE: Weird and unexpected behavior.
Posted by Knut Aksnes-NOR <Kn...@jeppesen.com>.
Unfortunately I doesn't have it in its orginal form, it was small, have a working version now using a small processor converting and assigning. I will dig it out later and posting it. Basically the route went straight into a choice picking out the POST's processing only those, returning an error string for GET- etc.
POSTS were logged to a file (via a wiretap) then sent to another route (a direct route).
This other route is essentially ActiveMQ->Composed Message Processor->ActiveMQ
Everything InOut.
At the end we got everything worked and within far too tight time constraints, so far a big success for Camel and ActiveMQ. At the time being we are quite pleased with Camel, it is not perfect but for this kind of project better than the alternatives we know about. This have been the kind of project where we normally would have said no to. Meeting the deadlines with the resources available would be very difficult using traditional approaches, consultants would be difficult due to the massive domain knowledge needed. With Camel and ActiveMQ we said that it can be done, risky, but can be done. We have had lots of problems unrelated to this technology choice and still made it.
-----Original Message-----
From: Raul Kripalani [mailto:raul@fusesource.com]
Sent: 4. april 2012 11:14
To: users@camel.apache.org
Subject: Re: Weird and unexpected behavior.
Hi,
Can we see the route where you use convertBodyTo?
Thanks,
Raul.
On 4 Apr 2012, at 07:24, Knut Aksnes-NOR <Kn...@jeppesen.com> wrote:
> Isn't this more or less what convertBodyTo(byte[].class) would do?
>
> -----Original Message-----
> From: Raul Kripalani [mailto:raul@fusesource.com]
> Sent: 4. april 2012 03:56
> To: users@camel.apache.org
> Subject: Re: Weird and unexpected behavior.
>
> Sorry to say this behaviour is *not* weird at all ;)
>
> You are converting the Stream to a byte[] the first time around, but you are not storing the result on the Exchange.
>
> Therefore, the Exchange still contains a Stream, which happens to already be exhausted. Hence the byte[] size 0 after the second processor.
>
> Just set the IN message to the byte[] resulting from the conversion, by using Exchange.getIn().setBody(byte[]). And you should be good to go (unless I have failed to spot something else!).
>
> Regards,
> Raul.
>
> On 3 Apr 2012, at 22:31, "Knut-Håvard Aksnes" <kh...@gmail.com> wrote:
>
>> Shouldn't .convertBodyTo(byte[].class) convert the stream content to
>> a re-readable byte array?
>>
>> (I am the same person as Knut Aksnes-NOR, different accounts due to
>> recent changes in mail setup at job.)
>>
>> Are there an easy way of tracing type conversions and/or list which
>> type converters are available, when debugging routes in camel? Both
>> options would ease debugging a lot. Such options might be there, but
>> I am not aware of how to use them. So I have had to fall back to
>> processors and tracer. I have been searching for such tools on the web pages and in Camel in Action.
>>
>> --
>> View this message in context:
>> http://camel.465427.n5.nabble.com/Weird-and-unexpected-behavior-tp561
>> 5 358p5616461.html Sent from the Camel - Users mailing list archive
>> at Nabble.com.
Re: Weird and unexpected behavior.
Posted by Raul Kripalani <ra...@fusesource.com>.
Hi,
Can we see the route where you use convertBodyTo?
Thanks,
Raul.
On 4 Apr 2012, at 07:24, Knut Aksnes-NOR <Kn...@jeppesen.com> wrote:
> Isn't this more or less what convertBodyTo(byte[].class) would do?
>
> -----Original Message-----
> From: Raul Kripalani [mailto:raul@fusesource.com]
> Sent: 4. april 2012 03:56
> To: users@camel.apache.org
> Subject: Re: Weird and unexpected behavior.
>
> Sorry to say this behaviour is *not* weird at all ;)
>
> You are converting the Stream to a byte[] the first time around, but you are not storing the result on the Exchange.
>
> Therefore, the Exchange still contains a Stream, which happens to already be exhausted. Hence the byte[] size 0 after the second processor.
>
> Just set the IN message to the byte[] resulting from the conversion, by using Exchange.getIn().setBody(byte[]). And you should be good to go (unless I have failed to spot something else!).
>
> Regards,
> Raul.
>
> On 3 Apr 2012, at 22:31, "Knut-Håvard Aksnes" <kh...@gmail.com> wrote:
>
>> Shouldn't .convertBodyTo(byte[].class) convert the stream content to a
>> re-readable byte array?
>>
>> (I am the same person as Knut Aksnes-NOR, different accounts due to
>> recent changes in mail setup at job.)
>>
>> Are there an easy way of tracing type conversions and/or list which
>> type converters are available, when debugging routes in camel? Both
>> options would ease debugging a lot. Such options might be there, but I
>> am not aware of how to use them. So I have had to fall back to
>> processors and tracer. I have been searching for such tools on the web pages and in Camel in Action.
>>
>> --
>> View this message in context:
>> http://camel.465427.n5.nabble.com/Weird-and-unexpected-behavior-tp5615
>> 358p5616461.html Sent from the Camel - Users mailing list archive at
>> Nabble.com.
RE: Weird and unexpected behavior.
Posted by Knut Aksnes-NOR <Kn...@jeppesen.com>.
Isn't this more or less what convertBodyTo(byte[].class) would do?
-----Original Message-----
From: Raul Kripalani [mailto:raul@fusesource.com]
Sent: 4. april 2012 03:56
To: users@camel.apache.org
Subject: Re: Weird and unexpected behavior.
Sorry to say this behaviour is *not* weird at all ;)
You are converting the Stream to a byte[] the first time around, but you are not storing the result on the Exchange.
Therefore, the Exchange still contains a Stream, which happens to already be exhausted. Hence the byte[] size 0 after the second processor.
Just set the IN message to the byte[] resulting from the conversion, by using Exchange.getIn().setBody(byte[]). And you should be good to go (unless I have failed to spot something else!).
Regards,
Raul.
On 3 Apr 2012, at 22:31, "Knut-Håvard Aksnes" <kh...@gmail.com> wrote:
> Shouldn't .convertBodyTo(byte[].class) convert the stream content to a
> re-readable byte array?
>
> (I am the same person as Knut Aksnes-NOR, different accounts due to
> recent changes in mail setup at job.)
>
> Are there an easy way of tracing type conversions and/or list which
> type converters are available, when debugging routes in camel? Both
> options would ease debugging a lot. Such options might be there, but I
> am not aware of how to use them. So I have had to fall back to
> processors and tracer. I have been searching for such tools on the web pages and in Camel in Action.
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Weird-and-unexpected-behavior-tp5615
> 358p5616461.html Sent from the Camel - Users mailing list archive at
> Nabble.com.
Re: Weird and unexpected behavior.
Posted by Claus Ibsen <cl...@gmail.com>.
On Wed, Apr 4, 2012 at 3:55 AM, Raul Kripalani <ra...@fusesource.com> wrote:
> Sorry to say this behaviour is *not* weird at all ;)
>
> You are converting the Stream to a byte[] the first time around, but
> you are not storing the result on the Exchange.
>
> Therefore, the Exchange still contains a Stream, which happens to
> already be exhausted. Hence the byte[] size 0 after the second
> processor.
>
> Just set the IN message to the byte[] resulting from the conversion,
> by using Exchange.getIn().setBody(byte[]). And you should be good to
> go (unless I have failed to spot something else!).
>
Yes Raul is spot on.
> Regards,
> Raul.
>
> On 3 Apr 2012, at 22:31, "Knut-Håvard Aksnes" <kh...@gmail.com> wrote:
>
>> Shouldn't .convertBodyTo(byte[].class) convert the stream content to a
>> re-readable byte array?
>>
>> (I am the same person as Knut Aksnes-NOR, different accounts due to recent
>> changes in mail setup at job.)
>>
>> Are there an easy way of tracing type conversions and/or list which type
>> converters are available, when debugging routes in camel? Both options would
>> ease debugging a lot. Such options might be there, but I am not aware of how
>> to use them. So I have had to fall back to processors and tracer. I have
>> been searching for such tools on the web pages and in Camel in Action.
>>
>> --
>> View this message in context: http://camel.465427.n5.nabble.com/Weird-and-unexpected-behavior-tp5615358p5616461.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: Weird and unexpected behavior.
Posted by Raul Kripalani <ra...@fusesource.com>.
Sorry to say this behaviour is *not* weird at all ;)
You are converting the Stream to a byte[] the first time around, but
you are not storing the result on the Exchange.
Therefore, the Exchange still contains a Stream, which happens to
already be exhausted. Hence the byte[] size 0 after the second
processor.
Just set the IN message to the byte[] resulting from the conversion,
by using Exchange.getIn().setBody(byte[]). And you should be good to
go (unless I have failed to spot something else!).
Regards,
Raul.
On 3 Apr 2012, at 22:31, "Knut-Håvard Aksnes" <kh...@gmail.com> wrote:
> Shouldn't .convertBodyTo(byte[].class) convert the stream content to a
> re-readable byte array?
>
> (I am the same person as Knut Aksnes-NOR, different accounts due to recent
> changes in mail setup at job.)
>
> Are there an easy way of tracing type conversions and/or list which type
> converters are available, when debugging routes in camel? Both options would
> ease debugging a lot. Such options might be there, but I am not aware of how
> to use them. So I have had to fall back to processors and tracer. I have
> been searching for such tools on the web pages and in Camel in Action.
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Weird-and-unexpected-behavior-tp5615358p5616461.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Weird and unexpected behavior.
Posted by Knut-Håvard Aksnes <kh...@gmail.com>.
I am already testing tracer, my main problem being that I work a lot
with non-textual data in the current project, relatively few routes but
still room for lots of weirdness. What we implement is essentially a
Composed message processor as well as processing for one new element.
Using HTTP against an existing legacy C++ server (fronted by ASP) for
the old part. And using .NET for the actual splitting and merging due to
reuse of code (via ActiveMQ. ) The splitter component takes a binary
message in, returning a map. The result is converted to a a collection
using a trivial processor for input to the camel splitter. The
aggregator is producing a map forwarded to .Net for conversion to a pure
binary result. At various places there are several wiretaps to feed the
incoming data into various back-ends. All in all a relatively typical
integration project. Not the nicest architecture, but it allows us to
reuse lots of legacy stuff without having to modify much of it.
On 04/03/2012 11:36 PM, David Karlsen wrote:
> Try the tracer: http://camel.apache.org/tracer.html
>
> 2012/4/3 Knut-Håvard Aksnes<kh...@gmail.com>:
>> Shouldn't .convertBodyTo(byte[].class) convert the stream content to a
>> re-readable byte array?
>>
>> (I am the same person as Knut Aksnes-NOR, different accounts due to recent
>> changes in mail setup at job.)
>>
>> Are there an easy way of tracing type conversions and/or list which type
>> converters are available, when debugging routes in camel? Both options would
>> ease debugging a lot. Such options might be there, but I am not aware of how
>> to use them. So I have had to fall back to processors and tracer. I have
>> been searching for such tools on the web pages and in Camel in Action.
>>
>> --
>> View this message in context: http://camel.465427.n5.nabble.com/Weird-and-unexpected-behavior-tp5615358p5616461.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
Re: Weird and unexpected behavior.
Posted by David Karlsen <da...@gmail.com>.
Try the tracer: http://camel.apache.org/tracer.html
2012/4/3 Knut-Håvard Aksnes <kh...@gmail.com>:
> Shouldn't .convertBodyTo(byte[].class) convert the stream content to a
> re-readable byte array?
>
> (I am the same person as Knut Aksnes-NOR, different accounts due to recent
> changes in mail setup at job.)
>
> Are there an easy way of tracing type conversions and/or list which type
> converters are available, when debugging routes in camel? Both options would
> ease debugging a lot. Such options might be there, but I am not aware of how
> to use them. So I have had to fall back to processors and tracer. I have
> been searching for such tools on the web pages and in Camel in Action.
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Weird-and-unexpected-behavior-tp5615358p5616461.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
--
David J. M. Karlsen - http://www.linkedin.com/in/davidkarlsen
Re: Weird and unexpected behavior.
Posted by Knut-Håvard Aksnes <kh...@gmail.com>.
Shouldn't .convertBodyTo(byte[].class) convert the stream content to a
re-readable byte array?
(I am the same person as Knut Aksnes-NOR, different accounts due to recent
changes in mail setup at job.)
Are there an easy way of tracing type conversions and/or list which type
converters are available, when debugging routes in camel? Both options would
ease debugging a lot. Such options might be there, but I am not aware of how
to use them. So I have had to fall back to processors and tracer. I have
been searching for such tools on the web pages and in Camel in Action.
--
View this message in context: http://camel.465427.n5.nabble.com/Weird-and-unexpected-behavior-tp5615358p5616461.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Weird and unexpected behavior.
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Read about stream caching
http://camel.apache.org/stream-caching.html
We have a notice on the jetty page
http://camel.apache.org/jetty.html
We should have that notice on the servlet as well, as most servlet
implementations is also stream based.
On Tue, Apr 3, 2012 at 4:39 PM, Knut Aksnes-NOR
<Kn...@jeppesen.com> wrote:
> In one of my routes I have seen a for me at least extremely strange behavior. It is either a bug or a potentially very dangerous feature.
> First some background:
> 1. I am running Camel 2.9.1 from using war packaging and Geronimo 2.2.1 with Jetty.
> 2. The source of my problem route is a servlet.
> 3. The requests arrive via http post and are binary data.
> 4. I have written a small helper processor shown here:
>
> public static final class BodyDebugger implements Processor {
> final String text;
>
> public BodyDebugger(String arg) {
> text = arg;
> }
>
> @Override
> public void process(Exchange exchange) throws Exception {
> byte[] body = exchange.getIn().getBody(byte[].class);
> LogFactory.getLog(getClass()).warn(
> text
> + ": "
> + (body == null ? "(NULL)" : Integer
> .toString(body.length)));
> }
> }
>
> It's only purpose is to convert the data to byte[] and show the array length. I do this conversion as I need the data block as byte array at a later stage (As input to another route, this time using ActiveMQ.)
>
> Now to the strange behavior: Invoking this processor twice in a row this way .process(new BodyDebugger("Foo")).process(new BodyDebugger("Bar") results in two different results the first one might be correct, the second time around the body length is 0, that is a body consisting of an empty array, not the null pointer.
--
Claus Ibsen
-----------------
CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/