You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Jason Burkhardt <jb...@gmail.com> on 2012/04/19 20:32:51 UTC
custom component synchronous processing
Ugh, tough time deciding a proper title for this one.
I have a custom component that basically implements a
com.sun.net.httpserver.HttpServer - as such, it is consumer only. My
HttpConsumer component extends DefaultConsumer and basically takes the
HttpServer and sets an HttpHandler object on it. The HttpHandler object is
created with a reference to the Processor from my HttpEndpoint (extends
DefaultEndpoint). When the handler receives a message it calls
endpoint.createExchange and then processor.process(exchange).
This is all well and good and a typical route looks like this:
<route>
<from uri="httpserver:myserver?someoptions"/>
<log message="message ${body}"/>
</route>
The problem I'm having is we are receiving posts to the server at a rate of
>= 3 per second and the log endpoint is getting them out of order from time
to time. I've debugged it to the point where the messages are in order
right up until the point processor.process(exchange) is called, and then it
falls into the realm of the InstrumentationProcessor, which processes it
asynchronously.
I've tried setting synchronous=true on my endpoint to no effect.
Is there a better way to accomplish what I want? I really need the messages
to be in the same order they were received for processing later in the
route, but thus far I have not been able to guarantee their order past the
processor.process call.
Any pointers/help would be appreciated. If it makes a difference, I'm
currently running camel 2.6.0
Thanks,
Jason Burkhardt
--
View this message in context: http://camel.465427.n5.nabble.com/custom-component-synchronous-processing-tp5652471p5652471.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: custom component synchronous processing
Posted by Claus Ibsen <cl...@gmail.com>.
On Tue, Apr 24, 2012 at 2:43 AM, Jason Burkhardt <jb...@gmail.com> wrote:
> Hi Claus,
> Thanks for pointing me in the proper direction.
> In most cases I want the http server to remain multithreaded, but in this
> specific case I really need the messages in order so out of your ideas I
> figured the most appropriate would be to limit the individual route using
> the threads dsl.
> I changed my route to:
> <route>
> <from uri="httpserver:myserver?someoptions"/>
> <threads poolSize="1" maxPoolSize="1">
> <log message="message ${body}"/>
> </threads>
> </route>
>
> But I will still occasionally log messages out of order (generally happens
> if one is much larger than a following one, eg 358kb message followed by a
> 0.5kb message, the 0.5kb message is logged first sometimes). They're all
> still logged in the proper order inside the http component, just like
> before, but once I call processor.process on my newly created exchange I
> lose the order.
>
> Did I miss the spirit of your post or am I using the threads dsl improperly?
>
Well the thread pool should process the messages in the order the
message is handed over to the thread pool.
But since your http server is multi threaded there is no guarantee
that what you see incoming in the http server
is the same order that is being handed over to the thread pool in Camel.
The JVM controls the threading and it may change CPU processing of
threads in between what you
see incoming at HTTP server and what gets send to the thread pool.
If you can control the threading model of the http server you can
possible configure it to use a single worker thread.
And then queue up incoming requests internally. Where the worker
thread is the thread that Camel uses, and thus it would become
sequential.
An alternative is that if the incoming requests have some sort of data
that can be used to re-sequence the messages.
Then you can use the resequencer EIP.
> --
> View this message in context: http://camel.465427.n5.nabble.com/custom-component-synchronous-processing-tp5652471p5660809.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: custom component synchronous processing
Posted by Jason Burkhardt <jb...@gmail.com>.
Hi Claus,
Thanks for pointing me in the proper direction.
In most cases I want the http server to remain multithreaded, but in this
specific case I really need the messages in order so out of your ideas I
figured the most appropriate would be to limit the individual route using
the threads dsl.
I changed my route to:
<route>
<from uri="httpserver:myserver?someoptions"/>
<threads poolSize="1" maxPoolSize="1">
<log message="message ${body}"/>
</threads>
</route>
But I will still occasionally log messages out of order (generally happens
if one is much larger than a following one, eg 358kb message followed by a
0.5kb message, the 0.5kb message is logged first sometimes). They're all
still logged in the proper order inside the http component, just like
before, but once I call processor.process on my newly created exchange I
lose the order.
Did I miss the spirit of your post or am I using the threads dsl improperly?
--
View this message in context: http://camel.465427.n5.nabble.com/custom-component-synchronous-processing-tp5652471p5660809.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: custom component synchronous processing
Posted by Claus Ibsen <cl...@gmail.com>.
On Thu, Apr 19, 2012 at 8:32 PM, Jason Burkhardt <jb...@gmail.com> wrote:
> Ugh, tough time deciding a proper title for this one.
> I have a custom component that basically implements a
> com.sun.net.httpserver.HttpServer - as such, it is consumer only. My
> HttpConsumer component extends DefaultConsumer and basically takes the
> HttpServer and sets an HttpHandler object on it. The HttpHandler object is
> created with a reference to the Processor from my HttpEndpoint (extends
> DefaultEndpoint). When the handler receives a message it calls
> endpoint.createExchange and then processor.process(exchange).
> This is all well and good and a typical route looks like this:
>
> <route>
> <from uri="httpserver:myserver?someoptions"/>
> <log message="message ${body}"/>
> </route>
>
> The problem I'm having is we are receiving posts to the server at a rate of
>>= 3 per second and the log endpoint is getting them out of order from time
> to time. I've debugged it to the point where the messages are in order
> right up until the point processor.process(exchange) is called, and then it
> falls into the realm of the InstrumentationProcessor, which processes it
> asynchronously.
> I've tried setting synchronous=true on my endpoint to no effect.
> Is there a better way to accomplish what I want? I really need the messages
> to be in the same order they were received for processing later in the
> route, but thus far I have not been able to guarantee their order past the
> processor.process call.
> Any pointers/help would be appreciated. If it makes a difference, I'm
> currently running camel 2.6.0
>
A http server is multi threaded and thus if you want any kind of
ordered processing further down by Camel.
You would need to process the message in sequence.
Either by sending the messages to a SEDA queue, and then have a route
that processes from the SEDA queue using
a single consumer.
Or you can use the threads EIP to limit the route to use a single thread.
Or let your consumer process the messages in sequence using a single thread.
But this will limit the scalability of your http service.
> Thanks,
> Jason Burkhardt
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/custom-component-synchronous-processing-tp5652471p5652471.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/