You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by mcrive <mc...@optasportsdata.com> on 2010/01/25 12:18:04 UTC
recipientList + AggregationStrategy
Hi,
I am using recipientList to send a message to a list of endpoints (mostly
FTP).
I am using an AggregationStrategy to compute few properties of the
exchanges:
- delivered
- how many retries
- time taken
I've been able to get 'delivered' and 'how many retries' for each endpoint
but I can't find a way to get the time it took to send the message.
I've tried to use an interceptor and store 'start-time' in the header and
then check the timestamp when the exchange hits the aggregation strategy but
this is not working as all exchanges are sent to the aggregationstrategy as
soon as the last endpoint is processed.
Do you have any suggestion on how I could get the transfer time for each
exchange?
I am using Camel 2.2
route sample:
//INTERCEPTOR
interceptSendToEndpoint("(ftp|direct|smtp):.*").process(new Processor() {
// in start-time header we store the time in which the message is sent to
the endpoint
public void process(Exchange exchange) throws Exception {
long currentTime = System.currentTimeMillis();
exchange.getIn().setHeader("start-time", currentTime );
}
});
//ROUTE
from("jms-test:queue:test.queue").process(processor)
.onException(Exception.class).maximumRedeliveries(2).redeliverDelay(60L).end()
.recipientList(header("recipientListHeader").tokenize(","))
.parallelProcessing().executorService(customThreadPoolExecutor)
.aggregationStrategy(new RecipientAggregationStrategy(deliveryEndpoints,
_endpointDeliveredBaseUri))
.to("direct:chunk.completed");
thanks
--
View this message in context: http://old.nabble.com/recipientList-%2B-AggregationStrategy-tp27305176p27305176.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: recipientList + AggregationStrategy
Posted by mcrive <mc...@optasportsdata.com>.
Wonderful news! thanks a lot
Camel rocks!
Claus Ibsen-2 wrote:
>
> After the 2.2 release which is happening soon. I think Hadrian will
> start to build the 2.2 release early this week.
>
--
View this message in context: http://old.nabble.com/recipientList-%2B-AggregationStrategy-tp27305176p27401314.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: recipientList + AggregationStrategy
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Feb 8, 2010 at 10:12 PM, mcrive <mc...@optasportsdata.com> wrote:
>
> Hi Claus,
> I saw trunk is for 2.3 now, do you have a plan about when you will check in
> the patch?
> https://issues.apache.org/activemq/browse/CAMEL-2429
>
Yeah I have just committed it now.
>
>
> Claus Ibsen-2 wrote:
>>
>> After the 2.2 release which is happening soon. I think Hadrian will
>> start to build the 2.2 release early this week.
>>
>
> --
> View this message in context: http://old.nabble.com/recipientList-%2B-AggregationStrategy-tp27305176p27506355.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: recipientList + AggregationStrategy
Posted by mcrive <mc...@optasportsdata.com>.
Hi Claus,
I saw trunk is for 2.3 now, do you have a plan about when you will check in
the patch?
https://issues.apache.org/activemq/browse/CAMEL-2429
Claus Ibsen-2 wrote:
>
> After the 2.2 release which is happening soon. I think Hadrian will
> start to build the 2.2 release early this week.
>
--
View this message in context: http://old.nabble.com/recipientList-%2B-AggregationStrategy-tp27305176p27506355.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: recipientList + AggregationStrategy
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Feb 1, 2010 at 9:53 AM, mcrive <mc...@optasportsdata.com> wrote:
>
> Hi,
> I saw you attached a patch to the ticket which will go to Camel 2.3,
> is there a plan on when you will start developing 2.3?
>
After the 2.2 release which is happening soon. I think Hadrian will
start to build the 2.2 release early this week.
> Thanks again,
> Marco
>
>
> Claus Ibsen-2 wrote:
>>
>>
>> I have created a new ticket as I think it can make Camel better
>> https://issues.apache.org/activemq/browse/CAMEL-2429
>>
>>
>
> --
> View this message in context: http://old.nabble.com/recipientList-%2B-AggregationStrategy-tp27305176p27401218.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: recipientList + AggregationStrategy
Posted by mcrive <mc...@optasportsdata.com>.
Hi,
I saw you attached a patch to the ticket which will go to Camel 2.3,
is there a plan on when you will start developing 2.3?
Thanks again,
Marco
Claus Ibsen-2 wrote:
>
>
> I have created a new ticket as I think it can make Camel better
> https://issues.apache.org/activemq/browse/CAMEL-2429
>
>
--
View this message in context: http://old.nabble.com/recipientList-%2B-AggregationStrategy-tp27305176p27401218.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: recipientList + AggregationStrategy
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Jan 29, 2010 at 1:06 PM, mcrive <mc...@optasportsdata.com> wrote:
>
> Hi,
> the question have been made in this conversation but unfortunately only you
> are replying me.
Well you have not said that you are looking for paid consultancy etc.
At the end I also have to draw a line somewhere. My family dont get
feed if nobody every in one way
or the other buys services from my employer. There is no such thing as
free lunch.
> Maybe I am looking into a very particular thing...
>
I have already laid out the answer in one of the earlier mails.
Just that if I have to write a specific sample for YOUR problem for
free is not something I can justify for my employer.
> It would be great if you can integrate such a feature in 2.2 (as I am
> already running it)
> I really appreciate your help, you already fix/enhanced Camel a lot
> following my issues!!
> That's the reason why I am on Camel 2.2 ;)
>
> Is there a JIRA ticket where you mention such a feature? If so I will keep
> an eye on it.
>
I have created a new ticket as I think it can make Camel better
https://issues.apache.org/activemq/browse/CAMEL-2429
> In the meantime I'll have a look to JMX to see if I can find a way to get
> the piece of information I need.
>
> Thanks again,
> Marco
>
>
>
> Claus Ibsen-2 wrote:
>>
>> There is a lot of stats in JMX, but since you use recipientList you
>> may want to filter that JMX stat to only be FTP related etc.
>>
>>> You told me I should consider consultancy, can you give me any direction?
>>> Who should I contact?
>>>
>>
>> You could ask at the Camel user forum for consultancy. There may be
>> people around wanting a gig.
>>
>> Obviously there is my employer but we do not do small jobs, which may
>> be what you are on the lookout for.
>>
>> But you are always welcome to contact them. There is a get support
>> button or a form to fill out under Contact Us:
>> http://fusesource.com/
>>
>>
>> On the other hand I have though of adding an event to the new
>> notification system in 2.2 to emit notifications for "send to
>> endpoint" which then could contain details such as endpoint and time
>> taken etc.
>>
>> Then you could just register a notification listener and grab that
>> stats as they come flying in.
>>
>>
>>
>>
>>
>>
>>>
>>> mcrive wrote:
>>>>
>>>> If you need to have people creating such a sample then consider
>>>> looking for consultancy.
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/recipientList-%2B-AggregationStrategy-tp27305176p27370252.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>>
>
> --
> View this message in context: http://old.nabble.com/recipientList-%2B-AggregationStrategy-tp27305176p27371092.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: recipientList + AggregationStrategy
Posted by mcrive <mc...@optasportsdata.com>.
Hi,
the question have been made in this conversation but unfortunately only you
are replying me.
Maybe I am looking into a very particular thing...
It would be great if you can integrate such a feature in 2.2 (as I am
already running it)
I really appreciate your help, you already fix/enhanced Camel a lot
following my issues!!
That's the reason why I am on Camel 2.2 ;)
Is there a JIRA ticket where you mention such a feature? If so I will keep
an eye on it.
In the meantime I'll have a look to JMX to see if I can find a way to get
the piece of information I need.
Thanks again,
Marco
Claus Ibsen-2 wrote:
>
> There is a lot of stats in JMX, but since you use recipientList you
> may want to filter that JMX stat to only be FTP related etc.
>
>> You told me I should consider consultancy, can you give me any direction?
>> Who should I contact?
>>
>
> You could ask at the Camel user forum for consultancy. There may be
> people around wanting a gig.
>
> Obviously there is my employer but we do not do small jobs, which may
> be what you are on the lookout for.
>
> But you are always welcome to contact them. There is a get support
> button or a form to fill out under Contact Us:
> http://fusesource.com/
>
>
> On the other hand I have though of adding an event to the new
> notification system in 2.2 to emit notifications for "send to
> endpoint" which then could contain details such as endpoint and time
> taken etc.
>
> Then you could just register a notification listener and grab that
> stats as they come flying in.
>
>
>
>
>
>
>>
>> mcrive wrote:
>>>
>>> If you need to have people creating such a sample then consider
>>> looking for consultancy.
>>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/recipientList-%2B-AggregationStrategy-tp27305176p27370252.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
>
--
View this message in context: http://old.nabble.com/recipientList-%2B-AggregationStrategy-tp27305176p27371092.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: recipientList + AggregationStrategy
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Jan 29, 2010 at 11:47 AM, mcrive <mc...@optasportsdata.com> wrote:
>
> Hi Claus,
> having a way to retrieve transfer time to an endpoint (FTP mostly) is key
> feature of the application I am writing (as a part of a bigger software
> system).
There is a lot of stats in JMX, but since you use recipientList you
may want to filter that JMX stat to only be FTP related etc.
> You told me I should consider consultancy, can you give me any direction?
> Who should I contact?
>
You could ask at the Camel user forum for consultancy. There may be
people around wanting a gig.
Obviously there is my employer but we do not do small jobs, which may
be what you are on the lookout for.
But you are always welcome to contact them. There is a get support
button or a form to fill out under Contact Us:
http://fusesource.com/
On the other hand I have though of adding an event to the new
notification system in 2.2 to emit notifications for "send to
endpoint" which then could contain details such as endpoint and time
taken etc.
Then you could just register a notification listener and grab that
stats as they come flying in.
>
> mcrive wrote:
>>
>> If you need to have people creating such a sample then consider
>> looking for consultancy.
>>
>
> --
> View this message in context: http://old.nabble.com/recipientList-%2B-AggregationStrategy-tp27305176p27370252.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: recipientList + AggregationStrategy
Posted by mcrive <mc...@optasportsdata.com>.
Hi Claus,
having a way to retrieve transfer time to an endpoint (FTP mostly) is key
feature of the application I am writing (as a part of a bigger software
system).
You told me I should consider consultancy, can you give me any direction?
Who should I contact?
mcrive wrote:
>
> If you need to have people creating such a sample then consider
> looking for consultancy.
>
--
View this message in context: http://old.nabble.com/recipientList-%2B-AggregationStrategy-tp27305176p27370252.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: recipientList + AggregationStrategy
Posted by mcrive <mc...@optasportsdata.com>.
if I put a destination after the recipientList its gets processed once all
endpoints got processed that means I can't compute transfer time of a single
endpoint.
I can't understand why you would use the interceptor, the interceptor isn't
called when the message has been delivered to the endpoint but when the
delivery starts so I can only have the start time, I can't see ways of
finding the end time.
Claus Ibsen-2 wrote:
>
>
> Sorry I am too busy to create such a sample. I think you just need to
> play a bit more with it and try to
> invoke manually that intercepted endpoint from a Processor where you
> can do the timing also.
>
> If you need to have people creating such a sample then consider
> looking for consultancy.
>
--
View this message in context: http://old.nabble.com/recipientList-%2B-AggregationStrategy-tp27305176p27337725.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: recipientList + AggregationStrategy
Posted by Claus Ibsen <cl...@gmail.com>.
On Tue, Jan 26, 2010 at 5:40 PM, mcrive <mc...@optasportsdata.com> wrote:
>
> Hi Claus,
> I am sorry but I don't understand what you mean...
>
> would it be possible for you to send me a sample (just the route)
>
Sorry I am too busy to create such a sample. I think you just need to
play a bit more with it and try to
invoke manually that intercepted endpoint from a Processor where you
can do the timing also.
If you need to have people creating such a sample then consider
looking for consultancy.
>
> Claus Ibsen-2 wrote:
>>
>> Hi
>>
>> Interesting use case you got.
>> I got to go over this again at another time. I am at a business
>> convention in the US which means I am on a 9h timezone difference
>> which makes me a bit dizzy at this moment.
>>
>> However just wanted to pitch in that JMX stores performance stats for
>> nearly all processors in a Camel route. However I know the JMX API is
>> clumpsy and a pain in the ...... to use.
>>
>> Using the interceptSendToEndpoint appears at first sight convenient to
>> use in your situations but you need to have a hook to the after
>> sending which it does not provides currently.
>>
>> We may be able to add something to the DSL so you can control when to
>> send to the original designated endpoint. Currently we only get the
>> skipSendToOriginalEndpoint(). However you could use this fact and send
>> the message yourself.
>>
>> For example from a bean or processor you could just
>>
>> long start = ....
>>
>> // send to intended endpoint
>> String uri = exchange.getHeader(Exchange.INTERCEPTED_ENDPOINT,
>> String.class);
>> // use eg producer template to send to that uri
>> long end = ...
>>
>> And you can use try .. finally so the end is always computed.
>>
>>
>>
>
> --
> View this message in context: http://old.nabble.com/recipientList-%2B-AggregationStrategy-tp27305176p27325684.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: recipientList + AggregationStrategy
Posted by mcrive <mc...@optasportsdata.com>.
Hi Claus,
I am sorry but I don't understand what you mean...
would it be possible for you to send me a sample (just the route)
Claus Ibsen-2 wrote:
>
> Hi
>
> Interesting use case you got.
> I got to go over this again at another time. I am at a business
> convention in the US which means I am on a 9h timezone difference
> which makes me a bit dizzy at this moment.
>
> However just wanted to pitch in that JMX stores performance stats for
> nearly all processors in a Camel route. However I know the JMX API is
> clumpsy and a pain in the ...... to use.
>
> Using the interceptSendToEndpoint appears at first sight convenient to
> use in your situations but you need to have a hook to the after
> sending which it does not provides currently.
>
> We may be able to add something to the DSL so you can control when to
> send to the original designated endpoint. Currently we only get the
> skipSendToOriginalEndpoint(). However you could use this fact and send
> the message yourself.
>
> For example from a bean or processor you could just
>
> long start = ....
>
> // send to intended endpoint
> String uri = exchange.getHeader(Exchange.INTERCEPTED_ENDPOINT,
> String.class);
> // use eg producer template to send to that uri
> long end = ...
>
> And you can use try .. finally so the end is always computed.
>
>
>
--
View this message in context: http://old.nabble.com/recipientList-%2B-AggregationStrategy-tp27305176p27325684.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: recipientList + AggregationStrategy
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Interesting use case you got.
I got to go over this again at another time. I am at a business
convention in the US which means I am on a 9h timezone difference
which makes me a bit dizzy at this moment.
However just wanted to pitch in that JMX stores performance stats for
nearly all processors in a Camel route. However I know the JMX API is
clumpsy and a pain in the ...... to use.
Using the interceptSendToEndpoint appears at first sight convenient to
use in your situations but you need to have a hook to the after
sending which it does not provides currently.
We may be able to add something to the DSL so you can control when to
send to the original designated endpoint. Currently we only get the
skipSendToOriginalEndpoint(). However you could use this fact and send
the message yourself.
For example from a bean or processor you could just
long start = ....
// send to intended endpoint
String uri = exchange.getHeader(Exchange.INTERCEPTED_ENDPOINT, String.class);
// use eg producer template to send to that uri
long end = ...
And you can use try .. finally so the end is always computed.
On Mon, Jan 25, 2010 at 12:18 PM, mcrive <mc...@optasportsdata.com> wrote:
>
> Hi,
> I am using recipientList to send a message to a list of endpoints (mostly
> FTP).
> I am using an AggregationStrategy to compute few properties of the
> exchanges:
> - delivered
> - how many retries
> - time taken
>
> I've been able to get 'delivered' and 'how many retries' for each endpoint
> but I can't find a way to get the time it took to send the message.
>
> I've tried to use an interceptor and store 'start-time' in the header and
> then check the timestamp when the exchange hits the aggregation strategy but
> this is not working as all exchanges are sent to the aggregationstrategy as
> soon as the last endpoint is processed.
>
> Do you have any suggestion on how I could get the transfer time for each
> exchange?
>
> I am using Camel 2.2
>
> route sample:
>
> //INTERCEPTOR
> interceptSendToEndpoint("(ftp|direct|smtp):.*").process(new Processor() {
> // in start-time header we store the time in which the message is sent to
> the endpoint
> public void process(Exchange exchange) throws Exception {
> long currentTime = System.currentTimeMillis();
> exchange.getIn().setHeader("start-time", currentTime );
> }
> });
>
> //ROUTE
> from("jms-test:queue:test.queue").process(processor)
> .onException(Exception.class).maximumRedeliveries(2).redeliverDelay(60L).end()
> .recipientList(header("recipientListHeader").tokenize(","))
> .parallelProcessing().executorService(customThreadPoolExecutor)
> .aggregationStrategy(new RecipientAggregationStrategy(deliveryEndpoints,
> _endpointDeliveredBaseUri))
> .to("direct:chunk.completed");
>
> thanks
> --
> View this message in context: http://old.nabble.com/recipientList-%2B-AggregationStrategy-tp27305176p27305176.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus