You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Debraj Manna <su...@gmail.com> on 2016/10/18 10:53:29 UTC
Camel - Making parallel GET requests and aggregating the results via
Dynamic Routes using Java DSL
I am receiving a request on a Jetty Http Endpoint. The request body
contains some urls in the request body. I have to make a GET request to
those urls. Then aggregate the results from each of the GET request and
return it to the caller.
Request Body:-
{
"data" : [
{"name" : "Hello", "url" : "http://server1"}
{"name" : "Hello2", "url" : "http://server2"}
]
}
One way I can think of doing this is like below:-
from("jetty:http://localhost:8888/hello").process(new Processor() {
public void process(Exchange exchange) throws Exception {
// 1. Make the GET request in parallel using ThreadPoolExecutor
// 2. Wait for all calls to finish. Collate the response
// 3. Write it to exchange.getOut().setBody
}
})
Can some one let me know if this can be achieved via Java DSL using camel
dynamic Routes, splitter & aggregator so that my Processor remains
relatively small?
I am using camel 2.16.3.
Re: Camel - Making parallel GET requests and aggregating the results
via Dynamic Routes using Java DSL
Posted by Debraj Manna <su...@gmail.com>.
Thanks Avnish.
On Tue, Oct 18, 2016 at 8:38 PM, Avnish Pundir <av...@tekmindz.com>
wrote:
> You can use JsonPathExpression in your split body and allow parallel
> execution using executorServiceRef, following is a sample skeleton -
>
> from("jetty:http://localhost:8888/hello").split(new
> JsonPathExpression("$.data[*]")).executorServiceRef(MY_EXECU
> TOR_REF).to(...)
>
> This will run your individual json tokens in parallel which you can use to
> send out independent GET requests to desired endpoints. You'll have to use
> aggregationStrategy though at end to accumulate the results (if you wish to
> combine some data) using strategyRef option. There are number of examples
> at http://camel.apache.org/splitter.html - towards the end of this
> documentation you'll see custome aggregationStrategy example as well.
>
> Just to clarify, JsonPathExpression comes from
> org.apache.camel.model.language.JsonPathExpression class (You'll have to
> import it in your route builder).
>
> Thanks,
> Avnish Pundir
>
>
> On 18-10-2016 16:23, Debraj Manna wrote:
>
>> I am receiving a request on a Jetty Http Endpoint. The request body
>> contains some urls in the request body. I have to make a GET request to
>> those urls. Then aggregate the results from each of the GET request and
>> return it to the caller.
>>
>> Request Body:-
>>
>> {
>> "data" : [
>> {"name" : "Hello", "url" : "http://server1"}
>> {"name" : "Hello2", "url" : "http://server2"}
>>
>> ]
>> }
>>
>> One way I can think of doing this is like below:-
>>
>> from("jetty:http://localhost:8888/hello").process(new Processor() {
>> public void process(Exchange exchange) throws Exception {
>> // 1. Make the GET request in parallel using ThreadPoolExecutor
>> // 2. Wait for all calls to finish. Collate the response
>> // 3. Write it to exchange.getOut().setBody
>> }
>> })
>>
>> Can some one let me know if this can be achieved via Java DSL using camel
>> dynamic Routes, splitter & aggregator so that my Processor remains
>> relatively small?
>>
>> I am using camel 2.16.3.
>>
>
> --
>
> ------------------------------
> Disclaimer: The information contained in this communication is
> confidential, private, proprietary, or otherwise privileged and is intended
> only for the use of the addressee.Unauthorized use, disclosure,
> distribution or copying is strictly prohibited and may be unlawful. If you
> have received this communication in error, please delete this message and
> notify the sender immediately - Samin TekMindz India Pvt.Ltd.
> ------------------------------
>
Re: Camel - Making parallel GET requests and aggregating the results
via Dynamic Routes using Java DSL
Posted by Avnish Pundir <av...@tekmindz.com>.
You can use JsonPathExpression in your split body and allow parallel
execution using executorServiceRef, following is a sample skeleton -
from("jetty:http://localhost:8888/hello").split(new
JsonPathExpression("$.data[*]")).executorServiceRef(MY_EXECUTOR_REF).to(...)
This will run your individual json tokens in parallel which you can use
to send out independent GET requests to desired endpoints. You'll have
to use aggregationStrategy though at end to accumulate the results (if
you wish to combine some data) using strategyRef option. There are
number of examples at http://camel.apache.org/splitter.html - towards
the end of this documentation you'll see custome aggregationStrategy
example as well.
Just to clarify, JsonPathExpression comes from
org.apache.camel.model.language.JsonPathExpression class (You'll have to
import it in your route builder).
Thanks,
Avnish Pundir
On 18-10-2016 16:23, Debraj Manna wrote:
> I am receiving a request on a Jetty Http Endpoint. The request body
> contains some urls in the request body. I have to make a GET request to
> those urls. Then aggregate the results from each of the GET request and
> return it to the caller.
>
> Request Body:-
>
> {
> "data" : [
> {"name" : "Hello", "url" : "http://server1"}
> {"name" : "Hello2", "url" : "http://server2"}
>
> ]
> }
>
> One way I can think of doing this is like below:-
>
> from("jetty:http://localhost:8888/hello").process(new Processor() {
> public void process(Exchange exchange) throws Exception {
> // 1. Make the GET request in parallel using ThreadPoolExecutor
> // 2. Wait for all calls to finish. Collate the response
> // 3. Write it to exchange.getOut().setBody
> }
> })
>
> Can some one let me know if this can be achieved via Java DSL using camel
> dynamic Routes, splitter & aggregator so that my Processor remains
> relatively small?
>
> I am using camel 2.16.3.
--
------------------------------
Disclaimer: The information contained in this communication is
confidential, private, proprietary, or otherwise privileged and is intended
only for the use of the addressee.Unauthorized use, disclosure,
distribution or copying is strictly prohibited and may be unlawful. If you
have received this communication in error, please delete this message and
notify the sender immediately - Samin TekMindz India Pvt.Ltd.
------------------------------