You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by humayun0156 <hu...@gmail.com> on 2013/07/20 23:15:09 UTC
Camel HTTP component vs. Apache HTTP client
Hi,
currently i'm working on a project which needs to download a lots of XML by
HTTP request. when i use camel's HTTP component it took to download per
request 600-800 ms but if i used for same request using Apache HTTP client
then it took 300-400 ms. Is it expected behavior or i misuse of camel's HTTP
component?
Any idea?
Thanks in advance.
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-HTTP-component-vs-Apache-HTTP-client-tp5735943.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel HTTP component vs. Apache HTTP client
Posted by humayun0156 <hu...@gmail.com>.
yes @Partha is right. i used the HttpEndpoints options
"httpClient.authenticationPreemptive=true" in my url and the download time
per xml reduced to 250-300ms.
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-HTTP-component-vs-Apache-HTTP-client-tp5735943p5736209.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel HTTP component vs. Apache HTTP client
Posted by partha <pa...@gmail.com>.
>From the code in DefaultHTTPClient, it seems that you are using preemptive
authentication. Could this be the problem with camel? If preemptive
authentication is not used, each request would go twice. Once without the
auth headers and then again with the auth headers. This could contribute to
the time to download.
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-HTTP-component-vs-Apache-HTTP-client-tp5735943p5736197.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel HTTP component vs. Apache HTTP client
Posted by humayun0156 <hu...@gmail.com>.
HelperBean.java <http://paste.ubuntu.com/5899832/>
ObjectCreatoinProcessor.java <http://paste.ubuntu.com/5899836/>
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-HTTP-component-vs-Apache-HTTP-client-tp5735943p5736005.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel HTTP component vs. Apache HTTP client
Posted by Raul Kripalani <ra...@evosent.com>.
Camel does a fair degree of lazy loading. Try to measure performance
on a second or third invocation of the route, rather than the first
time.
Also, pease post the source your HelperBean and ObjectCreationProcess class.
Thanks,
Raúl.
On 21 Jul 2013, at 18:51, humayun0156 <hu...@gmail.com> wrote:
> i've used http4 component and test my code. but it still took 600-800 ms per
> request.
> my route looks like :
> from("direct:npGet")
> .setHeader(Exchange.HTTP_METHOD, constant("GET"))
> .setProperty("url", simple("${body}"))
> .setHeader(Exchange.HTTP_URI, simple("${body}"))
> .to("http4://dummyHost.com?" + npAuthenticationString)
> .convertBodyTo(Document.class)
> .bean(HelperBean.class, "setUrlETagMapping")
> .process(new ObjectCreationProcess(xmlToObjectMapping));
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Camel-HTTP-component-vs-Apache-HTTP-client-tp5735943p5735974.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel HTTP component vs. Apache HTTP client
Posted by Willem Jiang <wi...@gmail.com>.
It makes sense that camel has lots of things to do beside processing the
message as you do with HttpClient.
Willem Jiang
Red Hat, Inc.
FuseSource is now part of Red Hat
Web: http://www.fusesource.com | http://www.redhat.com
Blog: http://willemjiang.blogspot.com (http://willemjiang.blogspot.com/)
(English)
http://jnn.iteye.com (http://jnn.javaeye.com/) (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On Mon, Jul 22, 2013 at 11:26 AM, humayun0156 <hu...@gmail.com> wrote:
> my xml files are not more than 5kb and i don't want to cache them.
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Camel-HTTP-component-vs-Apache-HTTP-client-tp5735943p5735995.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Re: Camel HTTP component vs. Apache HTTP client
Posted by humayun0156 <hu...@gmail.com>.
my xml files are not more than 5kb and i don't want to cache them.
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-HTTP-component-vs-Apache-HTTP-client-tp5735943p5735995.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel HTTP component vs. Apache HTTP client
Posted by Willem Jiang <wi...@gmail.com>.
Do you know what's the size of the XML file?
Camel will try to cache the XML input stream into a file if the input
stream size is big than 64K.
You can set the size bigger to avoid caching the input stream into a file
by setting the properties in CamelContext like
context.getProperties().put(CachedOutputStream.THRESHOLD, "1048576");
You can find more information here[1]
[1]http://camel.apache.org/stream-caching.html
On Mon, Jul 22, 2013 at 1:50 AM, humayun0156 <hu...@gmail.com> wrote:
> i've used http4 component and test my code. but it still took 600-800 ms
> per
> request.
> my route looks like :
> from("direct:npGet")
> .setHeader(Exchange.HTTP_METHOD, constant("GET"))
> .setProperty("url", simple("${body}"))
> .setHeader(Exchange.HTTP_URI, simple("${body}"))
> .to("http4://dummyHost.com?" + npAuthenticationString)
> .convertBodyTo(Document.class)
> .bean(HelperBean.class, "setUrlETagMapping")
> .process(new ObjectCreationProcess(xmlToObjectMapping));
>
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Camel-HTTP-component-vs-Apache-HTTP-client-tp5735943p5735974.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Re: Camel HTTP component vs. Apache HTTP client
Posted by humayun0156 <hu...@gmail.com>.
i've used http4 component and test my code. but it still took 600-800 ms per
request.
my route looks like :
from("direct:npGet")
.setHeader(Exchange.HTTP_METHOD, constant("GET"))
.setProperty("url", simple("${body}"))
.setHeader(Exchange.HTTP_URI, simple("${body}"))
.to("http4://dummyHost.com?" + npAuthenticationString)
.convertBodyTo(Document.class)
.bean(HelperBean.class, "setUrlETagMapping")
.process(new ObjectCreationProcess(xmlToObjectMapping));
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-HTTP-component-vs-Apache-HTTP-client-tp5735943p5735974.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel HTTP component vs. Apache HTTP client
Posted by Raul Kripalani <ra...@evosent.com>.
Many thanks.
As a first step, can you switch to the Camel HTTP4 component?
With the current setup, you're comparing v3 vs. v4.
Thanks,
Raúl.
On 21 Jul 2013 13:24, "humayun0156" <hu...@gmail.com> wrote:
> Here is my Apache HTTP client version to download a URL. all three files
> are
> here
> Resource.java <http://paste.ubuntu.com/5897131/>
> DefaultHTTPClient.java <http://paste.ubuntu.com/5897147/>
> XMLEntityParser <http://paste.ubuntu.com/5897144/>
>
> Performance issue is very imporntat. we should decide whether we should
> port
> our project to camel or not.
>
> Thanks
>
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Camel-HTTP-component-vs-Apache-HTTP-client-tp5735943p5735960.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Re: Camel HTTP component vs. Apache HTTP client
Posted by humayun0156 <hu...@gmail.com>.
Here is my Apache HTTP client version to download a URL. all three files are
here
Resource.java <http://paste.ubuntu.com/5897131/>
DefaultHTTPClient.java <http://paste.ubuntu.com/5897147/>
XMLEntityParser <http://paste.ubuntu.com/5897144/>
Performance issue is very imporntat. we should decide whether we should port
our project to camel or not.
Thanks
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-HTTP-component-vs-Apache-HTTP-client-tp5735943p5735960.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel HTTP component vs. Apache HTTP client
Posted by Raul Kripalani <ra...@evosent.com>.
Hi,
Can you also post your other version with the Apache HTTP client only?
I'd like to compare them to know where the performance hit is coming from.
Thanks,
Raúl.
On 21 Jul 2013 06:10, "humayun0156" <hu...@gmail.com> wrote:
> from("direct:npGet")
> .setHeader(Exchange.HTTP_METHOD, constant("GET"))
> .setProperty("url", simple("${body}"))
> .recipientList(simple("${body}?" + npAuthenticationString))
> .convertBodyTo(Document.class)
> .bean(HelperBean.class, "setUrlETagMapping")
> .process(new ObjectCreationProcess(xmlToObjectMapping));
>
>
>
> stopWatch.start();
> Exchange ex = producerTemplate.request("direct:npGet", new
> StringBodyProcessor(resourceURI.toString()));
> stopWatch.stop();
>
>
> The StringBodyProcessor is simply to set the url in the exchange body.
> The ObjectCreationgProcess is simply to create a new Object of the
> downloaded xml.
>
> My producer template is in a recursive loop.
>
> and my resourceURI is looks like : http://mysite.com/users/{id}
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Camel-HTTP-component-vs-Apache-HTTP-client-tp5735943p5735954.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Re: Camel HTTP component vs. Apache HTTP client
Posted by humayun0156 <hu...@gmail.com>.
from("direct:npGet")
.setHeader(Exchange.HTTP_METHOD, constant("GET"))
.setProperty("url", simple("${body}"))
.recipientList(simple("${body}?" + npAuthenticationString))
.convertBodyTo(Document.class)
.bean(HelperBean.class, "setUrlETagMapping")
.process(new ObjectCreationProcess(xmlToObjectMapping));
stopWatch.start();
Exchange ex = producerTemplate.request("direct:npGet", new
StringBodyProcessor(resourceURI.toString()));
stopWatch.stop();
The StringBodyProcessor is simply to set the url in the exchange body.
The ObjectCreationgProcess is simply to create a new Object of the
downloaded xml.
My producer template is in a recursive loop.
and my resourceURI is looks like : http://mysite.com/users/{id}
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-HTTP-component-vs-Apache-HTTP-client-tp5735943p5735954.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel HTTP component vs. Apache HTTP client
Posted by Christian Müller <ch...@gmail.com>.
How does your route looks like?
Best,
Christian
Am 20.07.2013 23:15 schrieb "humayun0156" <hu...@gmail.com>:
> Hi,
> currently i'm working on a project which needs to download a lots of XML by
> HTTP request. when i use camel's HTTP component it took to download per
> request 600-800 ms but if i used for same request using Apache HTTP client
> then it took 300-400 ms. Is it expected behavior or i misuse of camel's
> HTTP
> component?
> Any idea?
>
> Thanks in advance.
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Camel-HTTP-component-vs-Apache-HTTP-client-tp5735943.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>