You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Ke Ren <ke...@spaceapegames.com> on 2013/10/14 23:22:33 UTC
HttpClient IO hits the same core
Hi,
We are sending massive http client requests on amazon ec2 instance with 8
cores (m3.xlarge). We notice always one core is hammered the most (80%) but
others are just used around 30% to 40%. The httpclient we are using is
4.2.5. I reproduced this issue with very simple code on ec2 4 core
instance. Basically I created one http client instance new
DefaultHttpClient(mgr, params); mgr is a PoolingClientConnectionManager.
This http client instance is shared by multiple threads. In this test, each
thread worker just gets the same 700k file from s3. The result is the same.
always just one core is hit heavily but very low usage on others. From
jprofiler profiling, it looks the most cpu usage is on IO read from
InputStream. I used EntityUtils.toByteArray to read data from entity. From
netstat, there are 50 connections. The thing confuses me is why it always
hits the same core when http responses are processed from multiple threads.
Any ideas?
Thanks in advance,
Ke
Re: HttpClient IO hits the same core
Posted by S Ahmed <sa...@gmail.com>.
Interesting. Have you tried the same test on a dedicated server?
On Mon, Oct 14, 2013 at 5:22 PM, Ke Ren <ke...@spaceapegames.com> wrote:
> Hi,
>
> We are sending massive http client requests on amazon ec2 instance with 8
> cores (m3.xlarge). We notice always one core is hammered the most (80%) but
> others are just used around 30% to 40%. The httpclient we are using is
> 4.2.5. I reproduced this issue with very simple code on ec2 4 core
> instance. Basically I created one http client instance new
> DefaultHttpClient(mgr, params); mgr is a PoolingClientConnectionManager.
> This http client instance is shared by multiple threads. In this test, each
> thread worker just gets the same 700k file from s3. The result is the same.
> always just one core is hit heavily but very low usage on others. From
> jprofiler profiling, it looks the most cpu usage is on IO read from
> InputStream. I used EntityUtils.toByteArray to read data from entity. From
> netstat, there are 50 connections. The thing confuses me is why it always
> hits the same core when http responses are processed from multiple threads.
> Any ideas?
>
> Thanks in advance,
>
> Ke
>
Re: HttpClient IO hits the same core
Posted by Ke Ren <ke...@spaceapegames.com>.
I tested with different configs. Threads are not the issue. From profiling,
net io was processed cross multi-threads. executorService is used by
ExecutionContext for future in implicit way.
I can confirm it's nothing with http client. It's an issue related to
multi-queue on OS network interface. From linux interrupt, we can see all
network interrupts hit the same core but not distributed. We are closing to
resolve this issue. Thanks everyone's help.
On Tue, Oct 15, 2013 at 2:57 PM, Matt Brown <Ma...@citrix.com> wrote:
> You might want to try changing the maxTotal and defaultMaxPerRoute
> settings on the connection pool used by PoolingClientConnectionManager, as
> the defaults are 20 and 2 respectively, and all of your threads are making
> requests to the same http route.
>
> Also, I don't think it has anything to do with the behavior you are
> seeing, but it looks like the executorService you create is never actually
> used as you start each Worker thread manually?
>
> On Oct 15, 2013, at 6:17 AM, Ke Ren <ke...@spaceapegames.com> wrote:
>
> > yes, that was with http client 4.2.5. I tried to see if there is any
> > different with 4.3. The following is the code with 4.2.5
> >
> > import concurrent.{ExecutionContext, Await, Future, Promise}
> > import scala.concurrent.duration._
> > import java.util.concurrent.Executors
> > import org.apache.http.impl.client.{DefaultHttpClient, HttpClients}
> > import org.apache.http.util.EntityUtils
> > import org.apache.http.conn.scheme.{PlainSocketFactory, Scheme,
> > SchemeRegistry}
> > import org.apache.http.conn.ssl.SSLSocketFactory
> > import org.apache.http.impl.conn.PoolingClientConnectionManager
> > import org.apache.http.params.SyncBasicHttpParams
> > import org.apache.http.client.methods.HttpGet
> >
> > object MainTest {
> > def main(args: Array[String]) {
> > var num = 100
> > var jobs = 1000
> > if (args.size == 2) {
> > num = args(0).toInt
> > jobs = args(1).toInt
> > }
> > val executorService = Executors.newCachedThreadPool()
> > implicit val ec =
> ExecutionContext.fromExecutorService(executorService)
> > val schemeRegistry = new SchemeRegistry();
> > schemeRegistry.register(new Scheme("http", 80,
> > PlainSocketFactory.getSocketFactory()));
> > schemeRegistry.register(new Scheme("https", 443,
> > SSLSocketFactory.getSocketFactory()));
> > val mgr = new PoolingClientConnectionManager(schemeRegistry);
> > val params = new SyncBasicHttpParams();
> > val httpclient = new DefaultHttpClient(mgr, params);
> >
> > var workers = List.empty[Worker]
> > var futures = List.empty[Future[Boolean]]
> > for (i <- 0 to num) {
> > val p = Promise[Boolean]
> > workers = workers.::(new Worker(httpclient, p, jobs))
> > futures = futures.::(p.future)
> > }
> >
> > val allF = futures.tail.foldLeft(futures.head) {
> > case (future, rowFuture) => future.flatMap {
> > status =>
> > rowFuture.map(finished => status && finished)
> > }
> > }
> >
> > workers foreach {
> > worker =>
> > worker.start()
> > }
> >
> > Await.result(allF, 1 hour)
> > println("finished")
> > }
> > }
> >
> > class Worker(httpclient: DefaultHttpClient, p: Promise[Boolean], jobs:
> Int)
> > extends Thread {
> > override def run {
> > for (i <- 0 to jobs) {
> > try {
> > var httpget = new HttpGet("xxxx")
> > val response = httpclient.execute(httpget);
> > try {
> > val entity = response.getEntity();
> > if (entity != null) {
> > EntityUtils.toByteArray(entity)
> > }
> > } finally {
> > response.close();
> > httpclient.close()
> > }
> > } catch {
> > case e: Throwable =>
> > e.printStackTrace()
> > }
> > }
> > p.success(true)
> > }
> > }
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
Re: HttpClient IO hits the same core
Posted by Matt Brown <Ma...@citrix.com>.
You might want to try changing the maxTotal and defaultMaxPerRoute settings on the connection pool used by PoolingClientConnectionManager, as the defaults are 20 and 2 respectively, and all of your threads are making requests to the same http route.
Also, I don't think it has anything to do with the behavior you are seeing, but it looks like the executorService you create is never actually used as you start each Worker thread manually?
On Oct 15, 2013, at 6:17 AM, Ke Ren <ke...@spaceapegames.com> wrote:
> yes, that was with http client 4.2.5. I tried to see if there is any
> different with 4.3. The following is the code with 4.2.5
>
> import concurrent.{ExecutionContext, Await, Future, Promise}
> import scala.concurrent.duration._
> import java.util.concurrent.Executors
> import org.apache.http.impl.client.{DefaultHttpClient, HttpClients}
> import org.apache.http.util.EntityUtils
> import org.apache.http.conn.scheme.{PlainSocketFactory, Scheme,
> SchemeRegistry}
> import org.apache.http.conn.ssl.SSLSocketFactory
> import org.apache.http.impl.conn.PoolingClientConnectionManager
> import org.apache.http.params.SyncBasicHttpParams
> import org.apache.http.client.methods.HttpGet
>
> object MainTest {
> def main(args: Array[String]) {
> var num = 100
> var jobs = 1000
> if (args.size == 2) {
> num = args(0).toInt
> jobs = args(1).toInt
> }
> val executorService = Executors.newCachedThreadPool()
> implicit val ec = ExecutionContext.fromExecutorService(executorService)
> val schemeRegistry = new SchemeRegistry();
> schemeRegistry.register(new Scheme("http", 80,
> PlainSocketFactory.getSocketFactory()));
> schemeRegistry.register(new Scheme("https", 443,
> SSLSocketFactory.getSocketFactory()));
> val mgr = new PoolingClientConnectionManager(schemeRegistry);
> val params = new SyncBasicHttpParams();
> val httpclient = new DefaultHttpClient(mgr, params);
>
> var workers = List.empty[Worker]
> var futures = List.empty[Future[Boolean]]
> for (i <- 0 to num) {
> val p = Promise[Boolean]
> workers = workers.::(new Worker(httpclient, p, jobs))
> futures = futures.::(p.future)
> }
>
> val allF = futures.tail.foldLeft(futures.head) {
> case (future, rowFuture) => future.flatMap {
> status =>
> rowFuture.map(finished => status && finished)
> }
> }
>
> workers foreach {
> worker =>
> worker.start()
> }
>
> Await.result(allF, 1 hour)
> println("finished")
> }
> }
>
> class Worker(httpclient: DefaultHttpClient, p: Promise[Boolean], jobs: Int)
> extends Thread {
> override def run {
> for (i <- 0 to jobs) {
> try {
> var httpget = new HttpGet("xxxx")
> val response = httpclient.execute(httpget);
> try {
> val entity = response.getEntity();
> if (entity != null) {
> EntityUtils.toByteArray(entity)
> }
> } finally {
> response.close();
> httpclient.close()
> }
> } catch {
> case e: Throwable =>
> e.printStackTrace()
> }
> }
> p.success(true)
> }
> }
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: HttpClient IO hits the same core
Posted by Ke Ren <ke...@spaceapegames.com>.
[?], thanks. I did more experiments. It looks it's not http client lib but
more like network IO task scheduling on OS although we didn't do anything
special. Thanks anyway.
On Tue, Oct 15, 2013 at 1:58 PM, Oleg Kalnichevski <ol...@apache.org> wrote:
> On Tue, 2013-10-15 at 11:17 +0100, Ke Ren wrote:
> > yes, that was with http client 4.2.5. I tried to see if there is any
> > different with 4.3. The following is the code with 4.2.5
> >
> > import concurrent.{ExecutionContext, Await, Future, Promise}
> > import scala.concurrent.duration._
> > import java.util.concurrent.Executors
> > import org.apache.http.impl.client.{DefaultHttpClient, HttpClients}
> > import org.apache.http.util.EntityUtils
> > import org.apache.http.conn.scheme.{PlainSocketFactory, Scheme,
> > SchemeRegistry}
> > import org.apache.http.conn.ssl.SSLSocketFactory
> > import org.apache.http.impl.conn.PoolingClientConnectionManager
> > import org.apache.http.params.SyncBasicHttpParams
> > import org.apache.http.client.methods.HttpGet
> >
> > object MainTest {
> > def main(args: Array[String]) {
> > var num = 100
> > var jobs = 1000
> > if (args.size == 2) {
> > num = args(0).toInt
> > jobs = args(1).toInt
> > }
> > val executorService = Executors.newCachedThreadPool()
> > implicit val ec =
> ExecutionContext.fromExecutorService(executorService)
> > val schemeRegistry = new SchemeRegistry();
> > schemeRegistry.register(new Scheme("http", 80,
> > PlainSocketFactory.getSocketFactory()));
> > schemeRegistry.register(new Scheme("https", 443,
> > SSLSocketFactory.getSocketFactory()));
> > val mgr = new PoolingClientConnectionManager(schemeRegistry);
> > val params = new SyncBasicHttpParams();
> > val httpclient = new DefaultHttpClient(mgr, params);
> >
> > var workers = List.empty[Worker]
> > var futures = List.empty[Future[Boolean]]
> > for (i <- 0 to num) {
> > val p = Promise[Boolean]
> > workers = workers.::(new Worker(httpclient, p, jobs))
> > futures = futures.::(p.future)
> > }
> >
> > val allF = futures.tail.foldLeft(futures.head) {
> > case (future, rowFuture) => future.flatMap {
> > status =>
> > rowFuture.map(finished => status && finished)
> > }
> > }
> >
> > workers foreach {
> > worker =>
> > worker.start()
> > }
> >
> > Await.result(allF, 1 hour)
> > println("finished")
> > }
> > }
> >
> > class Worker(httpclient: DefaultHttpClient, p: Promise[Boolean], jobs:
> Int)
> > extends Thread {
> > override def run {
> > for (i <- 0 to jobs) {
> > try {
> > var httpget = new HttpGet("xxxx")
> > val response = httpclient.execute(httpget);
> > try {
> > val entity = response.getEntity();
> > if (entity != null) {
> > EntityUtils.toByteArray(entity)
> > }
> > } finally {
> > response.close();
> > httpclient.close()
> > }
> > } catch {
> > case e: Throwable =>
> > e.printStackTrace()
> > }
> > }
> > p.success(true)
> > }
> > }
>
> I see nothing wrong with your code. I am not familiar with Scala though.
> HttpClient related aspects look sane to me.
>
> Oleg
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
Re: HttpClient IO hits the same core
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2013-10-15 at 11:17 +0100, Ke Ren wrote:
> yes, that was with http client 4.2.5. I tried to see if there is any
> different with 4.3. The following is the code with 4.2.5
>
> import concurrent.{ExecutionContext, Await, Future, Promise}
> import scala.concurrent.duration._
> import java.util.concurrent.Executors
> import org.apache.http.impl.client.{DefaultHttpClient, HttpClients}
> import org.apache.http.util.EntityUtils
> import org.apache.http.conn.scheme.{PlainSocketFactory, Scheme,
> SchemeRegistry}
> import org.apache.http.conn.ssl.SSLSocketFactory
> import org.apache.http.impl.conn.PoolingClientConnectionManager
> import org.apache.http.params.SyncBasicHttpParams
> import org.apache.http.client.methods.HttpGet
>
> object MainTest {
> def main(args: Array[String]) {
> var num = 100
> var jobs = 1000
> if (args.size == 2) {
> num = args(0).toInt
> jobs = args(1).toInt
> }
> val executorService = Executors.newCachedThreadPool()
> implicit val ec = ExecutionContext.fromExecutorService(executorService)
> val schemeRegistry = new SchemeRegistry();
> schemeRegistry.register(new Scheme("http", 80,
> PlainSocketFactory.getSocketFactory()));
> schemeRegistry.register(new Scheme("https", 443,
> SSLSocketFactory.getSocketFactory()));
> val mgr = new PoolingClientConnectionManager(schemeRegistry);
> val params = new SyncBasicHttpParams();
> val httpclient = new DefaultHttpClient(mgr, params);
>
> var workers = List.empty[Worker]
> var futures = List.empty[Future[Boolean]]
> for (i <- 0 to num) {
> val p = Promise[Boolean]
> workers = workers.::(new Worker(httpclient, p, jobs))
> futures = futures.::(p.future)
> }
>
> val allF = futures.tail.foldLeft(futures.head) {
> case (future, rowFuture) => future.flatMap {
> status =>
> rowFuture.map(finished => status && finished)
> }
> }
>
> workers foreach {
> worker =>
> worker.start()
> }
>
> Await.result(allF, 1 hour)
> println("finished")
> }
> }
>
> class Worker(httpclient: DefaultHttpClient, p: Promise[Boolean], jobs: Int)
> extends Thread {
> override def run {
> for (i <- 0 to jobs) {
> try {
> var httpget = new HttpGet("xxxx")
> val response = httpclient.execute(httpget);
> try {
> val entity = response.getEntity();
> if (entity != null) {
> EntityUtils.toByteArray(entity)
> }
> } finally {
> response.close();
> httpclient.close()
> }
> } catch {
> case e: Throwable =>
> e.printStackTrace()
> }
> }
> p.success(true)
> }
> }
I see nothing wrong with your code. I am not familiar with Scala though.
HttpClient related aspects look sane to me.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: HttpClient IO hits the same core
Posted by Ke Ren <ke...@spaceapegames.com>.
yes, that was with http client 4.2.5. I tried to see if there is any
different with 4.3. The following is the code with 4.2.5
import concurrent.{ExecutionContext, Await, Future, Promise}
import scala.concurrent.duration._
import java.util.concurrent.Executors
import org.apache.http.impl.client.{DefaultHttpClient, HttpClients}
import org.apache.http.util.EntityUtils
import org.apache.http.conn.scheme.{PlainSocketFactory, Scheme,
SchemeRegistry}
import org.apache.http.conn.ssl.SSLSocketFactory
import org.apache.http.impl.conn.PoolingClientConnectionManager
import org.apache.http.params.SyncBasicHttpParams
import org.apache.http.client.methods.HttpGet
object MainTest {
def main(args: Array[String]) {
var num = 100
var jobs = 1000
if (args.size == 2) {
num = args(0).toInt
jobs = args(1).toInt
}
val executorService = Executors.newCachedThreadPool()
implicit val ec = ExecutionContext.fromExecutorService(executorService)
val schemeRegistry = new SchemeRegistry();
schemeRegistry.register(new Scheme("http", 80,
PlainSocketFactory.getSocketFactory()));
schemeRegistry.register(new Scheme("https", 443,
SSLSocketFactory.getSocketFactory()));
val mgr = new PoolingClientConnectionManager(schemeRegistry);
val params = new SyncBasicHttpParams();
val httpclient = new DefaultHttpClient(mgr, params);
var workers = List.empty[Worker]
var futures = List.empty[Future[Boolean]]
for (i <- 0 to num) {
val p = Promise[Boolean]
workers = workers.::(new Worker(httpclient, p, jobs))
futures = futures.::(p.future)
}
val allF = futures.tail.foldLeft(futures.head) {
case (future, rowFuture) => future.flatMap {
status =>
rowFuture.map(finished => status && finished)
}
}
workers foreach {
worker =>
worker.start()
}
Await.result(allF, 1 hour)
println("finished")
}
}
class Worker(httpclient: DefaultHttpClient, p: Promise[Boolean], jobs: Int)
extends Thread {
override def run {
for (i <- 0 to jobs) {
try {
var httpget = new HttpGet("xxxx")
val response = httpclient.execute(httpget);
try {
val entity = response.getEntity();
if (entity != null) {
EntityUtils.toByteArray(entity)
}
} finally {
response.close();
httpclient.close()
}
} catch {
case e: Throwable =>
e.printStackTrace()
}
}
p.success(true)
}
}
Re: HttpClient IO hits the same core
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2013-10-15 at 10:41 +0100, Ke Ren wrote:
> @Ahmed, don't have a dedicated server to test.
> @Oleg, yes, that's my confusion. I changed to http client 4.3 and still get
> the same result. Here is my test code. Can you see if I used anything
> wrongly? It's in scala but very similar to Java
>
> import concurrent.{ExecutionContext, Await, Future, Promise}
> import scala.concurrent.duration._
> import java.util.concurrent.Executors
> import org.apache.http.impl.client.HttpClients
> import org.apache.http.client.methods.HttpGet
> import org.apache.http.util.EntityUtils
>
> object MainTest {
> def main(args: Array[String]) {
> var num = 100
> var jobs = 1000
> if (args.size == 2) {
> num = args(0).toInt
> jobs = args(1).toInt
> }
> val executorService = Executors.newCachedThreadPool()
> implicit val ec = ExecutionContext.fromExecutorService(executorService)
>
> var workers = List.empty[Worker]
> var futures = List.empty[Future[Boolean]]
> for (i <- 0 to num) {
> val p = Promise[Boolean]
> workers = workers.::(new Worker(p, jobs))
> futures = futures.::(p.future)
> }
>
> val allF = futures.tail.foldLeft(futures.head) {
> case (future, rowFuture) => future.flatMap {
> status =>
> rowFuture.map(finished => status && finished)
> }
> }
>
> workers foreach {
> worker =>
> worker.start()
> }
>
> Await.result(allF, 1 hour)
> println("finished")
> }
> }
>
> class Worker(p: Promise[Boolean], jobs: Int) extends Thread {
> override def run {
> for (i <- 0 to jobs) {
> try {
> val httpclient = HttpClients.createDefault();
> val httpget = new HttpGet("
> http://s3.amazonaws.com/panda.apelabs.net/gamecontent-unittest/Current-Config.json
> ");
> val response = httpclient.execute(httpget);
> try {
> val entity = response.getEntity();
> if (entity != null) {
> EntityUtils.toByteArray(entity)
> }
> } finally {
> response.close();
> httpclient.close()
> }
> } catch {
> case e: Throwable =>
> e.printStackTrace()
> }
> }
> p.success(true)
> }
> }
>
>
You mentioned in your previous post you had been using one instance of
HttpClient shared by multiple threads. This is clearly not the case with
the code snippet above.
Oleg
> On Tue, Oct 15, 2013 at 10:07 AM, Oleg Kalnichevski <ol...@apache.org>wrote:
>
> > On Mon, 2013-10-14 at 22:22 +0100, Ke Ren wrote:
> > > Hi,
> > >
> > > We are sending massive http client requests on amazon ec2 instance with 8
> > > cores (m3.xlarge). We notice always one core is hammered the most (80%)
> > but
> > > others are just used around 30% to 40%. The httpclient we are using is
> > > 4.2.5. I reproduced this issue with very simple code on ec2 4 core
> > > instance. Basically I created one http client instance new
> > > DefaultHttpClient(mgr, params); mgr is a PoolingClientConnectionManager.
> > > This http client instance is shared by multiple threads. In this test,
> > each
> > > thread worker just gets the same 700k file from s3. The result is the
> > same.
> > > always just one core is hit heavily but very low usage on others. From
> > > jprofiler profiling, it looks the most cpu usage is on IO read from
> > > InputStream. I used EntityUtils.toByteArray to read data from entity.
> > From
> > > netstat, there are 50 connections. The thing confuses me is why it always
> > > hits the same core when http responses are processed from multiple
> > threads.
> > > Any ideas?
> > >
> > > Thanks in advance,
> > >
> > > Ke
> >
> > The only rational theory I can think of is that the ReentrantLock
> > guarding the connection pool favors certain threads due to 'unfair'
> > behavior used by default. Still, I would presume that threads should not
> > be bound to one core only and the load should get distributed evenly.
> >
> > Oleg
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: HttpClient IO hits the same core
Posted by Ke Ren <ke...@spaceapegames.com>.
@Ahmed, don't have a dedicated server to test.
@Oleg, yes, that's my confusion. I changed to http client 4.3 and still get
the same result. Here is my test code. Can you see if I used anything
wrongly? It's in scala but very similar to Java
import concurrent.{ExecutionContext, Await, Future, Promise}
import scala.concurrent.duration._
import java.util.concurrent.Executors
import org.apache.http.impl.client.HttpClients
import org.apache.http.client.methods.HttpGet
import org.apache.http.util.EntityUtils
object MainTest {
def main(args: Array[String]) {
var num = 100
var jobs = 1000
if (args.size == 2) {
num = args(0).toInt
jobs = args(1).toInt
}
val executorService = Executors.newCachedThreadPool()
implicit val ec = ExecutionContext.fromExecutorService(executorService)
var workers = List.empty[Worker]
var futures = List.empty[Future[Boolean]]
for (i <- 0 to num) {
val p = Promise[Boolean]
workers = workers.::(new Worker(p, jobs))
futures = futures.::(p.future)
}
val allF = futures.tail.foldLeft(futures.head) {
case (future, rowFuture) => future.flatMap {
status =>
rowFuture.map(finished => status && finished)
}
}
workers foreach {
worker =>
worker.start()
}
Await.result(allF, 1 hour)
println("finished")
}
}
class Worker(p: Promise[Boolean], jobs: Int) extends Thread {
override def run {
for (i <- 0 to jobs) {
try {
val httpclient = HttpClients.createDefault();
val httpget = new HttpGet("
http://s3.amazonaws.com/panda.apelabs.net/gamecontent-unittest/Current-Config.json
");
val response = httpclient.execute(httpget);
try {
val entity = response.getEntity();
if (entity != null) {
EntityUtils.toByteArray(entity)
}
} finally {
response.close();
httpclient.close()
}
} catch {
case e: Throwable =>
e.printStackTrace()
}
}
p.success(true)
}
}
On Tue, Oct 15, 2013 at 10:07 AM, Oleg Kalnichevski <ol...@apache.org>wrote:
> On Mon, 2013-10-14 at 22:22 +0100, Ke Ren wrote:
> > Hi,
> >
> > We are sending massive http client requests on amazon ec2 instance with 8
> > cores (m3.xlarge). We notice always one core is hammered the most (80%)
> but
> > others are just used around 30% to 40%. The httpclient we are using is
> > 4.2.5. I reproduced this issue with very simple code on ec2 4 core
> > instance. Basically I created one http client instance new
> > DefaultHttpClient(mgr, params); mgr is a PoolingClientConnectionManager.
> > This http client instance is shared by multiple threads. In this test,
> each
> > thread worker just gets the same 700k file from s3. The result is the
> same.
> > always just one core is hit heavily but very low usage on others. From
> > jprofiler profiling, it looks the most cpu usage is on IO read from
> > InputStream. I used EntityUtils.toByteArray to read data from entity.
> From
> > netstat, there are 50 connections. The thing confuses me is why it always
> > hits the same core when http responses are processed from multiple
> threads.
> > Any ideas?
> >
> > Thanks in advance,
> >
> > Ke
>
> The only rational theory I can think of is that the ReentrantLock
> guarding the connection pool favors certain threads due to 'unfair'
> behavior used by default. Still, I would presume that threads should not
> be bound to one core only and the load should get distributed evenly.
>
> Oleg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
Re: HttpClient IO hits the same core
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Mon, 2013-10-14 at 22:22 +0100, Ke Ren wrote:
> Hi,
>
> We are sending massive http client requests on amazon ec2 instance with 8
> cores (m3.xlarge). We notice always one core is hammered the most (80%) but
> others are just used around 30% to 40%. The httpclient we are using is
> 4.2.5. I reproduced this issue with very simple code on ec2 4 core
> instance. Basically I created one http client instance new
> DefaultHttpClient(mgr, params); mgr is a PoolingClientConnectionManager.
> This http client instance is shared by multiple threads. In this test, each
> thread worker just gets the same 700k file from s3. The result is the same.
> always just one core is hit heavily but very low usage on others. From
> jprofiler profiling, it looks the most cpu usage is on IO read from
> InputStream. I used EntityUtils.toByteArray to read data from entity. From
> netstat, there are 50 connections. The thing confuses me is why it always
> hits the same core when http responses are processed from multiple threads.
> Any ideas?
>
> Thanks in advance,
>
> Ke
The only rational theory I can think of is that the ReentrantLock
guarding the connection pool favors certain threads due to 'unfair'
behavior used by default. Still, I would presume that threads should not
be bound to one core only and the load should get distributed evenly.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org