You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by Ben Wong <bw...@adaptiveinternational.com> on 2004/02/16 06:40:54 UTC

GetMethod Performance

Hi,

I have noticed significant performance difference between using
HttpClient and Socket.

I tried to use GetMethod to download a 2MB file from a Webserver sitting
in the LAN. When I do it with HttpClient, it takes around 13-15 seconds
while it will only take less than half a second with Socket.

I was running the code below on a Sun Blade 100 with Solaris 8
installed. J2SDK1.4.2_03 and HttpClient 2.0 final were used.

Any help would be appreciated.

Thanks,
Ben

HttpClient code:
----------------
HttpClient client = new HttpClient();
GetMethod get = new
GetMethod("http://192.168.0.1/commons-httpclient-2.0-final.zip");
	
int statusCode = client.executeMethod(get);
System.out.println("Status Code: " + statusCode);
int size = 0;
	
InputStream in = get.getResponseBodyAsStream();
byte [] data = new byte[10000];
int read = 0;
	
while ((read = in.read(data)) > 0) {
	size += read;
}

in.close();
get.releaseConnection();

Socket Code:
------------
Socket soc = new Socket("192.168.0.11", 80);
InputStream in = soc.getInputStream();
OutputStream out = soc.getOutputStream();

String command = "GET
http://192.168.0.1/commons-httpclient-2.0-final.zip
HTTP/1.0\nUser-Agent: Jakarta Commons-HttpClient/2.0final\nHost:
10.0.3.11\n\n";
byte [] send = command.getBytes();

out.write(send);
byte b[] = new byte[4096];
int size = 0;
int count = 0;
while( (size = in.read(b)) >= 0) {
	count += size;
}
in.close();
out.close();

soc.close();





---------------------------------------------------------------------
To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org


Re: GetMethod Performance

Posted by Michael Becke <be...@u.washington.edu>.
Hi Ben,

I believe your HttpClient numbers are being inflated by one-time 
initialization costs.  I was able to get approximately the same numbers 
with HttpClient and Socket.  Please see my examples attached below:

Mike


HttpClient code:

         HttpClient client = new HttpClient();
         GetMethod get = new GetMethod("http://www.google.com");

         client.executeMethod(get);
         get.getResponseBodyAsString();
         get.releaseConnection();

         client = new HttpClient();
         get = new GetMethod("SOME_URL");

         long start = System.currentTimeMillis();

         int statusCode = client.executeMethod(get);
         int size = 0;

         InputStream in = get.getResponseBodyAsStream();
         byte [] data = new byte[4096];
         int read = 0;

         while ((read = in.read(data)) > 0) {
             size += read;
         }

         in.close();
         get.releaseConnection();

         long end = System.currentTimeMillis();

         System.out.println(end-start);


Socket code:

         HttpClient client = new HttpClient();
         GetMethod get = new GetMethod("http://www.google.com");

         client.executeMethod(get);
         get.getResponseBodyAsString();
         get.releaseConnection();

         long start = System.currentTimeMillis();

         Socket soc = new Socket("HOST", 80);
         InputStream in = soc.getInputStream();
         OutputStream out = soc.getOutputStream();

         String command = "GET /SOME_FILE\n\n";
         byte [] send = command.getBytes();

         out.write(send);
         byte b[] = new byte[4096];
         int size = 0;
         int count = 0;
         while( (size = in.read(b)) >= 0) {
             count += size;
         }
         in.close();
         out.close();

         soc.close();

         long end = System.currentTimeMillis();

         System.out.println(end-start);


On Feb 16, 2004, at 12:40 AM, Ben Wong wrote:

> Hi,
>
> I have noticed significant performance difference between using
> HttpClient and Socket.
>
> I tried to use GetMethod to download a 2MB file from a Webserver 
> sitting
> in the LAN. When I do it with HttpClient, it takes around 13-15 seconds
> while it will only take less than half a second with Socket.
>
> I was running the code below on a Sun Blade 100 with Solaris 8
> installed. J2SDK1.4.2_03 and HttpClient 2.0 final were used.
>
> Any help would be appreciated.
>
> Thanks,
> Ben
>
> HttpClient code:
> ----------------
> HttpClient client = new HttpClient();
> GetMethod get = new
> GetMethod("http://192.168.0.1/commons-httpclient-2.0-final.zip");
> 	
> int statusCode = client.executeMethod(get);
> System.out.println("Status Code: " + statusCode);
> int size = 0;
> 	
> InputStream in = get.getResponseBodyAsStream();
> byte [] data = new byte[10000];
> int read = 0;
> 	
> while ((read = in.read(data)) > 0) {
> 	size += read;
> }
>
> in.close();
> get.releaseConnection();
>
> Socket Code:
> ------------
> Socket soc = new Socket("192.168.0.11", 80);
> InputStream in = soc.getInputStream();
> OutputStream out = soc.getOutputStream();
>
> String command = "GET
> http://192.168.0.1/commons-httpclient-2.0-final.zip
> HTTP/1.0\nUser-Agent: Jakarta Commons-HttpClient/2.0final\nHost:
> 10.0.3.11\n\n";
> byte [] send = command.getBytes();
>
> out.write(send);
> byte b[] = new byte[4096];
> int size = 0;
> int count = 0;
> while( (size = in.read(b)) >= 0) {
> 	count += size;
> }
> in.close();
> out.close();
>
> soc.close();
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: 
> commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: 
> commons-httpclient-dev-help@jakarta.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org


RE: GetMethod Performance

Posted by Roland Weber <RO...@de.ibm.com>.
Hello Ben,

I remember reading about problems with particular JVM versions,
and with "stale connection checks" in this mailing list. But I didn't
track these issues, so someone else will have to help out. Or you
check the mailing list archive and the bugzilla database for known
issues. There's a good chance you'll find something about your
problem.

cheers,
  Roland






"Ben Wong" <bw...@adaptiveinternational.com>
16.02.2004 07:30
Please respond to "Commons HttpClient Project"
 
        To:     "'Commons HttpClient Project'" 
<co...@jakarta.apache.org>
        cc: 
        Subject:        RE: GetMethod Performance


Thanks Michael and Roland for the quick responses.

Roland, in fact, that was a typo and it was not caused by the
initialization of HttpClient AFAIK.

I had very close result when I ran the whole thing on my laptop, which
runs WinXP. Both could finish in less than a second.

However, when I put exactly the same code to Sun Solaris 8 on a Sun
Blade 100, I got a totally different story. Test finishes quick when I
used Socket, but it takes significantly longer when httpClient was used.

Did I miss out something that I should do when I execute Java class from
console on Sun box? Or do they have a different JVM configuration?

-----Original Message-----
From: Roland Weber [mailto:ROLWEBER@de.ibm.com] 
Sent: Monday, 16 February 2004 5:25 PM
To: Commons HttpClient Project
Subject: Re: GetMethod Performance

Hi Ben,

that is indeed a big difference. Two questions:

1. The HttpClient example uses IP address 192.168.0.1,
   the Socket example connects to 192.168.0.11
   Is that a typo, or do you indeed access a different host?
   If that different host is a caching proxy, that would be an
   explanation.

2. Did you check how much of the delay is due to the
   initialization of HttpClient? Please set a checkpoint
   after "new HttpClient()" and before "new GetMethod".
   Since the client only has to be created once for your
   application, that would be a fairer comparison.

I trust you ran your examples immediately after one
another while there was not much load on the network.

cheers,
  Roland






"Ben Wong" <bw...@adaptiveinternational.com>
16.02.2004 06:40
Please respond to "Commons HttpClient Project"
 
        To:     <co...@jakarta.apache.org>
        cc: 
        Subject:        GetMethod Performance


Hi,

I have noticed significant performance difference between using
HttpClient and Socket.

I tried to use GetMethod to download a 2MB file from a Webserver sitting
in the LAN. When I do it with HttpClient, it takes around 13-15 seconds
while it will only take less than half a second with Socket.

I was running the code below on a Sun Blade 100 with Solaris 8
installed. J2SDK1.4.2_03 and HttpClient 2.0 final were used.

Any help would be appreciated.

Thanks,
Ben

HttpClient code:
----------------
HttpClient client = new HttpClient();
GetMethod get = new
GetMethod("http://192.168.0.1/commons-httpclient-2.0-final.zip");
 
int statusCode = client.executeMethod(get);
System.out.println("Status Code: " + statusCode);
int size = 0;
 
InputStream in = get.getResponseBodyAsStream();
byte [] data = new byte[10000];
int read = 0;
 
while ((read = in.read(data)) > 0) {
                 size += read;
}

in.close();
get.releaseConnection();

Socket Code:
------------
Socket soc = new Socket("192.168.0.11", 80);
InputStream in = soc.getInputStream();
OutputStream out = soc.getOutputStream();

String command = "GET
http://192.168.0.1/commons-httpclient-2.0-final.zip
HTTP/1.0\nUser-Agent: Jakarta Commons-HttpClient/2.0final\nHost:
10.0.3.11\n\n";
byte [] send = command.getBytes();

out.write(send);
byte b[] = new byte[4096];
int size = 0;
int count = 0;
while( (size = in.read(b)) >= 0) {
                 count += size;
}
in.close();
out.close();

soc.close();





---------------------------------------------------------------------
To unsubscribe, e-mail: 
commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: 
commons-httpclient-dev-help@jakarta.apache.org





---------------------------------------------------------------------
To unsubscribe, e-mail: 
commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: 
commons-httpclient-dev-help@jakarta.apache.org



RE: GetMethod Performance

Posted by Ben Wong <bw...@adaptiveinternational.com>.
Thanks Michael and Roland for the quick responses.

Roland, in fact, that was a typo and it was not caused by the
initialization of HttpClient AFAIK.

I had very close result when I ran the whole thing on my laptop, which
runs WinXP. Both could finish in less than a second.

However, when I put exactly the same code to Sun Solaris 8 on a Sun
Blade 100, I got a totally different story. Test finishes quick when I
used Socket, but it takes significantly longer when httpClient was used.

Did I miss out something that I should do when I execute Java class from
console on Sun box? Or do they have a different JVM configuration?

-----Original Message-----
From: Roland Weber [mailto:ROLWEBER@de.ibm.com] 
Sent: Monday, 16 February 2004 5:25 PM
To: Commons HttpClient Project
Subject: Re: GetMethod Performance

Hi Ben,

that is indeed a big difference. Two questions:

1. The HttpClient example uses IP address 192.168.0.1,
   the Socket example connects to 192.168.0.11
   Is that a typo, or do you indeed access a different host?
   If that different host is a caching proxy, that would be an
   explanation.

2. Did you check how much of the delay is due to the
   initialization of HttpClient? Please set a checkpoint
   after "new HttpClient()" and before "new GetMethod".
   Since the client only has to be created once for your
   application, that would be a fairer comparison.

I trust you ran your examples immediately after one
another while there was not much load on the network.

cheers,
  Roland






"Ben Wong" <bw...@adaptiveinternational.com>
16.02.2004 06:40
Please respond to "Commons HttpClient Project"
 
        To:     <co...@jakarta.apache.org>
        cc: 
        Subject:        GetMethod Performance


Hi,

I have noticed significant performance difference between using
HttpClient and Socket.

I tried to use GetMethod to download a 2MB file from a Webserver sitting
in the LAN. When I do it with HttpClient, it takes around 13-15 seconds
while it will only take less than half a second with Socket.

I was running the code below on a Sun Blade 100 with Solaris 8
installed. J2SDK1.4.2_03 and HttpClient 2.0 final were used.

Any help would be appreciated.

Thanks,
Ben

HttpClient code:
----------------
HttpClient client = new HttpClient();
GetMethod get = new
GetMethod("http://192.168.0.1/commons-httpclient-2.0-final.zip");
 
int statusCode = client.executeMethod(get);
System.out.println("Status Code: " + statusCode);
int size = 0;
 
InputStream in = get.getResponseBodyAsStream();
byte [] data = new byte[10000];
int read = 0;
 
while ((read = in.read(data)) > 0) {
                 size += read;
}

in.close();
get.releaseConnection();

Socket Code:
------------
Socket soc = new Socket("192.168.0.11", 80);
InputStream in = soc.getInputStream();
OutputStream out = soc.getOutputStream();

String command = "GET
http://192.168.0.1/commons-httpclient-2.0-final.zip
HTTP/1.0\nUser-Agent: Jakarta Commons-HttpClient/2.0final\nHost:
10.0.3.11\n\n";
byte [] send = command.getBytes();

out.write(send);
byte b[] = new byte[4096];
int size = 0;
int count = 0;
while( (size = in.read(b)) >= 0) {
                 count += size;
}
in.close();
out.close();

soc.close();





---------------------------------------------------------------------
To unsubscribe, e-mail: 
commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: 
commons-httpclient-dev-help@jakarta.apache.org





---------------------------------------------------------------------
To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org


Re: GetMethod Performance

Posted by Roland Weber <RO...@de.ibm.com>.
Hi Ben,

that is indeed a big difference. Two questions:

1. The HttpClient example uses IP address 192.168.0.1,
   the Socket example connects to 192.168.0.11
   Is that a typo, or do you indeed access a different host?
   If that different host is a caching proxy, that would be an
   explanation.

2. Did you check how much of the delay is due to the
   initialization of HttpClient? Please set a checkpoint
   after "new HttpClient()" and before "new GetMethod".
   Since the client only has to be created once for your
   application, that would be a fairer comparison.

I trust you ran your examples immediately after one
another while there was not much load on the network.

cheers,
  Roland






"Ben Wong" <bw...@adaptiveinternational.com>
16.02.2004 06:40
Please respond to "Commons HttpClient Project"
 
        To:     <co...@jakarta.apache.org>
        cc: 
        Subject:        GetMethod Performance


Hi,

I have noticed significant performance difference between using
HttpClient and Socket.

I tried to use GetMethod to download a 2MB file from a Webserver sitting
in the LAN. When I do it with HttpClient, it takes around 13-15 seconds
while it will only take less than half a second with Socket.

I was running the code below on a Sun Blade 100 with Solaris 8
installed. J2SDK1.4.2_03 and HttpClient 2.0 final were used.

Any help would be appreciated.

Thanks,
Ben

HttpClient code:
----------------
HttpClient client = new HttpClient();
GetMethod get = new
GetMethod("http://192.168.0.1/commons-httpclient-2.0-final.zip");
 
int statusCode = client.executeMethod(get);
System.out.println("Status Code: " + statusCode);
int size = 0;
 
InputStream in = get.getResponseBodyAsStream();
byte [] data = new byte[10000];
int read = 0;
 
while ((read = in.read(data)) > 0) {
                 size += read;
}

in.close();
get.releaseConnection();

Socket Code:
------------
Socket soc = new Socket("192.168.0.11", 80);
InputStream in = soc.getInputStream();
OutputStream out = soc.getOutputStream();

String command = "GET
http://192.168.0.1/commons-httpclient-2.0-final.zip
HTTP/1.0\nUser-Agent: Jakarta Commons-HttpClient/2.0final\nHost:
10.0.3.11\n\n";
byte [] send = command.getBytes();

out.write(send);
byte b[] = new byte[4096];
int size = 0;
int count = 0;
while( (size = in.read(b)) >= 0) {
                 count += size;
}
in.close();
out.close();

soc.close();





---------------------------------------------------------------------
To unsubscribe, e-mail: 
commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: 
commons-httpclient-dev-help@jakarta.apache.org