You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Bert Verhees <be...@rosa.nl> on 2012/01/16 17:15:14 UTC
Unable to sendViaPost to url...... .... java.net.SocketTimeoutException:
Read timed out
Hi, I hope you can help me analyzing this problem:
I am using Tomcat6 6.0.32-5ubuntu1.1 together with Axis2 1.6.1
I am not sure if it is an Axis2 problem or a Tomcat-problem.
I have an client-application for test-purposes which runs inside the
Eclipse-IDE.
The problem only occurs when running Axis2 inside Tomcat6, when running
in Axis2 as stand-alone server, it runs better.
What I do is posting arrays of strings (about 50 elements, about 5Kb
max) to an webservice-call.
After some validation, the webservice posts the data to a postgresql
database.
I have some console-output to see how long it takes.
storing..... Person:26, ID:27
Creating EHR for:213d7c09-949c-4757-a91d-da5510215472
GUIDs saved to table:213d7c09-949c-4757-a91d-da5510215472,
73b60dd4-d745-4267-9c6f-fc8b067bfeb9
stored..... Person:26, UID:213d7c09-949c-4757-a91d-da5510215472
PT0.647S
As you can see, it starts with less then a second, sometime 0.3 seconds,
sometimes 1.1 second, depending on the number of elements in the array
to store.
After 1000 persons stored, it takes 4 seconds
storing..... Person:998, ID:989
Creating EHR for:49c96d87-38db-491d-a824-78800615e80a
GUIDs saved to table:49c96d87-38db-491d-a824-78800615e80a,
3554514f-ccb6-40dd-92f2-d9140760870e
stored..... Person:998, UID:49c96d87-38db-491d-a824-78800615e80a
PT4.147S
When running in Axis2 standalone, it stays about 4 seconds until 8000
persons stored (which is the complete group to store)
This is slow, I must investigate the library directly without webservice
if that is a problem of my coding. So we forget that for the moment.
The problem which bothers me is when I am using Tomcat.
The problem start to occur after posting 1200 patients, it takes 8
seconds. When it reaches 1300, it already takes 17 seconds.
When at Person 1330, it is already 29 seconds, and at 1340 it is 39
seconds, and than a few persons more and a time-out exception occurs.
I receive a java.net.SocketTimeoutException: Read timed out from the
client side-code. The default time-out time for Axis2-code is 40 seconds.
The printstacktrace is below, but it is client side stacktrace
I try a lot of things to solve the problem. I put a Thread.sleep(1000);
in my client-code between to postings, I thought, maybe the webservice
needs some rest to do garbage-clean up. Because it looks very much like
a memory blowing up. But it didn't help.
Of course, if you need any information, I will be happy to provide it,
please give some hints how to provide it
---
Kind regards and much thanks
Bert Verhees
---
I don't know if it is of any help, but I put the exception-trace below
14.01.2012 13:33:52 INFO (HTTPSender.java:196) - Unable to sendViaPost
to url[http://localhost:8080/axis2/services/DemographicService/]
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:146)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
at
org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:78)
at
org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:106)
at
org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1116)
at
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.readLine(MultiThreadedHttpConnectionManager.java:1413)
at
org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1973)
at
org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735)
at
org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098)
at
org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
at
org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at
org.apache.axis2.transport.http.AbstractHTTPSender.executeMethod(AbstractHTTPSender.java:621)
at
org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:193)
at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:75)
at
org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:404)
at
org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:231)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:443)
at
org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:406)
at
org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:229)
at
org.apache.axis2.client.OperationClient.execute(OperationClient.java:165)
at
openehr.rosa.demographicservice.DemographicServiceStub.saveDemographicNew(DemographicServiceStub.java:2668)
at nl.rosa.persons.StorePersons.storePerson(StorePersons.java:77)
at nl.rosa.persons.StorePersons.storePersons(StorePersons.java:49)
at run.FillDatabase.main(FillDatabase.java:35)
org.apache.axis2.AxisFault: Read timed out
at org.apache.axis2.AxisFault.makeFault(AxisFault.java:430)
at
org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:197)
at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:75)
at
org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:404)
at
org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:231)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:443)
at
org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:406)
at
org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:229)
at
org.apache.axis2.client.OperationClient.execute(OperationClient.java:165)
at
openehr.rosa.demographicservice.DemographicServiceStub.saveDemographicNew(DemographicServiceStub.java:2668)
at nl.rosa.persons.StorePersons.storePerson(StorePersons.java:77)
at nl.rosa.persons.StorePersons.storePersons(StorePersons.java:49)
at run.FillDatabase.main(FillDatabase.java:35)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:146)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
at
org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:78)
at
org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:106)
at
org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1116)
at
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.readLine(MultiThreadedHttpConnectionManager.java:1413)
at
org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1973)
at
org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735)
at
org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098)
at
org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
at
org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at
org.apache.axis2.transport.http.AbstractHTTPSender.executeMethod(AbstractHTTPSender.java:621)
at
org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:193)
... 11 more
Exception in thread "main" org.apache.axis2.AxisFault: Read timed out
at org.apache.axis2.AxisFault.makeFault(AxisFault.java:430)
at
org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:197)
at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:75)
at
org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:404)
at
org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:231)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:443)
at
org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:406)
at
org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:229)
at
org.apache.axis2.client.OperationClient.execute(OperationClient.java:165)
at
openehr.rosa.demographicservice.DemographicServiceStub.saveDemographicNew(DemographicServiceStub.java:2668)
at nl.rosa.persons.StorePersons.storePerson(StorePersons.java:77)
at nl.rosa.persons.StorePersons.storePersons(StorePersons.java:49)
at run.FillDatabase.main(FillDatabase.java:35)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:146)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
at
org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:78)
at
org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:106)
at
org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1116)
at
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.readLine(MultiThreadedHttpConnectionManager.java:1413)
at
org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1973)
at
org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735)
at
org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098)
at
org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
at
org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at
org.apache.axis2.transport.http.AbstractHTTPSender.executeMethod(AbstractHTTPSender.java:621)
at
org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:193)
... 11 more
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Unable to sendViaPost to url...... .... java.net.SocketTimeoutException:
Read timed out
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Bert,
On 1/18/12 4:22 PM, Bert Verhees wrote:
> I have done my testing, and at this moment, it seems that I found
> the problem, I forgot to close a databaseconnection in a function
> which is called a zillion times. Just one line of code, and the
> problem was solved.
Consider doing the following:
In your development environment, set maxActive in your connection pool
to "1". Also set the "logAbandoned" and similar settings to catch
resources that you don't properly close.
With maxActive="1", your webapp will stop dead for several minutes
while the connection pool starves waiting for the one missing
connection to (never) be returned to the pool before killing it and
creating a new one. Trust me: you'll notice when that happens.
We have the logAbandoned stuff even set in production just in case we
don't notice something in development.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk8ZhQoACgkQ9CaO5/Lv0PAd9ACfXiOQju6eLIMpB14bZhbkip/B
WjoAoJYJLf+vpYqh3IGYumI7zizO6Ggp
=cCmq
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Unable to sendViaPost to url...... .... java.net.SocketTimeoutException:
Read timed out
Posted by Bert Verhees <be...@rosa.nl>.
I have done my testing, and at this moment, it seems that I found the
problem, I forgot to close a databaseconnection in a function which is
called a zillion times.
Just one line of code, and the problem was solved.
At least I am so honest I admit it freely, and it will follow my career
until the end of times.
Thanks Pid an André for thinking with me.
The good news is that tomcat6/axis2 are good products, at least in my
situation.
Kind regards
Bert Verhees
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Unable to sendViaPost to url...... .... java.net.SocketTimeoutException:
Read timed out
Posted by Bert Verhees <be...@rosa.nl>.
Sorry, readers of this list for not replying, I am very busy
investigating the problem, I try replying later.
Bert
On 16-01-12 23:55, André Warnier wrote:
> Hi.
> I'll follow-up with an "out of band" message just for now.
>
> It was not very clear from your initial explanations whether you were
> sending all of the 1000, 2000, .. 8000 "person's data records" at
> once, in one HTTP POST, or sending the data in a person-by-person way.
> So my wild guess was based on the symptoms you were providing, which
> looked very much like a memory problem caused by the processing of
> increasingly large XML documents using a DOM-like parser.
> And sorry for missing your references to the platform and Tomcat used.
>
> Jumping ahead however, as Pid noted after your memory figures, it
> would appear that on your system, about 1 GB of swap space on disk
> /is/ being used, which would be relatively unusual for a Linux system
> if this application is the only one that is running.
> If indeed this swap space is being used by the Tomcat process, then it
> would definitely explain why the processing is slow (but the reason
> why that is so would still need to be found).
> (As an idea : as soon as an application starts to need the swap area,
> you can easily divide its performance by a factor 10).
>
> It would be interesting to see what the memory usage of the
> Tomcat-related processes is, while processing these requests. Your
> "top" utility would also give an example of this (use CTRL-F, n , <CR>
> to sort the processes by memory usage).
>
> The general point is : there is no specific reason, per se, why Tomcat
> would slow down dramatically when processing 1300 requests, as
> compared to 1000 similar requests. One would expect the 1300 requests
> to need approximately 130% of the time needed to process 1000
> requests, but not the double amount of time.
>
> So if it is not the back-end database, and it is not the Tomcat code
> per se, that leaves
> - the web application (including its Axis2 components)
> - or some fundamentally different way of handling the data, between
> - on one side the client + the stand-alone Axis2 server
> - on the other side the client + the Axis2 Web Service webapp
> this causing, perhaps, Tomcat to become "saturated" and start trashing
>
>
> I do not really know Axis2, other than what I just read on the
> Wikipedia page.
> So I can only make some further guesses at this stage :
>
> You mention that when running Axis2 stand-alone, the processing is
> relatively fast (8000 "persons" in 4 seconds), while when running "the
> same thing" as a Web service under Tomcat, it is infinitely slower.
> I will assume for now that the background database updates are the
> same in both cases, and that this is thus not a factor.
>
> You also seem to say that when running Axis2 stand-alone, the 8000
> "persons" data is only one POST, while through HTTP it would be 8000
> POSTs (or maybe I misunderstood that also).
>
> In any case, can it be that you are comparing apples with oranges ?
>
> As a loose collection of suggestions :
> - when you send SOAP requests via HTTP, there is an extra layer of
> wrapping around the SOAP data, both on the client side (wrap as
> SOAP-HTTP request) and on the server side (unwrap the SOAP-HTTP
> wrapping). One would expect some impact of these in any case, if
> repeated several thousand times in one case, and not in the other.
> - Axis2 seems to allow for configuring entirely different styles of
> XML parsing (DOM or stream/event-based). When your client talks to
> the Axis2 stand-alone server, is it really sending the data the same
> way, and is the Axis2 server library parsing the data the same way ?
> Asked another way : is it perhaps so that when you send the data via
> HTTP to the Web Service, the Web Service code uses a DOM-like parser,
> while when running in stand-alone mode, the Axis2 server uses a
> stream-like parsing method ?
> - maybe the Axis2 Web Service webapp library leaks memory ?
> - maybe your webapp using that library leaks memory ?
> - are your client and the server running on the same host ? SOAP is
> XML. XML processing, particularly in DOM-mode, can use a significant
> amount of memory (*). If the client (to create the XML data) and the
> server (to parse it) are running on the same host, and more or less
> synchronously, you are doubling the amount of memory required to
> process your request/response, and that could explain the onset of
> swapping.
> - when your client sends requests to the Web Servive over HTTP, does
> it send several requests in parallel ? How many Tomcat threads are
> available for processing requests, and how many are effectively
> processing requests in parallel ?
> - maybe indeed Tomcat is spending a lot of its time doing memory
> garbage collection, because of one of the above circumstances
>
>
> (*) an estimate is that to represent an XML document in memory using a
> DOM representation, you easily need 10 times the original amount of
> data in terms of RAM.
> Stream-style parsing and processing is much faster and much leaner in
> terms of memory requirements.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Unable to sendViaPost to url...... .... java.net.SocketTimeoutException:
Read timed out
Posted by André Warnier <aw...@ice-sa.com>.
Hi.
I'll follow-up with an "out of band" message just for now.
It was not very clear from your initial explanations whether you were sending all of the
1000, 2000, .. 8000 "person's data records" at once, in one HTTP POST, or sending the data
in a person-by-person way.
So my wild guess was based on the symptoms you were providing, which looked very much like
a memory problem caused by the processing of increasingly large XML documents using a
DOM-like parser.
And sorry for missing your references to the platform and Tomcat used.
Jumping ahead however, as Pid noted after your memory figures, it would appear that on
your system, about 1 GB of swap space on disk /is/ being used, which would be relatively
unusual for a Linux system if this application is the only one that is running.
If indeed this swap space is being used by the Tomcat process, then it would definitely
explain why the processing is slow (but the reason why that is so would still need to be
found).
(As an idea : as soon as an application starts to need the swap area, you can easily
divide its performance by a factor 10).
It would be interesting to see what the memory usage of the Tomcat-related processes is,
while processing these requests. Your "top" utility would also give an example of this
(use CTRL-F, n , <CR> to sort the processes by memory usage).
The general point is : there is no specific reason, per se, why Tomcat would slow down
dramatically when processing 1300 requests, as compared to 1000 similar requests. One
would expect the 1300 requests to need approximately 130% of the time needed to process
1000 requests, but not the double amount of time.
So if it is not the back-end database, and it is not the Tomcat code per se, that leaves
- the web application (including its Axis2 components)
- or some fundamentally different way of handling the data, between
- on one side the client + the stand-alone Axis2 server
- on the other side the client + the Axis2 Web Service webapp
this causing, perhaps, Tomcat to become "saturated" and start trashing
I do not really know Axis2, other than what I just read on the Wikipedia page.
So I can only make some further guesses at this stage :
You mention that when running Axis2 stand-alone, the processing is relatively fast (8000
"persons" in 4 seconds), while when running "the same thing" as a Web service under
Tomcat, it is infinitely slower.
I will assume for now that the background database updates are the same in both cases, and
that this is thus not a factor.
You also seem to say that when running Axis2 stand-alone, the 8000 "persons" data is only
one POST, while through HTTP it would be 8000 POSTs (or maybe I misunderstood that also).
In any case, can it be that you are comparing apples with oranges ?
As a loose collection of suggestions :
- when you send SOAP requests via HTTP, there is an extra layer of wrapping around the
SOAP data, both on the client side (wrap as SOAP-HTTP request) and on the server side
(unwrap the SOAP-HTTP wrapping). One would expect some impact of these in any case, if
repeated several thousand times in one case, and not in the other.
- Axis2 seems to allow for configuring entirely different styles of XML parsing (DOM or
stream/event-based). When your client talks to the Axis2 stand-alone server, is it really
sending the data the same way, and is the Axis2 server library parsing the data the same
way ? Asked another way : is it perhaps so that when you send the data via HTTP to the Web
Service, the Web Service code uses a DOM-like parser, while when running in stand-alone
mode, the Axis2 server uses a stream-like parsing method ?
- maybe the Axis2 Web Service webapp library leaks memory ?
- maybe your webapp using that library leaks memory ?
- are your client and the server running on the same host ? SOAP is XML. XML processing,
particularly in DOM-mode, can use a significant amount of memory (*). If the client (to
create the XML data) and the server (to parse it) are running on the same host, and more
or less synchronously, you are doubling the amount of memory required to process your
request/response, and that could explain the onset of swapping.
- when your client sends requests to the Web Servive over HTTP, does it send several
requests in parallel ? How many Tomcat threads are available for processing requests, and
how many are effectively processing requests in parallel ?
- maybe indeed Tomcat is spending a lot of its time doing memory garbage collection,
because of one of the above circumstances
(*) an estimate is that to represent an XML document in memory using a DOM representation,
you easily need 10 times the original amount of data in terms of RAM.
Stream-style parsing and processing is much faster and much leaner in terms of memory
requirements.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Unable to sendViaPost to url...... .... java.net.SocketTimeoutException:
Read timed out
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Bert,
On 1/16/12 4:39 PM, Bert Verhees wrote:
> On 16-01-12 22:26, Pid wrote:
>> The JConsole and VisualVM applications provided with the JDK
>> will tell you how much memory is being consumed by the
>> application in various areas of the Java program's process heap.
>>
>> You may use them to observe behaviour during the operation you
>> are experimenting with.
>
> Thanks Pid, I will certainly do that. I did not know about these
> possibilities.
You might also want to take some thread dumps during the
slow-processing to see where your code is spending its time.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk8W77AACgkQ9CaO5/Lv0PASygCeNQ848/auhyuTzLfohr0vYQH4
vTIAoJ3jNa0mgMD7riVibFVFqngFcrXj
=eJcg
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Unable to sendViaPost to url...... .... java.net.SocketTimeoutException:
Read timed out
Posted by Bert Verhees <be...@rosa.nl>.
On 16-01-12 22:26, Pid wrote:
> The JConsole and VisualVM applications provided with the JDK will tell
> you how much memory is being consumed by the application in various
> areas of the Java program's process heap.
>
> You may use them to observe behaviour during the operation you are
> experimenting with.
Thanks Pid, I will certainly do that. I did not know about these
possibilities.
best regards
Bert Verhees
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Unable to sendViaPost to url...... .... java.net.SocketTimeoutException:
Read timed out
Posted by Pid <pi...@pidster.com>.
On 16/01/2012 20:12, Bert Verhees wrote:
> Thanks, André, for your reply
>
>>
>> Is that per "person", or per POST ?
>
> It is per Person AND per post. The post creates a person-object which is
> stored in a database, and the person-object is formed out of an array of
> strings.
>>
>> to an webservice-call.
>>> After some validation, the webservice
>>
>> of which we here know nothing..
>
> It has nothing to do with the problem. But you may, of course know what
> it is all about. It is about forming OpenEHR-objects, the string arrays
> serve to create path-strings-array and value-string array. The problem
> is not there, I know, because there are three ways of running the code.
>
> One is to call the libraries behind the webservices directly, and also
> do the complete run of posting and validating. It is exact the same
> code, it is only passing the webservices. This runs fine.
> They other one is using axis2 as a standalone service, and the third one
> is using tomcat as a container for axis.
>
>>
>> posts the data to a postgresql
>>> database.
>>>
>>> I have some console-output to see how long it takes.
>>>
>>> storing..... Person:26, ID:27
>>> Creating EHR for:213d7c09-949c-4757-a91d-da5510215472
>>> GUIDs saved to table:213d7c09-949c-4757-a91d-da5510215472,
>>> 73b60dd4-d745-4267-9c6f-fc8b067bfeb9
>>> stored..... Person:26, UID:213d7c09-949c-4757-a91d-da5510215472
>>> PT0.647S
>>>
>>> As you can see, it starts with less then a second, sometime 0.3
>>> seconds, sometimes 1.1 second, depending on the number of elements in
>>> the array to store.
>>
>> What array, where ?
>>
>>>
>>> After 1000 persons stored, it takes 4 seconds
>>
>> Do you mean that you send data in one POST about 1000 persons (each
>> with 50 text data elements) ?
>
> No, it is each time a separate call.
>
>
>>
>>>
>>> storing..... Person:998, ID:989
>>> Creating EHR for:49c96d87-38db-491d-a824-78800615e80a
>>> GUIDs saved to table:49c96d87-38db-491d-a824-78800615e80a,
>>> 3554514f-ccb6-40dd-92f2-d9140760870e
>>> stored..... Person:998, UID:49c96d87-38db-491d-a824-78800615e80a
>>> PT4.147S
>>>
>>> When running in Axis2 standalone, it stays about 4 seconds until 8000
>>> persons stored (which is the complete group to store)
>>
>> is that 1 POST, or 8000 POSTs to the server ?
>
> This is one post.
>>
>>> This is slow, I must investigate the library directly without
>>> webservice if that is a problem of my coding. So we forget that for
>>> the moment.
>>>
>>> The problem which bothers me is when I am using Tomcat.
>>>
>>> The problem start to occur after posting 1200 patients, it takes 8
>>> seconds. When it reaches 1300, it already takes 17 seconds.
>>> When at Person 1330, it is already 29 seconds, and at 1340 it is 39
>>> seconds, and than a few persons more and a time-out exception occurs.
>>>
>>> I receive a java.net.SocketTimeoutException: Read timed out from the
>>> client side-code. The default time-out time for Axis2-code is 40
>>> seconds.
>>> The printstacktrace is below, but it is client side stacktrace
>>>
>>> I try a lot of things to solve the problem. I put a
>>> Thread.sleep(1000); in my client-code between to postings, I thought,
>>> maybe the webservice needs some rest to do garbage-clean up. Because
>>> it looks very much like a memory blowing up. But it didn't help.
>>>
>>> Of course, if you need any information, I will be happy to provide
>>> it, please give some hints how to provide it
>>>
>>
>> You could start by telling us which version of Tomcat this is, under
>> with Java JVM version, and on which platform (Windows, Linux,..).
>> Then tell us with how much available memory this Tomcat is running.
>
> I did tell in the message which Tomcat version I use, and I also did
> tell which platform I use
> I don't mind telling you again
>> I am using Tomcat6 6.0.32-5ubuntu1.1 together with Axis2 1.6.1
>
> I did not tell you which JVM I use, that is
> OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre11-0ubuntu1.11.10)
> OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
>
> I don't know how much memory is avalaible for tomcat, but this is what
> "top" tells me
> Mem: 4055608k total, 3277200k used, 778408k free, 2876k buffers
> Swap: 9764860k total, 1102476k used, 8662384k free, 243536k cached
The system is using swap. What else is running on the system?
The top output should also be able to tell you how much memory Tomcat is
consuming.
> I see there is 8 Gb swap unused, so Tomcat does not bother claiming it.
That is not clear from the above.
The JConsole and VisualVM applications provided with the JDK will tell
you how much memory is being consumed by the application in various
areas of the Java program's process heap.
You may use them to observe behaviour during the operation you are
experimenting with.
p
>> Making some wild guesses here, because you do not really provide any
>> precise information about what exactly you are trying to do :
>
> Excuse me for putting you in this situation. I did not know which
> information to provide.
>
>>
>> Let's suppose that your client application puts together one POST
>> request, consisting of XML data composed of 1000, 2000, 3000 elements
>> "person", each composed of up to 50 text data sub-elements, and then
>> sends that POST to the server.
>> And that the server application receives this big POST request, reads
>> the XML body of it, parses this XML in memory as a large "XML document
>> object", before it starts splitting it into 1000, 2000, 3000 "person"
>> elements in order to update records in the database..
>
> This suggestion is not right, every person is posted with a separate
> webservice-call. And every person consists of an array with maximum 50
> values (house number, city-name, short strings), and another array with
> the same number of value-descriptors (which are in fact openEHR
> archetype-path-strings of about 200 characters), so every person is
> maximum 15 Kb.
>
> I hope with this more precise information, you can give me some clue
> about where to look where the things break down.
>
> Thanks again for your reply and your questions for more information.
>
> Bert Verhees
>
>>
>> Then, if all the above roughly matches what you are doing, I would not
>> be very surprised if you got into serious memory usage problems,
>> forcing the server to swap this data and slowing down your process
>> dramatically (and exponentially) as the quantity of data increases.
>>
>> Have you tried a design where you send the data to the server in more
>> manageable chunks ?
>> What about a loop where you would send 100 persons's data at one time,
>> and run the loop 80 times ?
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
--
[key:62590808]
Re: Unable to sendViaPost to url...... .... java.net.SocketTimeoutException:
Read timed out
Posted by Bert Verhees <be...@rosa.nl>.
Thanks, André, for your reply
>
> Is that per "person", or per POST ?
It is per Person AND per post. The post creates a person-object which is
stored in a database, and the person-object is formed out of an array of
strings.
>
> to an webservice-call.
>> After some validation, the webservice
>
> of which we here know nothing..
It has nothing to do with the problem. But you may, of course know what
it is all about. It is about forming OpenEHR-objects, the string arrays
serve to create path-strings-array and value-string array. The problem
is not there, I know, because there are three ways of running the code.
One is to call the libraries behind the webservices directly, and also
do the complete run of posting and validating. It is exact the same
code, it is only passing the webservices. This runs fine.
They other one is using axis2 as a standalone service, and the third one
is using tomcat as a container for axis.
>
> posts the data to a postgresql
>> database.
>>
>> I have some console-output to see how long it takes.
>>
>> storing..... Person:26, ID:27
>> Creating EHR for:213d7c09-949c-4757-a91d-da5510215472
>> GUIDs saved to table:213d7c09-949c-4757-a91d-da5510215472,
>> 73b60dd4-d745-4267-9c6f-fc8b067bfeb9
>> stored..... Person:26, UID:213d7c09-949c-4757-a91d-da5510215472
>> PT0.647S
>>
>> As you can see, it starts with less then a second, sometime 0.3
>> seconds, sometimes 1.1 second, depending on the number of elements in
>> the array to store.
>
> What array, where ?
>
>>
>> After 1000 persons stored, it takes 4 seconds
>
> Do you mean that you send data in one POST about 1000 persons (each
> with 50 text data elements) ?
No, it is each time a separate call.
>
>>
>> storing..... Person:998, ID:989
>> Creating EHR for:49c96d87-38db-491d-a824-78800615e80a
>> GUIDs saved to table:49c96d87-38db-491d-a824-78800615e80a,
>> 3554514f-ccb6-40dd-92f2-d9140760870e
>> stored..... Person:998, UID:49c96d87-38db-491d-a824-78800615e80a
>> PT4.147S
>>
>> When running in Axis2 standalone, it stays about 4 seconds until 8000
>> persons stored (which is the complete group to store)
>
> is that 1 POST, or 8000 POSTs to the server ?
This is one post.
>
>> This is slow, I must investigate the library directly without
>> webservice if that is a problem of my coding. So we forget that for
>> the moment.
>>
>> The problem which bothers me is when I am using Tomcat.
>>
>> The problem start to occur after posting 1200 patients, it takes 8
>> seconds. When it reaches 1300, it already takes 17 seconds.
>> When at Person 1330, it is already 29 seconds, and at 1340 it is 39
>> seconds, and than a few persons more and a time-out exception occurs.
>>
>> I receive a java.net.SocketTimeoutException: Read timed out from the
>> client side-code. The default time-out time for Axis2-code is 40
>> seconds.
>> The printstacktrace is below, but it is client side stacktrace
>>
>> I try a lot of things to solve the problem. I put a
>> Thread.sleep(1000); in my client-code between to postings, I thought,
>> maybe the webservice needs some rest to do garbage-clean up. Because
>> it looks very much like a memory blowing up. But it didn't help.
>>
>> Of course, if you need any information, I will be happy to provide
>> it, please give some hints how to provide it
>>
>
> You could start by telling us which version of Tomcat this is, under
> with Java JVM version, and on which platform (Windows, Linux,..).
> Then tell us with how much available memory this Tomcat is running.
I did tell in the message which Tomcat version I use, and I also did
tell which platform I use
I don't mind telling you again
> I am using Tomcat6 6.0.32-5ubuntu1.1 together with Axis2 1.6.1
I did not tell you which JVM I use, that is
OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre11-0ubuntu1.11.10)
OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
I don't know how much memory is avalaible for tomcat, but this is what
"top" tells me
Mem: 4055608k total, 3277200k used, 778408k free, 2876k buffers
Swap: 9764860k total, 1102476k used, 8662384k free, 243536k cached
I see there is 8 Gb swap unused, so Tomcat does not bother claiming it.
>
> Making some wild guesses here, because you do not really provide any
> precise information about what exactly you are trying to do :
Excuse me for putting you in this situation. I did not know which
information to provide.
>
> Let's suppose that your client application puts together one POST
> request, consisting of XML data composed of 1000, 2000, 3000 elements
> "person", each composed of up to 50 text data sub-elements, and then
> sends that POST to the server.
> And that the server application receives this big POST request, reads
> the XML body of it, parses this XML in memory as a large "XML document
> object", before it starts splitting it into 1000, 2000, 3000 "person"
> elements in order to update records in the database..
This suggestion is not right, every person is posted with a separate
webservice-call. And every person consists of an array with maximum 50
values (house number, city-name, short strings), and another array with
the same number of value-descriptors (which are in fact openEHR
archetype-path-strings of about 200 characters), so every person is
maximum 15 Kb.
I hope with this more precise information, you can give me some clue
about where to look where the things break down.
Thanks again for your reply and your questions for more information.
Bert Verhees
>
> Then, if all the above roughly matches what you are doing, I would not
> be very surprised if you got into serious memory usage problems,
> forcing the server to swap this data and slowing down your process
> dramatically (and exponentially) as the quantity of data increases.
>
> Have you tried a design where you send the data to the server in more
> manageable chunks ?
> What about a loop where you would send 100 persons's data at one time,
> and run the loop 80 times ?
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Unable to sendViaPost to url...... .... java.net.SocketTimeoutException:
Read timed out
Posted by André Warnier <aw...@ice-sa.com>.
Bert Verhees wrote:
> Hi, I hope you can help me analyzing this problem:
>
> I am using Tomcat6 6.0.32-5ubuntu1.1 together with Axis2 1.6.1
>
> I am not sure if it is an Axis2 problem or a Tomcat-problem.
>
> I have an client-application for test-purposes which runs inside the
> Eclipse-IDE.
> The problem only occurs when running Axis2 inside Tomcat6, when running
> in Axis2 as stand-alone server, it runs better.
>
> What I do is posting arrays of strings (about 50 elements, about 5Kb
> max)
Is that per "person", or per POST ?
to an webservice-call.
> After some validation, the webservice
of which we here know nothing..
posts the data to a postgresql
> database.
>
> I have some console-output to see how long it takes.
>
> storing..... Person:26, ID:27
> Creating EHR for:213d7c09-949c-4757-a91d-da5510215472
> GUIDs saved to table:213d7c09-949c-4757-a91d-da5510215472,
> 73b60dd4-d745-4267-9c6f-fc8b067bfeb9
> stored..... Person:26, UID:213d7c09-949c-4757-a91d-da5510215472
> PT0.647S
>
> As you can see, it starts with less then a second, sometime 0.3 seconds,
> sometimes 1.1 second, depending on the number of elements in the array
> to store.
What array, where ?
>
> After 1000 persons stored, it takes 4 seconds
Do you mean that you send data in one POST about 1000 persons (each with 50 text data
elements) ?
>
> storing..... Person:998, ID:989
> Creating EHR for:49c96d87-38db-491d-a824-78800615e80a
> GUIDs saved to table:49c96d87-38db-491d-a824-78800615e80a,
> 3554514f-ccb6-40dd-92f2-d9140760870e
> stored..... Person:998, UID:49c96d87-38db-491d-a824-78800615e80a
> PT4.147S
>
> When running in Axis2 standalone, it stays about 4 seconds until 8000
> persons stored (which is the complete group to store)
is that 1 POST, or 8000 POSTs to the server ?
> This is slow, I must investigate the library directly without webservice
> if that is a problem of my coding. So we forget that for the moment.
>
> The problem which bothers me is when I am using Tomcat.
>
> The problem start to occur after posting 1200 patients, it takes 8
> seconds. When it reaches 1300, it already takes 17 seconds.
> When at Person 1330, it is already 29 seconds, and at 1340 it is 39
> seconds, and than a few persons more and a time-out exception occurs.
>
> I receive a java.net.SocketTimeoutException: Read timed out from the
> client side-code. The default time-out time for Axis2-code is 40 seconds.
> The printstacktrace is below, but it is client side stacktrace
>
> I try a lot of things to solve the problem. I put a Thread.sleep(1000);
> in my client-code between to postings, I thought, maybe the webservice
> needs some rest to do garbage-clean up. Because it looks very much like
> a memory blowing up. But it didn't help.
>
> Of course, if you need any information, I will be happy to provide it,
> please give some hints how to provide it
>
You could start by telling us which version of Tomcat this is, under with Java JVM
version, and on which platform (Windows, Linux,..).
Then tell us with how much available memory this Tomcat is running.
Making some wild guesses here, because you do not really provide any precise information
about what exactly you are trying to do :
Let's suppose that your client application puts together one POST request, consisting of
XML data composed of 1000, 2000, 3000 elements "person", each composed of up to 50 text
data sub-elements, and then sends that POST to the server.
And that the server application receives this big POST request, reads the XML body of it,
parses this XML in memory as a large "XML document object", before it starts splitting it
into 1000, 2000, 3000 "person" elements in order to update records in the database..
Then, if all the above roughly matches what you are doing, I would not be very surprised
if you got into serious memory usage problems, forcing the server to swap this data and
slowing down your process dramatically (and exponentially) as the quantity of data increases.
Have you tried a design where you send the data to the server in more manageable chunks ?
What about a loop where you would send 100 persons's data at one time, and run the loop 80
times ?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Unable to sendViaPost to url...... .... java.net.SocketTimeoutException:
Read timed out
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Bert,
On 1/16/12 11:15 AM, Bert Verhees wrote:
> I am not sure if it is an Axis2 problem or a Tomcat-problem.
>
> I have an client-application for test-purposes which runs inside
> the Eclipse-IDE. The problem only occurs when running Axis2 inside
> Tomcat6, when running in Axis2 as stand-alone server, it runs
> better.
SO, running Axis2 using it's built-in HTTP server (I didn't know it
had one) has acceptable performance, but running Axis2 within Tomcat
does not have acceptable performance?
> I have some console-output to see how long it takes.
How about real profiling data?
> When running in Axis2 standalone, it stays about 4 seconds until
> 8000 persons stored (which is the complete group to store) This is
> slow, I must investigate the library directly without webservice if
> that is a problem of my coding. So we forget that for the moment.
So, Axis2 performance is also unaccaptable.
> The problem which bothers me is when I am using Tomcat.
>
> The problem start to occur after posting 1200 patients, it takes 8
> seconds.
So your observation is that, when running within Tomcat, your code
appears to perform more slowly. Is that consistent? I wonder if your
code / database slows down considerably as data is added and you are
running your in-Tomcat tests after the out-of-Tomcat tests.
> I try a lot of things to solve the problem.
Like what?
> I put a Thread.sleep(1000); in my client-code between to postings,
> I thought, maybe the webservice needs some rest to do garbage-clean
> up.
Obviously, pausing for a second per record will limit performance. GC
will run when it has to -- you don't have to "give it time" to do its job.
> Because it looks very much like a memory blowing up. But it didn't
> help.
What makes you think that memory is "blowing up"?
It probably didn't help because you have mis-identified the problem.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEUEARECAAYFAk8W7QUACgkQ9CaO5/Lv0PDBEQCfTaLMZE7TPBdwAVaFe56MRBDC
OccAmNgqaj57ji+z11Z/hVourbydlS0=
=H9YI
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org