You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cocoon.apache.org by Peter Flynn <pf...@ucc.ie> on 2010/12/16 13:58:29 UTC
Too many open files
I'm getting this error in apparently random circumstances on
http://publish.ucc.ie/researchprofiles/ using Cocoon 2.1.11 under
Tomcat5 on RHEL5.
This application is reprocessing some HTML feeds from another server in
order to present the information in a different layout, and do some
extensive cleaning-up of the data. Most of the time it works just fine.
The full stacktrace says:
> org.apache.cocoon.ProcessingException: Failed to process pipeline
> at <map:serialize> - file:///var/www/xml/profiles/sitemap.xmap:74:18
> at <map:transform> - file:///var/www/xml/profiles/sitemap.xmap:69:36
> at <map:generate type="html"> - file:///var/www/xml/profiles/sitemap.xmap:68:17
The line numbers vary according to the page type (and thus which piece
of the pipeline).
All the templates generate the page layout by retrieving a HTML page via
Tidy, and slotting different pieces of data into different places: a
fairly typical mashup.
In the process, the XSLT will open five internal pipelines via
document() calls to retrieve data, each of which opens one or two other
document()s and two local (static) XML lookup tables (very small).
> at <map:mount> - file:///usr/share/tomcat5/webapps/ROOT/sitemap.xmap:1115:57
> at org.apache.cocoon.ProcessingException.throwLocated(ProcessingException.java:145)
Further down the stacktrace, it says:
> Caused by: java.net.SocketException: Too many open files
> at java.net.Socket.createImpl(Socket.java:388)
> at java.net.Socket.connect(Socket.java:517)
> at java.net.Socket.connect(Socket.java:469)
> at sun.net.NetworkClient.doConnect(NetworkClient.java:157)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
In my ignorance of the internals of Cocoon and Tomcat and Java, is it
running out of file handles when making a HTTP call?
The server is lightly loaded, but each page does open a significant
number of document()s: is there a setting that will increase the number
of open files allowed?
Is there any way to get it to report *what* it was trying to open (the
URI) when it failed?
///Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: Too many open files
Posted by Peter Flynn <pf...@ucc.ie>.
On 16/12/10 13:03, Johan Cwiklinski wrote:
> Hello,
>
> Le 16/12/2010 13:58, Peter Flynn a écrit :
>>> Caused by: java.net.SocketException: Too many open files
>>> at java.net.Socket.createImpl(Socket.java:388)
>>> at java.net.Socket.connect(Socket.java:517)
>>> at java.net.Socket.connect(Socket.java:469)
>>> at sun.net.NetworkClient.doConnect(NetworkClient.java:157)
>>> at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
>>
>> In my ignorance of the internals of Cocoon and Tomcat and Java, is it
>> running out of file handles when making a HTTP call?
>>
>> The server is lightly loaded, but each page does open a significant
>> number of document()s: is there a setting that will increase the number
>> of open files allowed?
>
> We already have seen this issue on some cocoon applications ; most of
> the time, that was due to intensive use of xpath "document" function
> with. We observe that using saxon as xslt processor genrerally solved
> that issue.
I have had that on my TODO list for some time. About time to do it...thanks.
///Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
RE: Too many open files
Posted by Laurent Medioni <lm...@odyssey-group.com>.
Yes, had that too in the past, ulimit -H -n 65535 cured the issue.
Laurent
____________________________________________________________
• This email and any files transmitted with it are CONFIDENTIAL and intended
solely for the use of the individual or entity to which they are addressed.
• Any unauthorized copying, disclosure, or distribution of the material within
this email is strictly forbidden.
• Any views or opinions presented within this e-mail are solely those of the
author and do not necessarily represent those of Odyssey Financial
Technologies SA unless otherwise specifically stated.
• An electronic message is not binding on its sender. Any message referring to
a binding engagement must be confirmed in writing and duly signed.
• If you have received this email in error, please notify the sender immediately
and delete the original.
Re: Too many open files
Posted by Peter Flynn <pf...@ucc.ie>.
On 16/12/10 16:49, john muth wrote:
>
> I hit this problem not too long ago and decided the o/s default limit of 1024 was too low for what we were trying to do.
>
> For me, on Redhat, the solution was to edit /etc/security/limits.conf
>
> tomcatuser hard nofile 65535
> tomcatuser soft nofile 65535
On 17/12/10 08:41, Laurent Medioni wrote:
> Yes, had that too in the past, ulimit -H -n 65535 cured the issue.
Thanks, I'll try. And presumably reboot the server.
///Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: Too many open files
Posted by john muth <jo...@yahoo.com>.
I hit this problem not too long ago and decided the o/s default limit of 1024 was too low for what we were trying to do.
For me, on Redhat, the solution was to edit /etc/security/limits.conf
tomcatuser hard nofile 65535
tomcatuser soft nofile 65535
--- On Thu, 12/16/10, Peter Flynn <pf...@ucc.ie> wrote:
> From: Peter Flynn <pf...@ucc.ie>
> Subject: Re: Too many open files
> To: users@cocoon.apache.org
> Date: Thursday, December 16, 2010, 4:00 PM
> On 16/12/10 14:39, Andreas Kuehne
> wrote:
> > Hi all,
> >
> > if you're in a Cocoon environment, it's always a good
> idea to avoid the
> > document function. Cocoon offers so many better ways
> like map:aggregate
> > or the CInclude transformer preserving the cache
> functionalities.
>
> Thank you, that sounds like a useful tip, although I wasn't
> aware of
> either of them. The problem is that the XSLT I am using
> also needs to
> run standalone (ie outside Cocoon) for other reasons, so I
> was trying to
> avoid too much Cocoon dependency.
>
> ///Peter
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
> For additional commands, e-mail: users-help@cocoon.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: Too many open files
Posted by Peter Flynn <pf...@ucc.ie>.
On 16/12/10 14:39, Andreas Kuehne wrote:
> Hi all,
>
> if you're in a Cocoon environment, it's always a good idea to avoid the
> document function. Cocoon offers so many better ways like map:aggregate
> or the CInclude transformer preserving the cache functionalities.
Thank you, that sounds like a useful tip, although I wasn't aware of
either of them. The problem is that the XSLT I am using also needs to
run standalone (ie outside Cocoon) for other reasons, so I was trying to
avoid too much Cocoon dependency.
///Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: Too many open files
Posted by Andreas Kuehne <ak...@yahoo.com>.
Hi all,
if you're in a Cocoon environment, it's always a good idea to avoid the document
function. Cocoon offers so many better ways like map:aggregate or the CInclude
transformer preserving the cache functionalities.
Greetings
Andreas
________________________________
From: Johan Cwiklinski <jo...@ajlsm.com>
To: users@cocoon.apache.org
Sent: Thu, December 16, 2010 2:03:25 PM
Subject: Re: Too many open files
Hello,
Le 16/12/2010 13:58, Peter Flynn a écrit :
>> Caused by: java.net.SocketException: Too many open files
>> at java.net.Socket.createImpl(Socket.java:388)
>> at java.net.Socket.connect(Socket.java:517)
>> at java.net.Socket.connect(Socket.java:469)
>> at sun.net.NetworkClient.doConnect(NetworkClient.java:157)
>> at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
>
> In my ignorance of the internals of Cocoon and Tomcat and Java, is it
> running out of file handles when making a HTTP call?
>
> The server is lightly loaded, but each page does open a significant
> number of document()s: is there a setting that will increase the number
> of open files allowed?
We already have seen this issue on some cocoon applications ; most of
the time, that was due to intensive use of xpath "document" function
with. We observe that using saxon as xslt processor genrerally solved
that issue.
Most of these issues has been resolved for us doing that ; some are
always present but I did not have time to further investigate yet.
>
> Is there any way to get it to report *what* it was trying to open (the
> URI) when it failed?
>
> ///Peter
>
Hope that could help you.
Regards,
--
Johan Cwiklinski
AJLSM
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: Too many open files
Posted by Johan Cwiklinski <jo...@ajlsm.com>.
Hello,
Le 16/12/2010 13:58, Peter Flynn a écrit :
>> Caused by: java.net.SocketException: Too many open files
>> at java.net.Socket.createImpl(Socket.java:388)
>> at java.net.Socket.connect(Socket.java:517)
>> at java.net.Socket.connect(Socket.java:469)
>> at sun.net.NetworkClient.doConnect(NetworkClient.java:157)
>> at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
>
> In my ignorance of the internals of Cocoon and Tomcat and Java, is it
> running out of file handles when making a HTTP call?
>
> The server is lightly loaded, but each page does open a significant
> number of document()s: is there a setting that will increase the number
> of open files allowed?
We already have seen this issue on some cocoon applications ; most of
the time, that was due to intensive use of xpath "document" function
with. We observe that using saxon as xslt processor genrerally solved
that issue.
Most of these issues has been resolved for us doing that ; some are
always present but I did not have time to further investigate yet.
>
> Is there any way to get it to report *what* it was trying to open (the
> URI) when it failed?
>
> ///Peter
>
Hope that could help you.
Regards,
--
Johan Cwiklinski
AJLSM
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org