You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Michael Priess <di...@googlemail.com> on 2014/01/13 10:09:11 UTC
Too many open files
Hi,
hi I actually use Apache Camel 2.10.1 in combination with ActiveMQ 5.7.0.
To build a connection to ActiveMQ I use the ActiveMQ ConnectionFactory.
After a while I get this exception:
2013-12-21 04:16:51,442 | ERROR | Could not accept connection :
java.net.SocketException: Too many open files |
org.apache.activemq.broker.TransportConnector | ActiveMQ Transport Server:
tcp://
0.0.0.0:61616?maximumConnections=1000&wireformat.maxFrameSize=104857600
Is there a way how to find which client open so much connections? Is there
a reason why ActiveMQ doesn't close this connections?
Any idea how to debug such an issue?
Regards,
Michael
Re: Too many open files
Posted by Torsten Mielke <tm...@redhat.com>.
In general the use of a pooling aware JMS ConnectionFactory such as ActiveMQs PooledConnectionFactory is *always* recommended. Particularly in Camel which uses Spring JMS underneath.
Regards,
Torsten Mielke
tmielke.blogspot.com
On 16 Jan 2014, at 10:54 am, Jose María Zaragoza <de...@gmail.com> wrote:
> 2014/1/16 Arjen van der Meijden <ac...@tweakers.net>:
>> I think he was referring to your clients which could be adjusted to pool a
>> small amount of connections, rather than opening many.
>> You obviously can't control this from the server-side.
>
> Well, I don't know so much about Apache Camel , but in
> http://camel.apache.org/activemq.html
>
> you can use PooledConnectionFactory
>
>
> Regards
Re: Too many open files
Posted by Jose María Zaragoza <de...@gmail.com>.
2014/1/16 Arjen van der Meijden <ac...@tweakers.net>:
> I think he was referring to your clients which could be adjusted to pool a
> small amount of connections, rather than opening many.
> You obviously can't control this from the server-side.
Well, I don't know so much about Apache Camel , but in
http://camel.apache.org/activemq.html
you can use PooledConnectionFactory
Regards
Re: Too many open files
Posted by Arjen van der Meijden <ac...@tweakers.net>.
I think he was referring to your clients which could be adjusted to pool
a small amount of connections, rather than opening many.
You obviously can't control this from the server-side. Still its very
uncommon to really need 1000 connections at the same time, it generally
suggests your clients open way to many connections. Or they're opened
and closed very fast, so the server can't keep up in its default
configuration and has many "waiting to be closed connections".
For each connection to your server there is a single file pointer. You
can, amongsts others, use 'lsof' to see how many for your java-proces
(see man lsof for usage examples).
You can also see connections in CLOSE_WAIT using lsof.
If you really have that many clients, there isn't much you can do.
You're limited to adjusting the server in such a way it can hold more
file pointers (i.e. use ulimit) or change your server-architecture to
have more servers (with that many clients, some redundancy is a good
thing anyway).
Best regards,
Arjen
On 16-1-2014 9:54 Michael Priess wrote:
> Hi,
>
> correct me if I'm wrong but if I listen to a topic the connection must be
> always open? In my thinking a PooledConnectionFactory make only sense if
> I'm sending a lot of messages to the broker.
> BTW. know someone the exactly number of file handles which are open per
> connection on the server?
>
>
> 2014/1/15 Jose María Zaragoza <de...@gmail.com>
>
>> 2014/1/13 Michael Priess <di...@googlemail.com>:
>>>
>>> Is there a way how to find which client open so much connections? Is
>> there
>>> a reason why ActiveMQ doesn't close this connections?
>>> Any idea how to debug such an issue?
>>
>>
>> Why dont use PooledConnectionFactory?
>>
>> Regards
>>
>
Re: Too many open files
Posted by Michael Priess <di...@googlemail.com>.
Hi,
correct me if I'm wrong but if I listen to a topic the connection must be
always open? In my thinking a PooledConnectionFactory make only sense if
I'm sending a lot of messages to the broker.
BTW. know someone the exactly number of file handles which are open per
connection on the server?
2014/1/15 Jose María Zaragoza <de...@gmail.com>
> 2014/1/13 Michael Priess <di...@googlemail.com>:
> >
> > Is there a way how to find which client open so much connections? Is
> there
> > a reason why ActiveMQ doesn't close this connections?
> > Any idea how to debug such an issue?
>
>
> Why dont use PooledConnectionFactory?
>
> Regards
>
Re: Too many open files
Posted by Jose María Zaragoza <de...@gmail.com>.
2014/1/13 Michael Priess <di...@googlemail.com>:
>
> Is there a way how to find which client open so much connections? Is there
> a reason why ActiveMQ doesn't close this connections?
> Any idea how to debug such an issue?
Why dont use PooledConnectionFactory?
Regards
Re: Too many open files
Posted by Torsten Mielke <tm...@redhat.com>.
Hi,
You can use JMX tools like console and connect to the broker, check the connections and their IP addresses.
I suggest you first find out to which client process these many connections belong and then look at the client code/configuration to find out why connections are not closed.
I am not aware of any bugs in this area, so its likely a user / configuration problem.
Regards,
Torsten Mielke
tmielke.blogspot.com
On 13 Jan 2014, at 10:09 am, Michael Priess <di...@googlemail.com> wrote:
> Hi,
>
> hi I actually use Apache Camel 2.10.1 in combination with ActiveMQ 5.7.0.
>
> To build a connection to ActiveMQ I use the ActiveMQ ConnectionFactory.
> After a while I get this exception:
>
> 2013-12-21 04:16:51,442 | ERROR | Could not accept connection :
> java.net.SocketException: Too many open files |
> org.apache.activemq.broker.TransportConnector | ActiveMQ Transport Server:
> tcp://
> 0.0.0.0:61616?maximumConnections=1000&wireformat.maxFrameSize=104857600
>
> Is there a way how to find which client open so much connections? Is there
> a reason why ActiveMQ doesn't close this connections?
> Any idea how to debug such an issue?
>
> Regards,
>
> Michael
Re: Too many open files
Posted by artnaseef <ar...@artnaseef.com>.
The 1000 maximum connections is an insane number for a single connection
pool. I'm hard-pressed to imagine a scenario in which such a large number
of connections can improve performance.
Why the large number?
--
View this message in context: http://activemq.2283324.n4.nabble.com/Too-many-open-files-tp4676228p4676259.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Too many open files
Posted by Arjen van der Meijden <ac...@tweakers.net>.
By default, ActiveMQ closes connections asynchronously. That is normally
not a problem, unless you open and close many connections (in our case,
hundreds per minute).
In that case, the asynchronous closing may not be able to keep up with
your opening/closing-rate and you start accumulating file descriptors.
There is a transport-level configuration switch that you can set in your
activemq.xml to switch that to synchronous:
We have it for stomp:
<transportConnector name="stomp"
uri="stomp://0.0.0.0:61613?transport.closeAsync=false" />
But as said, this should not happen when you have just a low open/close
rate.
By the way, if you allow 1000 connections at maximum and expect to get
close to that number... you'll need to raise the limit on most
Linux-distro's. The default tends to be 1024 (but that's not enough for
1000 connections, since you need to account for file descriptors for
libraries, storage files, etc).
We have added this to our /etc/init.d/activemq-script
ulimit -n 102400
Best regards,
Arjen
On 13-1-2014 10:09, Michael Priess wrote:
> Hi,
>
> hi I actually use Apache Camel 2.10.1 in combination with ActiveMQ 5.7.0.
>
> To build a connection to ActiveMQ I use the ActiveMQ ConnectionFactory.
> After a while I get this exception:
>
> 2013-12-21 04:16:51,442 | ERROR | Could not accept connection :
> java.net.SocketException: Too many open files |
> org.apache.activemq.broker.TransportConnector | ActiveMQ Transport Server:
> tcp://
> 0.0.0.0:61616?maximumConnections=1000&wireformat.maxFrameSize=104857600
>
> Is there a way how to find which client open so much connections? Is there
> a reason why ActiveMQ doesn't close this connections?
> Any idea how to debug such an issue?
>
> Regards,
>
> Michael
>