[jira] [Commented] (AMQ-6260) ActiveMQ 100% memory usage. Too many open files

["Pankaj Takawale (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AMQ-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15248109#comment-15248109 ] 

Pankaj Takawale commented on AMQ-6260:
--------------------------------------

We have ActiveMQ running on other envs. Those servers has similar ulimits and file counts, and they are running fine.

After couple of restarts, it is not showing "too many open files" now but memory is reaching to 100% in around 30 mins.

> ActiveMQ 100% memory usage. Too many open files
> -----------------------------------------------
>
>                 Key: AMQ-6260
>                 URL: https://issues.apache.org/jira/browse/AMQ-6260
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.11.3
>            Reporter: Pankaj Takawale
>
> Version: 5.11.3
> OS: Centos 6.5
> Persistence: JDBC  (Postgresql)
> ActiveMQ is running on 16 vCPU, 29 GB ram. It has around 700K pending messages left on one of the queues. This queue has around 240 to 360 consumers. 
> There are hundreds of other queues, but with less number of pending messages.
> Producer flow control is turned off.
> I'm getting "Too many open files" error. Opened files are much less than the overall limit. See below the details.
> Memory usage reaches to 100% after few minutes of restart. We experienced total 1+ million messages across multiple queues, but did not face the 100% memory or too many opened files at that time.
> Apr 19 08:04:12.232-ActiveMQ NIO Worker 2687 - ERROR TransportConnector   - deliverybroker                        - Could not accept connection : java.io.IOException: Too many open files
> lsof shows: total file counts 1724
> netsta -an|wc  : 374
> ulimit -a for a user that is running activemq process
> core file size          (blocks, -c) 0
> data seg size           (kbytes, -d) unlimited
> scheduling priority             (-e) 0
> file size               (blocks, -f) unlimited
> pending signals                 (-i) 240441
> max locked memory       (kbytes, -l) 64
> max memory size         (kbytes, -m) unlimited
> open files                      (-n) 65536
> pipe size            (512 bytes, -p) 8
> POSIX message queues     (bytes, -q) 819200
> real-time priority              (-r) 0
> stack size              (kbytes, -s) 10240
> cpu time               (seconds, -t) unlimited
> max user processes              (-u) 1024
> virtual memory          (kbytes, -v) unlimited
> file locks                      (-x) unlimited
> dmesg:
> Out of memory: Kill process 102774 (java) score 945 or sacrifice child
> Killed process 102774, UID 505, (java) total-vm:50435392kB, anon-rss:28966224kB, file-rss:20kB



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)