[jira] [Commented] (ARTEMIS-1036) Streaming huge messages between cluster nodes causes java.lang.OutOfMemoryError: Direct buffer memory

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARTEMIS-1036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15937815#comment-15937815 ] 

ASF GitHub Bot commented on ARTEMIS-1036:
-----------------------------------------

GitHub user franz1981 opened a pull request:

    https://github.com/apache/activemq-artemis/pull/1123

    ARTEMIS-1036 Streaming huge messages between cluster nodes causes java.lang.OutOfMemoryError: Direct buffer memory

    While streaming huge messages, it avoid any resize in the Netty buffers due to an wrong initial capacity, reducing the pressure on the Netty pool.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/franz1981/activemq-artemis oom_cluster_huge_msg

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/activemq-artemis/pull/1123.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1123
    
----
commit 1686b3545d14cdf591e00e6d04228b48b2b74a9f
Author: Francesco Nigro <ni...@gmail.com>
Date:   2017-03-15T15:59:57Z

    ARTEMIS-1036 Streaming huge messages between cluster nodes causes java.lang.OutOfMemoryError: Direct buffer memory

----


> Streaming huge messages between cluster nodes causes java.lang.OutOfMemoryError: Direct buffer memory
> -----------------------------------------------------------------------------------------------------
>
>                 Key: ARTEMIS-1036
>                 URL: https://issues.apache.org/jira/browse/ARTEMIS-1036
>             Project: ActiveMQ Artemis
>          Issue Type: Bug
>          Components: Broker
>            Reporter: Francesco Nigro
>            Assignee: Francesco Nigro
>
> _Scenario_
> * 2 servers are started in cluster.
> * Send one 1GB message to node1. Message is sent as described in subchapter [Streaming over JMS in Artemis docs | https://activemq.apache.org/artemis/docs/1.0.0/large-messages.html], it is sent like stream from file : 
> {code}
> BytesMessage message = session.createBytesMessage();
> message.setObjectProperty("JMS_AMQ_InputStream", bufferedInput);
> producer.send(message);
> {code}
> * Connect consumer on node2 and wait for message to redistribute to node2.
> * Receive 1GB stream message from node2 as described in subchapter [Streaming over JMS in Artemis docs | https://activemq.apache.org/artemis/docs/1.0.0/large-messages.html], it is saved to file. 
> {code}
>  BytesMessage messageReceived = (BytesMessage) consumer.receive(timeout);
> // This will block until the entire content is saved on disk
> messageReceived.setObjectProperty("JMS_AMQ_SaveStream", bufferedOutput);
> {code}
> _What is wrong_
> During message redistribution from node1 to node2 following error appears. It is thrown on node1.
> {code:title=OOM on node1 during message redistribution}
> 17:52:31,171 WARN  [org.apache.activemq.artemis.utils.OrderedExecutorFactory] (Thread-23 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@58d509f4-431458715)) failed to allocate 16777216 byte(s) of direct memory (used: 721420295, max: 734527488): io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 16777216 byte(s) of direct memory (used: 721420295, max: 734527488)
> 	at ...
> {code}
> _Notes_
> * This scenario only with one server (no cluster, just send/receive on one node) passes OK.
> * Message size is larger than server JVM memory. Artemis docs says following : "The only realistic limit to the size of a message that can be sent or consumed is the amount of disk space you have available".



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)