You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2015/10/16 17:53:05 UTC
[jira] [Commented] (ARTEMIS-262) Core Bridge throws OutOfMemory
exception when transferring large messages.
[ https://issues.apache.org/jira/browse/ARTEMIS-262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14960919#comment-14960919 ]
ASF GitHub Bot commented on ARTEMIS-262:
----------------------------------------
GitHub user mtaylor opened a pull request:
https://github.com/apache/activemq-artemis/pull/199
ARTEMIS-262 Fix Bridge OOM exception
Netty 4.x uses pooled buffers. These buffers can run out of memory when
transferring large amounts of data over connection. This was causing an
OutOfMemory exception to be thrown on the CoreBridge when tranferring
large messages. Netty provides a callback handler to notify listeners
when a Connection is writable. This patch adds the ability to register
connection writable listeners to the Netty connection and registers the
relevant callback from the Bridge to avoid writing when the buffers are
full.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mtaylor/activemq-artemis ARTEMIS-262
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/activemq-artemis/pull/199.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #199
----
commit 11228e6cf8d26e84cf2ad01867a8dcb1f47080aa
Author: Andy Taylor <an...@gmail.com>
Date: 2015-10-15T13:39:51Z
ARTEMIS-262 Fix Bridge OOM exception
Netty 4.x uses pooled buffers. These buffers can run out of memory when
transferring large amounts of data over connection. This was causing an
OutOfMemory exception to be thrown on the CoreBridge when tranferring
large messages. Netty provides a callback handler to notify listeners
when a Connection is writable. This patch adds the ability to register
connection writable listeners to the Netty connection and registers the
relevant callback from the Bridge to avoid writing when the buffers are
full.
----
> Core Bridge throws OutOfMemory exception when transferring large messages.
> --------------------------------------------------------------------------
>
> Key: ARTEMIS-262
> URL: https://issues.apache.org/jira/browse/ARTEMIS-262
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
> Affects Versions: 1.1.0
> Reporter: Martyn Taylor
> Fix For: 1.1.1
>
>
> To reproduce:
> 1. Start two servers
> 2. Send 100 messages with size 5MB to the first server
> 3. Restart the first server
> 4. Create a core bridge, which will forward messages to the second server
> 5. Receive messages from the second server
> The following exception is thrown:
> 16:02:52,112 ERROR [org.apache.activemq.artemis.core.client] (Thread-23 (ActiveMQ-server-ActiveMQServerImpl::serverUUID=1d6a38c6-44e8-11e5-9786-154a71a87770-460139716)) AMQ214017: Caught unexpected Throwable: java.lang.OutOfMemoryError: Direct buffer memory
> at java.nio.Bits.reserveMemory(Bits.java:658) [rt.jar:1.8.0_51]
> at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) [rt.jar:1.8.0_51]
> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311) [rt.jar:1.8.0_51]
> at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:437) [netty-all-4.0.26.Final.jar:4.0.26.Final]
> at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:179) [netty-all-4.0.26.Final.jar:4.0.26.Final]
> at io.netty.buffer.PoolArena.allocate(PoolArena.java:168) [netty-all-4.0.26.Final.jar:4.0.26.Final]
> at io.netty.buffer.PoolArena.reallocate(PoolArena.java:280) [netty-all-4.0.26.Final.jar:4.0.26.Final]
> at io.netty.buffer.PooledByteBuf.capacity(PooledByteBuf.java:110) [netty-all-4.0.26.Final.jar:4.0.26.Final]
> at io.netty.buffer.AbstractByteBuf.ensureWritable(AbstractByteBuf.java:251) [netty-all-4.0.26.Final.jar:4.0.26.Final]
> at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:817) [netty-all-4.0.26.Final.jar:4.0.26.Final]
> at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:825) [netty-all-4.0.26.Final.jar:4.0.26.Final]
> at org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.writeBytes(ChannelBufferWrapper.java:575) [artemis-commons-1.0.0.jar:1.0.0]
> at org.apache.activemq.artemis.core.protocol.core.impl.wireformat.SessionContinuationMessage.encodeRest(SessionContinuationMessage.java:76) [artemis-core-client-1.0.0.jar:1.0.0]
> at org.apache.activemq.artemis.core.protocol.core.impl.wireformat.SessionSendContinuationMessage.encodeRest(SessionSendContinuationMessage.java:100) [artemis-core-client-1.0.0.jar:1.0.0]
> at org.apache.activemq.artemis.core.protocol.core.impl.PacketImpl.encode(PacketImpl.java:283) [artemis-core-client-1.0.0.jar:1.0.0]
> at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.send(ChannelImpl.java:246) [artemis-core-client-1.0.0.jar:1.0.0]
> at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.send(ChannelImpl.java:216) [artemis-core-client-1.0.0.jar:1.0.0]
> at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQSessionContext.sendLargeMessageChunk(ActiveMQSessionContext.java:441) [artemis-core-client-1.0.0.jar:1.0.0]
> at org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.largeMessageSendServer(ClientProducerImpl.java:433) [artemis-core-client-1.0.0.jar:1.0.0]
> at org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.largeMessageSend(ClientProducerImpl.java:367) [artemis-core-client-1.0.0.jar:1.0.0]
> at org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.doSend(ClientProducerImpl.java:297) [artemis-core-client-1.0.0.jar:1.0.0]
> at org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:132) [artemis-core-client-1.0.0.jar:1.0.0]
> at org.apache.activemq.artemis.core.server.cluster.impl.BridgeImpl$2.run(BridgeImpl.java:711) [artemis-server-1.0.0.jar:1.0.0]
> at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:105) [artemis-core-client-1.0.0.jar:1.0.0]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_51]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_51]
> at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_51]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)