You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "Clebert Suconic (Jira)" <ji...@apache.org> on 2021/02/09 21:23:05 UTC
[jira] [Closed] (ARTEMIS-3105) Large message file not closed on
backup side
[ https://issues.apache.org/jira/browse/ARTEMIS-3105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Clebert Suconic closed ARTEMIS-3105.
------------------------------------
Fix Version/s: 2.17.0
Resolution: Fixed
> Large message file not closed on backup side
> --------------------------------------------
>
> Key: ARTEMIS-3105
> URL: https://issues.apache.org/jira/browse/ARTEMIS-3105
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
> Affects Versions: 2.16.0
> Reporter: yuebao
> Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> We send a large number of large messages to the cluster, on the backup side, we found error message of too many open files. The detailed log is as follows:
> {code:java}
> 2021-02-06 17:40:29,793 WARN [org.apache.activemq.artemis.core.server] AMQ222010: Critical IO Error, shutting down the server. file=NIOSequentialFile /opt/apache/activemq/brokers/habroker01/./data/large-messages/21475041553.msg, message=/opt/apache/activemq/brokers/habroker01/./data/large-messages/21475041553.msg (Too many open files): ActiveMQIOErrorException[errorType=IO_ERROR message=/opt/apache/activemq/brokers/habroker01/./data/large-messages/21475041553.msg (Too many open files)]
> at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.open(NIOSequentialFile.java:87) [artemis-journal-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.open(NIOSequentialFile.java:73) [artemis-journal-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.persistence.impl.journal.LargeServerMessageImpl.openFile(LargeServerMessageImpl.java:385) [artemis-server-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.persistence.impl.journal.LargeServerMessageImpl.validateFile(LargeServerMessageImpl.java:364) [artemis-server-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.persistence.impl.journal.LargeServerMessageImpl.addBytes(LargeServerMessageImpl.java:120) [artemis-server-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.replication.ReplicationEndpoint.handleLargeMessageWrite(ReplicationEndpoint.java:623) [artemis-server-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.replication.ReplicationEndpoint.handlePacket(ReplicationEndpoint.java:216) [artemis-server-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:638) [artemis-core-client-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:392) [artemis-core-client-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:374) [artemis-core-client-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1148) [artemis-core-client-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:68) [artemis-core-client-2.4.0.jar:2.4.0]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:797) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:404) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:304) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_92-SUNINGJVM]
> Caused by: java.io.FileNotFoundException: /opt/apache/activemq/brokers/habroker01/./data/large-messages/21475041553.msg (Too many open files)
> at java.io.RandomAccessFile.open0(Native Method) [rt.jar:1.8.0_92-SUNINGJVM]
> at java.io.RandomAccessFile.open(RandomAccessFile.java:316) [rt.jar:1.8.0_92-SUNINGJVM]
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243) [rt.jar:1.8.0_92-SUNINGJVM]
> at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.open(NIOSequentialFile.java:79) [artemis-journal-2.4.0.jar:2.4.0]
> ... 28 more
> {code}
> We think that large message is released on live side but not released on the backup side, when receive the last chunk.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)