You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/02/07 16:16:00 UTC
[jira] [Commented] (ARTEMIS-3105) Large message file not closed on
backup side
[ https://issues.apache.org/jira/browse/ARTEMIS-3105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17280602#comment-17280602 ]
ASF subversion and git services commented on ARTEMIS-3105:
----------------------------------------------------------
Commit 9c5ec1b07ccf908adab1bec3c5ef7d3f778f94a4 in activemq-artemis's branch refs/heads/master from 岳豹
[ https://gitbox.apache.org/repos/asf?p=activemq-artemis.git;h=9c5ec1b ]
ARTEMIS-3105 large message file not closed on backup side
> Large message file not closed on backup side
> --------------------------------------------
>
> Key: ARTEMIS-3105
> URL: https://issues.apache.org/jira/browse/ARTEMIS-3105
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
> Affects Versions: 2.16.0
> Reporter: yuebao
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> We send a large number of large messages to the cluster, on the backup side, we found error message of too many open files. The detailed log is as follows:
> {code:java}
> 2021-02-06 17:40:29,793 WARN [org.apache.activemq.artemis.core.server] AMQ222010: Critical IO Error, shutting down the server. file=NIOSequentialFile /opt/apache/activemq/brokers/habroker01/./data/large-messages/21475041553.msg, message=/opt/apache/activemq/brokers/habroker01/./data/large-messages/21475041553.msg (Too many open files): ActiveMQIOErrorException[errorType=IO_ERROR message=/opt/apache/activemq/brokers/habroker01/./data/large-messages/21475041553.msg (Too many open files)]
> at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.open(NIOSequentialFile.java:87) [artemis-journal-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.open(NIOSequentialFile.java:73) [artemis-journal-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.persistence.impl.journal.LargeServerMessageImpl.openFile(LargeServerMessageImpl.java:385) [artemis-server-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.persistence.impl.journal.LargeServerMessageImpl.validateFile(LargeServerMessageImpl.java:364) [artemis-server-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.persistence.impl.journal.LargeServerMessageImpl.addBytes(LargeServerMessageImpl.java:120) [artemis-server-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.replication.ReplicationEndpoint.handleLargeMessageWrite(ReplicationEndpoint.java:623) [artemis-server-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.replication.ReplicationEndpoint.handlePacket(ReplicationEndpoint.java:216) [artemis-server-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:638) [artemis-core-client-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:392) [artemis-core-client-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:374) [artemis-core-client-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1148) [artemis-core-client-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:68) [artemis-core-client-2.4.0.jar:2.4.0]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:797) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:404) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:304) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_92-SUNINGJVM]
> Caused by: java.io.FileNotFoundException: /opt/apache/activemq/brokers/habroker01/./data/large-messages/21475041553.msg (Too many open files)
> at java.io.RandomAccessFile.open0(Native Method) [rt.jar:1.8.0_92-SUNINGJVM]
> at java.io.RandomAccessFile.open(RandomAccessFile.java:316) [rt.jar:1.8.0_92-SUNINGJVM]
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243) [rt.jar:1.8.0_92-SUNINGJVM]
> at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.open(NIOSequentialFile.java:79) [artemis-journal-2.4.0.jar:2.4.0]
> ... 28 more
> {code}
> We think that large message is released on live side but not released on the backup side, when receive the last chunk.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)