You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/02/06 11:34:00 UTC
[jira] [Work logged] (ARTEMIS-3105) Large message file not closed
on backup side
[ https://issues.apache.org/jira/browse/ARTEMIS-3105?focusedWorklogId=549020&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549020 ]
ASF GitHub Bot logged work on ARTEMIS-3105:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 06/Feb/21 11:33
Start Date: 06/Feb/21 11:33
Worklog Time Spent: 10m
Work Description: CNNJYB opened a new pull request #3435:
URL: https://github.com/apache/activemq-artemis/pull/3435
We send a large number of large messages to the cluster, on the backup side, we found error message of too many open files.
We think that large message is released on live side but not released on the backup side, when receive the last chunk.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 549020)
Remaining Estimate: 0h
Time Spent: 10m
> Large message file not closed on backup side
> --------------------------------------------
>
> Key: ARTEMIS-3105
> URL: https://issues.apache.org/jira/browse/ARTEMIS-3105
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
> Affects Versions: 2.16.0
> Reporter: yuebao
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> We send a large number of large messages to the cluster, on the backup side, we found error message of too many open files. The detailed log is as follows:
> {code:java}
> 2021-02-06 17:40:29,793 WARN [org.apache.activemq.artemis.core.server] AMQ222010: Critical IO Error, shutting down the server. file=NIOSequentialFile /opt/apache/activemq/brokers/habroker01/./data/large-messages/21475041553.msg, message=/opt/apache/activemq/brokers/habroker01/./data/large-messages/21475041553.msg (Too many open files): ActiveMQIOErrorException[errorType=IO_ERROR message=/opt/apache/activemq/brokers/habroker01/./data/large-messages/21475041553.msg (Too many open files)]
> at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.open(NIOSequentialFile.java:87) [artemis-journal-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.open(NIOSequentialFile.java:73) [artemis-journal-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.persistence.impl.journal.LargeServerMessageImpl.openFile(LargeServerMessageImpl.java:385) [artemis-server-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.persistence.impl.journal.LargeServerMessageImpl.validateFile(LargeServerMessageImpl.java:364) [artemis-server-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.persistence.impl.journal.LargeServerMessageImpl.addBytes(LargeServerMessageImpl.java:120) [artemis-server-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.replication.ReplicationEndpoint.handleLargeMessageWrite(ReplicationEndpoint.java:623) [artemis-server-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.replication.ReplicationEndpoint.handlePacket(ReplicationEndpoint.java:216) [artemis-server-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:638) [artemis-core-client-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:392) [artemis-core-client-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:374) [artemis-core-client-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1148) [artemis-core-client-2.4.0.jar:2.4.0]
> at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:68) [artemis-core-client-2.4.0.jar:2.4.0]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:797) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:404) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:304) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-all-4.1.16.Final.jar:4.1.16.Final]
> at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_92-SUNINGJVM]
> Caused by: java.io.FileNotFoundException: /opt/apache/activemq/brokers/habroker01/./data/large-messages/21475041553.msg (Too many open files)
> at java.io.RandomAccessFile.open0(Native Method) [rt.jar:1.8.0_92-SUNINGJVM]
> at java.io.RandomAccessFile.open(RandomAccessFile.java:316) [rt.jar:1.8.0_92-SUNINGJVM]
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243) [rt.jar:1.8.0_92-SUNINGJVM]
> at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.open(NIOSequentialFile.java:79) [artemis-journal-2.4.0.jar:2.4.0]
> ... 28 more
> {code}
> We think that large message is released on live side but not released on the backup size, when receive the last chunk.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)