You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Bill Wan (JIRA)" <ji...@apache.org> on 2012/11/12 09:36:14 UTC
[jira] [Created] (AMQ-4170) Crash when clean inactive destination:
java.io.EOFException: Chunk stream does not exist at page
Bill Wan created AMQ-4170:
-----------------------------
Summary: Crash when clean inactive destination: java.io.EOFException: Chunk stream does not exist at page
Key: AMQ-4170
URL: https://issues.apache.org/jira/browse/AMQ-4170
Project: ActiveMQ
Issue Type: Bug
Affects Versions: 5.5.1
Reporter: Bill Wan
I add below configuration to clean inactive destination and expired message for every 2 hours
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" producerFlowControl="true" memoryLimit="1mb" gcInactiveDestinations="true" inactiveTimoutBeforeGC="7200000" expireMessagesPeriod="7200000">
<pendingSubscriberPolicy>
<vmCursor />
</pendingSubscriberPolicy>
</policyEntry>
<policyEntry queue=">" producerFlowControl="true" memoryLimit="1mb" gcInactiveDestinations="true" inactiveTimoutBeforeGC="7200000" expireMessagesPeriod="7200000">
<pendingQueuePolicy>
<vmQueueCursor/>
</pendingQueuePolicy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
After MQ run for several days, it will crash when scheduler do such clean job, the crash exception is:
2012-11-12 01:03:58,917 | INFO | 5099ae91f17021cb20b0b157 Inactive for longer than 7200000 ms - removing ... | org.apache.activemq.broker.region.Queue | ActiveMQ Broker[broker-cluster-194] Scheduler
2012-11-12 01:04:01,907 | INFO | Transport failed: java.net.SocketException: Connection reset | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///183.177.33.93:39617
2012-11-12 01:04:02,288 | ERROR | KahaDB failed to store to Journal | org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Broker[broker-cluster-194] Scheduler
java.io.EOFException: Chunk stream does not exist at page: 1868
at org.apache.kahadb.page.Transaction$2.readPage(Transaction.java:454)
at org.apache.kahadb.page.Transaction$2.<init>(Transaction.java:431)
at org.apache.kahadb.page.Transaction.openInputStream(Transaction.java:428)
at org.apache.kahadb.page.Transaction.load(Transaction.java:404)
at org.apache.kahadb.page.Transaction.load(Transaction.java:361)
at org.apache.kahadb.index.BTreeIndex.loadNode(BTreeIndex.java:262)
at org.apache.kahadb.index.BTreeIndex.getRoot(BTreeIndex.java:174)
at org.apache.kahadb.index.BTreeIndex.clear(BTreeIndex.java:202)
at org.apache.activemq.store.kahadb.MessageDatabase$MessageOrderIndex.remove(MessageDatabase.java:2177)
at org.apache.activemq.store.kahadb.MessageDatabase.updateIndex(MessageDatabase.java:1105)
at org.apache.activemq.store.kahadb.MessageDatabase$16.execute(MessageDatabase.java:936)
at org.apache.kahadb.page.Transaction.execute(Transaction.java:728)
at org.apache.activemq.store.kahadb.MessageDatabase.process(MessageDatabase.java:934)
at org.apache.activemq.store.kahadb.MessageDatabase$13.visit(MessageDatabase.java:884)
at org.apache.activemq.store.kahadb.data.KahaRemoveDestinationCommand.visit(KahaRemoveDestinationCommand.java:130)
at org.apache.activemq.store.kahadb.MessageDatabase.process(MessageDatabase.java:856)
at org.apache.activemq.store.kahadb.MessageDatabase.store(MessageDatabase.java:783)
at org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore.removeAllMessages(KahaDBStore.java:384)
at org.apache.activemq.store.ProxyMessageStore.removeAllMessages(ProxyMessageStore.java:56)
at org.apache.activemq.broker.region.BaseDestination.dispose(BaseDestination.java:503)
at org.apache.activemq.broker.region.AbstractRegion.dispose(AbstractRegion.java:511)
at org.apache.activemq.broker.region.AbstractRegion.removeDestination(AbstractRegion.java:198)
at org.apache.activemq.broker.jmx.ManagedQueueRegion.removeDestination(ManagedQueueRegion.java:62)
at org.apache.activemq.broker.region.RegionBroker.removeDestination(RegionBroker.java:324)
at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:149)
at org.apache.activemq.advisory.AdvisoryBroker.removeDestination(AdvisoryBroker.java:173)
at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:149)
at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:149)
at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:149)
at org.apache.activemq.broker.MutableBrokerFilter.removeDestination(MutableBrokerFilter.java:155)
at org.apache.activemq.broker.region.RegionBroker.purgeInactiveDestinations(RegionBroker.java:959)
at org.apache.activemq.broker.region.RegionBroker$1.run(RegionBroker.java:110)
at org.apache.activemq.thread.SchedulerTimerTask.run(SchedulerTimerTask.java:33)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQ-4170) Crash when clean inactive
destination: java.io.EOFException: Chunk stream does not exist at page
Posted by "Timothy Bish (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506852#comment-13506852 ]
Timothy Bish commented on AMQ-4170:
-----------------------------------
Do you have a test case that reproduce this?
Have you tried with a newer broker version? There's been a lot of work done since v5.5.1 to harden KahaDB.
> Crash when clean inactive destination: java.io.EOFException: Chunk stream does not exist at page
> -------------------------------------------------------------------------------------------------
>
> Key: AMQ-4170
> URL: https://issues.apache.org/jira/browse/AMQ-4170
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.5.1
> Reporter: Bill Wan
> Attachments: activemq.log
>
>
> I add below configuration to clean inactive destination and expired message for every 2 hours
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry topic=">" producerFlowControl="true" memoryLimit="1mb" gcInactiveDestinations="true" inactiveTimoutBeforeGC="7200000" expireMessagesPeriod="7200000">
> <pendingSubscriberPolicy>
> <vmCursor />
> </pendingSubscriberPolicy>
> </policyEntry>
> <policyEntry queue=">" producerFlowControl="true" memoryLimit="1mb" gcInactiveDestinations="true" inactiveTimoutBeforeGC="7200000" expireMessagesPeriod="7200000">
> <pendingQueuePolicy>
> <vmQueueCursor/>
> </pendingQueuePolicy>
> </policyEntry>
> </policyEntries>
> </policyMap>
> </destinationPolicy>
> After MQ run for several days, it will crash when scheduler do such clean job, the crash exception is:
> 2012-11-12 01:03:58,917 | INFO | 5099ae91f17021cb20b0b157 Inactive for longer than 7200000 ms - removing ... | org.apache.activemq.broker.region.Queue | ActiveMQ Broker[broker-cluster-194] Scheduler
> 2012-11-12 01:04:01,907 | INFO | Transport failed: java.net.SocketException: Connection reset | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///183.177.33.93:39617
> 2012-11-12 01:04:02,288 | ERROR | KahaDB failed to store to Journal | org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Broker[broker-cluster-194] Scheduler
> java.io.EOFException: Chunk stream does not exist at page: 1868
> at org.apache.kahadb.page.Transaction$2.readPage(Transaction.java:454)
> at org.apache.kahadb.page.Transaction$2.<init>(Transaction.java:431)
> at org.apache.kahadb.page.Transaction.openInputStream(Transaction.java:428)
> at org.apache.kahadb.page.Transaction.load(Transaction.java:404)
> at org.apache.kahadb.page.Transaction.load(Transaction.java:361)
> at org.apache.kahadb.index.BTreeIndex.loadNode(BTreeIndex.java:262)
> at org.apache.kahadb.index.BTreeIndex.getRoot(BTreeIndex.java:174)
> at org.apache.kahadb.index.BTreeIndex.clear(BTreeIndex.java:202)
> at org.apache.activemq.store.kahadb.MessageDatabase$MessageOrderIndex.remove(MessageDatabase.java:2177)
> at org.apache.activemq.store.kahadb.MessageDatabase.updateIndex(MessageDatabase.java:1105)
> at org.apache.activemq.store.kahadb.MessageDatabase$16.execute(MessageDatabase.java:936)
> at org.apache.kahadb.page.Transaction.execute(Transaction.java:728)
> at org.apache.activemq.store.kahadb.MessageDatabase.process(MessageDatabase.java:934)
> at org.apache.activemq.store.kahadb.MessageDatabase$13.visit(MessageDatabase.java:884)
> at org.apache.activemq.store.kahadb.data.KahaRemoveDestinationCommand.visit(KahaRemoveDestinationCommand.java:130)
> at org.apache.activemq.store.kahadb.MessageDatabase.process(MessageDatabase.java:856)
> at org.apache.activemq.store.kahadb.MessageDatabase.store(MessageDatabase.java:783)
> at org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore.removeAllMessages(KahaDBStore.java:384)
> at org.apache.activemq.store.ProxyMessageStore.removeAllMessages(ProxyMessageStore.java:56)
> at org.apache.activemq.broker.region.BaseDestination.dispose(BaseDestination.java:503)
> at org.apache.activemq.broker.region.AbstractRegion.dispose(AbstractRegion.java:511)
> at org.apache.activemq.broker.region.AbstractRegion.removeDestination(AbstractRegion.java:198)
> at org.apache.activemq.broker.jmx.ManagedQueueRegion.removeDestination(ManagedQueueRegion.java:62)
> at org.apache.activemq.broker.region.RegionBroker.removeDestination(RegionBroker.java:324)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:149)
> at org.apache.activemq.advisory.AdvisoryBroker.removeDestination(AdvisoryBroker.java:173)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:149)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:149)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:149)
> at org.apache.activemq.broker.MutableBrokerFilter.removeDestination(MutableBrokerFilter.java:155)
> at org.apache.activemq.broker.region.RegionBroker.purgeInactiveDestinations(RegionBroker.java:959)
> at org.apache.activemq.broker.region.RegionBroker$1.run(RegionBroker.java:110)
> at org.apache.activemq.thread.SchedulerTimerTask.run(SchedulerTimerTask.java:33)
> at java.util.TimerThread.mainLoop(Timer.java:555)
> at java.util.TimerThread.run(Timer.java:505)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AMQ-4170) Crash when clean inactive destination:
java.io.EOFException: Chunk stream does not exist at page
Posted by "Bill Wan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQ-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Wan updated AMQ-4170:
--------------------------
Attachment: activemq.log
add exception log
> Crash when clean inactive destination: java.io.EOFException: Chunk stream does not exist at page
> -------------------------------------------------------------------------------------------------
>
> Key: AMQ-4170
> URL: https://issues.apache.org/jira/browse/AMQ-4170
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.5.1
> Reporter: Bill Wan
> Attachments: activemq.log
>
>
> I add below configuration to clean inactive destination and expired message for every 2 hours
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry topic=">" producerFlowControl="true" memoryLimit="1mb" gcInactiveDestinations="true" inactiveTimoutBeforeGC="7200000" expireMessagesPeriod="7200000">
> <pendingSubscriberPolicy>
> <vmCursor />
> </pendingSubscriberPolicy>
> </policyEntry>
> <policyEntry queue=">" producerFlowControl="true" memoryLimit="1mb" gcInactiveDestinations="true" inactiveTimoutBeforeGC="7200000" expireMessagesPeriod="7200000">
> <pendingQueuePolicy>
> <vmQueueCursor/>
> </pendingQueuePolicy>
> </policyEntry>
> </policyEntries>
> </policyMap>
> </destinationPolicy>
> After MQ run for several days, it will crash when scheduler do such clean job, the crash exception is:
> 2012-11-12 01:03:58,917 | INFO | 5099ae91f17021cb20b0b157 Inactive for longer than 7200000 ms - removing ... | org.apache.activemq.broker.region.Queue | ActiveMQ Broker[broker-cluster-194] Scheduler
> 2012-11-12 01:04:01,907 | INFO | Transport failed: java.net.SocketException: Connection reset | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///183.177.33.93:39617
> 2012-11-12 01:04:02,288 | ERROR | KahaDB failed to store to Journal | org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Broker[broker-cluster-194] Scheduler
> java.io.EOFException: Chunk stream does not exist at page: 1868
> at org.apache.kahadb.page.Transaction$2.readPage(Transaction.java:454)
> at org.apache.kahadb.page.Transaction$2.<init>(Transaction.java:431)
> at org.apache.kahadb.page.Transaction.openInputStream(Transaction.java:428)
> at org.apache.kahadb.page.Transaction.load(Transaction.java:404)
> at org.apache.kahadb.page.Transaction.load(Transaction.java:361)
> at org.apache.kahadb.index.BTreeIndex.loadNode(BTreeIndex.java:262)
> at org.apache.kahadb.index.BTreeIndex.getRoot(BTreeIndex.java:174)
> at org.apache.kahadb.index.BTreeIndex.clear(BTreeIndex.java:202)
> at org.apache.activemq.store.kahadb.MessageDatabase$MessageOrderIndex.remove(MessageDatabase.java:2177)
> at org.apache.activemq.store.kahadb.MessageDatabase.updateIndex(MessageDatabase.java:1105)
> at org.apache.activemq.store.kahadb.MessageDatabase$16.execute(MessageDatabase.java:936)
> at org.apache.kahadb.page.Transaction.execute(Transaction.java:728)
> at org.apache.activemq.store.kahadb.MessageDatabase.process(MessageDatabase.java:934)
> at org.apache.activemq.store.kahadb.MessageDatabase$13.visit(MessageDatabase.java:884)
> at org.apache.activemq.store.kahadb.data.KahaRemoveDestinationCommand.visit(KahaRemoveDestinationCommand.java:130)
> at org.apache.activemq.store.kahadb.MessageDatabase.process(MessageDatabase.java:856)
> at org.apache.activemq.store.kahadb.MessageDatabase.store(MessageDatabase.java:783)
> at org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore.removeAllMessages(KahaDBStore.java:384)
> at org.apache.activemq.store.ProxyMessageStore.removeAllMessages(ProxyMessageStore.java:56)
> at org.apache.activemq.broker.region.BaseDestination.dispose(BaseDestination.java:503)
> at org.apache.activemq.broker.region.AbstractRegion.dispose(AbstractRegion.java:511)
> at org.apache.activemq.broker.region.AbstractRegion.removeDestination(AbstractRegion.java:198)
> at org.apache.activemq.broker.jmx.ManagedQueueRegion.removeDestination(ManagedQueueRegion.java:62)
> at org.apache.activemq.broker.region.RegionBroker.removeDestination(RegionBroker.java:324)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:149)
> at org.apache.activemq.advisory.AdvisoryBroker.removeDestination(AdvisoryBroker.java:173)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:149)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:149)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:149)
> at org.apache.activemq.broker.MutableBrokerFilter.removeDestination(MutableBrokerFilter.java:155)
> at org.apache.activemq.broker.region.RegionBroker.purgeInactiveDestinations(RegionBroker.java:959)
> at org.apache.activemq.broker.region.RegionBroker$1.run(RegionBroker.java:110)
> at org.apache.activemq.thread.SchedulerTimerTask.run(SchedulerTimerTask.java:33)
> at java.util.TimerThread.mainLoop(Timer.java:555)
> at java.util.TimerThread.run(Timer.java:505)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira