You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "jk@penguinsfan.com (JIRA)" <ji...@apache.org> on 2006/12/01 04:37:02 UTC
[jira] Created: (AMQ-1082) Improve Kaha freeList
Improve Kaha freeList
---------------------
Key: AMQ-1082
URL: https://issues.apache.org/activemq/browse/AMQ-1082
Project: ActiveMQ
Issue Type: Improvement
Affects Versions: 4.2.0
Reporter: jk@penguinsfan.com
This is in kaha.impl.index.IndexManager.java.
The index currently maintains an in-memory free list. Each entry costs at least 51 bytes of RAM. Issue is that the more free space that there is in the file, the more RAM is used (oddly enough).
I am worried about the case where suppose a producer enqueues millions of messages and then stops. Now, suppose a consumer dequeues all of the messages (no producers are starting up during this time, so the free entries are not being reclaimed). Seems like this could use a huge amount of memory. Yes, 51 MB is not that much, but suppose there are a bunch of consumers and hence a ConsumerMessageRef for each one. (Maybe I'm missing something....)
Perhaps this linked list could be pushed into the Kaha store itself, by rewriting the IndexItems on disk to point previous/next free item, thus maintaining this on disk and in place. For perf reasons, maybe supplement with some caching but that might not really be necessary because the OS cache should help out.
A lot of the other places in Kaha (such as ListContainerImpl.java) have caches which limit to say 100 cached items, but the free list seems to obviate all of that stuff due to it being essentially unbounded.
Any thoughts?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/activemq/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (AMQ-1082) Improve Kaha freeList
Posted by "Albert Strasheim (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Albert Strasheim reopened AMQ-1082:
-----------------------------------
Hope I have the right issue here.
When running our fast producer, slow consumer unit test for a while, I'm getting the following stack trace. The missing file it is looking for has indeed been deleted. There's still plenty of free space on the disk.
ERROR DataManagerImpl - Looking for key 1 but not found in fileMap: {2=data-TopicSubscription-2 number = 2 , length = 33554006 refCount = 12755, 4=data-TopicSubscription-4 number = 4 , length = 33554158 refCount = 17435, 6=data-TopicSubscription-6 number = 6 , length = 20247053 refCount = 10538, 3=data-TopicSubscription-3 number = 3 , length = 33553411 refCount = 17037, 5=data-TopicSubscription-5 number = 5 , length = 33552256 refCount = 16291}
ERROR ListContainerImpl - Failed to get value for offset=102, key=(-1, -1, 0), value=(1, 2001, 2022), previousItem=-1, nextItem=204
java.io.IOException: Could not locate data file data-TopicSubscription-1
at org.apache.activemq.kaha.impl.data.DataManagerImpl.getDataFile(DataManagerImpl.java:121)
at org.apache.activemq.kaha.impl.data.SyncDataFileReader.readItem(SyncDataFileReader.java:60)
at org.apache.activemq.kaha.impl.data.DataManagerImpl.readItem(DataManagerImpl.java:130)
at org.apache.activemq.kaha.impl.container.ListContainerImpl.getValue(ListContainerImpl.java:841)
at org.apache.activemq.kaha.impl.container.ContainerValueCollectionIterator.next(ContainerValueCollectionIterator.java:48)
at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.next(FilePendingMessageCursor.java:186)
at org.apache.activemq.broker.region.TopicSubscription.dispatchMatched(TopicSubscription.java:347)
at org.apache.activemq.broker.region.TopicSubscription.acknowledge(TopicSubscription.java:191)
at org.apache.activemq.broker.region.AbstractRegion.acknowledge(AbstractRegion.java:312)
at org.apache.activemq.broker.region.RegionBroker.acknowledge(RegionBroker.java:401)
at org.apache.activemq.broker.TransactionBroker.acknowledge(TransactionBroker.java:181)
at org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:74)
at org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:74)
at org.apache.activemq.broker.MutableBrokerFilter.acknowledge(MutableBrokerFilter.java:88)
at org.apache.activemq.broker.TransportConnection.processMessageAck(TransportConnection.java:456)
at org.apache.activemq.command.MessageAck.visit(MessageAck.java:179)
at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:258)
at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:164)
at org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:95)
at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:65)
at org.apache.activemq.transport.vm.VMTransport.syncOneWay(VMTransport.java:99)
at org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:86)
at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
at org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1165)
at org.apache.activemq.ActiveMQSession.asyncSendPacket(ActiveMQSession.java:1647)
at org.apache.activemq.ActiveMQMessageConsumer.ackLater(ActiveMQMessageConsumer.java:748)
at org.apache.activemq.ActiveMQMessageConsumer.afterMessageIsConsumed(ActiveMQMessageConsumer.java:705)
at org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:871)
at org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:99)
at org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:166)
at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:111)
at org.apache.activemq.thread.PooledTaskRunner.access$100(PooledTaskRunner.java:26)
at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:44)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
ERROR Service - Async error occurred: org.apache.activemq.kaha.RuntimeStoreException: java.io.IOException: Could not locate data file data-TopicSubscription-1
org.apache.activemq.kaha.RuntimeStoreException: java.io.IOException: Could not locate data file data-TopicSubscription-1
at org.apache.activemq.kaha.impl.container.ListContainerImpl.getValue(ListContainerImpl.java:844)
at org.apache.activemq.kaha.impl.container.ContainerValueCollectionIterator.next(ContainerValueCollectionIterator.java:48)
at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.next(FilePendingMessageCursor.java:186)
at org.apache.activemq.broker.region.TopicSubscription.dispatchMatched(TopicSubscription.java:347)
at org.apache.activemq.broker.region.TopicSubscription.acknowledge(TopicSubscription.java:191)
at org.apache.activemq.broker.region.AbstractRegion.acknowledge(AbstractRegion.java:312)
at org.apache.activemq.broker.region.RegionBroker.acknowledge(RegionBroker.java:401)
at org.apache.activemq.broker.TransactionBroker.acknowledge(TransactionBroker.java:181)
at org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:74)
at org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:74)
at org.apache.activemq.broker.MutableBrokerFilter.acknowledge(MutableBrokerFilter.java:88)
at org.apache.activemq.broker.TransportConnection.processMessageAck(TransportConnection.java:456)
at org.apache.activemq.command.MessageAck.visit(MessageAck.java:179)
at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:258)
at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:164)
at org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:95)
at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:65)
at org.apache.activemq.transport.vm.VMTransport.syncOneWay(VMTransport.java:99)
at org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:86)
at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
at org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1165)
at org.apache.activemq.ActiveMQSession.asyncSendPacket(ActiveMQSession.java:1647)
at org.apache.activemq.ActiveMQMessageConsumer.ackLater(ActiveMQMessageConsumer.java:748)
at org.apache.activemq.ActiveMQMessageConsumer.afterMessageIsConsumed(ActiveMQMessageConsumer.java:705)
at org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:871)
at org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:99)
at org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:166)
at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:111)
at org.apache.activemq.thread.PooledTaskRunner.access$100(PooledTaskRunner.java:26)
at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:44)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
Caused by: java.io.IOException: Could not locate data file data-TopicSubscription-1
at org.apache.activemq.kaha.impl.data.DataManagerImpl.getDataFile(DataManagerImpl.java:121)
at org.apache.activemq.kaha.impl.data.SyncDataFileReader.readItem(SyncDataFileReader.java:60)
at org.apache.activemq.kaha.impl.data.DataManagerImpl.readItem(DataManagerImpl.java:130)
at org.apache.activemq.kaha.impl.container.ListContainerImpl.getValue(ListContainerImpl.java:841)
... 33 more
java.io.IOException: Could not locate data file data-TopicSubscription-1
javax.jms.JMSException: java.io.IOException: Could not locate data file data-TopicSubscription-1
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:46)
at org.apache.activemq.ActiveMQConnection.onAsyncException(ActiveMQConnection.java:1527)
at org.apache.activemq.ActiveMQConnection$2.run(ActiveMQConnection.java:1500)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
Caused by: org.apache.activemq.kaha.RuntimeStoreException: java.io.IOException: Could not locate data file data-TopicSubscription-1
at org.apache.activemq.kaha.impl.container.ListContainerImpl.getValue(ListContainerImpl.java:844)
at org.apache.activemq.kaha.impl.container.ContainerValueCollectionIterator.next(ContainerValueCollectionIterator.java:48)
at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.next(FilePendingMessageCursor.java:186)
at org.apache.activemq.broker.region.TopicSubscription.dispatchMatched(TopicSubscription.java:347)
at org.apache.activemq.broker.region.TopicSubscription.acknowledge(TopicSubscription.java:191)
at org.apache.activemq.broker.region.AbstractRegion.acknowledge(AbstractRegion.java:312)
at org.apache.activemq.broker.region.RegionBroker.acknowledge(RegionBroker.java:401)
at org.apache.activemq.broker.TransactionBroker.acknowledge(TransactionBroker.java:181)
at org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:74)
at org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:74)
at org.apache.activemq.broker.MutableBrokerFilter.acknowledge(MutableBrokerFilter.java:88)
at org.apache.activemq.broker.TransportConnection.processMessageAck(TransportConnection.java:456)
at org.apache.activemq.command.MessageAck.visit(MessageAck.java:179)
at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:258)
at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:164)
at org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:95)
at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:65)
at org.apache.activemq.transport.vm.VMTransport.syncOneWay(VMTransport.java:99)
at org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:86)
at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
at org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1165)
at org.apache.activemq.ActiveMQSession.asyncSendPacket(ActiveMQSession.java:1647)
at org.apache.activemq.ActiveMQMessageConsumer.ackLater(ActiveMQMessageConsumer.java:748)
at org.apache.activemq.ActiveMQMessageConsumer.afterMessageIsConsumed(ActiveMQMessageConsumer.java:705)
at org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:871)
at org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:99)
at org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:166)
at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:111)
at org.apache.activemq.thread.PooledTaskRunner.access$100(PooledTaskRunner.java:26)
at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:44)
... 3 more
Caused by: java.io.IOException: Could not locate data file data-TopicSubscription-1
at org.apache.activemq.kaha.impl.data.DataManagerImpl.getDataFile(DataManagerImpl.java:121)
at org.apache.activemq.kaha.impl.data.SyncDataFileReader.readItem(SyncDataFileReader.java:60)
at org.apache.activemq.kaha.impl.data.DataManagerImpl.readItem(DataManagerImpl.java:130)
at org.apache.activemq.kaha.impl.container.ListContainerImpl.getValue(ListContainerImpl.java:841)
... 33 more
> Improve Kaha freeList
> ---------------------
>
> Key: AMQ-1082
> URL: https://issues.apache.org/activemq/browse/AMQ-1082
> Project: ActiveMQ
> Issue Type: Improvement
> Affects Versions: 4.2.0
> Reporter: jk@penguinsfan.com
> Assigned To: Rob Davies
> Fix For: 4.2.0
>
>
> This is in kaha.impl.index.IndexManager.java.
> The index currently maintains an in-memory free list. Each entry costs at least 51 bytes of RAM. Issue is that the more free space that there is in the file, the more RAM is used (oddly enough).
> I am worried about the case where suppose a producer enqueues millions of messages and then stops. Now, suppose a consumer dequeues all of the messages (no producers are starting up during this time, so the free entries are not being reclaimed). Seems like this could use a huge amount of memory. Yes, 51 MB is not that much, but suppose there are a bunch of consumers and hence a ConsumerMessageRef for each one. (Maybe I'm missing something....)
> Perhaps this linked list could be pushed into the Kaha store itself, by rewriting the IndexItems on disk to point previous/next free item, thus maintaining this on disk and in place. For perf reasons, maybe supplement with some caching but that might not really be necessary because the OS cache should help out.
> A lot of the other places in Kaha (such as ListContainerImpl.java) have caches which limit to say 100 cached items, but the free list seems to obviate all of that stuff due to it being essentially unbounded.
> Any thoughts?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (AMQ-1082) Improve Kaha freeList
Posted by "Rob Davies (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rob Davies resolved AMQ-1082.
-----------------------------
Resolution: Fixed
Fix Version/s: 4.2.0
Fixed by SVN revision 499798
> Improve Kaha freeList
> ---------------------
>
> Key: AMQ-1082
> URL: https://issues.apache.org/activemq/browse/AMQ-1082
> Project: ActiveMQ
> Issue Type: Improvement
> Affects Versions: 4.2.0
> Reporter: jk@penguinsfan.com
> Assigned To: Rob Davies
> Fix For: 4.2.0
>
>
> This is in kaha.impl.index.IndexManager.java.
> The index currently maintains an in-memory free list. Each entry costs at least 51 bytes of RAM. Issue is that the more free space that there is in the file, the more RAM is used (oddly enough).
> I am worried about the case where suppose a producer enqueues millions of messages and then stops. Now, suppose a consumer dequeues all of the messages (no producers are starting up during this time, so the free entries are not being reclaimed). Seems like this could use a huge amount of memory. Yes, 51 MB is not that much, but suppose there are a bunch of consumers and hence a ConsumerMessageRef for each one. (Maybe I'm missing something....)
> Perhaps this linked list could be pushed into the Kaha store itself, by rewriting the IndexItems on disk to point previous/next free item, thus maintaining this on disk and in place. For perf reasons, maybe supplement with some caching but that might not really be necessary because the OS cache should help out.
> A lot of the other places in Kaha (such as ListContainerImpl.java) have caches which limit to say 100 cached items, but the free list seems to obviate all of that stuff due to it being essentially unbounded.
> Any thoughts?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (AMQ-1082) Improve Kaha freeList
Posted by "Rob Davies (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rob Davies reassigned AMQ-1082:
-------------------------------
Assignee: Rob Davies
> Improve Kaha freeList
> ---------------------
>
> Key: AMQ-1082
> URL: https://issues.apache.org/activemq/browse/AMQ-1082
> Project: ActiveMQ
> Issue Type: Improvement
> Affects Versions: 4.2.0
> Reporter: jk@penguinsfan.com
> Assigned To: Rob Davies
>
> This is in kaha.impl.index.IndexManager.java.
> The index currently maintains an in-memory free list. Each entry costs at least 51 bytes of RAM. Issue is that the more free space that there is in the file, the more RAM is used (oddly enough).
> I am worried about the case where suppose a producer enqueues millions of messages and then stops. Now, suppose a consumer dequeues all of the messages (no producers are starting up during this time, so the free entries are not being reclaimed). Seems like this could use a huge amount of memory. Yes, 51 MB is not that much, but suppose there are a bunch of consumers and hence a ConsumerMessageRef for each one. (Maybe I'm missing something....)
> Perhaps this linked list could be pushed into the Kaha store itself, by rewriting the IndexItems on disk to point previous/next free item, thus maintaining this on disk and in place. For perf reasons, maybe supplement with some caching but that might not really be necessary because the OS cache should help out.
> A lot of the other places in Kaha (such as ListContainerImpl.java) have caches which limit to say 100 cached items, but the free list seems to obviate all of that stuff due to it being essentially unbounded.
> Any thoughts?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.