[jira] Commented: (AMQ-1797) persistent messages for durable subscribers are not purged from disc storage

["Juraj Kuruc (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/activemq/browse/AMQ-1797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=49976#action_49976 ] 

Juraj Kuruc commented on AMQ-1797:
----------------------------------

Tests on amq revision number 729038 with amqPersistenceAdapter as well as with kahaPersistenceAdapter but none of them was sucessful
from different reasons.

With amqPersistenceAdapter, 100k msgs have been sent and data directory size is 1.1 GB. 
Further message sending make any difference - message store still grows. For addition following error occures several times:
ERROR RecoveryListenerAdapter        - Message id ... could not be recovered from the data store - already dispatched

Using kahaPersistenceAdapter, after 100k msgs one of the consumer receives 100k second 99999 msgs.After another 300k msgs data directory size is 410MB. Also some errors are logged by AMQ:

ERROR DataManagerImpl                - Looking for key 15 but not found in fileMap: {17=data-queue-data-17 number = 17 , length = 177868 refCount = 6, 16=data-queue-data-16 number = 16 , length = 33546816 refCount = 86}
ERROR MapContainerImpl               - Failed to get value for offset=4845, key=(15, 29641420, 47), value=(15, 29641472, 8839), previousItem=-1, nextItem=-1
java.io.IOException: Could not locate data file data-queue-data-15
        at org.apache.activemq.kaha.impl.data.DataManagerImpl.getDataFile(DataManagerImpl.java:129)
        at org.apache.activemq.kaha.impl.data.SyncDataFileReader.readItem(SyncDataFileReader.java:65)
        at org.apache.activemq.kaha.impl.data.DataManagerImpl.readItem(DataManagerImpl.java:141)
        at org.apache.activemq.kaha.impl.container.MapContainerImpl.getValue(MapContainerImpl.java:481)
        at org.apache.activemq.store.kahadaptor.KahaMessageStore.recoverNextMessages(KahaMessageStore.java:166)
        at org.apache.activemq.store.ProxyMessageStore.recoverNextMessages(ProxyMessageStore.java:83)
        at org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:92)
        at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:236)
        at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.hasNext(AbstractStoreCursor.java:136)
        at org.apache.activemq.broker.region.cursors.StoreQueueCursor.hasNext(StoreQueueCursor.java:131)
        at org.apache.activemq.broker.region.Queue.doPageIn(Queue.java:1180)
        at org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:1308)
        at org.apache.activemq.broker.region.Queue.iterate(Queue.java:1008)
        at org.apache.activemq.thread.DeterministicTaskRunner.runTask(DeterministicTaskRunner.java:84)
        at org.apache.activemq.thread.DeterministicTaskRunner$1.run(DeterministicTaskRunner.java:41)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:619)
ERROR AbstractStoreCursor            - Failed to fill batch for more see atached log - kaha_amq.log

For this tests it was necessary to adjust memory usage and limit settings - view attached activemq.xml.
Tested on SUSE Linux Enterprise Server 10.


> persistent messages for durable subscribers are not purged from disc storage
> ----------------------------------------------------------------------------
>
>                 Key: AMQ-1797
>                 URL: https://issues.apache.org/activemq/browse/AMQ-1797
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Message Store
>    Affects Versions: 5.1.0
>         Environment: WinXP,
> java version "1.6.0_05"
> Java(TM) SE Runtime Environment (build 1.6.0_05-b13)
> Java HotSpot(TM) Client VM (build 10.0-b19, mixed mode, sharing)
>            Reporter: Juraj Kuruc
>            Assignee: Rob Davies
>             Fix For: 5.3.0
>
>         Attachments: kaha_amq.zip, Storage_cleanup_Redelivery_test.zip, textMessageFile.txt, TopicConsumers.java, TopicProducer.java
>
>
> I have some problems with multiple durable subscribers while persistent delivery is used.
> Message should be deleted from disc storage (periodically) after all interested subscribers have received it. In case there was one durable subscriber everything woks just fine. However when I tested this scenerio with two durable subscribers on topic,  the messages were never deleted (disc usage is continously consumed and never purged). 
> Test case description:
> -using default activemq.xml configuration file from binary distribution 
> -TopicConsumers.java creates two separetes durable consumers
> -textMessageFile.txt is file which text is sent by producer (place it on classpath)
> Result:
> -all messages there were send were also received by both subscribers
> -after more then 100k messages data directory is about 1GB. 
> -restarting activemq with so big persistent storage takes very long time

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.