You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by Márcio Moraes <mm...@gmail.com> on 2007/10/11 15:59:26 UTC
ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old messages
Hi,
I'm testing ActiveMQ-5.00-RC1 under the following scenario:
- Sending N messages to a Durable Topic
- Two consumers identified with its clientId
- Both has a selector and only one can accept the test messages
All works fine, except the messages are never removed from the data
directory, it grows infinitely.
Is it a bug?
My work around may be cron a script to clean that.
Att,
Márcio Moraes
Re: ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old messages
Posted by Rob Davies <ra...@gmail.com>.
Hi Marcio - this is fixed in trunk
On Oct 17, 2007, at 7:01 PM, Márcio Moraes wrote:
> Rob,
>
> May I open a issue about it?
>
> Att,
>
> Márcio Moraes
>
> 2007/10/15, Márcio Moraes <mm...@gmail.com>:
>>
>> Thanks,
>>
>> You right it does not solve my problem.
>>
>> I am think to workaround it with a script to cleanup old data files.
>>
>> 2007/10/12, Rob Davies < rajdavies@gmail.com>:
>>>
>>> looks like it might be a bug - though it doesn't solve your problem
>>> - which is another bug btw - will fix
>>>
>>>
>>>
>>> cheers,
>>>
>>> Rob
>>>
>>> http://rajdavies.blogspot.com/
>>>
>>>
>>>
>>> On Oct 12, 2007, at 12:05 AM, Márcio Moraes wrote:
>>>
>>>> I thought found the bug.
>>>>
>>>> --- DataFileAccessorPool.java ---
>>>>
>>>> public void closeDataFileReader(DataFileAccessor reader) {
>>>> openCounter--;
>>>> used = true; // HERE
>>>> if (pool.size() >= maxOpenReadersPerFile || disposed) {
>>>> reader.dispose();
>>>> } else {
>>>> pool.add(reader);
>>>> }
>>>> }
>>>>
>>>> Att,
>>>>
>>>> Márcio Moraes
>>>>
>>>> 2007/10/11, Márcio Moraes <mm...@gmail.com>:
>>>>>
>>>>> I see ...
>>>>>
>>>>> The point is it seems not cleanup until a restart.
>>>>>
>>>>> I put a log into DataFileAccessorPool.java to see the
>>>>> disposeUnused
>>>>> method.
>>>>>
>>>>> Then i sent a lot of messages, until my data files look like this:
>>>>>
>>>>> ls -lh data/localhost/journal/
>>>>> total 50M
>>>>> -rw-r--r-- 1 root root 32M 2007-10-11 18:21 data-3
>>>>> -rw-r--r-- 1 root root 32M 2007-10-11 18:31 data-4
>>>>> -rw-r--r-- 1 root root 2.1K 2007-10-11 18:31 data-control
>>>>>
>>>>> and
>>>>>
>>>>> ls -lh data/localhost/kr-store/data/
>>>>> total 40M
>>>>> -rw-r--r-- 1 root root 949 2007-10-11 17:57 data-blob-1
>>>>> -rw-r--r-- 1 root root 3.3K 2007-10-11 17:59 data-kaha-1
>>>>> -rw-r--r-- 1 root root 3.2M 2007-10-11 18:31 data-topic-acks-1
>>>>> -rw-r--r-- 1 root root 6.1M 2007-10-11 18:31 data-topic-data-1
>>>>> -rw-r--r-- 1 root root 17M 2007-10-11 18:31 data-topic-subs-
>>>>> references-1
>>>>> -rw-r--r-- 1 root root 0 2007-10-11 17:57
>>>>> hash-index-blob_topic___ActiveMQ.Agent-Subscriptions
>>>>> -rw-r--r-- 1 root root 33K 2007-10-11 17:57
>>>>> hash-index-blob_topic___auction.admin-Subscriptions
>>>>> -rw-r--r-- 1 root root 33K 2007-10-11 17:57
>>>>> hash-index-blob_topic___auction.negotiation-Subscriptions
>>>>> -rw-r--r-- 1 root root 17K 2007-10-11 17:57
>>>>> hash-index-blob_topic___auction.notification-Subscriptions
>>>>> -rw-r--r-- 1 root root 0 2007-10-11 17:59
>>>>> hash-index-blob_topic___auction.response-Subscriptions
>>>>> -rw-r--r-- 1 root root 0 2007-10-11 17:57
>>>>> hash-index-topic-data_topic___ActiveMQ.Agent
>>>>> -rw-r--r-- 1 root root 17M 2007-10-11 18:31
>>>>> hash-index-topic-data_topic___auction.admin
>>>>> -rw-r--r-- 1 root root 0 2007-10-11 17:57
>>>>> hash-index-topic-data_topic___auction.negotiation
>>>>> -rw-r--r-- 1 root root 0 2007-10-11 17:57
>>>>> hash-index-topic-data_topic___auction.notification
>>>>> -rw-r--r-- 1 root root 0 2007-10-11 17:59
>>>>> hash-index-topic-data_topic___auction.response
>>>>> -rw-r--r-- 1 root root 510 2007-10-11 17:59 index-blob
>>>>> -rw-r--r-- 1 root root 1.1K 2007-10-11 17:59 index-kaha
>>>>> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-acks
>>>>> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-data
>>>>> -rw-r--r-- 1 root root 4.4M 2007-10-11 18:31 index-topic-subs-
>>>>> references
>>>>> -rw-r--r-- 1 root root 0 2007-10-11 17:57 lock
>>>>>
>>>>>
>>>>> I thought data-3 should be removed, am i right?
>>>>>
>>>>> But my log says:
>>>>>
>>>>> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
>>>>> DataFileAccessorPool:112 - file:data-4 number = 4 , length =
>>>>> 20506094
>>>>> refCount = 89444 pool.isUsed():true
>>>>> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
>>>>> DataFileAccessorPool:112 - file:data-3 number = 3 , length =
>>>>> 33554297
>>>>> refCount = 108283 pool.isUsed():true
>>>>> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
>>>>> DataFileAccessorPool:112 - file:data-4 number = 4 , length =
>>>>> 21730205
>>>>> refCount = 94782 pool.isUsed():true
>>>>> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
>>>>> DataFileAccessorPool:112 - file:data-3 number = 3 , length =
>>>>> 33554297
>>>>> refCount = 108283 pool.isUsed():true
>>>>> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,382
>>>>> DataFileAccessorPool:112 - file:data-4 number = 4 , length =
>>>>> 22957070
>>>>> refCount = 100132 pool.isUsed():true
>>>>> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,383
>>>>> DataFileAccessorPool:112 - file:data-3 number = 3 , length =
>>>>> 33554297
>>>>> refCount = 108283 pool.isUsed():true
>>>>>
>>>>> if pool.isUsed() is true nothing is cleanup.
>>>>>
>>>>> It seems that the file into the pool never get unused.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> Att,
>>>>>
>>>>> Márcio Moraes
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> 2007/10/11, Rob Davies < rajdavies@gmail.com>:
>>>>>>
>>>>>> that still looks ok - the default data file length is 32mb - I'll
>>>>>> expose the property to make that configurable
>>>>>>
>>>>>>
>>>>>> cheers,
>>>>>>
>>>>>> Rob
>>>>>>
>>>>>> http://rajdavies.blogspot.com/
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Oct 11, 2007, at 8:56 PM, Márcio Moraes wrote:
>>>>>>
>>>>>>> I build a new version from SNAPSHOT (revision 583595) and run
>>>>>>> again my
>>>>>>> tests.
>>>>>>>
>>>>>>> After sent ~15k messages my data dir was 34mb
>>>>>>>
>>>>>>> I wait 10 minutes but data still with 34mb ...
>>>>>>>
>>>>>>> then i restart activemq and data dir was now with 20mb
>>>>>>>
>>>>>>> Is this a ok behavior? How log it takes to clean up? Always do i
>>>>>>> need
>>>>>>> restart?
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Márcio Moraes
>>>>>>>
>>>>>>> 2007/10/11, Márcio Moraes <mm...@gmail.com>:
>>>>>>>>
>>>>>>>> Ok, I will try and let you know ...
>>>>>>>>
>>>>>>>> Thanks.
>>>>>>>>
>>>>>>>> 2007/10/11, Rob Davies <ra...@gmail.com>:
>>>>>>>>>
>>>>>>>>> ah - in that case - please use a later snapshot - there was
>>>>>>>>> a bug
>>>>>>>>> with selectors which is now fixed
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> cheers,
>>>>>>>>>
>>>>>>>>> Rob
>>>>>>>>>
>>>>>>>>> http://rajdavies.blogspot.com/
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Oct 11, 2007, at 6:59 PM, Márcio Moraes wrote:
>>>>>>>>>
>>>>>>>>>> I think messages have been readed.
>>>>>>>>>>
>>>>>>>>>> Because i have 2 durable consumers with selector.
>>>>>>>>>>
>>>>>>>>>> The first consumer/selector accept the messages the other one
>>>>>>>>>> does
>>>>>>>>>> not.
>>>>>>>>>>
>>>>>>>>>> I think message should be considered readed even to the
>>>>>>>>>> second
>>>>>>>>>> consumer.
>>>>>>>>>>
>>>>>>>>>> Am i right?
>>>>>>>>>>
>>>>>>>>>> Att,
>>>>>>>>>>
>>>>>>>>>> Márcio Moraes
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2007/10/11, Rob Davies < rajdavies@gmail.com>:
>>>>>>>>>>>
>>>>>>>>>>> Hi Marcio,
>>>>>>>>>>>
>>>>>>>>>>> the messages are stored in a journal file log - the whole
>>>>>>>>>>> log
>>>>>>>>>>> will be
>>>>>>>>>>> removed when all the messages have been read. This is
>>>>>>>>>>> done for
>>>>>>>>>>> performance
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> cheers,
>>>>>>>>>>>
>>>>>>>>>>> Rob
>>>>>>>>>>>
>>>>>>>>>>> http://rajdavies.blogspot.com/
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Oct 11, 2007, at 2:59 PM, Márcio Moraes wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> I'm testing ActiveMQ-5.00-RC1 under the following scenario:
>>>>>>>>>>>>
>>>>>>>>>>>> - Sending N messages to a Durable Topic
>>>>>>>>>>>> - Two consumers identified with its clientId
>>>>>>>>>>>> - Both has a selector and only one can accept the test
>>>>>>>>>>>> messages
>>>>>>>>>>>>
>>>>>>>>>>>> All works fine, except the messages are never removed
>>>>>>>>>>>> from the
>>>>>>>>>>>> data
>>>>>>>>>>>> directory, it grows infinitely.
>>>>>>>>>>>>
>>>>>>>>>>>> Is it a bug?
>>>>>>>>>>>>
>>>>>>>>>>>> My work around may be cron a script to clean that.
>>>>>>>>>>>>
>>>>>>>>>>>> Att,
>>>>>>>>>>>>
>>>>>>>>>>>> Márcio Moraes
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>>>
>>
Re: ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old messages
Posted by Márcio Moraes <mm...@gmail.com>.
Rob,
May I open a issue about it?
Att,
Márcio Moraes
2007/10/15, Márcio Moraes <mm...@gmail.com>:
>
> Thanks,
>
> You right it does not solve my problem.
>
> I am think to workaround it with a script to cleanup old data files.
>
> 2007/10/12, Rob Davies < rajdavies@gmail.com>:
> >
> > looks like it might be a bug - though it doesn't solve your problem
> > - which is another bug btw - will fix
> >
> >
> >
> > cheers,
> >
> > Rob
> >
> > http://rajdavies.blogspot.com/
> >
> >
> >
> > On Oct 12, 2007, at 12:05 AM, Márcio Moraes wrote:
> >
> > > I thought found the bug.
> > >
> > > --- DataFileAccessorPool.java ---
> > >
> > > public void closeDataFileReader(DataFileAccessor reader) {
> > > openCounter--;
> > > used = true; // HERE
> > > if (pool.size() >= maxOpenReadersPerFile || disposed) {
> > > reader.dispose();
> > > } else {
> > > pool.add(reader);
> > > }
> > > }
> > >
> > > Att,
> > >
> > > Márcio Moraes
> > >
> > > 2007/10/11, Márcio Moraes <mm...@gmail.com>:
> > >>
> > >> I see ...
> > >>
> > >> The point is it seems not cleanup until a restart.
> > >>
> > >> I put a log into DataFileAccessorPool.java to see the disposeUnused
> > >> method.
> > >>
> > >> Then i sent a lot of messages, until my data files look like this:
> > >>
> > >> ls -lh data/localhost/journal/
> > >> total 50M
> > >> -rw-r--r-- 1 root root 32M 2007-10-11 18:21 data-3
> > >> -rw-r--r-- 1 root root 32M 2007-10-11 18:31 data-4
> > >> -rw-r--r-- 1 root root 2.1K 2007-10-11 18:31 data-control
> > >>
> > >> and
> > >>
> > >> ls -lh data/localhost/kr-store/data/
> > >> total 40M
> > >> -rw-r--r-- 1 root root 949 2007-10-11 17:57 data-blob-1
> > >> -rw-r--r-- 1 root root 3.3K 2007-10-11 17:59 data-kaha-1
> > >> -rw-r--r-- 1 root root 3.2M 2007-10-11 18:31 data-topic-acks-1
> > >> -rw-r--r-- 1 root root 6.1M 2007-10-11 18:31 data-topic-data-1
> > >> -rw-r--r-- 1 root root 17M 2007-10-11 18:31 data-topic-subs-
> > >> references-1
> > >> -rw-r--r-- 1 root root 0 2007-10-11 17:57
> > >> hash-index-blob_topic___ActiveMQ.Agent-Subscriptions
> > >> -rw-r--r-- 1 root root 33K 2007-10-11 17:57
> > >> hash-index-blob_topic___auction.admin-Subscriptions
> > >> -rw-r--r-- 1 root root 33K 2007-10-11 17:57
> > >> hash-index-blob_topic___auction.negotiation-Subscriptions
> > >> -rw-r--r-- 1 root root 17K 2007-10-11 17:57
> > >> hash-index-blob_topic___auction.notification-Subscriptions
> > >> -rw-r--r-- 1 root root 0 2007-10-11 17:59
> > >> hash-index-blob_topic___auction.response-Subscriptions
> > >> -rw-r--r-- 1 root root 0 2007-10-11 17:57
> > >> hash-index-topic-data_topic___ActiveMQ.Agent
> > >> -rw-r--r-- 1 root root 17M 2007-10-11 18:31
> > >> hash-index-topic-data_topic___auction.admin
> > >> -rw-r--r-- 1 root root 0 2007-10-11 17:57
> > >> hash-index-topic-data_topic___auction.negotiation
> > >> -rw-r--r-- 1 root root 0 2007-10-11 17:57
> > >> hash-index-topic-data_topic___auction.notification
> > >> -rw-r--r-- 1 root root 0 2007-10-11 17:59
> > >> hash-index-topic-data_topic___auction.response
> > >> -rw-r--r-- 1 root root 510 2007-10-11 17:59 index-blob
> > >> -rw-r--r-- 1 root root 1.1K 2007-10-11 17:59 index-kaha
> > >> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-acks
> > >> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-data
> > >> -rw-r--r-- 1 root root 4.4M 2007-10-11 18:31 index-topic-subs-
> > >> references
> > >> -rw-r--r-- 1 root root 0 2007-10-11 17:57 lock
> > >>
> > >>
> > >> I thought data-3 should be removed, am i right?
> > >>
> > >> But my log says:
> > >>
> > >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
> > >> DataFileAccessorPool:112 - file:data-4 number = 4 , length =
> > >> 20506094
> > >> refCount = 89444 pool.isUsed():true
> > >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
> > >> DataFileAccessorPool:112 - file:data-3 number = 3 , length =
> > >> 33554297
> > >> refCount = 108283 pool.isUsed():true
> > >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
> > >> DataFileAccessorPool:112 - file:data-4 number = 4 , length =
> > >> 21730205
> > >> refCount = 94782 pool.isUsed():true
> > >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
> > >> DataFileAccessorPool:112 - file:data-3 number = 3 , length =
> > >> 33554297
> > >> refCount = 108283 pool.isUsed():true
> > >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,382
> > >> DataFileAccessorPool:112 - file:data-4 number = 4 , length =
> > >> 22957070
> > >> refCount = 100132 pool.isUsed():true
> > >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,383
> > >> DataFileAccessorPool:112 - file:data-3 number = 3 , length =
> > >> 33554297
> > >> refCount = 108283 pool.isUsed():true
> > >>
> > >> if pool.isUsed() is true nothing is cleanup.
> > >>
> > >> It seems that the file into the pool never get unused.
> > >>
> > >> Thanks.
> > >>
> > >> Att,
> > >>
> > >> Márcio Moraes
> > >>
> > >>
> > >>
> > >>
> > >> 2007/10/11, Rob Davies < rajdavies@gmail.com>:
> > >>>
> > >>> that still looks ok - the default data file length is 32mb - I'll
> > >>> expose the property to make that configurable
> > >>>
> > >>>
> > >>> cheers,
> > >>>
> > >>> Rob
> > >>>
> > >>> http://rajdavies.blogspot.com/
> > >>>
> > >>>
> > >>>
> > >>> On Oct 11, 2007, at 8:56 PM, Márcio Moraes wrote:
> > >>>
> > >>>> I build a new version from SNAPSHOT (revision 583595) and run
> > >>>> again my
> > >>>> tests.
> > >>>>
> > >>>> After sent ~15k messages my data dir was 34mb
> > >>>>
> > >>>> I wait 10 minutes but data still with 34mb ...
> > >>>>
> > >>>> then i restart activemq and data dir was now with 20mb
> > >>>>
> > >>>> Is this a ok behavior? How log it takes to clean up? Always do i
> > >>>> need
> > >>>> restart?
> > >>>>
> > >>>> Thanks,
> > >>>>
> > >>>> Márcio Moraes
> > >>>>
> > >>>> 2007/10/11, Márcio Moraes <mm...@gmail.com>:
> > >>>>>
> > >>>>> Ok, I will try and let you know ...
> > >>>>>
> > >>>>> Thanks.
> > >>>>>
> > >>>>> 2007/10/11, Rob Davies <ra...@gmail.com>:
> > >>>>>>
> > >>>>>> ah - in that case - please use a later snapshot - there was a bug
> > >>>>>> with selectors which is now fixed
> > >>>>>>
> > >>>>>>
> > >>>>>> cheers,
> > >>>>>>
> > >>>>>> Rob
> > >>>>>>
> > >>>>>> http://rajdavies.blogspot.com/
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> On Oct 11, 2007, at 6:59 PM, Márcio Moraes wrote:
> > >>>>>>
> > >>>>>>> I think messages have been readed.
> > >>>>>>>
> > >>>>>>> Because i have 2 durable consumers with selector.
> > >>>>>>>
> > >>>>>>> The first consumer/selector accept the messages the other one
> > >>>>>>> does
> > >>>>>>> not.
> > >>>>>>>
> > >>>>>>> I think message should be considered readed even to the second
> > >>>>>>> consumer.
> > >>>>>>>
> > >>>>>>> Am i right?
> > >>>>>>>
> > >>>>>>> Att,
> > >>>>>>>
> > >>>>>>> Márcio Moraes
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> 2007/10/11, Rob Davies < rajdavies@gmail.com>:
> > >>>>>>>>
> > >>>>>>>> Hi Marcio,
> > >>>>>>>>
> > >>>>>>>> the messages are stored in a journal file log - the whole log
> > >>>>>>>> will be
> > >>>>>>>> removed when all the messages have been read. This is done for
> > >>>>>>>> performance
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> cheers,
> > >>>>>>>>
> > >>>>>>>> Rob
> > >>>>>>>>
> > >>>>>>>> http://rajdavies.blogspot.com/
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Oct 11, 2007, at 2:59 PM, Márcio Moraes wrote:
> > >>>>>>>>
> > >>>>>>>>> Hi,
> > >>>>>>>>>
> > >>>>>>>>> I'm testing ActiveMQ-5.00-RC1 under the following scenario:
> > >>>>>>>>>
> > >>>>>>>>> - Sending N messages to a Durable Topic
> > >>>>>>>>> - Two consumers identified with its clientId
> > >>>>>>>>> - Both has a selector and only one can accept the test
> > >>>>>>>>> messages
> > >>>>>>>>>
> > >>>>>>>>> All works fine, except the messages are never removed from the
> > >>>>>>>>> data
> > >>>>>>>>> directory, it grows infinitely.
> > >>>>>>>>>
> > >>>>>>>>> Is it a bug?
> > >>>>>>>>>
> > >>>>>>>>> My work around may be cron a script to clean that.
> > >>>>>>>>>
> > >>>>>>>>> Att,
> > >>>>>>>>>
> > >>>>>>>>> Márcio Moraes
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> > >>>
> > >>
> >
> >
>
Re: ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old messages
Posted by Márcio Moraes <mm...@gmail.com>.
Thanks,
You right it does not solve my problem.
I am think to workaround it with a script to cleanup old data files.
2007/10/12, Rob Davies <ra...@gmail.com>:
>
> looks like it might be a bug - though it doesn't solve your problem
> - which is another bug btw - will fix
>
>
>
> cheers,
>
> Rob
>
> http://rajdavies.blogspot.com/
>
>
>
> On Oct 12, 2007, at 12:05 AM, Márcio Moraes wrote:
>
> > I thought found the bug.
> >
> > --- DataFileAccessorPool.java ---
> >
> > public void closeDataFileReader(DataFileAccessor reader) {
> > openCounter--;
> > used = true; // HERE
> > if (pool.size() >= maxOpenReadersPerFile || disposed) {
> > reader.dispose();
> > } else {
> > pool.add(reader);
> > }
> > }
> >
> > Att,
> >
> > Márcio Moraes
> >
> > 2007/10/11, Márcio Moraes <mm...@gmail.com>:
> >>
> >> I see ...
> >>
> >> The point is it seems not cleanup until a restart.
> >>
> >> I put a log into DataFileAccessorPool.java to see the disposeUnused
> >> method.
> >>
> >> Then i sent a lot of messages, until my data files look like this:
> >>
> >> ls -lh data/localhost/journal/
> >> total 50M
> >> -rw-r--r-- 1 root root 32M 2007-10-11 18:21 data-3
> >> -rw-r--r-- 1 root root 32M 2007-10-11 18:31 data-4
> >> -rw-r--r-- 1 root root 2.1K 2007-10-11 18:31 data-control
> >>
> >> and
> >>
> >> ls -lh data/localhost/kr-store/data/
> >> total 40M
> >> -rw-r--r-- 1 root root 949 2007-10-11 17:57 data-blob-1
> >> -rw-r--r-- 1 root root 3.3K 2007-10-11 17:59 data-kaha-1
> >> -rw-r--r-- 1 root root 3.2M 2007-10-11 18:31 data-topic-acks-1
> >> -rw-r--r-- 1 root root 6.1M 2007-10-11 18:31 data-topic-data-1
> >> -rw-r--r-- 1 root root 17M 2007-10-11 18:31 data-topic-subs-
> >> references-1
> >> -rw-r--r-- 1 root root 0 2007-10-11 17:57
> >> hash-index-blob_topic___ActiveMQ.Agent-Subscriptions
> >> -rw-r--r-- 1 root root 33K 2007-10-11 17:57
> >> hash-index-blob_topic___auction.admin-Subscriptions
> >> -rw-r--r-- 1 root root 33K 2007-10-11 17:57
> >> hash-index-blob_topic___auction.negotiation-Subscriptions
> >> -rw-r--r-- 1 root root 17K 2007-10-11 17:57
> >> hash-index-blob_topic___auction.notification-Subscriptions
> >> -rw-r--r-- 1 root root 0 2007-10-11 17:59
> >> hash-index-blob_topic___auction.response-Subscriptions
> >> -rw-r--r-- 1 root root 0 2007-10-11 17:57
> >> hash-index-topic-data_topic___ActiveMQ.Agent
> >> -rw-r--r-- 1 root root 17M 2007-10-11 18:31
> >> hash-index-topic-data_topic___auction.admin
> >> -rw-r--r-- 1 root root 0 2007-10-11 17:57
> >> hash-index-topic-data_topic___auction.negotiation
> >> -rw-r--r-- 1 root root 0 2007-10-11 17:57
> >> hash-index-topic-data_topic___auction.notification
> >> -rw-r--r-- 1 root root 0 2007-10-11 17:59
> >> hash-index-topic-data_topic___auction.response
> >> -rw-r--r-- 1 root root 510 2007-10-11 17:59 index-blob
> >> -rw-r--r-- 1 root root 1.1K 2007-10-11 17:59 index-kaha
> >> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-acks
> >> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-data
> >> -rw-r--r-- 1 root root 4.4M 2007-10-11 18:31 index-topic-subs-
> >> references
> >> -rw-r--r-- 1 root root 0 2007-10-11 17:57 lock
> >>
> >>
> >> I thought data-3 should be removed, am i right?
> >>
> >> But my log says:
> >>
> >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
> >> DataFileAccessorPool:112 - file:data-4 number = 4 , length =
> >> 20506094
> >> refCount = 89444 pool.isUsed():true
> >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
> >> DataFileAccessorPool:112 - file:data-3 number = 3 , length =
> >> 33554297
> >> refCount = 108283 pool.isUsed():true
> >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
> >> DataFileAccessorPool:112 - file:data-4 number = 4 , length =
> >> 21730205
> >> refCount = 94782 pool.isUsed():true
> >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
> >> DataFileAccessorPool:112 - file:data-3 number = 3 , length =
> >> 33554297
> >> refCount = 108283 pool.isUsed():true
> >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,382
> >> DataFileAccessorPool:112 - file:data-4 number = 4 , length =
> >> 22957070
> >> refCount = 100132 pool.isUsed():true
> >> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,383
> >> DataFileAccessorPool:112 - file:data-3 number = 3 , length =
> >> 33554297
> >> refCount = 108283 pool.isUsed():true
> >>
> >> if pool.isUsed() is true nothing is cleanup.
> >>
> >> It seems that the file into the pool never get unused.
> >>
> >> Thanks.
> >>
> >> Att,
> >>
> >> Márcio Moraes
> >>
> >>
> >>
> >>
> >> 2007/10/11, Rob Davies <ra...@gmail.com>:
> >>>
> >>> that still looks ok - the default data file length is 32mb - I'll
> >>> expose the property to make that configurable
> >>>
> >>>
> >>> cheers,
> >>>
> >>> Rob
> >>>
> >>> http://rajdavies.blogspot.com/
> >>>
> >>>
> >>>
> >>> On Oct 11, 2007, at 8:56 PM, Márcio Moraes wrote:
> >>>
> >>>> I build a new version from SNAPSHOT (revision 583595) and run
> >>>> again my
> >>>> tests.
> >>>>
> >>>> After sent ~15k messages my data dir was 34mb
> >>>>
> >>>> I wait 10 minutes but data still with 34mb ...
> >>>>
> >>>> then i restart activemq and data dir was now with 20mb
> >>>>
> >>>> Is this a ok behavior? How log it takes to clean up? Always do i
> >>>> need
> >>>> restart?
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Márcio Moraes
> >>>>
> >>>> 2007/10/11, Márcio Moraes <mm...@gmail.com>:
> >>>>>
> >>>>> Ok, I will try and let you know ...
> >>>>>
> >>>>> Thanks.
> >>>>>
> >>>>> 2007/10/11, Rob Davies <ra...@gmail.com>:
> >>>>>>
> >>>>>> ah - in that case - please use a later snapshot - there was a bug
> >>>>>> with selectors which is now fixed
> >>>>>>
> >>>>>>
> >>>>>> cheers,
> >>>>>>
> >>>>>> Rob
> >>>>>>
> >>>>>> http://rajdavies.blogspot.com/
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Oct 11, 2007, at 6:59 PM, Márcio Moraes wrote:
> >>>>>>
> >>>>>>> I think messages have been readed.
> >>>>>>>
> >>>>>>> Because i have 2 durable consumers with selector.
> >>>>>>>
> >>>>>>> The first consumer/selector accept the messages the other one
> >>>>>>> does
> >>>>>>> not.
> >>>>>>>
> >>>>>>> I think message should be considered readed even to the second
> >>>>>>> consumer.
> >>>>>>>
> >>>>>>> Am i right?
> >>>>>>>
> >>>>>>> Att,
> >>>>>>>
> >>>>>>> Márcio Moraes
> >>>>>>>
> >>>>>>>
> >>>>>>> 2007/10/11, Rob Davies < rajdavies@gmail.com>:
> >>>>>>>>
> >>>>>>>> Hi Marcio,
> >>>>>>>>
> >>>>>>>> the messages are stored in a journal file log - the whole log
> >>>>>>>> will be
> >>>>>>>> removed when all the messages have been read. This is done for
> >>>>>>>> performance
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> cheers,
> >>>>>>>>
> >>>>>>>> Rob
> >>>>>>>>
> >>>>>>>> http://rajdavies.blogspot.com/
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Oct 11, 2007, at 2:59 PM, Márcio Moraes wrote:
> >>>>>>>>
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> I'm testing ActiveMQ-5.00-RC1 under the following scenario:
> >>>>>>>>>
> >>>>>>>>> - Sending N messages to a Durable Topic
> >>>>>>>>> - Two consumers identified with its clientId
> >>>>>>>>> - Both has a selector and only one can accept the test
> >>>>>>>>> messages
> >>>>>>>>>
> >>>>>>>>> All works fine, except the messages are never removed from the
> >>>>>>>>> data
> >>>>>>>>> directory, it grows infinitely.
> >>>>>>>>>
> >>>>>>>>> Is it a bug?
> >>>>>>>>>
> >>>>>>>>> My work around may be cron a script to clean that.
> >>>>>>>>>
> >>>>>>>>> Att,
> >>>>>>>>>
> >>>>>>>>> Márcio Moraes
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>
> >>>
> >>
>
>
Re: ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old messages
Posted by Rob Davies <ra...@gmail.com>.
looks like it might be a bug - though it doesn't solve your problem
- which is another bug btw - will fix
cheers,
Rob
http://rajdavies.blogspot.com/
On Oct 12, 2007, at 12:05 AM, Márcio Moraes wrote:
> I thought found the bug.
>
> --- DataFileAccessorPool.java ---
>
> public void closeDataFileReader(DataFileAccessor reader) {
> openCounter--;
> used = true; // HERE
> if (pool.size() >= maxOpenReadersPerFile || disposed) {
> reader.dispose();
> } else {
> pool.add(reader);
> }
> }
>
> Att,
>
> Márcio Moraes
>
> 2007/10/11, Márcio Moraes <mm...@gmail.com>:
>>
>> I see ...
>>
>> The point is it seems not cleanup until a restart.
>>
>> I put a log into DataFileAccessorPool.java to see the disposeUnused
>> method.
>>
>> Then i sent a lot of messages, until my data files look like this:
>>
>> ls -lh data/localhost/journal/
>> total 50M
>> -rw-r--r-- 1 root root 32M 2007-10-11 18:21 data-3
>> -rw-r--r-- 1 root root 32M 2007-10-11 18:31 data-4
>> -rw-r--r-- 1 root root 2.1K 2007-10-11 18:31 data-control
>>
>> and
>>
>> ls -lh data/localhost/kr-store/data/
>> total 40M
>> -rw-r--r-- 1 root root 949 2007-10-11 17:57 data-blob-1
>> -rw-r--r-- 1 root root 3.3K 2007-10-11 17:59 data-kaha-1
>> -rw-r--r-- 1 root root 3.2M 2007-10-11 18:31 data-topic-acks-1
>> -rw-r--r-- 1 root root 6.1M 2007-10-11 18:31 data-topic-data-1
>> -rw-r--r-- 1 root root 17M 2007-10-11 18:31 data-topic-subs-
>> references-1
>> -rw-r--r-- 1 root root 0 2007-10-11 17:57
>> hash-index-blob_topic___ActiveMQ.Agent-Subscriptions
>> -rw-r--r-- 1 root root 33K 2007-10-11 17:57
>> hash-index-blob_topic___auction.admin-Subscriptions
>> -rw-r--r-- 1 root root 33K 2007-10-11 17:57
>> hash-index-blob_topic___auction.negotiation-Subscriptions
>> -rw-r--r-- 1 root root 17K 2007-10-11 17:57
>> hash-index-blob_topic___auction.notification-Subscriptions
>> -rw-r--r-- 1 root root 0 2007-10-11 17:59
>> hash-index-blob_topic___auction.response-Subscriptions
>> -rw-r--r-- 1 root root 0 2007-10-11 17:57
>> hash-index-topic-data_topic___ActiveMQ.Agent
>> -rw-r--r-- 1 root root 17M 2007-10-11 18:31
>> hash-index-topic-data_topic___auction.admin
>> -rw-r--r-- 1 root root 0 2007-10-11 17:57
>> hash-index-topic-data_topic___auction.negotiation
>> -rw-r--r-- 1 root root 0 2007-10-11 17:57
>> hash-index-topic-data_topic___auction.notification
>> -rw-r--r-- 1 root root 0 2007-10-11 17:59
>> hash-index-topic-data_topic___auction.response
>> -rw-r--r-- 1 root root 510 2007-10-11 17:59 index-blob
>> -rw-r--r-- 1 root root 1.1K 2007-10-11 17:59 index-kaha
>> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-acks
>> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-data
>> -rw-r--r-- 1 root root 4.4M 2007-10-11 18:31 index-topic-subs-
>> references
>> -rw-r--r-- 1 root root 0 2007-10-11 17:57 lock
>>
>>
>> I thought data-3 should be removed, am i right?
>>
>> But my log says:
>>
>> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
>> DataFileAccessorPool:112 - file:data-4 number = 4 , length =
>> 20506094
>> refCount = 89444 pool.isUsed():true
>> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
>> DataFileAccessorPool:112 - file:data-3 number = 3 , length =
>> 33554297
>> refCount = 108283 pool.isUsed():true
>> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
>> DataFileAccessorPool:112 - file:data-4 number = 4 , length =
>> 21730205
>> refCount = 94782 pool.isUsed():true
>> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
>> DataFileAccessorPool:112 - file:data-3 number = 3 , length =
>> 33554297
>> refCount = 108283 pool.isUsed():true
>> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,382
>> DataFileAccessorPool:112 - file:data-4 number = 4 , length =
>> 22957070
>> refCount = 100132 pool.isUsed():true
>> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,383
>> DataFileAccessorPool:112 - file:data-3 number = 3 , length =
>> 33554297
>> refCount = 108283 pool.isUsed():true
>>
>> if pool.isUsed() is true nothing is cleanup.
>>
>> It seems that the file into the pool never get unused.
>>
>> Thanks.
>>
>> Att,
>>
>> Márcio Moraes
>>
>>
>>
>>
>> 2007/10/11, Rob Davies <ra...@gmail.com>:
>>>
>>> that still looks ok - the default data file length is 32mb - I'll
>>> expose the property to make that configurable
>>>
>>>
>>> cheers,
>>>
>>> Rob
>>>
>>> http://rajdavies.blogspot.com/
>>>
>>>
>>>
>>> On Oct 11, 2007, at 8:56 PM, Márcio Moraes wrote:
>>>
>>>> I build a new version from SNAPSHOT (revision 583595) and run
>>>> again my
>>>> tests.
>>>>
>>>> After sent ~15k messages my data dir was 34mb
>>>>
>>>> I wait 10 minutes but data still with 34mb ...
>>>>
>>>> then i restart activemq and data dir was now with 20mb
>>>>
>>>> Is this a ok behavior? How log it takes to clean up? Always do i
>>>> need
>>>> restart?
>>>>
>>>> Thanks,
>>>>
>>>> Márcio Moraes
>>>>
>>>> 2007/10/11, Márcio Moraes <mm...@gmail.com>:
>>>>>
>>>>> Ok, I will try and let you know ...
>>>>>
>>>>> Thanks.
>>>>>
>>>>> 2007/10/11, Rob Davies <ra...@gmail.com>:
>>>>>>
>>>>>> ah - in that case - please use a later snapshot - there was a bug
>>>>>> with selectors which is now fixed
>>>>>>
>>>>>>
>>>>>> cheers,
>>>>>>
>>>>>> Rob
>>>>>>
>>>>>> http://rajdavies.blogspot.com/
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Oct 11, 2007, at 6:59 PM, Márcio Moraes wrote:
>>>>>>
>>>>>>> I think messages have been readed.
>>>>>>>
>>>>>>> Because i have 2 durable consumers with selector.
>>>>>>>
>>>>>>> The first consumer/selector accept the messages the other one
>>>>>>> does
>>>>>>> not.
>>>>>>>
>>>>>>> I think message should be considered readed even to the second
>>>>>>> consumer.
>>>>>>>
>>>>>>> Am i right?
>>>>>>>
>>>>>>> Att,
>>>>>>>
>>>>>>> Márcio Moraes
>>>>>>>
>>>>>>>
>>>>>>> 2007/10/11, Rob Davies < rajdavies@gmail.com>:
>>>>>>>>
>>>>>>>> Hi Marcio,
>>>>>>>>
>>>>>>>> the messages are stored in a journal file log - the whole log
>>>>>>>> will be
>>>>>>>> removed when all the messages have been read. This is done for
>>>>>>>> performance
>>>>>>>>
>>>>>>>>
>>>>>>>> cheers,
>>>>>>>>
>>>>>>>> Rob
>>>>>>>>
>>>>>>>> http://rajdavies.blogspot.com/
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Oct 11, 2007, at 2:59 PM, Márcio Moraes wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I'm testing ActiveMQ-5.00-RC1 under the following scenario:
>>>>>>>>>
>>>>>>>>> - Sending N messages to a Durable Topic
>>>>>>>>> - Two consumers identified with its clientId
>>>>>>>>> - Both has a selector and only one can accept the test
>>>>>>>>> messages
>>>>>>>>>
>>>>>>>>> All works fine, except the messages are never removed from the
>>>>>>>>> data
>>>>>>>>> directory, it grows infinitely.
>>>>>>>>>
>>>>>>>>> Is it a bug?
>>>>>>>>>
>>>>>>>>> My work around may be cron a script to clean that.
>>>>>>>>>
>>>>>>>>> Att,
>>>>>>>>>
>>>>>>>>> Márcio Moraes
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>>>
>>
Re: ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old messages
Posted by Márcio Moraes <mm...@gmail.com>.
I thought found the bug.
--- DataFileAccessorPool.java ---
public void closeDataFileReader(DataFileAccessor reader) {
openCounter--;
used = true; // HERE
if (pool.size() >= maxOpenReadersPerFile || disposed) {
reader.dispose();
} else {
pool.add(reader);
}
}
Att,
Márcio Moraes
2007/10/11, Márcio Moraes <mm...@gmail.com>:
>
> I see ...
>
> The point is it seems not cleanup until a restart.
>
> I put a log into DataFileAccessorPool.java to see the disposeUnused
> method.
>
> Then i sent a lot of messages, until my data files look like this:
>
> ls -lh data/localhost/journal/
> total 50M
> -rw-r--r-- 1 root root 32M 2007-10-11 18:21 data-3
> -rw-r--r-- 1 root root 32M 2007-10-11 18:31 data-4
> -rw-r--r-- 1 root root 2.1K 2007-10-11 18:31 data-control
>
> and
>
> ls -lh data/localhost/kr-store/data/
> total 40M
> -rw-r--r-- 1 root root 949 2007-10-11 17:57 data-blob-1
> -rw-r--r-- 1 root root 3.3K 2007-10-11 17:59 data-kaha-1
> -rw-r--r-- 1 root root 3.2M 2007-10-11 18:31 data-topic-acks-1
> -rw-r--r-- 1 root root 6.1M 2007-10-11 18:31 data-topic-data-1
> -rw-r--r-- 1 root root 17M 2007-10-11 18:31 data-topic-subs-references-1
> -rw-r--r-- 1 root root 0 2007-10-11 17:57
> hash-index-blob_topic___ActiveMQ.Agent-Subscriptions
> -rw-r--r-- 1 root root 33K 2007-10-11 17:57
> hash-index-blob_topic___auction.admin-Subscriptions
> -rw-r--r-- 1 root root 33K 2007-10-11 17:57
> hash-index-blob_topic___auction.negotiation-Subscriptions
> -rw-r--r-- 1 root root 17K 2007-10-11 17:57
> hash-index-blob_topic___auction.notification-Subscriptions
> -rw-r--r-- 1 root root 0 2007-10-11 17:59
> hash-index-blob_topic___auction.response-Subscriptions
> -rw-r--r-- 1 root root 0 2007-10-11 17:57
> hash-index-topic-data_topic___ActiveMQ.Agent
> -rw-r--r-- 1 root root 17M 2007-10-11 18:31
> hash-index-topic-data_topic___auction.admin
> -rw-r--r-- 1 root root 0 2007-10-11 17:57
> hash-index-topic-data_topic___auction.negotiation
> -rw-r--r-- 1 root root 0 2007-10-11 17:57
> hash-index-topic-data_topic___auction.notification
> -rw-r--r-- 1 root root 0 2007-10-11 17:59
> hash-index-topic-data_topic___auction.response
> -rw-r--r-- 1 root root 510 2007-10-11 17:59 index-blob
> -rw-r--r-- 1 root root 1.1K 2007-10-11 17:59 index-kaha
> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-acks
> -rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-data
> -rw-r--r-- 1 root root 4.4M 2007-10-11 18:31 index-topic-subs-references
> -rw-r--r-- 1 root root 0 2007-10-11 17:57 lock
>
>
> I thought data-3 should be removed, am i right?
>
> But my log says:
>
> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
> DataFileAccessorPool:112 - file:data-4 number = 4 , length = 20506094
> refCount = 89444 pool.isUsed():true
> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381
> DataFileAccessorPool:112 - file:data-3 number = 3 , length = 33554297
> refCount = 108283 pool.isUsed():true
> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
> DataFileAccessorPool:112 - file:data-4 number = 4 , length = 21730205
> refCount = 94782 pool.isUsed():true
> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385
> DataFileAccessorPool:112 - file:data-3 number = 3 , length = 33554297
> refCount = 108283 pool.isUsed():true
> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,382
> DataFileAccessorPool:112 - file:data-4 number = 4 , length = 22957070
> refCount = 100132 pool.isUsed():true
> [DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,383
> DataFileAccessorPool:112 - file:data-3 number = 3 , length = 33554297
> refCount = 108283 pool.isUsed():true
>
> if pool.isUsed() is true nothing is cleanup.
>
> It seems that the file into the pool never get unused.
>
> Thanks.
>
> Att,
>
> Márcio Moraes
>
>
>
>
> 2007/10/11, Rob Davies <ra...@gmail.com>:
> >
> > that still looks ok - the default data file length is 32mb - I'll
> > expose the property to make that configurable
> >
> >
> > cheers,
> >
> > Rob
> >
> > http://rajdavies.blogspot.com/
> >
> >
> >
> > On Oct 11, 2007, at 8:56 PM, Márcio Moraes wrote:
> >
> > > I build a new version from SNAPSHOT (revision 583595) and run again my
> > > tests.
> > >
> > > After sent ~15k messages my data dir was 34mb
> > >
> > > I wait 10 minutes but data still with 34mb ...
> > >
> > > then i restart activemq and data dir was now with 20mb
> > >
> > > Is this a ok behavior? How log it takes to clean up? Always do i need
> > > restart?
> > >
> > > Thanks,
> > >
> > > Márcio Moraes
> > >
> > > 2007/10/11, Márcio Moraes <mm...@gmail.com>:
> > >>
> > >> Ok, I will try and let you know ...
> > >>
> > >> Thanks.
> > >>
> > >> 2007/10/11, Rob Davies <ra...@gmail.com>:
> > >>>
> > >>> ah - in that case - please use a later snapshot - there was a bug
> > >>> with selectors which is now fixed
> > >>>
> > >>>
> > >>> cheers,
> > >>>
> > >>> Rob
> > >>>
> > >>> http://rajdavies.blogspot.com/
> > >>>
> > >>>
> > >>>
> > >>> On Oct 11, 2007, at 6:59 PM, Márcio Moraes wrote:
> > >>>
> > >>>> I think messages have been readed.
> > >>>>
> > >>>> Because i have 2 durable consumers with selector.
> > >>>>
> > >>>> The first consumer/selector accept the messages the other one does
> > >>>> not.
> > >>>>
> > >>>> I think message should be considered readed even to the second
> > >>>> consumer.
> > >>>>
> > >>>> Am i right?
> > >>>>
> > >>>> Att,
> > >>>>
> > >>>> Márcio Moraes
> > >>>>
> > >>>>
> > >>>> 2007/10/11, Rob Davies < rajdavies@gmail.com>:
> > >>>>>
> > >>>>> Hi Marcio,
> > >>>>>
> > >>>>> the messages are stored in a journal file log - the whole log
> > >>>>> will be
> > >>>>> removed when all the messages have been read. This is done for
> > >>>>> performance
> > >>>>>
> > >>>>>
> > >>>>> cheers,
> > >>>>>
> > >>>>> Rob
> > >>>>>
> > >>>>> http://rajdavies.blogspot.com/
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On Oct 11, 2007, at 2:59 PM, Márcio Moraes wrote:
> > >>>>>
> > >>>>>> Hi,
> > >>>>>>
> > >>>>>> I'm testing ActiveMQ-5.00-RC1 under the following scenario:
> > >>>>>>
> > >>>>>> - Sending N messages to a Durable Topic
> > >>>>>> - Two consumers identified with its clientId
> > >>>>>> - Both has a selector and only one can accept the test messages
> > >>>>>>
> > >>>>>> All works fine, except the messages are never removed from the
> > >>>>>> data
> > >>>>>> directory, it grows infinitely.
> > >>>>>>
> > >>>>>> Is it a bug?
> > >>>>>>
> > >>>>>> My work around may be cron a script to clean that.
> > >>>>>>
> > >>>>>> Att,
> > >>>>>>
> > >>>>>> Márcio Moraes
> > >>>>>
> > >>>>>
> > >>>
> > >>>
> > >>
> >
> >
>
Re: ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old messages
Posted by Márcio Moraes <mm...@gmail.com>.
I see ...
The point is it seems not cleanup until a restart.
I put a log into DataFileAccessorPool.java to see the disposeUnused method.
Then i sent a lot of messages, until my data files look like this:
ls -lh data/localhost/journal/
total 50M
-rw-r--r-- 1 root root 32M 2007-10-11 18:21 data-3
-rw-r--r-- 1 root root 32M 2007-10-11 18:31 data-4
-rw-r--r-- 1 root root 2.1K 2007-10-11 18:31 data-control
and
ls -lh data/localhost/kr-store/data/
total 40M
-rw-r--r-- 1 root root 949 2007-10-11 17:57 data-blob-1
-rw-r--r-- 1 root root 3.3K 2007-10-11 17:59 data-kaha-1
-rw-r--r-- 1 root root 3.2M 2007-10-11 18:31 data-topic-acks-1
-rw-r--r-- 1 root root 6.1M 2007-10-11 18:31 data-topic-data-1
-rw-r--r-- 1 root root 17M 2007-10-11 18:31 data-topic-subs-references-1
-rw-r--r-- 1 root root 0 2007-10-11 17:57
hash-index-blob_topic___ActiveMQ.Agent-Subscriptions
-rw-r--r-- 1 root root 33K 2007-10-11 17:57
hash-index-blob_topic___auction.admin-Subscriptions
-rw-r--r-- 1 root root 33K 2007-10-11 17:57
hash-index-blob_topic___auction.negotiation-Subscriptions
-rw-r--r-- 1 root root 17K 2007-10-11 17:57
hash-index-blob_topic___auction.notification-Subscriptions
-rw-r--r-- 1 root root 0 2007-10-11 17:59
hash-index-blob_topic___auction.response-Subscriptions
-rw-r--r-- 1 root root 0 2007-10-11 17:57
hash-index-topic-data_topic___ActiveMQ.Agent
-rw-r--r-- 1 root root 17M 2007-10-11 18:31
hash-index-topic-data_topic___auction.admin
-rw-r--r-- 1 root root 0 2007-10-11 17:57
hash-index-topic-data_topic___auction.negotiation
-rw-r--r-- 1 root root 0 2007-10-11 17:57
hash-index-topic-data_topic___auction.notification
-rw-r--r-- 1 root root 0 2007-10-11 17:59
hash-index-topic-data_topic___auction.response
-rw-r--r-- 1 root root 510 2007-10-11 17:59 index-blob
-rw-r--r-- 1 root root 1.1K 2007-10-11 17:59 index-kaha
-rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-acks
-rw-r--r-- 1 root root 2.4M 2007-10-11 18:31 index-topic-data
-rw-r--r-- 1 root root 4.4M 2007-10-11 18:31 index-topic-subs-references
-rw-r--r-- 1 root root 0 2007-10-11 17:57 lock
I thought data-3 should be removed, am i right?
But my log says:
[DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381 DataFileAccessorPool:112
- file:data-4 number = 4 , length = 20506094 refCount = 89444 pool.isUsed
():true
[DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:11,381 DataFileAccessorPool:112
- file:data-3 number = 3 , length = 33554297 refCount = 108283 pool.isUsed
():true
[DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385 DataFileAccessorPool:112
- file:data-4 number = 4 , length = 21730205 refCount = 94782 pool.isUsed
():true
[DEBUG][ActiveMQ Scheduler]2007-10-11 18:32:41,385 DataFileAccessorPool:112
- file:data-3 number = 3 , length = 33554297 refCount = 108283 pool.isUsed
():true
[DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,382 DataFileAccessorPool:112
- file:data-4 number = 4 , length = 22957070 refCount = 100132 pool.isUsed
():true
[DEBUG][ActiveMQ Scheduler]2007-10-11 18:33:11,383 DataFileAccessorPool:112
- file:data-3 number = 3 , length = 33554297 refCount = 108283 pool.isUsed
():true
if pool.isUsed() is true nothing is cleanup.
It seems that the file into the pool never get unused.
Thanks.
Att,
Márcio Moraes
2007/10/11, Rob Davies <ra...@gmail.com>:
>
> that still looks ok - the default data file length is 32mb - I'll
> expose the property to make that configurable
>
>
> cheers,
>
> Rob
>
> http://rajdavies.blogspot.com/
>
>
>
> On Oct 11, 2007, at 8:56 PM, Márcio Moraes wrote:
>
> > I build a new version from SNAPSHOT (revision 583595) and run again my
> > tests.
> >
> > After sent ~15k messages my data dir was 34mb
> >
> > I wait 10 minutes but data still with 34mb ...
> >
> > then i restart activemq and data dir was now with 20mb
> >
> > Is this a ok behavior? How log it takes to clean up? Always do i need
> > restart?
> >
> > Thanks,
> >
> > Márcio Moraes
> >
> > 2007/10/11, Márcio Moraes <mm...@gmail.com>:
> >>
> >> Ok, I will try and let you know ...
> >>
> >> Thanks.
> >>
> >> 2007/10/11, Rob Davies <ra...@gmail.com>:
> >>>
> >>> ah - in that case - please use a later snapshot - there was a bug
> >>> with selectors which is now fixed
> >>>
> >>>
> >>> cheers,
> >>>
> >>> Rob
> >>>
> >>> http://rajdavies.blogspot.com/
> >>>
> >>>
> >>>
> >>> On Oct 11, 2007, at 6:59 PM, Márcio Moraes wrote:
> >>>
> >>>> I think messages have been readed.
> >>>>
> >>>> Because i have 2 durable consumers with selector.
> >>>>
> >>>> The first consumer/selector accept the messages the other one does
> >>>> not.
> >>>>
> >>>> I think message should be considered readed even to the second
> >>>> consumer.
> >>>>
> >>>> Am i right?
> >>>>
> >>>> Att,
> >>>>
> >>>> Márcio Moraes
> >>>>
> >>>>
> >>>> 2007/10/11, Rob Davies < rajdavies@gmail.com>:
> >>>>>
> >>>>> Hi Marcio,
> >>>>>
> >>>>> the messages are stored in a journal file log - the whole log
> >>>>> will be
> >>>>> removed when all the messages have been read. This is done for
> >>>>> performance
> >>>>>
> >>>>>
> >>>>> cheers,
> >>>>>
> >>>>> Rob
> >>>>>
> >>>>> http://rajdavies.blogspot.com/
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Oct 11, 2007, at 2:59 PM, Márcio Moraes wrote:
> >>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> I'm testing ActiveMQ-5.00-RC1 under the following scenario:
> >>>>>>
> >>>>>> - Sending N messages to a Durable Topic
> >>>>>> - Two consumers identified with its clientId
> >>>>>> - Both has a selector and only one can accept the test messages
> >>>>>>
> >>>>>> All works fine, except the messages are never removed from the
> >>>>>> data
> >>>>>> directory, it grows infinitely.
> >>>>>>
> >>>>>> Is it a bug?
> >>>>>>
> >>>>>> My work around may be cron a script to clean that.
> >>>>>>
> >>>>>> Att,
> >>>>>>
> >>>>>> Márcio Moraes
> >>>>>
> >>>>>
> >>>
> >>>
> >>
>
>
Re: ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old messages
Posted by Rob Davies <ra...@gmail.com>.
that still looks ok - the default data file length is 32mb - I'll
expose the property to make that configurable
cheers,
Rob
http://rajdavies.blogspot.com/
On Oct 11, 2007, at 8:56 PM, Márcio Moraes wrote:
> I build a new version from SNAPSHOT (revision 583595) and run again my
> tests.
>
> After sent ~15k messages my data dir was 34mb
>
> I wait 10 minutes but data still with 34mb ...
>
> then i restart activemq and data dir was now with 20mb
>
> Is this a ok behavior? How log it takes to clean up? Always do i need
> restart?
>
> Thanks,
>
> Márcio Moraes
>
> 2007/10/11, Márcio Moraes <mm...@gmail.com>:
>>
>> Ok, I will try and let you know ...
>>
>> Thanks.
>>
>> 2007/10/11, Rob Davies <ra...@gmail.com>:
>>>
>>> ah - in that case - please use a later snapshot - there was a bug
>>> with selectors which is now fixed
>>>
>>>
>>> cheers,
>>>
>>> Rob
>>>
>>> http://rajdavies.blogspot.com/
>>>
>>>
>>>
>>> On Oct 11, 2007, at 6:59 PM, Márcio Moraes wrote:
>>>
>>>> I think messages have been readed.
>>>>
>>>> Because i have 2 durable consumers with selector.
>>>>
>>>> The first consumer/selector accept the messages the other one does
>>>> not.
>>>>
>>>> I think message should be considered readed even to the second
>>>> consumer.
>>>>
>>>> Am i right?
>>>>
>>>> Att,
>>>>
>>>> Márcio Moraes
>>>>
>>>>
>>>> 2007/10/11, Rob Davies < rajdavies@gmail.com>:
>>>>>
>>>>> Hi Marcio,
>>>>>
>>>>> the messages are stored in a journal file log - the whole log
>>>>> will be
>>>>> removed when all the messages have been read. This is done for
>>>>> performance
>>>>>
>>>>>
>>>>> cheers,
>>>>>
>>>>> Rob
>>>>>
>>>>> http://rajdavies.blogspot.com/
>>>>>
>>>>>
>>>>>
>>>>> On Oct 11, 2007, at 2:59 PM, Márcio Moraes wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I'm testing ActiveMQ-5.00-RC1 under the following scenario:
>>>>>>
>>>>>> - Sending N messages to a Durable Topic
>>>>>> - Two consumers identified with its clientId
>>>>>> - Both has a selector and only one can accept the test messages
>>>>>>
>>>>>> All works fine, except the messages are never removed from the
>>>>>> data
>>>>>> directory, it grows infinitely.
>>>>>>
>>>>>> Is it a bug?
>>>>>>
>>>>>> My work around may be cron a script to clean that.
>>>>>>
>>>>>> Att,
>>>>>>
>>>>>> Márcio Moraes
>>>>>
>>>>>
>>>
>>>
>>
Re: ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old messages
Posted by Márcio Moraes <mm...@gmail.com>.
I build a new version from SNAPSHOT (revision 583595) and run again my
tests.
After sent ~15k messages my data dir was 34mb
I wait 10 minutes but data still with 34mb ...
then i restart activemq and data dir was now with 20mb
Is this a ok behavior? How log it takes to clean up? Always do i need
restart?
Thanks,
Márcio Moraes
2007/10/11, Márcio Moraes <mm...@gmail.com>:
>
> Ok, I will try and let you know ...
>
> Thanks.
>
> 2007/10/11, Rob Davies <ra...@gmail.com>:
> >
> > ah - in that case - please use a later snapshot - there was a bug
> > with selectors which is now fixed
> >
> >
> > cheers,
> >
> > Rob
> >
> > http://rajdavies.blogspot.com/
> >
> >
> >
> > On Oct 11, 2007, at 6:59 PM, Márcio Moraes wrote:
> >
> > > I think messages have been readed.
> > >
> > > Because i have 2 durable consumers with selector.
> > >
> > > The first consumer/selector accept the messages the other one does
> > > not.
> > >
> > > I think message should be considered readed even to the second
> > > consumer.
> > >
> > > Am i right?
> > >
> > > Att,
> > >
> > > Márcio Moraes
> > >
> > >
> > > 2007/10/11, Rob Davies < rajdavies@gmail.com>:
> > >>
> > >> Hi Marcio,
> > >>
> > >> the messages are stored in a journal file log - the whole log will be
> > >> removed when all the messages have been read. This is done for
> > >> performance
> > >>
> > >>
> > >> cheers,
> > >>
> > >> Rob
> > >>
> > >> http://rajdavies.blogspot.com/
> > >>
> > >>
> > >>
> > >> On Oct 11, 2007, at 2:59 PM, Márcio Moraes wrote:
> > >>
> > >>> Hi,
> > >>>
> > >>> I'm testing ActiveMQ-5.00-RC1 under the following scenario:
> > >>>
> > >>> - Sending N messages to a Durable Topic
> > >>> - Two consumers identified with its clientId
> > >>> - Both has a selector and only one can accept the test messages
> > >>>
> > >>> All works fine, except the messages are never removed from the data
> > >>> directory, it grows infinitely.
> > >>>
> > >>> Is it a bug?
> > >>>
> > >>> My work around may be cron a script to clean that.
> > >>>
> > >>> Att,
> > >>>
> > >>> Márcio Moraes
> > >>
> > >>
> >
> >
>
Re: ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old messages
Posted by Márcio Moraes <mm...@gmail.com>.
Ok, I will try and let you know ...
Thanks.
2007/10/11, Rob Davies <ra...@gmail.com>:
>
> ah - in that case - please use a later snapshot - there was a bug
> with selectors which is now fixed
>
>
> cheers,
>
> Rob
>
> http://rajdavies.blogspot.com/
>
>
>
> On Oct 11, 2007, at 6:59 PM, Márcio Moraes wrote:
>
> > I think messages have been readed.
> >
> > Because i have 2 durable consumers with selector.
> >
> > The first consumer/selector accept the messages the other one does
> > not.
> >
> > I think message should be considered readed even to the second
> > consumer.
> >
> > Am i right?
> >
> > Att,
> >
> > Márcio Moraes
> >
> >
> > 2007/10/11, Rob Davies <ra...@gmail.com>:
> >>
> >> Hi Marcio,
> >>
> >> the messages are stored in a journal file log - the whole log will be
> >> removed when all the messages have been read. This is done for
> >> performance
> >>
> >>
> >> cheers,
> >>
> >> Rob
> >>
> >> http://rajdavies.blogspot.com/
> >>
> >>
> >>
> >> On Oct 11, 2007, at 2:59 PM, Márcio Moraes wrote:
> >>
> >>> Hi,
> >>>
> >>> I'm testing ActiveMQ-5.00-RC1 under the following scenario:
> >>>
> >>> - Sending N messages to a Durable Topic
> >>> - Two consumers identified with its clientId
> >>> - Both has a selector and only one can accept the test messages
> >>>
> >>> All works fine, except the messages are never removed from the data
> >>> directory, it grows infinitely.
> >>>
> >>> Is it a bug?
> >>>
> >>> My work around may be cron a script to clean that.
> >>>
> >>> Att,
> >>>
> >>> Márcio Moraes
> >>
> >>
>
>
Re: ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old messages
Posted by Rob Davies <ra...@gmail.com>.
ah - in that case - please use a later snapshot - there was a bug
with selectors which is now fixed
cheers,
Rob
http://rajdavies.blogspot.com/
On Oct 11, 2007, at 6:59 PM, Márcio Moraes wrote:
> I think messages have been readed.
>
> Because i have 2 durable consumers with selector.
>
> The first consumer/selector accept the messages the other one does
> not.
>
> I think message should be considered readed even to the second
> consumer.
>
> Am i right?
>
> Att,
>
> Márcio Moraes
>
>
> 2007/10/11, Rob Davies <ra...@gmail.com>:
>>
>> Hi Marcio,
>>
>> the messages are stored in a journal file log - the whole log will be
>> removed when all the messages have been read. This is done for
>> performance
>>
>>
>> cheers,
>>
>> Rob
>>
>> http://rajdavies.blogspot.com/
>>
>>
>>
>> On Oct 11, 2007, at 2:59 PM, Márcio Moraes wrote:
>>
>>> Hi,
>>>
>>> I'm testing ActiveMQ-5.00-RC1 under the following scenario:
>>>
>>> - Sending N messages to a Durable Topic
>>> - Two consumers identified with its clientId
>>> - Both has a selector and only one can accept the test messages
>>>
>>> All works fine, except the messages are never removed from the data
>>> directory, it grows infinitely.
>>>
>>> Is it a bug?
>>>
>>> My work around may be cron a script to clean that.
>>>
>>> Att,
>>>
>>> Márcio Moraes
>>
>>
Re: ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old messages
Posted by Márcio Moraes <mm...@gmail.com>.
I think messages have been readed.
Because i have 2 durable consumers with selector.
The first consumer/selector accept the messages the other one does not.
I think message should be considered readed even to the second consumer.
Am i right?
Att,
Márcio Moraes
2007/10/11, Rob Davies <ra...@gmail.com>:
>
> Hi Marcio,
>
> the messages are stored in a journal file log - the whole log will be
> removed when all the messages have been read. This is done for
> performance
>
>
> cheers,
>
> Rob
>
> http://rajdavies.blogspot.com/
>
>
>
> On Oct 11, 2007, at 2:59 PM, Márcio Moraes wrote:
>
> > Hi,
> >
> > I'm testing ActiveMQ-5.00-RC1 under the following scenario:
> >
> > - Sending N messages to a Durable Topic
> > - Two consumers identified with its clientId
> > - Both has a selector and only one can accept the test messages
> >
> > All works fine, except the messages are never removed from the data
> > directory, it grows infinitely.
> >
> > Is it a bug?
> >
> > My work around may be cron a script to clean that.
> >
> > Att,
> >
> > Márcio Moraes
>
>
Re: ActiveMQ-5.00-RC1 - Default AMQMessageStore dont remove old messages
Posted by Rob Davies <ra...@gmail.com>.
Hi Marcio,
the messages are stored in a journal file log - the whole log will be
removed when all the messages have been read. This is done for
performance
cheers,
Rob
http://rajdavies.blogspot.com/
On Oct 11, 2007, at 2:59 PM, Márcio Moraes wrote:
> Hi,
>
> I'm testing ActiveMQ-5.00-RC1 under the following scenario:
>
> - Sending N messages to a Durable Topic
> - Two consumers identified with its clientId
> - Both has a selector and only one can accept the test messages
>
> All works fine, except the messages are never removed from the data
> directory, it grows infinitely.
>
> Is it a bug?
>
> My work around may be cron a script to clean that.
>
> Att,
>
> Márcio Moraes