ActiveMQ => Artemis mkahadb

[Jean-Yves Terrien <jy...@yahoo.fr.INVALID>]

Hello
We have a system with ActiveMQ which manages a few thousand queues.
We were faced with a difficult. If on an instance of ActiveMQ 1 persisted queue does not have a consumer, one message is enough in this queue for the size of the KahaDB database to increase significantly. Indeed a message not consumed in one of the queues of the base prevents the system from freeing space for all queues on this database. And we are hiding millions of messages. Except with so many tails this is a common situation. To avoid this we used mkahaDB. Thus only the queue without a consumer is impacted. The other queues can free up space.
We are considering switching to Artemis. But I don't find any information on this specific point in the documentation.
Won't the new journal clog the filesystem?
Where can I find the documentation to ensure that in the event of a consumer suspension, all messages consumed in other queues will be deleted.

Thank

Re: ActiveMQ => Artemis mkahadb

[Clebert Suconic <cl...@gmail.com>]

Let me explain this in different terms.. I just thought about this way
to explain how it works...

Artemis is optimized to deliver messages as fast as possible,
delivering them through just passing references in message with as
minimal encoding / decoding as possible.
Durable messages will be stored in the journal, but the journal is
mostly "write-only", only read in case of a restart of the broker.

So, the messages mostly stay in memory and the journal is just a
recovery tool in case of a crash.


Now, if the messages don't fit in the memory (governed by max-size on
address-size, global max -size). then messages are flowed through the
disk on paging.. ON paging we have a cache in memory that's evicted in
case other cursos need that memory on the same address.  So, paging is
more towards flow through disk with extra caching.


So, a message not consumed on a destination will use memory.  Artemis
is optimized as a message system... meaning message going through.. as
long as you configure paging accordingly you should be fine.


Hope that helps!

On Sat, Feb 27, 2021 at 7:10 PM Clebert Suconic
<cl...@gmail.com> wrote:
>
> There are two things that could happen..
>
> if you have a destination with one message, the message will just stay
> in the journal. without causing any issues.. you may eventually have
> some journal compression happening.
>
> ther other thing that could happen is one destination exploding to
> millions of messages.. on that case the destination goes into page
> mode.. where there is a separated structure for the messages, outside
> of the journal. Look for Paging on the documentation.
>
> On Sat, Feb 27, 2021 at 3:33 PM Jean-Yves Terrien
> <jy...@yahoo.fr.invalid> wrote:
> >
> > Hello
> > We have a system with ActiveMQ which manages a few thousand queues.
> > We were faced with a difficult. If on an instance of ActiveMQ 1 persisted queue does not have a consumer, one message is enough in this queue for the size of the KahaDB database to increase significantly. Indeed a message not consumed in one of the queues of the base prevents the system from freeing space for all queues on this database. And we are hiding millions of messages. Except with so many tails this is a common situation. To avoid this we used mkahaDB. Thus only the queue without a consumer is impacted. The other queues can free up space.
> > We are considering switching to Artemis. But I don't find any information on this specific point in the documentation.
> > Won't the new journal clog the filesystem?
> > Where can I find the documentation to ensure that in the event of a consumer suspension, all messages consumed in other queues will be deleted.
> >
> > Thank
>
>
>
> --
> Clebert Suconic



-- 
Clebert Suconic

Re: ActiveMQ => Artemis mkahadb

[Clebert Suconic <cl...@gmail.com>]

There are two things that could happen..

if you have a destination with one message, the message will just stay
in the journal. without causing any issues.. you may eventually have
some journal compression happening.

ther other thing that could happen is one destination exploding to
millions of messages.. on that case the destination goes into page
mode.. where there is a separated structure for the messages, outside
of the journal. Look for Paging on the documentation.

On Sat, Feb 27, 2021 at 3:33 PM Jean-Yves Terrien
<jy...@yahoo.fr.invalid> wrote:
>
> Hello
> We have a system with ActiveMQ which manages a few thousand queues.
> We were faced with a difficult. If on an instance of ActiveMQ 1 persisted queue does not have a consumer, one message is enough in this queue for the size of the KahaDB database to increase significantly. Indeed a message not consumed in one of the queues of the base prevents the system from freeing space for all queues on this database. And we are hiding millions of messages. Except with so many tails this is a common situation. To avoid this we used mkahaDB. Thus only the queue without a consumer is impacted. The other queues can free up space.
> We are considering switching to Artemis. But I don't find any information on this specific point in the documentation.
> Won't the new journal clog the filesystem?
> Where can I find the documentation to ensure that in the event of a consumer suspension, all messages consumed in other queues will be deleted.
>
> Thank



-- 
Clebert Suconic