You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by ak-dak <ku...@dakosy.de> on 2017/03/30 13:14:18 UTC
general problem with pending messages
Hi all,
we are using the ActiveMQ-Broker in version 5.13.1 for several projects and
it works fine. But there is one thing what I haven't understand correctly.
If we have a queue with pending messages the amount of log files in the
kahadb data folder grows permanantly. After some time when the folder is
full up to it's maximum (1GB), the health status of the broker is warning
and the consumption of messages is slow.
Sometimes it could be possible that messages will not be processed and thus
they are pending for several hours or days. What can we do to make the
ActiveMQ broker more robust with pending messages?
Best regards
Anton
--
View this message in context: http://activemq.2283324.n4.nabble.com/general-problem-with-pending-messages-tp4724359.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: general problem with pending messages
Posted by ak-dak <ku...@dakosy.de>.
Hi Tim,
thank you for your explanation. I will do the following things:
- increasing the store usage limit
- configure mKahaDb with several folders for several projects or use cases
It seems that AMQ in version 5.14.4 can handle log files much better than in
version 5.13.1 *if I separate the kahadb folders with mKahaDb*. This means
that version 5.14.4 produces a second log file after a while but remeves the
older file. In version 5.13.1 the amount of log files grows in the separated
kahadb folder. My test works with the same activemq-config params and the
same Java program for producing and consuming JMS messages.
Best regards
--
View this message in context: http://activemq.2283324.n4.nabble.com/general-problem-with-pending-messages-tp4724359p4724414.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: general problem with pending messages
Posted by Timothy Bish <ta...@gmail.com>.
On 03/30/2017 11:34 AM, ak-dak wrote:
> Hi Tim,
>
> ok I have tested it out.
>
> I'am using AMQ 5.14.4 for testing:
>
> I have configured the following:
> <kahaDB directory="${activemq.data}/kahadb" enableAckCompaction="true"
> compactAcksIgnoresStoreGrowth="true"/>
>
> My test appliacation creates one pending message and afterwards one message
> for every 10 milliseconds which will be consumed. Unfortunately the kahadb
> folder still increases with log files.
> According to this doc: http://activemq.apache.org/kahadb.html
>
> ... I expected that producing of log files will be minimised. What's wrong?
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/general-problem-with-pending-messages-tp4724359p4724369.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
ACK compaction isn't magic and will only help you in some very specific
cases. With your use case store growth is going to happen regardless.
You can review the documentation on debugging the KahaDB store to see
what is keeping your logs alive. A message broker is not a database and
treating it as one is bound to get you into trouble. If you leave a
live message in one log file it will keep that journal file around until
it is consumed. As I mentioned before if you have more than one queue
then consider mKahaDB.
--
Tim Bish
twitter: @tabish121
blog: http://timbish.blogspot.com/
Re: general problem with pending messages
Posted by ak-dak <ku...@dakosy.de>.
On the other hand I could configure with mKahaDb several kahadb folders for
specific queues. But the configuration is not changeable at runtime and
restarting of the broker if we detect problems is not an option.
--
View this message in context: http://activemq.2283324.n4.nabble.com/general-problem-with-pending-messages-tp4724359p4724370.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: general problem with pending messages
Posted by ak-dak <ku...@dakosy.de>.
Hi Tim,
ok I have tested it out.
I'am using AMQ 5.14.4 for testing:
I have configured the following:
<kahaDB directory="${activemq.data}/kahadb" enableAckCompaction="true"
compactAcksIgnoresStoreGrowth="true"/>
My test appliacation creates one pending message and afterwards one message
for every 10 milliseconds which will be consumed. Unfortunately the kahadb
folder still increases with log files.
According to this doc: http://activemq.apache.org/kahadb.html
... I expected that producing of log files will be minimised. What's wrong?
--
View this message in context: http://activemq.2283324.n4.nabble.com/general-problem-with-pending-messages-tp4724359p4724369.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: general problem with pending messages
Posted by Timothy Bish <ta...@gmail.com>.
On 03/30/2017 09:40 AM, ak-dak wrote:
> Hi Tim,
>
> increasing the store limit is a way but after 2 hours our kahadb folder
> reaches 1GB. For me it seems that this is too much because sometimes we have
> just 1 pending message and 1GB of log files.
>
> Maybe there is an other way? :-)
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/general-problem-with-pending-messages-tp4724359p4724364.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Depending on the use case mKahaDB may offer a solution:
http://activemq.apache.org/kahadb.html
--
Tim Bish
twitter: @tabish121
blog: http://timbish.blogspot.com/
Re: general problem with pending messages
Posted by ak-dak <ku...@dakosy.de>.
Hi Tim,
increasing the store limit is a way but after 2 hours our kahadb folder
reaches 1GB. For me it seems that this is too much because sometimes we have
just 1 pending message and 1GB of log files.
Maybe there is an other way? :-)
--
View this message in context: http://activemq.2283324.n4.nabble.com/general-problem-with-pending-messages-tp4724359p4724364.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: general problem with pending messages
Posted by Timothy Bish <ta...@gmail.com>.
On 03/30/2017 09:31 AM, ak-dak wrote:
> Hi Tim,
>
> an expiration value would result in lost messages if they are not processed
> before the expiration, right?
>
> But some times messages are not processed over the weekend because of a down
> time of the consuming system. In such a case we did not want lost pending
> messages.
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/general-problem-with-pending-messages-tp4724359p4724361.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
If you plan to have backlog and your store size is not making your
backlog then my logic dictates that you should think about increasing
the store limit.
--
Tim Bish
twitter: @tabish121
blog: http://timbish.blogspot.com/
Re: general problem with pending messages
Posted by ak-dak <ku...@dakosy.de>.
Hi Tim,
an expiration value would result in lost messages if they are not processed
before the expiration, right?
But some times messages are not processed over the weekend because of a down
time of the consuming system. In such a case we did not want lost pending
messages.
--
View this message in context: http://activemq.2283324.n4.nabble.com/general-problem-with-pending-messages-tp4724359p4724361.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: general problem with pending messages
Posted by Timothy Bish <ta...@gmail.com>.
On 03/30/2017 09:14 AM, ak-dak wrote:
> Hi all,
>
> we are using the ActiveMQ-Broker in version 5.13.1 for several projects and
> it works fine. But there is one thing what I haven't understand correctly.
>
> If we have a queue with pending messages the amount of log files in the
> kahadb data folder grows permanantly. After some time when the folder is
> full up to it's maximum (1GB), the health status of the broker is warning
> and the consumption of messages is slow.
>
> Sometimes it could be possible that messages will not be processed and thus
> they are pending for several hours or days. What can we do to make the
> ActiveMQ broker more robust with pending messages?
>
> Best regards
>
> Anton
>
>
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/general-problem-with-pending-messages-tp4724359.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
You could send messages with expiration values to ensure that they don't
linger and cause a jam in the brokers cleanup of old KahaDB logs.
--
Tim Bish
twitter: @tabish121
blog: http://timbish.blogspot.com/