Checkpoint worker; what's it doing?

[Raffi <ra...@gmail.com>]

It's my understanding each kahadb is assigned a checkpoint thread which,
every 5s (default), wakes up and does something; what is it doing, and is
there any negative impact if the interval is increased to 60s or 120s? We
are using persistent messages, jsyk



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Re: Checkpoint worker; what's it doing?

[Tim Bain <tb...@alumni.duke.edu>]

The org.apache.activemq.store.kahadb.MessageDatabase
<https://github.com/apache/activemq/blob/master/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/MessageDatabase.java>.CheckpointRunner
does the following:

   1. If the journal does periodic disk syncing rather than immediate
   syncing and the sync period has elapsed, it causes unsynced content to be
   written to disk.
   2. If the cleanup interval has elapsed, it performs cleanup (identifies
   and deletes files that contain no unacknowledged messages, performs ack
   compaction if enabled, flushes any pending index-modifying operations in
   the pagefile, and then deletes any files that are no longer referenced).
   3. If the checkpoint interval has elapsed, it performs a checkpoint
   (flushes any pending index-modifying operations in the pagefile, and then
   deletes any files that are no longer referenced) - basically a cleanup
   without the search for files that can be GC'ed and the ack compaction.

The consequences of increasing the period of these events are the following:

   1. If you're doing periodic disk syncing, you incur a greater risk of
   losing data. If not, there is no effect on this particular dimension.
   2. Because the pagefile stays the same but you're going longer before
   flushing, you're more likely to fill it before the next flush, in which
   case writes are done either in the background or synchronously within the
   current thread, which may be less efficient. So your throughput could drop,
   but how much would probably depend heavily on how heavily you're using the
   persistent store.

You asked whether there was any negative impact of having it run less
frequently, but I'll turn the question around: what positive impact do you
believe you'll achieve by having it run less frequently, such that it's
worth 1) your time, and 2) the increased odds of hitting edge cases caused
by running in an uncommonly-used configuration? Have you measured a
performance impact of the operations of that thread?

Tim

P.S. All information in this email comes from some time this evening
reading through this code for the first time, so it's entirely possible
that I've missed or misrepresented something here. YMMV.


On Fri, Oct 20, 2017 at 1:44 PM, Raffi <ra...@gmail.com> wrote:

> It's my understanding each kahadb is assigned a checkpoint thread which,
> every 5s (default), wakes up and does something; what is it doing, and is
> there any negative impact if the interval is increased to 60s or 120s? We
> are using persistent messages, jsyk
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>