How Do I Pragmatically Know When A Compaction Is Over? (i.e., how do I find hotspots)

[David Medinets <da...@gmail.com>]

I have a map-reduce job which uses AccumuloInputFormat. Some of the
mappers take 5 minutes while others take 40 minutes. Looking at the
entry count it seems like some of the tablets have more entries than
others. I'd like to generate a histogram of the number of entries per
tablet.

On the way to that goal, I learned that using Bulk Ingest does not
update the Number of Entries so that I need to perform a compaction
before I can learn the number of entries... Which leads me to how can
I tell that a compaction is complete?

Re: How Do I Pragmatically Know When A Compaction Is Over? (i.e., how do I find hotspots)

[Billie Rinaldi <bi...@apache.org>]

On Wed, Jan 2, 2013 at 7:12 AM, David Medinets <da...@gmail.com>wrote:

> I have a map-reduce job which uses AccumuloInputFormat. Some of the
> mappers take 5 minutes while others take 40 minutes. Looking at the
> entry count it seems like some of the tablets have more entries than
> others. I'd like to generate a histogram of the number of entries per
> tablet.
>
> On the way to that goal, I learned that using Bulk Ingest does not
> update the Number of Entries so that I need to perform a compaction
> before I can learn the number of entries... Which leads me to how can
> I tell that a compaction is complete?
>

If you use the "-w" flag to the compact command, the command won't return
until the compaction has completed.  There's also a boolean wait parameter
for one of the TableOperations.compact methods.  Otherwise, you have to
look at the monitor page to see when tablets are no longer being compacted.

Billie