[jira] [Comment Edited] (HBASE-20945) HBase JMX - timestamp of last Major Compaction (started, completed successfully)

["Hari Sekhon (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-20945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559548#comment-16559548 ] 

Hari Sekhon edited comment on HBASE-20945 at 7/27/18 10:38 AM:
---------------------------------------------------------------

Yes but the JMX API already exposes the following information per region in RegionServer JMX:
 * compactionsCompletedCount
 * numBytesCompactedCount
 * numFilesCompactedCount

and 12 other metrics per region next to them.

Adding one or two more for fields for timestamp of last compaction started and completed seems to be in line with what has already been done.

For HMaster you probably only want a summary JMX field per table similar to what is shown in the UI under table.jsp for whether a table is currently compacting:
{code:java}
Table Attributes
Attribute Name	Value	Description
Enabled	        true	Is the table enabled
Compaction	MINOR	Is the table compacting{code}
Adding a field next to Compaction showing Time of Last Compaction is in line with what has already been done.


was (Author: harisekhon):
Yes but the JMX API already exposes the following information per region in RegionServer JMX:
 * compactionsCompletedCount
 * numBytesCompactedCount
 * numFilesCompactedCount

and 12 other metrics per region next to them.

Adding one or two more for fields for timestamp of last compaction started and completed seems to be in line with what has already been done.

For HMaster you probably only want a summary JMX field per table similar to what is shown in the UI under table.jsp for whether a table is currently compacting:
{code:java}
Table Attributes
Attribute Name	Value	Description
Enabled	true	Is the table enabled
Compaction	MINOR	Is the table compacting{code}
Adding a field next to Compaction showing Time of Last Compaction is in line with what has already been done.

> HBase JMX - timestamp of last Major Compaction (started, completed successfully)
> --------------------------------------------------------------------------------
>
>                 Key: HBASE-20945
>                 URL: https://issues.apache.org/jira/browse/HBASE-20945
>             Project: HBase
>          Issue Type: Improvement
>          Components: API, Compaction, master, metrics, monitoring, regionserver, tooling
>            Reporter: Hari Sekhon
>            Priority: Major
>
> Request that the timestamp of the last major compaction be stored in JMX API available at /jmx.
> Major Compactions may be disabled to better control scheduling to trigger off peak (this is an old school recommendation), but there is a risk that the major compaction doesn't happen in that case. Also people may trigger major compactions manually and it's hard to see that (I've looked at graphs of storefile counts where it's not obvious but I can infer it from spikes in compaction queue length). Storing the last timestamps would allow all sorts of scripting checks against the API much more simply than trying to infer it from changes in graphs. Also with recent changes to allow compactions to be cancelled in HBASE-6028, the queue length doesn't tell the whole story as the compaction may not have happened if it got cancelled, so the compaction queue spike will be there even though major compaction did not in fact happen/complete.
> Since major compactions may take hours and can also now be cancelled in the latest versions of HBase, we need a few different fields added to JMX:
>  * HBase Master JMX:
>  ** timestamp that last major compaction was triggered, either manually via major_compact command or via schedule
>  ** timestamp that last major compaction completed successfully (since timestamp above could have been started and then later cancelled manually if load was too high)
>  * HBase Regionserver JMX:
>  ** timestamp per region that last major compaction was triggered (there are already compcationsCompletedCount, numBytesCompactedCount and numFilesCompactedCount so it makes sense to add this next to those for each region)
>  ** timestamp per region that last major compaction completed successfully



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)