[jira] [Updated] (HBASE-18135) Track file archival for low latency space quota with snapshots

["Josh Elser (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-18135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Josh Elser updated HBASE-18135:
-------------------------------
    Attachment: HBASE-18135.001.patch

.001 Parking an initial implementation for reviews to catch up.

This moves the logic for computing the size of a snapshot out of a Chore and into its own in-memory state in the Master. This state can be updated via the Chore or via RPCs from a RS. The computed snapshot sizes are still persisted to the hbase:quota table. The actual code change is much smaller than the patch lends itself because a bit of code was just moved directly.

> Track file archival for low latency space quota with snapshots
> --------------------------------------------------------------
>
>                 Key: HBASE-18135
>                 URL: https://issues.apache.org/jira/browse/HBASE-18135
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>             Fix For: 3.0.0
>
>         Attachments: HBASE-18135.001.patch
>
>
> Related to the work proposed on HBASE-17748 and building on the same idea as HBASE-18133, we can make the space quota tracking for HBase snapshots faster to respond.
> When snapshots are in play, the location of a file (whether in the {{data}} or {{archive}} directory) plays a factor in the realized size of a table. Like flushes, compactions, etc, moving files from the data directory to the archive directory is done by the RegionServer. We can hook into this call and send the necessary information to the Master so that it can more quickly update the size of a table when there are snapshots in play.
> This will require the RegionServer to report the full coordinates of the file being moved (table+region+family+file) so that the SnapshotQuotaObserverChore running in the master can avoid HDFS lookups in partial or total to compute the location of a Region's hfiles.
> This may also require some refactoring of the SnapshotQuotaObserverChore to de-couple the receipt of these file archival reports from RegionServers (e.g. {{HRegionFileSystem.removeStoreFiles(..)}}, and the Master processing the sizes of snapshots.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)