[jira] [Commented] (HBASE-21476) Support for nanosecond timestamps

["Andrey Elenskiy (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-21476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16861636#comment-16861636 ] 

Andrey Elenskiy commented on HBASE-21476:
-----------------------------------------

Hello, I'd like to restart trying to get this merged as we've been running this change in production for some time now and upgrades are tough because we need to recompile every time. However, I'm not a fan of all the "if" checks that I've added around and would like to make the implementation a bit more flexible.

What do you think about making java.time.Clock to be a field of HRegion (so each region would have its own clock as described here: [https://docs.google.com/a/arista.com/document/d/1LL2GAodiYi0waBz5ODGL4LDT4e_bXy8P9h6kWC05Bhw/edit?disco=AAAAAQ5zZuM])? Then most of the places within HRegion would use java.time.Instant and occasionally use a public helper function in HRegion that would convert Insant to either millisecond or nanoseconds (depending if region's table had NANOSECOND_TIMESTAMPS attribute). The nice thing about this is that nanosecond vs millisecond decisions would be contained to a single function and there's no need to pass around "isNanosecondTimestamps" everywhere. StoreScanner and various compaction routines would also use region's clock (instead of EnvironmentEdge) to make decisions which is also a good side effect. This approach can also be implemented in iterative steps.

> Support for nanosecond timestamps
> ---------------------------------
>
>                 Key: HBASE-21476
>                 URL: https://issues.apache.org/jira/browse/HBASE-21476
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 2.1.1
>            Reporter: Andrey Elenskiy
>            Assignee: Andrey Elenskiy
>            Priority: Major
>              Labels: features, patch
>         Attachments: Apache HBase - Nanosecond Timestamps v1.pdf, HBASE-21476.branch-2.1.0003.patch, HBASE-21476.branch-2.1.0004.patch, nanosecond_timestamps_v1.patch, nanosecond_timestamps_v2.patch
>
>
> Introducing a new table attribute "NANOSECOND_TIMESTAMPS" to tell HBase to handle timestamps with nanosecond precision. This is useful for applications that timestamp updates at the source with nanoseconds and still want features like column family TTL and "hbase.hstore.time.to.purge.deletes" to work.
> The attribute should be specified either on new tables or on existing tables which have timestamps only with nanosecond precision. There's no migration from milliseconds to nanoseconds for already existing tables. We could add this migration as part of compaction if you think that would be useful, but that would obviously make the change more complex.
> I've added a new EnvironmentEdge method "currentTimeNano()" that uses [java.time.Instant|https://docs.oracle.com/javase/8/docs/api/java/time/Instant.html] to get time in nanoseconds which means it will only work with Java 8. The idea is to gradually replace all places where "EnvironmentEdge.currentTime()" is used to have HBase working purely with nanoseconds (which is a prerequisite for HBASE-14070). Also, I've refactored ScanInfo and PartitionedMobCompactor to expect TableDescriptor as an argument which makes code a little cleaner and easier to extend.
> Couple more points:
> - column family TTL (specified in seconds) and "hbase.hstore.time.to.purge.deletes" (specified in milliseconds) options don't need to be changed, those are adjusted automatically.
> - Per cell TTL needs to be scaled by clients accordingly after "NANOSECOND_TIMESTAMPS" table attribute is specified.
> Looking for everyone's feedback to know if that's a worthwhile direction. Will add more comprehensive tests in a later patch.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)