You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Clara Xiong (Jira)" <ji...@apache.org> on 2020/05/27 17:55:00 UTC
[jira] [Commented] (HBASE-24289) Heterogeneous Storage for Date
Tiered Compaction
[ https://issues.apache.org/jira/browse/HBASE-24289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17117957#comment-17117957 ]
Clara Xiong commented on HBASE-24289:
-------------------------------------
This is a great idea to leverage multiple types of storage for data of different age. May I suggest a more generic design? In my opinion, this is orthogonal to date-tied compaction. It can be overlaid on top of DTC, any other date-based compaction policies or other writes (wals and cf maybe?). A writer factory that returns writers on different storage/folder based on date ranges would be a more reusable component. We can call it date-based HDFS storage assignment. And I would call it date ranges to differentiate from windows for date-tiered compaction. Another change I recommend is for the policy to take a list of date boundaries for ranges to assign (potentially) different HDFS storages. It can be empty so all are on the same storage, or multiple for different storages. This makes it backward compatible and easy for future extension or adoption.
> Heterogeneous Storage for Date Tiered Compaction
> ------------------------------------------------
>
> Key: HBASE-24289
> URL: https://issues.apache.org/jira/browse/HBASE-24289
> Project: HBase
> Issue Type: New Feature
> Components: Compaction
> Reporter: Mengqing Peng
> Assignee: Mengqing Peng
> Priority: Major
>
> Support DateTiredCompaction(HBASE-15181) for cold and hot data separation, support different storage policies for different time periods of data to get better performance, for example, we can configure the data of last 1 month in SSD, and 1 month ago data was in HDD.
> design doc:
> https://docs.google.com/document/d/1fk_EWLNnxniwt3gDjUS_apQ3cPzn90AmvDT1wkirvKE/edit?usp=sharing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)