[jira] [Comment Edited] (HBASE-20105) Allow flushes to target SSD storage

["Anoop Sam John (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-20105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16400088#comment-16400088 ] 

Anoop Sam John edited comment on HBASE-20105 at 3/15/18 8:46 AM:
-----------------------------------------------------------------

I see that is an existing code.  I was seeing how we make use of the storage policy been set at HCD level.
{code}
// set block storage policy for temp path
String policyName = this.conf.get(ColumnFamilyDescriptorBuilder.STORAGE_POLICY);
      if (null == policyName) {
        policyName = this.conf.get(HStore.BLOCK_STORAGE_POLICY_KEY);
      }
      FSUtils.setStoragePolicy(this.fs, dir, policyName);
{code}
Here dir is the temp dir path for this CF.  Now after your patch, we will end up setting the flush storage policy as the storage policy for this temp dir.  We should NOT do that.  We are any way setting the policy specifically on the file later.   
The path where we set this flush storage policy is the file path under the temp dir.  Later we will rename the file to come under the region cf dir.   On that file path as such the policy was not set. It would be best if we set on that also?

Not on your patch. But the above code try to set the policy on the temp dir every time a flush/compaction happen. We can avoid that. Call it only once after this temp dir is created first.

Instead of HConstants new constant can be added in StoreFlusher?


was (Author: anoop.hbase):
I see that is an existing code.  I was seeing how we make use of the storage policy been set at HCD level.
{code}
// set block storage policy for temp path
String policyName = this.conf.get(ColumnFamilyDescriptorBuilder.STORAGE_POLICY);
      if (null == policyName) {
        policyName = this.conf.get(HStore.BLOCK_STORAGE_POLICY_KEY);
      }
      FSUtils.setStoragePolicy(this.fs, dir, policyName);
{code}
Here dir is the temp dir path for this CF.  Now after your patch, we will end up setting the flush storage policy as the storage policy for this temp dir.  We should NOT do that.  We are any way setting the policy specifically on the file later.   
The path where we set this flush storage policy is the file path under the temp dir.  Later we will rename the file to come under the region cf dir.   On that file path as such the policy was not set. It would be best if we set on that also?

Not on your patch. But the above code try to set the policy on the temp dir every time a flush/compaction happen. We can avoid that. Call it only once after this temp dir is created first.

> Allow flushes to target SSD storage
> -----------------------------------
>
>                 Key: HBASE-20105
>                 URL: https://issues.apache.org/jira/browse/HBASE-20105
>             Project: HBase
>          Issue Type: New Feature
>          Components: Performance, regionserver
>    Affects Versions: hbase-2.0.0-alpha-4
>            Reporter: Jean-Marc Spaggiari
>            Assignee: Jean-Marc Spaggiari
>            Priority: Major
>             Fix For: 2.0.0
>
>         Attachments: HBASE-20105-v0.patch, HBASE-20105-v1.patch, HBASE-20105-v2.patch, HBASE-20105-v3.patch, HBASE-20105-v4.patch, HBASE-20105-v5.patch, HBASE-20105-v6.patch
>
>
> On heavy writes usecases, flushes are compactes together pretty quickly. Allowing flushes to go on SSD allows faster flush and faster first compactions. Subsequent compactions going on regular storage.
>  
> I will be interesting to have an option to target SSD for flushes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)