You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Ming <ov...@sina.com> on 2018/04/17 01:46:38 UTC
can we set a table to use a HDFS specific HSM Storage policy?
Hi, all,
HDFS support HSM, one can set a file or dir storage policy to use different
hardware disks. I am wondering is there a way in HBase, when create a table,
specify the policy that that table uses?
So that table can be put into SSD, or HDD defined by the storage policy.
I read the HBase manual, but not find related topics, so asked help here.
Thanks,
Ming
Re: can we set a table to use a HDFS specific HSM Storage policy?
Posted by Yu Li <ca...@gmail.com>.
(Belated) All HSM related work has been backported into branch-1 through
HBASE-19858, but not released yet (should be included in 1.5.0).
Notice that the current commit breaks compatibility with hadoop 2.4/2.5
(the reason that the jira is reopened), so if you are on hadoop 2.6+ and
need HSM support, just cherry-pick the HBASE-19858 commit from branch-1 and
try it in your own version.
Hope this information helps.
Best Regards,
Yu
On 18 April 2018 at 08:24, Ming <ov...@sina.com> wrote:
>
> Thank you Anoop for the answer, this is very helpful.
>
> Ming
>
> -----Original Message-----
> From: Anoop John <an...@gmail.com>
> Sent: Wednesday, April 18, 2018 12:50 AM
> To: user@hbase.apache.org
> Subject: Re: can we set a table to use a HDFS specific HSM Storage policy?
>
> Oh ya seems yes.. I was under the impression that it is there in 1.x.
> Some HBase users use it. Might be, they back ported it.
>
> -Anoop-
>
> On Tue, Apr 17, 2018 at 5:38 PM, Ming <ov...@sina.com> wrote:
> >
> > Hi, Anoop,
> >
> > In which release this API is supported? From the JIRA
> https://issues.apache.org/jira/browse/HBASE-14061, it seems this is only
> available in HBase 2.0?
> >
> > Thanks,
> > Ming
> >
> > -----Original Message-----
> > From: Anoop John <an...@gmail.com>
> > Sent: Tuesday, April 17, 2018 1:42 PM
> > To: user@hbase.apache.org
> > Subject: Re: can we set a table to use a HDFS specific HSM Storage
> policy?
> >
> > When creating table, user can set storage policy on column family.
> > Each of the CF can have different policy. Pls see
> > setStoragePolicy(String) API in HColumnDescriptor.
> >
> > -Anoop-
> >
> > On Tue, Apr 17, 2018 at 7:16 AM, Ming <ov...@sina.com> wrote:
> >> Hi, all,
> >>
> >>
> >>
> >> HDFS support HSM, one can set a file or dir storage policy to use
> different
> >> hardware disks. I am wondering is there a way in HBase, when create a
> table,
> >> specify the policy that that table uses?
> >>
> >> So that table can be put into SSD, or HDD defined by the storage policy.
> >>
> >> I read the HBase manual, but not find related topics, so asked help
> here.
> >>
> >>
> >>
> >> Thanks,
> >>
> >> Ming
> >>
> >>
> >>
> >
> >
>
>
>
RE: can we set a table to use a HDFS specific HSM Storage policy?
Posted by Ming <ov...@sina.com>.
Thank you Anoop for the answer, this is very helpful.
Ming
-----Original Message-----
From: Anoop John <an...@gmail.com>
Sent: Wednesday, April 18, 2018 12:50 AM
To: user@hbase.apache.org
Subject: Re: can we set a table to use a HDFS specific HSM Storage policy?
Oh ya seems yes.. I was under the impression that it is there in 1.x.
Some HBase users use it. Might be, they back ported it.
-Anoop-
On Tue, Apr 17, 2018 at 5:38 PM, Ming <ov...@sina.com> wrote:
>
> Hi, Anoop,
>
> In which release this API is supported? From the JIRA https://issues.apache.org/jira/browse/HBASE-14061, it seems this is only available in HBase 2.0?
>
> Thanks,
> Ming
>
> -----Original Message-----
> From: Anoop John <an...@gmail.com>
> Sent: Tuesday, April 17, 2018 1:42 PM
> To: user@hbase.apache.org
> Subject: Re: can we set a table to use a HDFS specific HSM Storage policy?
>
> When creating table, user can set storage policy on column family.
> Each of the CF can have different policy. Pls see
> setStoragePolicy(String) API in HColumnDescriptor.
>
> -Anoop-
>
> On Tue, Apr 17, 2018 at 7:16 AM, Ming <ov...@sina.com> wrote:
>> Hi, all,
>>
>>
>>
>> HDFS support HSM, one can set a file or dir storage policy to use different
>> hardware disks. I am wondering is there a way in HBase, when create a table,
>> specify the policy that that table uses?
>>
>> So that table can be put into SSD, or HDD defined by the storage policy.
>>
>> I read the HBase manual, but not find related topics, so asked help here.
>>
>>
>>
>> Thanks,
>>
>> Ming
>>
>>
>>
>
>
Re: can we set a table to use a HDFS specific HSM Storage policy?
Posted by Anoop John <an...@gmail.com>.
Oh ya seems yes.. I was under the impression that it is there in 1.x.
Some HBase users use it. Might be, they back ported it.
-Anoop-
On Tue, Apr 17, 2018 at 5:38 PM, Ming <ov...@sina.com> wrote:
>
> Hi, Anoop,
>
> In which release this API is supported? From the JIRA https://issues.apache.org/jira/browse/HBASE-14061, it seems this is only available in HBase 2.0?
>
> Thanks,
> Ming
>
> -----Original Message-----
> From: Anoop John <an...@gmail.com>
> Sent: Tuesday, April 17, 2018 1:42 PM
> To: user@hbase.apache.org
> Subject: Re: can we set a table to use a HDFS specific HSM Storage policy?
>
> When creating table, user can set storage policy on column family.
> Each of the CF can have different policy. Pls see
> setStoragePolicy(String) API in HColumnDescriptor.
>
> -Anoop-
>
> On Tue, Apr 17, 2018 at 7:16 AM, Ming <ov...@sina.com> wrote:
>> Hi, all,
>>
>>
>>
>> HDFS support HSM, one can set a file or dir storage policy to use different
>> hardware disks. I am wondering is there a way in HBase, when create a table,
>> specify the policy that that table uses?
>>
>> So that table can be put into SSD, or HDD defined by the storage policy.
>>
>> I read the HBase manual, but not find related topics, so asked help here.
>>
>>
>>
>> Thanks,
>>
>> Ming
>>
>>
>>
>
>
RE: can we set a table to use a HDFS specific HSM Storage policy?
Posted by Ming <ov...@sina.com>.
Hi, Anoop,
In which release this API is supported? From the JIRA https://issues.apache.org/jira/browse/HBASE-14061, it seems this is only available in HBase 2.0?
Thanks,
Ming
-----Original Message-----
From: Anoop John <an...@gmail.com>
Sent: Tuesday, April 17, 2018 1:42 PM
To: user@hbase.apache.org
Subject: Re: can we set a table to use a HDFS specific HSM Storage policy?
When creating table, user can set storage policy on column family.
Each of the CF can have different policy. Pls see
setStoragePolicy(String) API in HColumnDescriptor.
-Anoop-
On Tue, Apr 17, 2018 at 7:16 AM, Ming <ov...@sina.com> wrote:
> Hi, all,
>
>
>
> HDFS support HSM, one can set a file or dir storage policy to use different
> hardware disks. I am wondering is there a way in HBase, when create a table,
> specify the policy that that table uses?
>
> So that table can be put into SSD, or HDD defined by the storage policy.
>
> I read the HBase manual, but not find related topics, so asked help here.
>
>
>
> Thanks,
>
> Ming
>
>
>
RE: can we set a table to use a HDFS specific HSM Storage policy?
Posted by Ming <ov...@sina.com>.
Thank you Anoop!
-----Original Message-----
From: Anoop John <an...@gmail.com>
Sent: Tuesday, April 17, 2018 1:42 PM
To: user@hbase.apache.org
Subject: Re: can we set a table to use a HDFS specific HSM Storage policy?
When creating table, user can set storage policy on column family.
Each of the CF can have different policy. Pls see
setStoragePolicy(String) API in HColumnDescriptor.
-Anoop-
On Tue, Apr 17, 2018 at 7:16 AM, Ming <ov...@sina.com> wrote:
> Hi, all,
>
>
>
> HDFS support HSM, one can set a file or dir storage policy to use different
> hardware disks. I am wondering is there a way in HBase, when create a table,
> specify the policy that that table uses?
>
> So that table can be put into SSD, or HDD defined by the storage policy.
>
> I read the HBase manual, but not find related topics, so asked help here.
>
>
>
> Thanks,
>
> Ming
>
>
>
Re: can we set a table to use a HDFS specific HSM Storage policy?
Posted by Anoop John <an...@gmail.com>.
When creating table, user can set storage policy on column family.
Each of the CF can have different policy. Pls see
setStoragePolicy(String) API in HColumnDescriptor.
-Anoop-
On Tue, Apr 17, 2018 at 7:16 AM, Ming <ov...@sina.com> wrote:
> Hi, all,
>
>
>
> HDFS support HSM, one can set a file or dir storage policy to use different
> hardware disks. I am wondering is there a way in HBase, when create a table,
> specify the policy that that table uses?
>
> So that table can be put into SSD, or HDD defined by the storage policy.
>
> I read the HBase manual, but not find related topics, so asked help here.
>
>
>
> Thanks,
>
> Ming
>
>
>