You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Ming <ov...@sina.com> on 2018/04/20 12:38:25 UTC
How to get the HDFS path for a given HBase table?
Hello,
I am trying to use Java API to get the HDFS path for a given table, but I
cannot find that method.
For some version, I notice it is under hbase.rootdir, then
'data/namespace/tablename'; for some version of hbase, it is
'data/default/tablename'.
So I wish there is a programming way to get that Path string.
Any help will be very appreciated!
Thanks,
Ming
RE: How to get the HDFS path for a given HBase table?
Posted by "Liu, Ming (Ming)" <mi...@esgyn.cn>.
Thank you Yu , I am using a commercial distro of Hadoop/HBase, not able to change the src code. So I have to use HBase/HDFS User API to achieve the goal.
Since there will be this feature, so I will temporally guess the Path and invoke HDFS API directly. When HBase new release integrated with my used distro, I can change my code to use new HBase API, which is much more elegant.
Thanks,
Ming
-----Original Message-----
From: Yu Li <ca...@gmail.com>
Sent: Saturday, April 21, 2018 6:15 PM
To: Hbase-User <us...@hbase.apache.org>
Subject: Re: How to get the HDFS path for a given HBase table?
Maybe HBASE-19858 could help, more information replied to the original
thread on HSM, FYI.
Best Regards,
Yu
On 20 April 2018 at 21:04, Ming <ov...@sina.com> wrote:
>
> Thanks Sean,
>
> HDFS support HSM, so some table can use SSD and some use HDD by setting
> the storage policy of a given HDFS directory or file.
> As Anoop kindly replied few days earlier, HBase support set storage policy
> via ColumnDescriptor, that is nice, but it can only work in HBase 2.0.
>
> We are using HBase 1.2.0, so I want to directly use HDFS API to set the
> storage policy for a given HBase Table, but I have to know its path.
>
> Ming
>
> -----Original Message-----
> From: Sean Busbey <bu...@apache.org>
> Sent: Friday, April 20, 2018 8:49 PM
> To: user@hbase.apache.org
> Subject: Re: How to get the HDFS path for a given HBase table?
>
>
> The mapping from HBase table contents to HDFS Path(s) is an internal
> implementation detail. It's purposefully not included in any user
> facing API.
>
> The class JMS mentioned can be used to "peak behind the curtain", but
> it's an internal implementation class so you can't rely on it being
> present or the same in any given version of HBase.
>
> What are you trying to get done with the path? That might help us come
> up with a sustainable approach, if there is one.
>
> On Fri, Apr 20, 2018 at 7:42 AM, Jean-Marc Spaggiari
> <je...@spaggiari.org> wrote:
> > Hi Ming,
> >
> > Take a look at the FSUtils... There is plenty of very useful helpers
> > there...
> >
> > JMS
> >
> > 2018-04-20 8:38 GMT-04:00 Ming <ov...@sina.com>:
> >
> >> Hello,
> >>
> >>
> >>
> >> I am trying to use Java API to get the HDFS path for a given table, but
> I
> >> cannot find that method.
> >>
> >> For some version, I notice it is under hbase.rootdir, then
> >> 'data/namespace/tablename'; for some version of hbase, it is
> >> 'data/default/tablename'.
> >>
> >> So I wish there is a programming way to get that Path string.
> >>
> >>
> >>
> >> Any help will be very appreciated!
> >>
> >>
> >>
> >> Thanks,
> >>
> >> Ming
> >>
> >>
>
>
>
>
Re: How to get the HDFS path for a given HBase table?
Posted by Yu Li <ca...@gmail.com>.
Maybe HBASE-19858 could help, more information replied to the original
thread on HSM, FYI.
Best Regards,
Yu
On 20 April 2018 at 21:04, Ming <ov...@sina.com> wrote:
>
> Thanks Sean,
>
> HDFS support HSM, so some table can use SSD and some use HDD by setting
> the storage policy of a given HDFS directory or file.
> As Anoop kindly replied few days earlier, HBase support set storage policy
> via ColumnDescriptor, that is nice, but it can only work in HBase 2.0.
>
> We are using HBase 1.2.0, so I want to directly use HDFS API to set the
> storage policy for a given HBase Table, but I have to know its path.
>
> Ming
>
> -----Original Message-----
> From: Sean Busbey <bu...@apache.org>
> Sent: Friday, April 20, 2018 8:49 PM
> To: user@hbase.apache.org
> Subject: Re: How to get the HDFS path for a given HBase table?
>
>
> The mapping from HBase table contents to HDFS Path(s) is an internal
> implementation detail. It's purposefully not included in any user
> facing API.
>
> The class JMS mentioned can be used to "peak behind the curtain", but
> it's an internal implementation class so you can't rely on it being
> present or the same in any given version of HBase.
>
> What are you trying to get done with the path? That might help us come
> up with a sustainable approach, if there is one.
>
> On Fri, Apr 20, 2018 at 7:42 AM, Jean-Marc Spaggiari
> <je...@spaggiari.org> wrote:
> > Hi Ming,
> >
> > Take a look at the FSUtils... There is plenty of very useful helpers
> > there...
> >
> > JMS
> >
> > 2018-04-20 8:38 GMT-04:00 Ming <ov...@sina.com>:
> >
> >> Hello,
> >>
> >>
> >>
> >> I am trying to use Java API to get the HDFS path for a given table, but
> I
> >> cannot find that method.
> >>
> >> For some version, I notice it is under hbase.rootdir, then
> >> 'data/namespace/tablename'; for some version of hbase, it is
> >> 'data/default/tablename'.
> >>
> >> So I wish there is a programming way to get that Path string.
> >>
> >>
> >>
> >> Any help will be very appreciated!
> >>
> >>
> >>
> >> Thanks,
> >>
> >> Ming
> >>
> >>
>
>
>
>
RE: How to get the HDFS path for a given HBase table?
Posted by Ming <ov...@sina.com>.
Thanks Sean,
HDFS support HSM, so some table can use SSD and some use HDD by setting the storage policy of a given HDFS directory or file.
As Anoop kindly replied few days earlier, HBase support set storage policy via ColumnDescriptor, that is nice, but it can only work in HBase 2.0.
We are using HBase 1.2.0, so I want to directly use HDFS API to set the storage policy for a given HBase Table, but I have to know its path.
Ming
-----Original Message-----
From: Sean Busbey <bu...@apache.org>
Sent: Friday, April 20, 2018 8:49 PM
To: user@hbase.apache.org
Subject: Re: How to get the HDFS path for a given HBase table?
The mapping from HBase table contents to HDFS Path(s) is an internal
implementation detail. It's purposefully not included in any user
facing API.
The class JMS mentioned can be used to "peak behind the curtain", but
it's an internal implementation class so you can't rely on it being
present or the same in any given version of HBase.
What are you trying to get done with the path? That might help us come
up with a sustainable approach, if there is one.
On Fri, Apr 20, 2018 at 7:42 AM, Jean-Marc Spaggiari
<je...@spaggiari.org> wrote:
> Hi Ming,
>
> Take a look at the FSUtils... There is plenty of very useful helpers
> there...
>
> JMS
>
> 2018-04-20 8:38 GMT-04:00 Ming <ov...@sina.com>:
>
>> Hello,
>>
>>
>>
>> I am trying to use Java API to get the HDFS path for a given table, but I
>> cannot find that method.
>>
>> For some version, I notice it is under hbase.rootdir, then
>> 'data/namespace/tablename'; for some version of hbase, it is
>> 'data/default/tablename'.
>>
>> So I wish there is a programming way to get that Path string.
>>
>>
>>
>> Any help will be very appreciated!
>>
>>
>>
>> Thanks,
>>
>> Ming
>>
>>
Re: How to get the HDFS path for a given HBase table?
Posted by Sean Busbey <bu...@apache.org>.
The mapping from HBase table contents to HDFS Path(s) is an internal
implementation detail. It's purposefully not included in any user
facing API.
The class JMS mentioned can be used to "peak behind the curtain", but
it's an internal implementation class so you can't rely on it being
present or the same in any given version of HBase.
What are you trying to get done with the path? That might help us come
up with a sustainable approach, if there is one.
On Fri, Apr 20, 2018 at 7:42 AM, Jean-Marc Spaggiari
<je...@spaggiari.org> wrote:
> Hi Ming,
>
> Take a look at the FSUtils... There is plenty of very useful helpers
> there...
>
> JMS
>
> 2018-04-20 8:38 GMT-04:00 Ming <ov...@sina.com>:
>
>> Hello,
>>
>>
>>
>> I am trying to use Java API to get the HDFS path for a given table, but I
>> cannot find that method.
>>
>> For some version, I notice it is under hbase.rootdir, then
>> 'data/namespace/tablename'; for some version of hbase, it is
>> 'data/default/tablename'.
>>
>> So I wish there is a programming way to get that Path string.
>>
>>
>>
>> Any help will be very appreciated!
>>
>>
>>
>> Thanks,
>>
>> Ming
>>
>>
Re: How to get the HDFS path for a given HBase table?
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Hi Ming,
Take a look at the FSUtils... There is plenty of very useful helpers
there...
JMS
2018-04-20 8:38 GMT-04:00 Ming <ov...@sina.com>:
> Hello,
>
>
>
> I am trying to use Java API to get the HDFS path for a given table, but I
> cannot find that method.
>
> For some version, I notice it is under hbase.rootdir, then
> 'data/namespace/tablename'; for some version of hbase, it is
> 'data/default/tablename'.
>
> So I wish there is a programming way to get that Path string.
>
>
>
> Any help will be very appreciated!
>
>
>
> Thanks,
>
> Ming
>
>