You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by "Adaryl \"Bob\" Wakefield, MBA" <ad...@hotmail.com> on 2014/07/14 23:52:33 UTC
clarification on HBASE functionality
HBASE uses HDFS to store it's data correct?
B.
Re: clarification on HBASE functionality
Posted by Andrew Purtell <ap...@apache.org>.
HBase will take advantage of HDFS specific features if they are available
but can run on anything that has a Hadoop FileSystem driver. Gluster is an
option. Maybe Lustre and Ceph also.
If you plan on dedicating storage to Cassandra, then you don't have to
worry about managing a distributed filesystem. However, if you plan to
deploy and use a distributed filesystem for useful work already, and want
to run a scale out store on the same infrastructure, then HBase can be
convenient.
You can run into trouble if using a Hadoop FileSystem implementation that
doesn't guarantee you can immediately read what you've written, so don't
use S3.
On Tue, Jul 15, 2014 at 9:20 AM, Jay Vyas <ja...@gmail.com>
wrote:
> Hbase is not harcoded to hdfs: it works on any file system that implements
> the file system interface, we've run it on glusterfs for example. I assume
> some have also run it on s3 and other alternative file systems .
>
> ** However **
>
> For best performance, direct block io hooks on hdfs can boost high
> throughout applications on hdfs.
>
> Ultimately, the hbase root directory only needs a fully qualified
> FileSystem uri which maps to a FileSystem class.
>
> On Jul 14, 2014, at 5:59 PM, Ted Yu <yu...@gmail.com> wrote:
>
> Right.
> hbase is different from Cassandra in this regard.
>
>
> On Mon, Jul 14, 2014 at 2:57 PM,
>
> Adaryl "Bob" Wakefield, MBA <ad...@hotmail.com> wrote:
>
>> Now this is different from Cassandra which does NOT use HDFS correct?
>> (Sorry. Don’t know why that needed two emails.)
>>
>> B.
>>
>> *From:* Ted Yu <yu...@gmail.com>
>> *Sent:* Monday, July 14, 2014 4:53 PM
>> *To:* mailto:user@hadoop.apache.org <us...@hadoop.apache.org>
>> *Subject:* Re: clarification on HBASE functionality
>>
>> Yes.
>> See http://hbase.apache.org/book.html#arch.hdfs
>>
>>
>> On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <
>> adaryl.wakefield@hotmail.com> wrote:
>>
>>> HBASE uses HDFS to store it's data correct?
>>>
>>> B.
>>>
>>
>>
>
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: clarification on HBASE functionality
Posted by Andrew Purtell <ap...@apache.org>.
HBase will take advantage of HDFS specific features if they are available
but can run on anything that has a Hadoop FileSystem driver. Gluster is an
option. Maybe Lustre and Ceph also.
If you plan on dedicating storage to Cassandra, then you don't have to
worry about managing a distributed filesystem. However, if you plan to
deploy and use a distributed filesystem for useful work already, and want
to run a scale out store on the same infrastructure, then HBase can be
convenient.
You can run into trouble if using a Hadoop FileSystem implementation that
doesn't guarantee you can immediately read what you've written, so don't
use S3.
On Tue, Jul 15, 2014 at 9:20 AM, Jay Vyas <ja...@gmail.com>
wrote:
> Hbase is not harcoded to hdfs: it works on any file system that implements
> the file system interface, we've run it on glusterfs for example. I assume
> some have also run it on s3 and other alternative file systems .
>
> ** However **
>
> For best performance, direct block io hooks on hdfs can boost high
> throughout applications on hdfs.
>
> Ultimately, the hbase root directory only needs a fully qualified
> FileSystem uri which maps to a FileSystem class.
>
> On Jul 14, 2014, at 5:59 PM, Ted Yu <yu...@gmail.com> wrote:
>
> Right.
> hbase is different from Cassandra in this regard.
>
>
> On Mon, Jul 14, 2014 at 2:57 PM,
>
> Adaryl "Bob" Wakefield, MBA <ad...@hotmail.com> wrote:
>
>> Now this is different from Cassandra which does NOT use HDFS correct?
>> (Sorry. Don’t know why that needed two emails.)
>>
>> B.
>>
>> *From:* Ted Yu <yu...@gmail.com>
>> *Sent:* Monday, July 14, 2014 4:53 PM
>> *To:* mailto:user@hadoop.apache.org <us...@hadoop.apache.org>
>> *Subject:* Re: clarification on HBASE functionality
>>
>> Yes.
>> See http://hbase.apache.org/book.html#arch.hdfs
>>
>>
>> On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <
>> adaryl.wakefield@hotmail.com> wrote:
>>
>>> HBASE uses HDFS to store it's data correct?
>>>
>>> B.
>>>
>>
>>
>
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: clarification on HBASE functionality
Posted by Andrew Purtell <ap...@apache.org>.
HBase will take advantage of HDFS specific features if they are available
but can run on anything that has a Hadoop FileSystem driver. Gluster is an
option. Maybe Lustre and Ceph also.
If you plan on dedicating storage to Cassandra, then you don't have to
worry about managing a distributed filesystem. However, if you plan to
deploy and use a distributed filesystem for useful work already, and want
to run a scale out store on the same infrastructure, then HBase can be
convenient.
You can run into trouble if using a Hadoop FileSystem implementation that
doesn't guarantee you can immediately read what you've written, so don't
use S3.
On Tue, Jul 15, 2014 at 9:20 AM, Jay Vyas <ja...@gmail.com>
wrote:
> Hbase is not harcoded to hdfs: it works on any file system that implements
> the file system interface, we've run it on glusterfs for example. I assume
> some have also run it on s3 and other alternative file systems .
>
> ** However **
>
> For best performance, direct block io hooks on hdfs can boost high
> throughout applications on hdfs.
>
> Ultimately, the hbase root directory only needs a fully qualified
> FileSystem uri which maps to a FileSystem class.
>
> On Jul 14, 2014, at 5:59 PM, Ted Yu <yu...@gmail.com> wrote:
>
> Right.
> hbase is different from Cassandra in this regard.
>
>
> On Mon, Jul 14, 2014 at 2:57 PM,
>
> Adaryl "Bob" Wakefield, MBA <ad...@hotmail.com> wrote:
>
>> Now this is different from Cassandra which does NOT use HDFS correct?
>> (Sorry. Don’t know why that needed two emails.)
>>
>> B.
>>
>> *From:* Ted Yu <yu...@gmail.com>
>> *Sent:* Monday, July 14, 2014 4:53 PM
>> *To:* mailto:user@hadoop.apache.org <us...@hadoop.apache.org>
>> *Subject:* Re: clarification on HBASE functionality
>>
>> Yes.
>> See http://hbase.apache.org/book.html#arch.hdfs
>>
>>
>> On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <
>> adaryl.wakefield@hotmail.com> wrote:
>>
>>> HBASE uses HDFS to store it's data correct?
>>>
>>> B.
>>>
>>
>>
>
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: clarification on HBASE functionality
Posted by Andrew Purtell <ap...@apache.org>.
HBase will take advantage of HDFS specific features if they are available
but can run on anything that has a Hadoop FileSystem driver. Gluster is an
option. Maybe Lustre and Ceph also.
If you plan on dedicating storage to Cassandra, then you don't have to
worry about managing a distributed filesystem. However, if you plan to
deploy and use a distributed filesystem for useful work already, and want
to run a scale out store on the same infrastructure, then HBase can be
convenient.
You can run into trouble if using a Hadoop FileSystem implementation that
doesn't guarantee you can immediately read what you've written, so don't
use S3.
On Tue, Jul 15, 2014 at 9:20 AM, Jay Vyas <ja...@gmail.com>
wrote:
> Hbase is not harcoded to hdfs: it works on any file system that implements
> the file system interface, we've run it on glusterfs for example. I assume
> some have also run it on s3 and other alternative file systems .
>
> ** However **
>
> For best performance, direct block io hooks on hdfs can boost high
> throughout applications on hdfs.
>
> Ultimately, the hbase root directory only needs a fully qualified
> FileSystem uri which maps to a FileSystem class.
>
> On Jul 14, 2014, at 5:59 PM, Ted Yu <yu...@gmail.com> wrote:
>
> Right.
> hbase is different from Cassandra in this regard.
>
>
> On Mon, Jul 14, 2014 at 2:57 PM,
>
> Adaryl "Bob" Wakefield, MBA <ad...@hotmail.com> wrote:
>
>> Now this is different from Cassandra which does NOT use HDFS correct?
>> (Sorry. Don’t know why that needed two emails.)
>>
>> B.
>>
>> *From:* Ted Yu <yu...@gmail.com>
>> *Sent:* Monday, July 14, 2014 4:53 PM
>> *To:* mailto:user@hadoop.apache.org <us...@hadoop.apache.org>
>> *Subject:* Re: clarification on HBASE functionality
>>
>> Yes.
>> See http://hbase.apache.org/book.html#arch.hdfs
>>
>>
>> On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <
>> adaryl.wakefield@hotmail.com> wrote:
>>
>>> HBASE uses HDFS to store it's data correct?
>>>
>>> B.
>>>
>>
>>
>
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: clarification on HBASE functionality
Posted by Jay Vyas <ja...@gmail.com>.
Hbase is not harcoded to hdfs: it works on any file system that implements the file system interface, we've run it on glusterfs for example. I assume some have also run it on s3 and other alternative file systems .
** However **
For best performance, direct block io hooks on hdfs can boost high throughout applications on hdfs.
Ultimately, the hbase root directory only needs a fully qualified FileSystem uri which maps to a FileSystem class.
> On Jul 14, 2014, at 5:59 PM, Ted Yu <yu...@gmail.com> wrote:
>
> Right.
> hbase is different from Cassandra in this regard.
>
>
>> On Mon, Jul 14, 2014 at 2:57 PM, Adaryl "Bob" Wakefield, MBA <ad...@hotmail.com> wrote:
>> Now this is different from Cassandra which does NOT use HDFS correct? (Sorry. Don’t know why that needed two emails.)
>>
>> B.
>>
>> From: Ted Yu
>> Sent: Monday, July 14, 2014 4:53 PM
>> To: mailto:user@hadoop.apache.org
>> Subject: Re: clarification on HBASE functionality
>>
>> Yes.
>> See http://hbase.apache.org/book.html#arch.hdfs
>>
>>
>>> On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <ad...@hotmail.com> wrote:
>>> HBASE uses HDFS to store it's data correct?
>>>
>>> B.
>
Re: clarification on HBASE functionality
Posted by Jay Vyas <ja...@gmail.com>.
Hbase is not harcoded to hdfs: it works on any file system that implements the file system interface, we've run it on glusterfs for example. I assume some have also run it on s3 and other alternative file systems .
** However **
For best performance, direct block io hooks on hdfs can boost high throughout applications on hdfs.
Ultimately, the hbase root directory only needs a fully qualified FileSystem uri which maps to a FileSystem class.
> On Jul 14, 2014, at 5:59 PM, Ted Yu <yu...@gmail.com> wrote:
>
> Right.
> hbase is different from Cassandra in this regard.
>
>
>> On Mon, Jul 14, 2014 at 2:57 PM, Adaryl "Bob" Wakefield, MBA <ad...@hotmail.com> wrote:
>> Now this is different from Cassandra which does NOT use HDFS correct? (Sorry. Don’t know why that needed two emails.)
>>
>> B.
>>
>> From: Ted Yu
>> Sent: Monday, July 14, 2014 4:53 PM
>> To: mailto:user@hadoop.apache.org
>> Subject: Re: clarification on HBASE functionality
>>
>> Yes.
>> See http://hbase.apache.org/book.html#arch.hdfs
>>
>>
>>> On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <ad...@hotmail.com> wrote:
>>> HBASE uses HDFS to store it's data correct?
>>>
>>> B.
>
Re: clarification on HBASE functionality
Posted by Jay Vyas <ja...@gmail.com>.
Hbase is not harcoded to hdfs: it works on any file system that implements the file system interface, we've run it on glusterfs for example. I assume some have also run it on s3 and other alternative file systems .
** However **
For best performance, direct block io hooks on hdfs can boost high throughout applications on hdfs.
Ultimately, the hbase root directory only needs a fully qualified FileSystem uri which maps to a FileSystem class.
> On Jul 14, 2014, at 5:59 PM, Ted Yu <yu...@gmail.com> wrote:
>
> Right.
> hbase is different from Cassandra in this regard.
>
>
>> On Mon, Jul 14, 2014 at 2:57 PM, Adaryl "Bob" Wakefield, MBA <ad...@hotmail.com> wrote:
>> Now this is different from Cassandra which does NOT use HDFS correct? (Sorry. Don’t know why that needed two emails.)
>>
>> B.
>>
>> From: Ted Yu
>> Sent: Monday, July 14, 2014 4:53 PM
>> To: mailto:user@hadoop.apache.org
>> Subject: Re: clarification on HBASE functionality
>>
>> Yes.
>> See http://hbase.apache.org/book.html#arch.hdfs
>>
>>
>>> On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <ad...@hotmail.com> wrote:
>>> HBASE uses HDFS to store it's data correct?
>>>
>>> B.
>
Re: clarification on HBASE functionality
Posted by Jay Vyas <ja...@gmail.com>.
Hbase is not harcoded to hdfs: it works on any file system that implements the file system interface, we've run it on glusterfs for example. I assume some have also run it on s3 and other alternative file systems .
** However **
For best performance, direct block io hooks on hdfs can boost high throughout applications on hdfs.
Ultimately, the hbase root directory only needs a fully qualified FileSystem uri which maps to a FileSystem class.
> On Jul 14, 2014, at 5:59 PM, Ted Yu <yu...@gmail.com> wrote:
>
> Right.
> hbase is different from Cassandra in this regard.
>
>
>> On Mon, Jul 14, 2014 at 2:57 PM, Adaryl "Bob" Wakefield, MBA <ad...@hotmail.com> wrote:
>> Now this is different from Cassandra which does NOT use HDFS correct? (Sorry. Don’t know why that needed two emails.)
>>
>> B.
>>
>> From: Ted Yu
>> Sent: Monday, July 14, 2014 4:53 PM
>> To: mailto:user@hadoop.apache.org
>> Subject: Re: clarification on HBASE functionality
>>
>> Yes.
>> See http://hbase.apache.org/book.html#arch.hdfs
>>
>>
>>> On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <ad...@hotmail.com> wrote:
>>> HBASE uses HDFS to store it's data correct?
>>>
>>> B.
>
Re: clarification on HBASE functionality
Posted by Ted Yu <yu...@gmail.com>.
Right.
hbase is different from Cassandra in this regard.
On Mon, Jul 14, 2014 at 2:57 PM, Adaryl "Bob" Wakefield, MBA <
adaryl.wakefield@hotmail.com> wrote:
> Now this is different from Cassandra which does NOT use HDFS correct?
> (Sorry. Don’t know why that needed two emails.)
>
> B.
>
> *From:* Ted Yu <yu...@gmail.com>
> *Sent:* Monday, July 14, 2014 4:53 PM
> *To:* mailto:user@hadoop.apache.org <us...@hadoop.apache.org>
> *Subject:* Re: clarification on HBASE functionality
>
> Yes.
> See http://hbase.apache.org/book.html#arch.hdfs
>
>
> On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <
> adaryl.wakefield@hotmail.com> wrote:
>
>> HBASE uses HDFS to store it's data correct?
>>
>> B.
>>
>
>
Re: clarification on HBASE functionality
Posted by Ted Yu <yu...@gmail.com>.
Right.
hbase is different from Cassandra in this regard.
On Mon, Jul 14, 2014 at 2:57 PM, Adaryl "Bob" Wakefield, MBA <
adaryl.wakefield@hotmail.com> wrote:
> Now this is different from Cassandra which does NOT use HDFS correct?
> (Sorry. Don’t know why that needed two emails.)
>
> B.
>
> *From:* Ted Yu <yu...@gmail.com>
> *Sent:* Monday, July 14, 2014 4:53 PM
> *To:* mailto:user@hadoop.apache.org <us...@hadoop.apache.org>
> *Subject:* Re: clarification on HBASE functionality
>
> Yes.
> See http://hbase.apache.org/book.html#arch.hdfs
>
>
> On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <
> adaryl.wakefield@hotmail.com> wrote:
>
>> HBASE uses HDFS to store it's data correct?
>>
>> B.
>>
>
>
Re: clarification on HBASE functionality
Posted by Ted Yu <yu...@gmail.com>.
Right.
hbase is different from Cassandra in this regard.
On Mon, Jul 14, 2014 at 2:57 PM, Adaryl "Bob" Wakefield, MBA <
adaryl.wakefield@hotmail.com> wrote:
> Now this is different from Cassandra which does NOT use HDFS correct?
> (Sorry. Don’t know why that needed two emails.)
>
> B.
>
> *From:* Ted Yu <yu...@gmail.com>
> *Sent:* Monday, July 14, 2014 4:53 PM
> *To:* mailto:user@hadoop.apache.org <us...@hadoop.apache.org>
> *Subject:* Re: clarification on HBASE functionality
>
> Yes.
> See http://hbase.apache.org/book.html#arch.hdfs
>
>
> On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <
> adaryl.wakefield@hotmail.com> wrote:
>
>> HBASE uses HDFS to store it's data correct?
>>
>> B.
>>
>
>
Re: clarification on HBASE functionality
Posted by Ted Yu <yu...@gmail.com>.
Right.
hbase is different from Cassandra in this regard.
On Mon, Jul 14, 2014 at 2:57 PM, Adaryl "Bob" Wakefield, MBA <
adaryl.wakefield@hotmail.com> wrote:
> Now this is different from Cassandra which does NOT use HDFS correct?
> (Sorry. Don’t know why that needed two emails.)
>
> B.
>
> *From:* Ted Yu <yu...@gmail.com>
> *Sent:* Monday, July 14, 2014 4:53 PM
> *To:* mailto:user@hadoop.apache.org <us...@hadoop.apache.org>
> *Subject:* Re: clarification on HBASE functionality
>
> Yes.
> See http://hbase.apache.org/book.html#arch.hdfs
>
>
> On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <
> adaryl.wakefield@hotmail.com> wrote:
>
>> HBASE uses HDFS to store it's data correct?
>>
>> B.
>>
>
>
Re: clarification on HBASE functionality
Posted by "Adaryl \"Bob\" Wakefield, MBA" <ad...@hotmail.com>.
Now this is different from Cassandra which does NOT use HDFS correct? (Sorry. Don’t know why that needed two emails.)
B.
From: Ted Yu
Sent: Monday, July 14, 2014 4:53 PM
To: mailto:user@hadoop.apache.org
Subject: Re: clarification on HBASE functionality
Yes.
See http://hbase.apache.org/book.html#arch.hdfs
On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <ad...@hotmail.com> wrote:
HBASE uses HDFS to store it's data correct?
B.
Re: clarification on HBASE functionality
Posted by "Adaryl \"Bob\" Wakefield, MBA" <ad...@hotmail.com>.
Now this is different from Cassandra which does NOT use HDFS correct? (Sorry. Don’t know why that needed two emails.)
B.
From: Ted Yu
Sent: Monday, July 14, 2014 4:53 PM
To: mailto:user@hadoop.apache.org
Subject: Re: clarification on HBASE functionality
Yes.
See http://hbase.apache.org/book.html#arch.hdfs
On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <ad...@hotmail.com> wrote:
HBASE uses HDFS to store it's data correct?
B.
Re: clarification on HBASE functionality
Posted by "Adaryl \"Bob\" Wakefield, MBA" <ad...@hotmail.com>.
Now this is different from Cassandra which does NOT use HDFS correct? (Sorry. Don’t know why that needed two emails.)
B.
From: Ted Yu
Sent: Monday, July 14, 2014 4:53 PM
To: mailto:user@hadoop.apache.org
Subject: Re: clarification on HBASE functionality
Yes.
See http://hbase.apache.org/book.html#arch.hdfs
On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <ad...@hotmail.com> wrote:
HBASE uses HDFS to store it's data correct?
B.
Re: clarification on HBASE functionality
Posted by "Adaryl \"Bob\" Wakefield, MBA" <ad...@hotmail.com>.
Now this is different from Cassandra which does NOT use HDFS correct? (Sorry. Don’t know why that needed two emails.)
B.
From: Ted Yu
Sent: Monday, July 14, 2014 4:53 PM
To: mailto:user@hadoop.apache.org
Subject: Re: clarification on HBASE functionality
Yes.
See http://hbase.apache.org/book.html#arch.hdfs
On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <ad...@hotmail.com> wrote:
HBASE uses HDFS to store it's data correct?
B.
Re: clarification on HBASE functionality
Posted by Ted Yu <yu...@gmail.com>.
Yes.
See http://hbase.apache.org/book.html#arch.hdfs
On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <
adaryl.wakefield@hotmail.com> wrote:
> HBASE uses HDFS to store it's data correct?
>
> B.
>
Re: clarification on HBASE functionality
Posted by Ted Yu <yu...@gmail.com>.
Yes.
See http://hbase.apache.org/book.html#arch.hdfs
On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <
adaryl.wakefield@hotmail.com> wrote:
> HBASE uses HDFS to store it's data correct?
>
> B.
>
Re: clarification on HBASE functionality
Posted by Ted Yu <yu...@gmail.com>.
Yes.
See http://hbase.apache.org/book.html#arch.hdfs
On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <
adaryl.wakefield@hotmail.com> wrote:
> HBASE uses HDFS to store it's data correct?
>
> B.
>
Re: clarification on HBASE functionality
Posted by Ted Yu <yu...@gmail.com>.
Yes.
See http://hbase.apache.org/book.html#arch.hdfs
On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <
adaryl.wakefield@hotmail.com> wrote:
> HBASE uses HDFS to store it's data correct?
>
> B.
>