would HBase benefit from a underlying file system that supports random r/w

[Sujee Maniyam <su...@sujee.net>]

i understand currently Hbase does append on HDFS, b/c HDFS doesn't allow
random seek + modification.

how will a file system that provides random seek+modify benefit hbase?

Does HBase do anything special to take advantage of MapR file sytem (which
I _think_ supports random r/w)

just curious

thanks
Sujee

http://sujee.net

Re: would HBase benefit from a underlying file system that supports random r/w

[Jason Frantz <jf...@maprtech.com>]

Hi Sujee,

You are correct that MapR's filesystem supports random read / write and
updating in place. However, HBase was written for HDFS interfaces which
don't allow modifying existing files.

That said, there are a number of features of our filesystem that can be
used via HBase (e.g. snapshots, mirroring), as well as our usual management
features. Here's a link to some HBase slides we presented a few months back
that provides a few more details:

http://www.slideshare.net/tshiran1/hbase-with-mapr

-Jason

On Mon, Jul 23, 2012 at 2:24 PM, Sujee Maniyam <su...@sujee.net> wrote:

> i understand currently Hbase does append on HDFS, b/c HDFS doesn't allow
> random seek + modification.
>
> how will a file system that provides random seek+modify benefit hbase?
>
> Does HBase do anything special to take advantage of MapR file sytem (which
> I _think_ supports random r/w)
>
> just curious
>
> thanks
> Sujee
>
> http://sujee.net
>

Re: would HBase benefit from a underlying file system that supports random r/w

[Amandeep Khurana <am...@gmail.com>]

On Monday, July 23, 2012 at 2:24 PM, Sujee Maniyam wrote:

> i understand currently Hbase does append on HDFS, b/c HDFS doesn't allow
> random seek + modification.
> 
> how will a file system that provides random seek+modify benefit hbase?
In the current architecture, I don't think there will be much benefit. One could argue that you could create mutable HFiles and do away with things like compactions if you had the ability to do random writes. But then the issue is that every time there is an insert, you'll be seeking and that's expensive. Think a step further - a FS with random writes running over SSDs might change the game a little bit more and take away the seek costs. That's an entirely different conversation. 
> 
> Does HBase do anything special to take advantage of MapR file sytem (which
> I _think_ supports random r/w)
> 
> just curious
> 
> thanks
> Sujee
> 
> http://sujee.net