hfile - Storefile

[Himanshu Vashishtha <hv...@cs.ualberta.ca>]

It will be great if anyone can point out the difference in their
functionality. What I understand is Storefile is made persistent in fs,
compactions occurred on these (to me, its an equivalent of SSTable).
Hfile is derived (evolved) from MapFile, giving access to faster random
reads (as it uses index to look for a key etc). Is it a handler sort of to
access the persisted Storefile? By their package structure, i think so
(Hfile is in io, storefile is in rs).

Now, the interesting thing is I know I am possibly wrong. As Ryan pointed
out recently:"Hfiles are write once read many. Once written they cannot be
modified so there is way to move things around."...
This is a SSTable property: they are immutable. And they are merged into one
by compactions (major).

I will appreciate if you can provide the missing link.

Thanks,
~Himanshu

Re: hfile - Storefile

[Stack <st...@duboce.net>]

This is my fault.  I'm moderator on the list.  Some stuff goes to
moderator first and I'm not fast enough clearing them.  Sorry about
that J-D and Himanshu.
St.Ack

On Wed, Sep 1, 2010 at 1:07 PM, Himanshu Vashishtha
<va...@gmail.com> wrote:
> yeah! it got posted twice, unintentionally. Thanks for the answer, J-D.
>
> On Wed, Sep 1, 2010 at 2:04 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
>
>> Like I answered this morning to the same question:
>>
>> StoreFile is just a wrapper for HFile to add HBase-sugar on top of it.
>>
>> J-D
>>
>> On Wed, Sep 1, 2010 at 7:59 AM, Himanshu Vashishtha
>> <hv...@cs.ualberta.ca> wrote:
>> > It will be great if anyone can point out the difference in their
>> > functionality. What I understand is Storefile is made persistent in fs,
>> > compactions occurred on these (to me, its an equivalent of SSTable).
>> > Hfile is derived (evolved) from MapFile, giving access to faster random
>> > reads (as it uses index to look for a key etc). Is it a handler sort of
>> to
>> > access the persisted Storefile? By their package structure, i think so
>> > (Hfile is in io, storefile is in rs).
>> >
>> > Now, the interesting thing is I know I am possibly wrong. As Ryan pointed
>> > out recently:"Hfiles are write once read many. Once written they cannot
>> be
>> > modified so there is way to move things around."...
>> > This is a SSTable property: they are immutable. And they are merged into
>> one
>> > by compactions (major).
>> >
>> > I will appreciate if you can provide the missing link.
>> >
>> > Thanks,
>> > ~Himanshu
>> >
>>
>

Re: hfile - Storefile

[Himanshu Vashishtha <va...@gmail.com>]

yeah! it got posted twice, unintentionally. Thanks for the answer, J-D.

On Wed, Sep 1, 2010 at 2:04 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:

> Like I answered this morning to the same question:
>
> StoreFile is just a wrapper for HFile to add HBase-sugar on top of it.
>
> J-D
>
> On Wed, Sep 1, 2010 at 7:59 AM, Himanshu Vashishtha
> <hv...@cs.ualberta.ca> wrote:
> > It will be great if anyone can point out the difference in their
> > functionality. What I understand is Storefile is made persistent in fs,
> > compactions occurred on these (to me, its an equivalent of SSTable).
> > Hfile is derived (evolved) from MapFile, giving access to faster random
> > reads (as it uses index to look for a key etc). Is it a handler sort of
> to
> > access the persisted Storefile? By their package structure, i think so
> > (Hfile is in io, storefile is in rs).
> >
> > Now, the interesting thing is I know I am possibly wrong. As Ryan pointed
> > out recently:"Hfiles are write once read many. Once written they cannot
> be
> > modified so there is way to move things around."...
> > This is a SSTable property: they are immutable. And they are merged into
> one
> > by compactions (major).
> >
> > I will appreciate if you can provide the missing link.
> >
> > Thanks,
> > ~Himanshu
> >
>

RE: hfile - Storefile

[Jonathan Gray <jg...@facebook.com>]

Himanshu,

Don't be afraid to check-out the HBase codebase from SVN and scope out HFile and StoreFile classes for yourself!

HBase has a fairly easy to follow codebase despite being rather large.

JG

> -----Original Message-----
> From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of Jean-
> Daniel Cryans
> Sent: Wednesday, September 01, 2010 1:04 PM
> To: dev@hbase.apache.org
> Subject: Re: hfile - Storefile
> 
> Like I answered this morning to the same question:
> 
> StoreFile is just a wrapper for HFile to add HBase-sugar on top of it.
> 
> J-D
> 
> On Wed, Sep 1, 2010 at 7:59 AM, Himanshu Vashishtha
> <hv...@cs.ualberta.ca> wrote:
> > It will be great if anyone can point out the difference in their
> > functionality. What I understand is Storefile is made persistent in
> fs,
> > compactions occurred on these (to me, its an equivalent of SSTable).
> > Hfile is derived (evolved) from MapFile, giving access to faster
> random
> > reads (as it uses index to look for a key etc). Is it a handler sort
> of to
> > access the persisted Storefile? By their package structure, i think
> so
> > (Hfile is in io, storefile is in rs).
> >
> > Now, the interesting thing is I know I am possibly wrong. As Ryan
> pointed
> > out recently:"Hfiles are write once read many. Once written they
> cannot be
> > modified so there is way to move things around."...
> > This is a SSTable property: they are immutable. And they are merged
> into one
> > by compactions (major).
> >
> > I will appreciate if you can provide the missing link.
> >
> > Thanks,
> > ~Himanshu
> >

Re: hfile - Storefile

[Jean-Daniel Cryans <jd...@apache.org>]

Like I answered this morning to the same question:

StoreFile is just a wrapper for HFile to add HBase-sugar on top of it.

J-D

On Wed, Sep 1, 2010 at 7:59 AM, Himanshu Vashishtha
<hv...@cs.ualberta.ca> wrote:
> It will be great if anyone can point out the difference in their
> functionality. What I understand is Storefile is made persistent in fs,
> compactions occurred on these (to me, its an equivalent of SSTable).
> Hfile is derived (evolved) from MapFile, giving access to faster random
> reads (as it uses index to look for a key etc). Is it a handler sort of to
> access the persisted Storefile? By their package structure, i think so
> (Hfile is in io, storefile is in rs).
>
> Now, the interesting thing is I know I am possibly wrong. As Ryan pointed
> out recently:"Hfiles are write once read many. Once written they cannot be
> modified so there is way to move things around."...
> This is a SSTable property: they are immutable. And they are merged into one
> by compactions (major).
>
> I will appreciate if you can provide the missing link.
>
> Thanks,
> ~Himanshu
>