BigTable

[Leen Toelen <to...@gmail.com>]

Hi,

are there any plans to create a structured data storage system like
Google's BigTable in Hadoop? Their implementation relies heavily on
mapreduce and gfs, so Hadoop is an ideal match. BigTable is what
drives google base, earth and news for instance.

http://blog.outer-court.com/archive/2005-10-23-n61.html

Regards,
Leen Toelen

Re: BigTable

[Stefan Groschupf <sg...@media-style.com>]

> Stefan, if you've got something in this area, perhaps we should
> share...

Yes I would love to! I was working on basic infrastructure like  
master - slave  - client but there are no well working node internals  
itself (write buffer, checkpoint writing etc.).
I can clean up stuff and put it somewhere if there is interest but I  
need some time since I will be not able to access the code until next  
days.

However beside that I in general very much interested to work with  
other people on this.
May be we can make a short general master plan and split the work?

Stefan
	

Re: BigTable

[Michael Cafarella <mi...@gmail.com>]

Hi Leen,

I've taken a look at this, and it's quite feasible.  I've been
working on a class I call HStore, which is a single-machine
row store.  This would be an important building block in any
such system.  (One would then need to design the table-distribution
aspect.)

Stefan, if you've got something in this area, perhaps we should
share...

--Mike



On 5/11/06, Leen Toelen <to...@gmail.com> wrote:
> Hi,
>
> are there any plans to create a structured data storage system like
> Google's BigTable in Hadoop? Their implementation relies heavily on
> mapreduce and gfs, so Hadoop is an ideal match. BigTable is what
> drives google base, earth and news for instance.
>
> http://blog.outer-court.com/archive/2005-10-23-n61.html
>
> Regards,
> Leen Toelen
>

Re: BigTable

[Stefan Groschupf <sg...@media-style.com>]

Hi,
I'm already have some very basic code but it does not work yet,  
however it is part of a project and I plan to publish  it somehow.
But I can not imaging that it will find the way to the hadoop svn  
once. However it uses  hadoop rcp, dfs and writables.
Stefan

Am 11.05.2006 um 00:08 schrieb Leen Toelen:

> Hi,
>
> are there any plans to create a structured data storage system like
> Google's BigTable in Hadoop? Their implementation relies heavily on
> mapreduce and gfs, so Hadoop is an ideal match. BigTable is what
> drives google base, earth and news for instance.
>
> http://blog.outer-court.com/archive/2005-10-23-n61.html
>
> Regards,
> Leen Toelen
>