You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Ding, Hui" <hu...@sap.com> on 2008/09/12 01:09:37 UTC
Is map-reduce-merge coming?
Just read the map-reduce-merge paper by the Yahoo! folks. It seems
there's nothing fancy:just combined two map-reduce together and use a
merge for coordination.
But still, it would be great to see how it works. I'm wondering if
there is on-going effort to put this into the hbase impl? Or does hadoop
has this already?
RE: Is map-reduce-merge coming?
Posted by Jim Kellerman <ji...@powerset.com>.
HBase can be a source or a sink of any Hadoop map/reduce operation. To do joins, or other relational operations requires a custom map/reduce application.
In the future, when there are connectors for PIG or HIVE, custom map/reduce connectors will not be required. However there are currently no connectors between HBase or PIG/HIVE.
---
Jim Kellerman, Senior Software Development Engineer
Powerset (Live Search, Microsoft Corporation)
> -----Original Message-----
> From: Ding, Hui [mailto:hui.ding@sap.com]
> Sent: Thursday, September 11, 2008 4:18 PM
> To: hbase-dev@hadoop.apache.org
> Subject: RE: Is map-reduce-merge coming?
>
> So?
> Hadoop already supports performing join using mapreduce, with Hbase as
> source or sink?
>
> -----Original Message-----
> From: Jim Kellerman [mailto:jim@powerset.com]
> Sent: Thursday, September 11, 2008 4:16 PM
> To: hbase-dev@hadoop.apache.org
> Subject: RE: Is map-reduce-merge coming?
>
> HBase can be a source or a sink for map/reduce operations. See
> org.apache.hadoop.hbase.mapred.*
>
> ---
> Jim Kellerman, Senior Software Development Engineer
> Powerset (Live Search, Microsoft Corporation)
>
>
> > -----Original Message-----
> > From: Ding, Hui [mailto:hui.ding@sap.com]
> > Sent: Thursday, September 11, 2008 4:10 PM
> > To: hbase-dev@hadoop.apache.org
> > Subject: Is map-reduce-merge coming?
> >
> > Just read the map-reduce-merge paper by the Yahoo! folks. It seems
> > there's nothing fancy:just combined two map-reduce together and use a
> > merge for coordination.
> > But still, it would be great to see how it works. I'm wondering if
> > there is on-going effort to put this into the hbase impl? Or does
> hadoop
> > has this already?
RE: Is map-reduce-merge coming?
Posted by "Ding, Hui" <hu...@sap.com>.
So?
Hadoop already supports performing join using mapreduce, with Hbase as
source or sink?
-----Original Message-----
From: Jim Kellerman [mailto:jim@powerset.com]
Sent: Thursday, September 11, 2008 4:16 PM
To: hbase-dev@hadoop.apache.org
Subject: RE: Is map-reduce-merge coming?
HBase can be a source or a sink for map/reduce operations. See
org.apache.hadoop.hbase.mapred.*
---
Jim Kellerman, Senior Software Development Engineer
Powerset (Live Search, Microsoft Corporation)
> -----Original Message-----
> From: Ding, Hui [mailto:hui.ding@sap.com]
> Sent: Thursday, September 11, 2008 4:10 PM
> To: hbase-dev@hadoop.apache.org
> Subject: Is map-reduce-merge coming?
>
> Just read the map-reduce-merge paper by the Yahoo! folks. It seems
> there's nothing fancy:just combined two map-reduce together and use a
> merge for coordination.
> But still, it would be great to see how it works. I'm wondering if
> there is on-going effort to put this into the hbase impl? Or does
hadoop
> has this already?
RE: Is map-reduce-merge coming?
Posted by Jim Kellerman <ji...@powerset.com>.
HBase can be a source or a sink for map/reduce operations. See org.apache.hadoop.hbase.mapred.*
---
Jim Kellerman, Senior Software Development Engineer
Powerset (Live Search, Microsoft Corporation)
> -----Original Message-----
> From: Ding, Hui [mailto:hui.ding@sap.com]
> Sent: Thursday, September 11, 2008 4:10 PM
> To: hbase-dev@hadoop.apache.org
> Subject: Is map-reduce-merge coming?
>
> Just read the map-reduce-merge paper by the Yahoo! folks. It seems
> there's nothing fancy:just combined two map-reduce together and use a
> merge for coordination.
> But still, it would be great to see how it works. I'm wondering if
> there is on-going effort to put this into the hbase impl? Or does hadoop
> has this already?