How to join 2 tables using hadoop?

[Pavan Sudheendra <pa...@gmail.com>]

Hi,

I know that HBase by default doesn't support table joins like RDBMS..
But anyway, I have a table who value contains a json with a particular
ID in it..
This id references another table where it is a key..

I want to fetch the id first from table A , query table 2 and get its
corresponding value..

What is the best way of achieving this using the MR framework?
Apologizes, i'm still new to Hadoop and HBase so please go easy on me.

Thanks for any help

--
Regards-
Pavan

Re: How to join 2 tables using hadoop?

[Shahab Yunus <sh...@gmail.com>]

These are two separate things. first is to read data from Json format and
then joining it. we can separate those. You first need to read the row
using JSon storage API for Pig then yes, you can use native Pig constructs
for joining.

Regards,
Shahab

On Saturday, July 20, 2013, Pavan Sudheendra wrote:

> If i have the value of a row in JSON format, would pig we able to
> parse it and join the fields as per my needs?
>
> On Fri, Jul 19, 2013 at 10:00 PM, Shahab Yunus <shahab.yunus@gmail.com<javascript:;>>
> wrote:
> > You can also look into Pig, if you already haven't. It supports various
> > kinds of joins and is simpler than writing your own M/R job (assuming
> that
> > you don't have complex or custom requirements.)
> >
> > Regards,
> > Shahab
> >
> >
> > On Fri, Jul 19, 2013 at 12:24 AM, Pavan Sudheendra <pavan0591@gmail.com<javascript:;>
> >wrote:
> >
> >> Hi,
> >>
> >> I know that HBase by default doesn't support table joins like RDBMS..
> >> But anyway, I have a table who value contains a json with a particular
> >> ID in it..
> >> This id references another table where it is a key..
> >>
> >> I want to fetch the id first from table A , query table 2 and get its
> >> corresponding value..
> >>
> >> What is the best way of achieving this using the MR framework?
> >> Apologizes, i'm still new to Hadoop and HBase so please go easy on me.
> >>
> >> Thanks for any help
> >>
> >> --
> >> Regards-
> >> Pavan
> >>
>
>
>
> --
> Regards-
> Pavan
>

Re: How to join 2 tables using hadoop?

[Pavan Sudheendra <pa...@gmail.com>]

If i have the value of a row in JSON format, would pig we able to
parse it and join the fields as per my needs?

On Fri, Jul 19, 2013 at 10:00 PM, Shahab Yunus <sh...@gmail.com> wrote:
> You can also look into Pig, if you already haven't. It supports various
> kinds of joins and is simpler than writing your own M/R job (assuming that
> you don't have complex or custom requirements.)
>
> Regards,
> Shahab
>
>
> On Fri, Jul 19, 2013 at 12:24 AM, Pavan Sudheendra <pa...@gmail.com>wrote:
>
>> Hi,
>>
>> I know that HBase by default doesn't support table joins like RDBMS..
>> But anyway, I have a table who value contains a json with a particular
>> ID in it..
>> This id references another table where it is a key..
>>
>> I want to fetch the id first from table A , query table 2 and get its
>> corresponding value..
>>
>> What is the best way of achieving this using the MR framework?
>> Apologizes, i'm still new to Hadoop and HBase so please go easy on me.
>>
>> Thanks for any help
>>
>> --
>> Regards-
>> Pavan
>>



-- 
Regards-
Pavan

Re: How to join 2 tables using hadoop?

[Shahab Yunus <sh...@gmail.com>]

You can also look into Pig, if you already haven't. It supports various
kinds of joins and is simpler than writing your own M/R job (assuming that
you don't have complex or custom requirements.)

Regards,
Shahab


On Fri, Jul 19, 2013 at 12:24 AM, Pavan Sudheendra <pa...@gmail.com>wrote:

> Hi,
>
> I know that HBase by default doesn't support table joins like RDBMS..
> But anyway, I have a table who value contains a json with a particular
> ID in it..
> This id references another table where it is a key..
>
> I want to fetch the id first from table A , query table 2 and get its
> corresponding value..
>
> What is the best way of achieving this using the MR framework?
> Apologizes, i'm still new to Hadoop and HBase so please go easy on me.
>
> Thanks for any help
>
> --
> Regards-
> Pavan
>

Re: How to join 2 tables using hadoop?

[yonghu <yo...@gmail.com>]

 You can write one MR job to finish this. First read two tables at Map
function, the output key will be the reference key for one table and
primary key for the other table. At the Reduce function, you can "join" the
tuples which contain the same key. Please note this is a very naive
approach, for more join optimization options,  you can take a look at the
strategies which Pig or Hive uses.




On Fri, Jul 19, 2013 at 10:17 AM, Nitin Pawar <ni...@gmail.com>wrote:

> Try hive with hbase storage handler
>
>
> On Fri, Jul 19, 2013 at 9:54 AM, Pavan Sudheendra <pavan0591@gmail.com
> >wrote:
>
> > Hi,
> >
> > I know that HBase by default doesn't support table joins like RDBMS..
> > But anyway, I have a table who value contains a json with a particular
> > ID in it..
> > This id references another table where it is a key..
> >
> > I want to fetch the id first from table A , query table 2 and get its
> > corresponding value..
> >
> > What is the best way of achieving this using the MR framework?
> > Apologizes, i'm still new to Hadoop and HBase so please go easy on me.
> >
> > Thanks for any help
> >
> > --
> > Regards-
> > Pavan
> >
>
>
>
> --
> Nitin Pawar
>

Re: How to join 2 tables using hadoop?

[Nitin Pawar <ni...@gmail.com>]

Try hive with hbase storage handler


On Fri, Jul 19, 2013 at 9:54 AM, Pavan Sudheendra <pa...@gmail.com>wrote:

> Hi,
>
> I know that HBase by default doesn't support table joins like RDBMS..
> But anyway, I have a table who value contains a json with a particular
> ID in it..
> This id references another table where it is a key..
>
> I want to fetch the id first from table A , query table 2 and get its
> corresponding value..
>
> What is the best way of achieving this using the MR framework?
> Apologizes, i'm still new to Hadoop and HBase so please go easy on me.
>
> Thanks for any help
>
> --
> Regards-
> Pavan
>



-- 
Nitin Pawar