can Hadoop writes to more than one table the same reduce?

[Jander g <ja...@gmail.com>]

Hi,

I want to use Hadoop to create index into mysql using inverted index. I can
do it when write to only one table, but now I need write to three tables,
how can I do it?

Furthermore, one table will use the primary key of the other two, which will
be known after inserting. Now, I create three table as follows:
Document: id(primary key), document_url;
Keywords: id(primary key), keyword;
Index: id, keyword_id, document_id.

Will is it feasible? Any help will be appreciated.

Thanks.
-- 
Regards,
Jander

Re: can Hadoop writes to more than one table the same reduce?

[Jander g <ja...@gmail.com>]

Hi, Hari

Thank you very much. I use DBOutputFormat to write mysql and write into one
table successfully. Well, I will try the ConnectionPoll.

Thanks again,
Jander

On Fri, Jan 14, 2011 at 11:32 PM, Hari Sreekumar
<hs...@clickable.com>wrote:

> I have never tried it, but just for my info... which outputformat do you
> use
> to write into one table? You could try using a ConnectionPool, innitiate
> conection to all 3 tables in setup() method and write to the tables in the
> reduce phase. Have you tried it?
>
> hope that helps,
> Hari
>
> On Thu, Jan 13, 2011 at 11:45 AM, Jander g <ja...@gmail.com> wrote:
>
> > Hi,
> >
> > I want to use Hadoop to create index into mysql using inverted index. I
> can
> > do it when write to only one table, but now I need write to three tables,
> > how can I do it?
> >
> > Furthermore, one table will use the primary key of the other two, which
> > will
> > be known after inserting. Now, I create three table as follows:
> > Document: id(primary key), document_url;
> > Keywords: id(primary key), keyword;
> > Index: id, keyword_id, document_id.
> >
> > Will is it feasible? Any help will be appreciated.
> >
> > Thanks.
> > --
> > Regards,
> > Jander
> >
>



-- 
Thanks,
Jander

Re: can Hadoop writes to more than one table the same reduce?

[Hari Sreekumar <hs...@clickable.com>]

I have never tried it, but just for my info... which outputformat do you use
to write into one table? You could try using a ConnectionPool, innitiate
conection to all 3 tables in setup() method and write to the tables in the
reduce phase. Have you tried it?

hope that helps,
Hari

On Thu, Jan 13, 2011 at 11:45 AM, Jander g <ja...@gmail.com> wrote:

> Hi,
>
> I want to use Hadoop to create index into mysql using inverted index. I can
> do it when write to only one table, but now I need write to three tables,
> how can I do it?
>
> Furthermore, one table will use the primary key of the other two, which
> will
> be known after inserting. Now, I create three table as follows:
> Document: id(primary key), document_url;
> Keywords: id(primary key), keyword;
> Index: id, keyword_id, document_id.
>
> Will is it feasible? Any help will be appreciated.
>
> Thanks.
> --
> Regards,
> Jander
>