You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by rajgopalv <ra...@gmail.com> on 2010/12/06 18:58:07 UTC
Maps sharing a common table.
Hi,
I'm writing a MR job to read values from a CSV file and insert it into hbase
using Htable.put()
So each map function will insert one row.
There is no reduce function.
But now, i open a htable instance inside every map function. This is bad..
i know...
but how can I share a common htable instance between multiple map jobs. ?
--
View this message in context: http://old.nabble.com/Maps-sharing-a-common-table.-tp30366727p30366727.html
Sent from the HBase User mailing list archive at Nabble.com.
Re: Maps sharing a common table.
Posted by rajgopalv <ra...@gmail.com>.
@all.. Thanks.
I created instance variable in the setup() function asd suggested.
rajgopalv wrote:
>
> Hi,
> I'm writing a MR job to read values from a CSV file and insert it into
> hbase using Htable.put()
> So each map function will insert one row.
> There is no reduce function.
>
> But now, i open a htable instance inside every map function. This is
> bad.. i know...
> but how can I share a common htable instance between multiple map jobs. ?
>
--
View this message in context: http://old.nabble.com/Maps-sharing-a-common-table.-tp30366727p30396247.html
Sent from the HBase User mailing list archive at Nabble.com.
RE: Maps sharing a common table.
Posted by Michael Segel <mi...@hotmail.com>.
> Date: Mon, 6 Dec 2010 10:03:02 -0800
> Subject: Re: Maps sharing a common table.
> From: jdcryans@apache.org
> To: user@hbase.apache.org
>
> You need to instantiate 1 HTable per Map task, then reuse it for every
> map() invocations. Sharing the actual object between JVMs isn't what
> you want to do.
>
> J-D
>
In Mapper class see Mapper.setup() and set up a class instance variable for the HTable connection.
> On Mon, Dec 6, 2010 at 9:58 AM, rajgopalv <ra...@gmail.com> wrote:
> >
> > Hi,
> > I'm writing a MR job to read values from a CSV file and insert it into hbase
> > using Htable.put()
> > So each map function will insert one row.
> > There is no reduce function.
> >
> > But now, i open a htable instance inside every map function. This is bad..
> > i know...
> > but how can I share a common htable instance between multiple map jobs. ?
> > --
> > View this message in context: http://old.nabble.com/Maps-sharing-a-common-table.-tp30366727p30366727.html
> > Sent from the HBase User mailing list archive at Nabble.com.
> >
> >
Re: Maps sharing a common table.
Posted by Jean-Daniel Cryans <jd...@apache.org>.
You need to instantiate 1 HTable per Map task, then reuse it for every
map() invocations. Sharing the actual object between JVMs isn't what
you want to do.
J-D
On Mon, Dec 6, 2010 at 9:58 AM, rajgopalv <ra...@gmail.com> wrote:
>
> Hi,
> I'm writing a MR job to read values from a CSV file and insert it into hbase
> using Htable.put()
> So each map function will insert one row.
> There is no reduce function.
>
> But now, i open a htable instance inside every map function. This is bad..
> i know...
> but how can I share a common htable instance between multiple map jobs. ?
> --
> View this message in context: http://old.nabble.com/Maps-sharing-a-common-table.-tp30366727p30366727.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
>