You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Jim Holloway <ji...@windstream.net> on 2013/10/21 18:44:28 UTC
TableMapReduceUtil, multiple scan objects, how to identify
originating scan object in mapper
Hello,
I’m using the new 94.6 multi scan feature to pull rows from different tables
into a single mapper.
ArrayList<Scan> scans = new ArrayList<Scan>();
scans.add(scanMain);
scans.add(scanJunction);
TableMapReduceUtil.initTableMapperJob(scans,
EI_ExtractType3Mapper1.class,
Text.class,
Text.class,
job1);
My problem is I don’t know how to identify the source (table) of a
particular row coming in to my mapper job. My first couple of map/reduce
jobs I had the luxury of being able to identify the table source based on a
specific characteristic of the row key (one of the tables having an
underscore(_) embedded). Knew that was sloppy but haven’t been able to find
a better way.
Is there a definitive way?
Jim
--
View this message in context: http://apache-hbase.679495.n3.nabble.com/TableMapReduceUtil-multiple-scan-objects-how-to-identify-originating-scan-object-in-mapper-tp4051954.html
Sent from the HBase User mailing list archive at Nabble.com.
Re: TableMapReduceUtil, multiple scan objects, how to identify
originating scan object in mapper
Posted by Jim Holloway <ji...@windstream.net>.
Yes, I can do that.
Thank you for ‘Hbase in Action’. It’s been very valuable to me. I also very much like the Manning model of being able to buy the hard copy and then have the title available on my Kindle as well.
Jim
From: Nick Dimiduk [via Apache HBase]
Sent: Monday, October 21, 2013 7:02 PM
To: Jim Holloway
Subject: Re: TableMapReduceUtil, multiple scan objects, how to identify originating scan object in mapper
Hi Jim,
I don't see an obvious way to gain access to this information. If you don't
find a clever way to get at this, would you mind opening a ticket for this
feature request?
Thanks,
Nick
On Mon, Oct 21, 2013 at 9:44 AM, Jim Holloway
<[hidden email]>wrote:
> Hello,
>
> I’m using the new 94.6 multi scan feature to pull rows from different
> tables
> into a single mapper.
>
> ArrayList<Scan> scans = new ArrayList<Scan>();
> scans.add(scanMain);
> scans.add(scanJunction);
>
> TableMapReduceUtil.initTableMapperJob(scans,
> EI_ExtractType3Mapper1.class,
> Text.class,
> Text.class,
> job1);
>
> My problem is I don’t know how to identify the source (table) of a
> particular row coming in to my mapper job. My first couple of map/reduce
> jobs I had the luxury of being able to identify the table source based on a
> specific characteristic of the row key (one of the tables having an
> underscore(_) embedded). Knew that was sloppy but haven’t been able to
> find
> a better way.
>
> Is there a definitive way?
>
> Jim
>
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/TableMapReduceUtil-multiple-scan-objects-how-to-identify-originating-scan-object-in-mapper-tp4051954.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
--------------------------------------------------------------------------------
If you reply to this email, your message will be added to the discussion below:
http://apache-hbase.679495.n3.nabble.com/TableMapReduceUtil-multiple-scan-objects-how-to-identify-originating-scan-object-in-mapper-tp4051954p4051973.html
To unsubscribe from TableMapReduceUtil, multiple scan objects, how to identify originating scan object in mapper, click here.
NAML
--
View this message in context: http://apache-hbase.679495.n3.nabble.com/TableMapReduceUtil-multiple-scan-objects-how-to-identify-originating-scan-object-in-mapper-tp4051954p4052025.html
Sent from the HBase User mailing list archive at Nabble.com.
Re: TableMapReduceUtil, multiple scan objects, how to identify
originating scan object in mapper
Posted by Nick Dimiduk <nd...@gmail.com>.
Hi Jim,
I don't see an obvious way to gain access to this information. If you don't
find a clever way to get at this, would you mind opening a ticket for this
feature request?
Thanks,
Nick
On Mon, Oct 21, 2013 at 9:44 AM, Jim Holloway
<ji...@windstream.net>wrote:
> Hello,
>
> I’m using the new 94.6 multi scan feature to pull rows from different
> tables
> into a single mapper.
>
> ArrayList<Scan> scans = new ArrayList<Scan>();
> scans.add(scanMain);
> scans.add(scanJunction);
>
> TableMapReduceUtil.initTableMapperJob(scans,
> EI_ExtractType3Mapper1.class,
> Text.class,
> Text.class,
> job1);
>
> My problem is I don’t know how to identify the source (table) of a
> particular row coming in to my mapper job. My first couple of map/reduce
> jobs I had the luxury of being able to identify the table source based on a
> specific characteristic of the row key (one of the tables having an
> underscore(_) embedded). Knew that was sloppy but haven’t been able to
> find
> a better way.
>
> Is there a definitive way?
>
> Jim
>
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/TableMapReduceUtil-multiple-scan-objects-how-to-identify-originating-scan-object-in-mapper-tp4051954.html
> Sent from the HBase User mailing list archive at Nabble.com.
>