You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Jim Holloway <ji...@windstream.net> on 2013/10/21 18:44:28 UTC

TableMapReduceUtil, multiple scan objects, how to identify originating scan object in mapper

Hello,

I’m using the new 94.6 multi scan feature to pull rows from different tables
into a single mapper.  

			ArrayList<Scan> scans = new ArrayList<Scan>();
			scans.add(scanMain);
			scans.add(scanJunction);

			TableMapReduceUtil.initTableMapperJob(scans,
					EI_ExtractType3Mapper1.class,
					Text.class,
					Text.class,
					job1);

My problem is I don’t know how to identify the source (table) of a
particular row coming in to my mapper job.  My first couple of map/reduce
jobs I had the luxury of being able to identify the table source based on a
specific characteristic of the row key (one of the tables having an
underscore(_) embedded).  Knew that was sloppy but haven’t been able to find
a better way.  

Is there a definitive way?  

Jim




--
View this message in context: http://apache-hbase.679495.n3.nabble.com/TableMapReduceUtil-multiple-scan-objects-how-to-identify-originating-scan-object-in-mapper-tp4051954.html
Sent from the HBase User mailing list archive at Nabble.com.

Re: TableMapReduceUtil, multiple scan objects, how to identify originating scan object in mapper

Posted by Jim Holloway <ji...@windstream.net>.
Yes, I can do that.

Thank you for ‘Hbase in Action’.  It’s been very valuable to me.  I also very much like the Manning model of being able to buy the hard copy and then have the title available on my Kindle as well.

Jim

From: Nick Dimiduk [via Apache HBase] 
Sent: Monday, October 21, 2013 7:02 PM
To: Jim Holloway 
Subject: Re: TableMapReduceUtil, multiple scan objects, how to identify originating scan object in mapper

Hi Jim, 

I don't see an obvious way to gain access to this information. If you don't 
find a clever way to get at this, would you mind opening a ticket for this 
feature request? 

Thanks, 
Nick 


On Mon, Oct 21, 2013 at 9:44 AM, Jim Holloway 
<[hidden email]>wrote: 


> Hello, 
> 
> I’m using the new 94.6 multi scan feature to pull rows from different 
> tables 
> into a single mapper. 
> 
>                         ArrayList<Scan> scans = new ArrayList<Scan>(); 
>                         scans.add(scanMain); 
>                         scans.add(scanJunction); 
> 
>                         TableMapReduceUtil.initTableMapperJob(scans, 
>                                         EI_ExtractType3Mapper1.class, 
>                                         Text.class, 
>                                         Text.class, 
>                                         job1); 
> 
> My problem is I don’t know how to identify the source (table) of a 
> particular row coming in to my mapper job.  My first couple of map/reduce 
> jobs I had the luxury of being able to identify the table source based on a 
> specific characteristic of the row key (one of the tables having an 
> underscore(_) embedded).  Knew that was sloppy but haven’t been able to 
> find 
> a better way. 
> 
> Is there a definitive way? 
> 
> Jim 
> 
> 
> 
> 
> -- 
> View this message in context: 
> http://apache-hbase.679495.n3.nabble.com/TableMapReduceUtil-multiple-scan-objects-how-to-identify-originating-scan-object-in-mapper-tp4051954.html
> Sent from the HBase User mailing list archive at Nabble.com. 
> 




--------------------------------------------------------------------------------

If you reply to this email, your message will be added to the discussion below:
http://apache-hbase.679495.n3.nabble.com/TableMapReduceUtil-multiple-scan-objects-how-to-identify-originating-scan-object-in-mapper-tp4051954p4051973.html 
To unsubscribe from TableMapReduceUtil, multiple scan objects, how to identify originating scan object in mapper, click here.
NAML



--
View this message in context: http://apache-hbase.679495.n3.nabble.com/TableMapReduceUtil-multiple-scan-objects-how-to-identify-originating-scan-object-in-mapper-tp4051954p4052025.html
Sent from the HBase User mailing list archive at Nabble.com.

Re: TableMapReduceUtil, multiple scan objects, how to identify originating scan object in mapper

Posted by Nick Dimiduk <nd...@gmail.com>.
Hi Jim,

I don't see an obvious way to gain access to this information. If you don't
find a clever way to get at this, would you mind opening a ticket for this
feature request?

Thanks,
Nick


On Mon, Oct 21, 2013 at 9:44 AM, Jim Holloway
<ji...@windstream.net>wrote:

> Hello,
>
> I’m using the new 94.6 multi scan feature to pull rows from different
> tables
> into a single mapper.
>
>                         ArrayList<Scan> scans = new ArrayList<Scan>();
>                         scans.add(scanMain);
>                         scans.add(scanJunction);
>
>                         TableMapReduceUtil.initTableMapperJob(scans,
>                                         EI_ExtractType3Mapper1.class,
>                                         Text.class,
>                                         Text.class,
>                                         job1);
>
> My problem is I don’t know how to identify the source (table) of a
> particular row coming in to my mapper job.  My first couple of map/reduce
> jobs I had the luxury of being able to identify the table source based on a
> specific characteristic of the row key (one of the tables having an
> underscore(_) embedded).  Knew that was sloppy but haven’t been able to
> find
> a better way.
>
> Is there a definitive way?
>
> Jim
>
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/TableMapReduceUtil-multiple-scan-objects-how-to-identify-originating-scan-object-in-mapper-tp4051954.html
> Sent from the HBase User mailing list archive at Nabble.com.
>