Best way to access same data into mapper/reducer class

[Shuja Rehman <sh...@gmail.com>]

Hi All,

I have a simple question. I have a arraylist which i am populating through
db in main class. now i want to use the same list in my map and reduce class
so the question is how to access/send this array list in map and reduce
class. one way is to again fetch the data in mapper and reducer classes
which means 3 times db access and seems not suitable and other way either
could be define arraylist as static in main class and then access it or
might be using distributed cache.

so kindly let me know the best way of doing it

Thanks
-- 
Regards
Shuja-ur-Rehman Baig
<http://pk.linkedin.com/in/shujamughal>

Re: Best way to access same data into mapper/reducer class

[Chase Bradford <ch...@gmail.com>]

I did a write up on this sort of thing, but it was based on HashSets.
You might find it useful:

http://chasebradford.wordpress.com/2011/02/05/distributed-cache-static-objects-and-fast-setup/

On Mon, Feb 21, 2011 at 10:57 AM, Shuja Rehman <sh...@gmail.com> wrote:
> Hi All,
>
> I have a simple question. I have a arraylist which i am populating through
> db in main class. now i want to use the same list in my map and reduce class
> so the question is how to access/send this array list in map and reduce
> class. one way is to again fetch the data in mapper and reducer classes
> which means 3 times db access and seems not suitable and other way either
> could be define arraylist as static in main class and then access it or
> might be using distributed cache.
>
> so kindly let me know the best way of doing it
>
> Thanks
> --
> Regards
> Shuja-ur-Rehman Baig
>
>
>



-- 
Chase Bradford


“If in physics there's something you don't understand, you can always
hide behind the uncharted depths of nature. But if your program
doesn't work, there is no obstinate nature. If it doesn't work, you've
messed up.”

- Edsger Dijkstra