Running MR steps from different jar files

["Razon, Oren" <or...@intel.com>]

hi,
I'm usng Mahout 0.7. I'm building my own jar file which depeand on Maohut code (Mahout-core.jar).
I'm using maven as my build tool and creating thin jar's, meaning my jar does not include Mahout jar inisde of it and it is only refrence it in it's classpath.

When building a simple application which act as a driver that activate a set of MR steps at Mahout everything works fine.

But, when I'm adding a new reducer in my jar file, and than try to activate a series of MR steps that uses Mahout code and in addition my own custom reducer (which is in my jar) it fails doing so.
I'm using the command setJarByClass to add mahout jar to be avialble at the nodes (and it works fine) and also added a setJarByClass to my own custom jar.

So, how can I make 2 different jar's to be avialble at all nodes for a specific application? It seem lie if the setJarByClass can work only for one jar only.
BTW... When I'm creating one "fat" jar that encapsulate Mahout jar inside of it and than do setJarByClass only to my big "fat" jar it works. But I don't want to create such "fat" jars.

Thanks,
Oren



---------------------------------------------------------------------
Intel Electronics Ltd.

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.