Map/Reduce from web servers

["Allen, Jeffrey" <ja...@ll.mit.edu>]

Hi,

I have an existing enterprise system using web services.  I'd like to have an event in the web service eventually result in a map/reduce being performed.  It would be very desirable to be able to package up the map reduce classes into a jar that gets deployed inside the war file for the web service.   (Primarily just to make redeployment easy.)  In effect, I believe what I'm asking is if there is a way I can stream the map/reduce code (ie, the jar file) down to the JobTracker.

Could someone please confirm whether I can do this or not, and if I can, help point me in the right direction?

Thanks in advance for any insight...

Regards
Jeff

Re: Map/Reduce from web servers

[Steve Loughran <st...@apache.org>]

Allen, Jeffrey wrote:
> Hi,
> 
> I have an existing enterprise system using web services.  I'd like to have an event in the web service eventually result in a map/reduce being performed.  It would be very desirable to be able to package up the map reduce classes into a jar that gets deployed inside the war file for the web service.   (Primarily just to make redeployment easy.)  In effect, I believe what I'm asking is if there is a way I can stream the map/reduce code (ie, the jar file) down to the JobTracker.
> 
> Could someone please confirm whether I can do this or not, and if I can, help point me in the right direction?
> 
> Thanks in advance for any insight...

You could embed the JAR in the WAR as a single resource, stream it out 
from there to the filesystem and then upload it. As a feature creep, you 
could see if the the copy operations handled jar: as a filesystem URI 
which would then enable a streamed upload.

-steve