PigServer memory leak due to calling File.deleteOnExit() for job jars.

[Ankit Malpani <an...@gmail.com>]

Hello Pig Gurus,

I am using PigServer (
http://pig.apache.org/docs/r0.10.0/api/org/apache/pig/PigServer.html) to
schedule jobs on production (~100 per day) and realized that the cleanup of
job jar files on the local filesystem is triggered by calling
java.io.file.deleteOnExit() - indicating that these tmp files get deleted
only when the jvm shuts down gracefully. If my understanding is correct,
even if I write a separate cron to delete the leftover job jars, a long
running process/jvm which relies on using PigServer for executing pig
scripts would eventually run out of memory.

Has anyone faced such issues and if so found a work-around?

Any help/pointers are appreciated.

Other references to similar issues
grokbase.com/t/pig/user/101ts7cbzd/how-to-cleanup-old-job-jars ,
http://stackoverflow.com/questions/12711419/pigserver-leaving-job-jars-on-local-filesystem,


Thanks,
Ankit