Optimisations

[Sylvain Menu <sy...@gmail.com>]

Hi,

I currently use Pig to send the same script every day. The only change are
inputs & output files + the parameter of my UDF to load inputs. So it is
possible to send the job and change only parameters in order to do faster
job ? Where is created the jar to send to Yarn ?

Thanks

Re: Optimisations

[Sylvain Menu <sy...@gmail.com>]

First step, I patch the code to not recreate the job.jar but use the last
generated. The jar always contains the same defaults classes defined in
JarManager + the pigContext. But the pigContext is no used because it is in
job.xml... Result: 10 sec less for each job :)

Maybe it is possible to give this possibility in a new release ?


2014-01-24 Sylvain Menu <sy...@gmail.com>

> Hi,
>
> I currently use Pig to send the same script every day. The only change are
> inputs & output files + the parameter of my UDF to load inputs. So it is
> possible to send the job and change only parameters in order to do faster
> job ? Where is created the jar to send to Yarn ?
>
> Thanks
>