You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Shrikrishna Shrin <kr...@cooliris.com> on 2009/07/11 02:10:32 UTC
PIG 0.3.0 cron job fail
Hi all,
This might be a naive question but we have a cron job that runs a bunch of
nightly PIG jobs and ever since we upgraded to PIG 0.3.0 the jobs started
failing because PIG tries to connect to the local file system instead of
HDFS. Running the job outside of cron works fine.
Example:
*script run via cron outputs:*
2009-07-10 01:08:26,122 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
to hadoop file system at:* file:///*
*
Running the same script outside of cron outputs:*
2009-07-10 16:57:58,543 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
to hadoop file system at: *hdfs://blah.com:54310*
Am I overlooking something? Should I be explicitly exporting some path in
cron? I haven't seen this issue with PIG 0.2.0.
Thanks,
Krishna
Re: PIG 0.3.0 cron job fail
Posted by Dmitriy Ryaboy <dv...@cloudera.com>.
Krishna,
Generally speaking, this kind of thing happens when your environment
is different in the interactive shell and in cron.
Try dumping your environment to a file and looking for differences.
What version of Hadoop are you running?
-Dmitriy
On Fri, Jul 10, 2009 at 5:10 PM, Shrikrishna Shrin<kr...@cooliris.com> wrote:
> Hi all,
>
> This might be a naive question but we have a cron job that runs a bunch of
> nightly PIG jobs and ever since we upgraded to PIG 0.3.0 the jobs started
> failing because PIG tries to connect to the local file system instead of
> HDFS. Running the job outside of cron works fine.
>
> Example:
> *script run via cron outputs:*
>
> 2009-07-10 01:08:26,122 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
> to hadoop file system at:* file:///*
> *
> Running the same script outside of cron outputs:*
>
> 2009-07-10 16:57:58,543 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
> to hadoop file system at: *hdfs://blah.com:54310*
>
> Am I overlooking something? Should I be explicitly exporting some path in
> cron? I haven't seen this issue with PIG 0.2.0.
> Thanks,
>
> Krishna
>