You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Mix Nin <pi...@gmail.com> on 2013/03/27 22:40:29 UTC
Commands not working properlry when stored in pig file
I wrote a pig script as follows and stored it in x.pig file
Data = LOAD '/....' as (,,,, )
NoNullData= FILTER Data by qe is not null;
STORE (foreach (group NoNullData all) generate flatten($1)) into
'exp/$inputDatePig';
evnt_dtl =LOAD 'exp/$inputDatePig/part-r-00000' AS (cust,,,,,)
I executed the command as follows
pig -f x.pig -param inputDatePig=03272013
And finally it says exp/03272013 tough the directory exists as it gets
created in STORE command.
What is wrong in this
Re: Commands not working properlry when stored in pig file
Posted by Mix Nin <pi...@gmail.com>.
This is the error I get
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 32% complete
2013-03-27 14:38:35,568 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 50% complete
2013-03-27 14:38:45,731 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- job null has failed! Stop running all dependent jobs
2013-03-27 14:38:45,731 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete
2013-03-27 14:38:45,734 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
recreate exception from backend error:
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input
path does not exist: hdfs://user/lnindrakrishna/exp/03272013/part-r-00000
But when I remove second LOAD command, everything runs fine
On Wed, Mar 27, 2013 at 2:40 PM, Mix Nin <pi...@gmail.com> wrote:
> I wrote a pig script as follows and stored it in x.pig file
>
> Data = LOAD '/....' as (,,,, )
> NoNullData= FILTER Data by qe is not null;
> STORE (foreach (group NoNullData all) generate flatten($1)) into
> 'exp/$inputDatePig';
>
>
> evnt_dtl =LOAD 'exp/$inputDatePig/part-r-00000' AS (cust,,,,,)
>
>
>
> I executed the command as follows
>
> pig -f x.pig -param inputDatePig=03272013
>
>
> And finally it says exp/03272013 tough the directory exists as it gets
> created in STORE command.
>
> What is wrong in this
>
Re: Commands not working properlry when stored in pig file
Posted by Harsh J <ha...@cloudera.com>.
Moving to Pig's user lists. Please use the right lists for your user questions.
On Thu, Mar 28, 2013 at 3:10 AM, Mix Nin <pi...@gmail.com> wrote:
> I wrote a pig script as follows and stored it in x.pig file
>
> Data = LOAD '/....' as (,,,, )
> NoNullData= FILTER Data by qe is not null;
> STORE (foreach (group NoNullData all) generate flatten($1)) into
> 'exp/$inputDatePig';
>
>
> evnt_dtl =LOAD 'exp/$inputDatePig/part-r-00000' AS (cust,,,,,)
>
>
>
> I executed the command as follows
>
> pig -f x.pig -param inputDatePig=03272013
>
>
> And finally it says exp/03272013 tough the directory exists as it gets
> created in STORE command.
>
> What is wrong in this
--
Harsh J
Re: Commands not working properlry when stored in pig file
Posted by Mix Nin <pi...@gmail.com>.
This is the error I get
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 32% complete
2013-03-27 14:38:35,568 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 50% complete
2013-03-27 14:38:45,731 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- job null has failed! Stop running all dependent jobs
2013-03-27 14:38:45,731 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete
2013-03-27 14:38:45,734 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
recreate exception from backend error:
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input
path does not exist: hdfs://user/lnindrakrishna/exp/03272013/part-r-00000
But when I remove second LOAD command, everything runs fine
On Wed, Mar 27, 2013 at 2:40 PM, Mix Nin <pi...@gmail.com> wrote:
> I wrote a pig script as follows and stored it in x.pig file
>
> Data = LOAD '/....' as (,,,, )
> NoNullData= FILTER Data by qe is not null;
> STORE (foreach (group NoNullData all) generate flatten($1)) into
> 'exp/$inputDatePig';
>
>
> evnt_dtl =LOAD 'exp/$inputDatePig/part-r-00000' AS (cust,,,,,)
>
>
>
> I executed the command as follows
>
> pig -f x.pig -param inputDatePig=03272013
>
>
> And finally it says exp/03272013 tough the directory exists as it gets
> created in STORE command.
>
> What is wrong in this
>
Re: Commands not working properlry when stored in pig file
Posted by Harsh J <ha...@cloudera.com>.
Moving to Pig's user lists. Please use the right lists for your user questions.
On Thu, Mar 28, 2013 at 3:10 AM, Mix Nin <pi...@gmail.com> wrote:
> I wrote a pig script as follows and stored it in x.pig file
>
> Data = LOAD '/....' as (,,,, )
> NoNullData= FILTER Data by qe is not null;
> STORE (foreach (group NoNullData all) generate flatten($1)) into
> 'exp/$inputDatePig';
>
>
> evnt_dtl =LOAD 'exp/$inputDatePig/part-r-00000' AS (cust,,,,,)
>
>
>
> I executed the command as follows
>
> pig -f x.pig -param inputDatePig=03272013
>
>
> And finally it says exp/03272013 tough the directory exists as it gets
> created in STORE command.
>
> What is wrong in this
--
Harsh J
Re: Commands not working properlry when stored in pig file
Posted by Prashant Kommireddi <pr...@gmail.com>.
Check hdfs to see if this really exists
hdfs://user/lnindrakrishna/exp/03272013/part-r-00000
On Wed, Mar 27, 2013 at 2:40 PM, Mix Nin <pi...@gmail.com> wrote:
> I wrote a pig script as follows and stored it in x.pig file
>
> Data = LOAD '/....' as (,,,, )
> NoNullData= FILTER Data by qe is not null;
> STORE (foreach (group NoNullData all) generate flatten($1)) into
> 'exp/$inputDatePig';
>
>
> evnt_dtl =LOAD 'exp/$inputDatePig/part-r-00000' AS (cust,,,,,)
>
>
>
> I executed the command as follows
>
> pig -f x.pig -param inputDatePig=03272013
>
>
> And finally it says exp/03272013 tough the directory exists as it gets
> created in STORE command.
>
> What is wrong in this
>
Re: Commands not working properlry when stored in pig file
Posted by Prashant Kommireddi <pr...@gmail.com>.
Check hdfs to see if this really exists
hdfs://user/lnindrakrishna/exp/03272013/part-r-00000
On Wed, Mar 27, 2013 at 2:40 PM, Mix Nin <pi...@gmail.com> wrote:
> I wrote a pig script as follows and stored it in x.pig file
>
> Data = LOAD '/....' as (,,,, )
> NoNullData= FILTER Data by qe is not null;
> STORE (foreach (group NoNullData all) generate flatten($1)) into
> 'exp/$inputDatePig';
>
>
> evnt_dtl =LOAD 'exp/$inputDatePig/part-r-00000' AS (cust,,,,,)
>
>
>
> I executed the command as follows
>
> pig -f x.pig -param inputDatePig=03272013
>
>
> And finally it says exp/03272013 tough the directory exists as it gets
> created in STORE command.
>
> What is wrong in this
>
Re: Commands not working properlry when stored in pig file
Posted by Harsh J <ha...@cloudera.com>.
Moving to Pig's user lists. Please use the right lists for your user questions.
On Thu, Mar 28, 2013 at 3:10 AM, Mix Nin <pi...@gmail.com> wrote:
> I wrote a pig script as follows and stored it in x.pig file
>
> Data = LOAD '/....' as (,,,, )
> NoNullData= FILTER Data by qe is not null;
> STORE (foreach (group NoNullData all) generate flatten($1)) into
> 'exp/$inputDatePig';
>
>
> evnt_dtl =LOAD 'exp/$inputDatePig/part-r-00000' AS (cust,,,,,)
>
>
>
> I executed the command as follows
>
> pig -f x.pig -param inputDatePig=03272013
>
>
> And finally it says exp/03272013 tough the directory exists as it gets
> created in STORE command.
>
> What is wrong in this
--
Harsh J
Re: Commands not working properlry when stored in pig file
Posted by Prashant Kommireddi <pr...@gmail.com>.
Check hdfs to see if this really exists
hdfs://user/lnindrakrishna/exp/03272013/part-r-00000
On Wed, Mar 27, 2013 at 2:40 PM, Mix Nin <pi...@gmail.com> wrote:
> I wrote a pig script as follows and stored it in x.pig file
>
> Data = LOAD '/....' as (,,,, )
> NoNullData= FILTER Data by qe is not null;
> STORE (foreach (group NoNullData all) generate flatten($1)) into
> 'exp/$inputDatePig';
>
>
> evnt_dtl =LOAD 'exp/$inputDatePig/part-r-00000' AS (cust,,,,,)
>
>
>
> I executed the command as follows
>
> pig -f x.pig -param inputDatePig=03272013
>
>
> And finally it says exp/03272013 tough the directory exists as it gets
> created in STORE command.
>
> What is wrong in this
>
Re: Commands not working properlry when stored in pig file
Posted by Mix Nin <pi...@gmail.com>.
This is the error I get
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 32% complete
2013-03-27 14:38:35,568 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 50% complete
2013-03-27 14:38:45,731 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- job null has failed! Stop running all dependent jobs
2013-03-27 14:38:45,731 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete
2013-03-27 14:38:45,734 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
recreate exception from backend error:
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input
path does not exist: hdfs://user/lnindrakrishna/exp/03272013/part-r-00000
But when I remove second LOAD command, everything runs fine
On Wed, Mar 27, 2013 at 2:40 PM, Mix Nin <pi...@gmail.com> wrote:
> I wrote a pig script as follows and stored it in x.pig file
>
> Data = LOAD '/....' as (,,,, )
> NoNullData= FILTER Data by qe is not null;
> STORE (foreach (group NoNullData all) generate flatten($1)) into
> 'exp/$inputDatePig';
>
>
> evnt_dtl =LOAD 'exp/$inputDatePig/part-r-00000' AS (cust,,,,,)
>
>
>
> I executed the command as follows
>
> pig -f x.pig -param inputDatePig=03272013
>
>
> And finally it says exp/03272013 tough the directory exists as it gets
> created in STORE command.
>
> What is wrong in this
>
Re: Commands not working properlry when stored in pig file
Posted by Harsh J <ha...@cloudera.com>.
Moving to Pig's user lists. Please use the right lists for your user questions.
On Thu, Mar 28, 2013 at 3:10 AM, Mix Nin <pi...@gmail.com> wrote:
> I wrote a pig script as follows and stored it in x.pig file
>
> Data = LOAD '/....' as (,,,, )
> NoNullData= FILTER Data by qe is not null;
> STORE (foreach (group NoNullData all) generate flatten($1)) into
> 'exp/$inputDatePig';
>
>
> evnt_dtl =LOAD 'exp/$inputDatePig/part-r-00000' AS (cust,,,,,)
>
>
>
> I executed the command as follows
>
> pig -f x.pig -param inputDatePig=03272013
>
>
> And finally it says exp/03272013 tough the directory exists as it gets
> created in STORE command.
>
> What is wrong in this
--
Harsh J
Re: Commands not working properlry when stored in pig file
Posted by Harsh J <ha...@cloudera.com>.
Moving to Pig's user lists. Please use the right lists for your user questions.
On Thu, Mar 28, 2013 at 3:10 AM, Mix Nin <pi...@gmail.com> wrote:
> I wrote a pig script as follows and stored it in x.pig file
>
> Data = LOAD '/....' as (,,,, )
> NoNullData= FILTER Data by qe is not null;
> STORE (foreach (group NoNullData all) generate flatten($1)) into
> 'exp/$inputDatePig';
>
>
> evnt_dtl =LOAD 'exp/$inputDatePig/part-r-00000' AS (cust,,,,,)
>
>
>
> I executed the command as follows
>
> pig -f x.pig -param inputDatePig=03272013
>
>
> And finally it says exp/03272013 tough the directory exists as it gets
> created in STORE command.
>
> What is wrong in this
--
Harsh J
Re: Commands not working properlry when stored in pig file
Posted by Prashant Kommireddi <pr...@gmail.com>.
Check hdfs to see if this really exists
hdfs://user/lnindrakrishna/exp/03272013/part-r-00000
On Wed, Mar 27, 2013 at 2:40 PM, Mix Nin <pi...@gmail.com> wrote:
> I wrote a pig script as follows and stored it in x.pig file
>
> Data = LOAD '/....' as (,,,, )
> NoNullData= FILTER Data by qe is not null;
> STORE (foreach (group NoNullData all) generate flatten($1)) into
> 'exp/$inputDatePig';
>
>
> evnt_dtl =LOAD 'exp/$inputDatePig/part-r-00000' AS (cust,,,,,)
>
>
>
> I executed the command as follows
>
> pig -f x.pig -param inputDatePig=03272013
>
>
> And finally it says exp/03272013 tough the directory exists as it gets
> created in STORE command.
>
> What is wrong in this
>
Re: Commands not working properlry when stored in pig file
Posted by Mix Nin <pi...@gmail.com>.
This is the error I get
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 32% complete
2013-03-27 14:38:35,568 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 50% complete
2013-03-27 14:38:45,731 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- job null has failed! Stop running all dependent jobs
2013-03-27 14:38:45,731 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete
2013-03-27 14:38:45,734 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
recreate exception from backend error:
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input
path does not exist: hdfs://user/lnindrakrishna/exp/03272013/part-r-00000
But when I remove second LOAD command, everything runs fine
On Wed, Mar 27, 2013 at 2:40 PM, Mix Nin <pi...@gmail.com> wrote:
> I wrote a pig script as follows and stored it in x.pig file
>
> Data = LOAD '/....' as (,,,, )
> NoNullData= FILTER Data by qe is not null;
> STORE (foreach (group NoNullData all) generate flatten($1)) into
> 'exp/$inputDatePig';
>
>
> evnt_dtl =LOAD 'exp/$inputDatePig/part-r-00000' AS (cust,,,,,)
>
>
>
> I executed the command as follows
>
> pig -f x.pig -param inputDatePig=03272013
>
>
> And finally it says exp/03272013 tough the directory exists as it gets
> created in STORE command.
>
> What is wrong in this
>