suggestion

[Yang <te...@gmail.com>]

let's say my pig script generates 2 MR jobs.

it seems that currently pig parser won't try to parse the second part until
it finishes running the first MR.

by that time 1 hour may have passed and now pig says:

blahblah does not exists in the schema of myvar_name : { x:int ,
y:chararray.......}

so a lot of time is wasted, particularly in debugging.

is there a mode like pig -c
(like perl -c ) to check syntax errors?

Thanks
Yang

Re: suggestion

[Yang <te...@gmail.com>]

perfect, thanks

On Fri, Jun 29, 2012 at 10:11 AM, Jie Li <ji...@cs.duke.edu> wrote:

> Pig does have a "-c" to check the syntax:
>
> pig -x local -c -f x.pig
>
> Jie
>
> On Fri, Jun 29, 2012 at 5:02 AM, Ruslan Al-Fakikh
> <ru...@jalent.ru> wrote:
> > Hey Yang,
> >
> > For debugging you may want the local mode, try
> > pig -x local
> >
> > Also there are some useful commands like, DESCRIBE, ILLUSTRATE
> >
> > Ruslan
> >
> > On Fri, Jun 29, 2012 at 7:38 AM, Jonathan Coveney <jc...@gmail.com>
> wrote:
> >> Do you have an example?
> >>
> >> 2012/6/28 Yang <te...@gmail.com>
> >>
> >>> thanks
> >>>
> >>>
> >>> it was simply "blahblah field does not existing in schema for my_var :
> >>> {......} "
> >>>
> >>>
> >>> On Thu, Jun 28, 2012 at 8:24 PM, Jonathan Coveney <jcoveney@gmail.com
> >>> >wrote:
> >>>
> >>> > Pig SHOULD parse the whole script, AFAIK. There are certain errors
> that
> >>> > will only surface at runtime, but in general, parsing errors should
> be
> >>> > surfacing early. Do you happen to have an example?
> >>> >
> >>> > 2012/6/28 Yang <te...@gmail.com>
> >>> >
> >>> > > let's say my pig script generates 2 MR jobs.
> >>> > >
> >>> > > it seems that currently pig parser won't try to parse the second
> part
> >>> > until
> >>> > > it finishes running the first MR.
> >>> > >
> >>> > > by that time 1 hour may have passed and now pig says:
> >>> > >
> >>> > > blahblah does not exists in the schema of myvar_name : { x:int ,
> >>> > > y:chararray.......}
> >>> > >
> >>> > > so a lot of time is wasted, particularly in debugging.
> >>> > >
> >>> > > is there a mode like pig -c
> >>> > > (like perl -c ) to check syntax errors?
> >>> > >
> >>> > > Thanks
> >>> > > Yang
> >>> > >
> >>> >
> >>>
> >
> >
> >
> > --
> > Best Regards,
> > Ruslan Al-Fakikh
> >
>

Re: suggestion

[Jie Li <ji...@cs.duke.edu>]

Pig does have a "-c" to check the syntax:

pig -x local -c -f x.pig

Jie

On Fri, Jun 29, 2012 at 5:02 AM, Ruslan Al-Fakikh
<ru...@jalent.ru> wrote:
> Hey Yang,
>
> For debugging you may want the local mode, try
> pig -x local
>
> Also there are some useful commands like, DESCRIBE, ILLUSTRATE
>
> Ruslan
>
> On Fri, Jun 29, 2012 at 7:38 AM, Jonathan Coveney <jc...@gmail.com> wrote:
>> Do you have an example?
>>
>> 2012/6/28 Yang <te...@gmail.com>
>>
>>> thanks
>>>
>>>
>>> it was simply "blahblah field does not existing in schema for my_var :
>>> {......} "
>>>
>>>
>>> On Thu, Jun 28, 2012 at 8:24 PM, Jonathan Coveney <jcoveney@gmail.com
>>> >wrote:
>>>
>>> > Pig SHOULD parse the whole script, AFAIK. There are certain errors that
>>> > will only surface at runtime, but in general, parsing errors should be
>>> > surfacing early. Do you happen to have an example?
>>> >
>>> > 2012/6/28 Yang <te...@gmail.com>
>>> >
>>> > > let's say my pig script generates 2 MR jobs.
>>> > >
>>> > > it seems that currently pig parser won't try to parse the second part
>>> > until
>>> > > it finishes running the first MR.
>>> > >
>>> > > by that time 1 hour may have passed and now pig says:
>>> > >
>>> > > blahblah does not exists in the schema of myvar_name : { x:int ,
>>> > > y:chararray.......}
>>> > >
>>> > > so a lot of time is wasted, particularly in debugging.
>>> > >
>>> > > is there a mode like pig -c
>>> > > (like perl -c ) to check syntax errors?
>>> > >
>>> > > Thanks
>>> > > Yang
>>> > >
>>> >
>>>
>
>
>
> --
> Best Regards,
> Ruslan Al-Fakikh
>

Re: suggestion

[Ruslan Al-Fakikh <ru...@jalent.ru>]

Hey Yang,

For debugging you may want the local mode, try
pig -x local

Also there are some useful commands like, DESCRIBE, ILLUSTRATE

Ruslan

On Fri, Jun 29, 2012 at 7:38 AM, Jonathan Coveney <jc...@gmail.com> wrote:
> Do you have an example?
>
> 2012/6/28 Yang <te...@gmail.com>
>
>> thanks
>>
>>
>> it was simply "blahblah field does not existing in schema for my_var :
>> {......} "
>>
>>
>> On Thu, Jun 28, 2012 at 8:24 PM, Jonathan Coveney <jcoveney@gmail.com
>> >wrote:
>>
>> > Pig SHOULD parse the whole script, AFAIK. There are certain errors that
>> > will only surface at runtime, but in general, parsing errors should be
>> > surfacing early. Do you happen to have an example?
>> >
>> > 2012/6/28 Yang <te...@gmail.com>
>> >
>> > > let's say my pig script generates 2 MR jobs.
>> > >
>> > > it seems that currently pig parser won't try to parse the second part
>> > until
>> > > it finishes running the first MR.
>> > >
>> > > by that time 1 hour may have passed and now pig says:
>> > >
>> > > blahblah does not exists in the schema of myvar_name : { x:int ,
>> > > y:chararray.......}
>> > >
>> > > so a lot of time is wasted, particularly in debugging.
>> > >
>> > > is there a mode like pig -c
>> > > (like perl -c ) to check syntax errors?
>> > >
>> > > Thanks
>> > > Yang
>> > >
>> >
>>



-- 
Best Regards,
Ruslan Al-Fakikh

Re: suggestion

[Jonathan Coveney <jc...@gmail.com>]

Do you have an example?

2012/6/28 Yang <te...@gmail.com>

> thanks
>
>
> it was simply "blahblah field does not existing in schema for my_var :
> {......} "
>
>
> On Thu, Jun 28, 2012 at 8:24 PM, Jonathan Coveney <jcoveney@gmail.com
> >wrote:
>
> > Pig SHOULD parse the whole script, AFAIK. There are certain errors that
> > will only surface at runtime, but in general, parsing errors should be
> > surfacing early. Do you happen to have an example?
> >
> > 2012/6/28 Yang <te...@gmail.com>
> >
> > > let's say my pig script generates 2 MR jobs.
> > >
> > > it seems that currently pig parser won't try to parse the second part
> > until
> > > it finishes running the first MR.
> > >
> > > by that time 1 hour may have passed and now pig says:
> > >
> > > blahblah does not exists in the schema of myvar_name : { x:int ,
> > > y:chararray.......}
> > >
> > > so a lot of time is wasted, particularly in debugging.
> > >
> > > is there a mode like pig -c
> > > (like perl -c ) to check syntax errors?
> > >
> > > Thanks
> > > Yang
> > >
> >
>

Re: suggestion

[Yang <te...@gmail.com>]

thanks


it was simply "blahblah field does not existing in schema for my_var :
{......} "


On Thu, Jun 28, 2012 at 8:24 PM, Jonathan Coveney <jc...@gmail.com>wrote:

> Pig SHOULD parse the whole script, AFAIK. There are certain errors that
> will only surface at runtime, but in general, parsing errors should be
> surfacing early. Do you happen to have an example?
>
> 2012/6/28 Yang <te...@gmail.com>
>
> > let's say my pig script generates 2 MR jobs.
> >
> > it seems that currently pig parser won't try to parse the second part
> until
> > it finishes running the first MR.
> >
> > by that time 1 hour may have passed and now pig says:
> >
> > blahblah does not exists in the schema of myvar_name : { x:int ,
> > y:chararray.......}
> >
> > so a lot of time is wasted, particularly in debugging.
> >
> > is there a mode like pig -c
> > (like perl -c ) to check syntax errors?
> >
> > Thanks
> > Yang
> >
>

Re: suggestion

[Jonathan Coveney <jc...@gmail.com>]

Pig SHOULD parse the whole script, AFAIK. There are certain errors that
will only surface at runtime, but in general, parsing errors should be
surfacing early. Do you happen to have an example?

2012/6/28 Yang <te...@gmail.com>

> let's say my pig script generates 2 MR jobs.
>
> it seems that currently pig parser won't try to parse the second part until
> it finishes running the first MR.
>
> by that time 1 hour may have passed and now pig says:
>
> blahblah does not exists in the schema of myvar_name : { x:int ,
> y:chararray.......}
>
> so a lot of time is wasted, particularly in debugging.
>
> is there a mode like pig -c
> (like perl -c ) to check syntax errors?
>
> Thanks
> Yang
>