You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by hc busy <hc...@gmail.com> on 2010/09/30 05:09:55 UTC
funny error
Guys, I'm seeing this one
2998
Unexpected internal error.
Can we be more specific or dump a stack trace when this happens?
RE: funny error
Posted by Santhosh Srinivasan <sm...@yahoo-inc.com>.
Are you sure that the 2998 and 1007 both popped up?
-----Original Message-----
From: hc busy [mailto:hc.busy@gmail.com]
Sent: Wednesday, September 29, 2010 8:21 PM
To: pig-user@hadoop.apache.org
Subject: Re: funny error
But having complained about error, I want to say this seems like a really courteous error message
ERROR 1007: Found duplicates in schema. : 2 columns, column_name: 2 columns.
*Please* alias the columns with unique names.
Where it explains nicely how to fix the error.
On Wed, Sep 29, 2010 at 8:18 PM, hc busy <hc...@gmail.com> wrote:
> "null" was the error. this 60k PigLatin script that I'm running hasn't
> changed that much, but suddenly started erroring out. I've rebuilt pig
> release 7 from scratch, checked java version, err... checked PiggyBank
> and our own libraries, not there changed.
>
> You know, some comercial software that has professional support will
> actually send emails when an "unhandled error" occurs. The email is
> received by the developer/support and diagnosed. And in that email, it
> would contain all the gory details that the product doesn't want to display to the user.
>
>
> I wonder if you guys are up to doing something like that for pig?
>
>
>
> On Wed, Sep 29, 2010 at 8:13 PM, Jeff Zhang <zj...@gmail.com> wrote:
>
>> No other stack trace ? And in what situation does this happen ?
>>
>>
>>
>> On Thu, Sep 30, 2010 at 11:09 AM, hc busy <hc...@gmail.com> wrote:
>> > Guys, I'm seeing this one
>> >
>> > 2998
>> >
>> > Unexpected internal error.
>> >
>> >
>> > Can we be more specific or dump a stack trace when this happens?
>> >
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
Re: funny error
Posted by hc busy <hc...@gmail.com>.
But having complained about error, I want to say this seems like a
really courteous error message
ERROR 1007: Found duplicates in schema. : 2 columns, column_name: 2 columns.
*Please* alias the columns with unique names.
Where it explains nicely how to fix the error.
On Wed, Sep 29, 2010 at 8:18 PM, hc busy <hc...@gmail.com> wrote:
> "null" was the error. this 60k PigLatin script that I'm running hasn't
> changed that much, but suddenly started erroring out. I've rebuilt pig
> release 7 from scratch, checked java version, err... checked PiggyBank and
> our own libraries, not there changed.
>
> You know, some comercial software that has professional support will
> actually send emails when an "unhandled error" occurs. The email is received
> by the developer/support and diagnosed. And in that email, it would contain
> all the gory details that the product doesn't want to display to the user.
>
>
> I wonder if you guys are up to doing something like that for pig?
>
>
>
> On Wed, Sep 29, 2010 at 8:13 PM, Jeff Zhang <zj...@gmail.com> wrote:
>
>> No other stack trace ? And in what situation does this happen ?
>>
>>
>>
>> On Thu, Sep 30, 2010 at 11:09 AM, hc busy <hc...@gmail.com> wrote:
>> > Guys, I'm seeing this one
>> >
>> > 2998
>> >
>> > Unexpected internal error.
>> >
>> >
>> > Can we be more specific or dump a stack trace when this happens?
>> >
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
Re: funny error
Posted by Thejas M Nair <te...@yahoo-inc.com>.
If it crashed while executing MR job , you might have more information in the logs of the failed hadoop tasks.
Its possible that for some reason (bug?) pig might have failed to get the complete error from hadoop task.
-Thejas
On 9/30/10 2:32 PM, "hc busy" <hc...@gmail.com> wrote:
Well, that's a great point! The only reason why I'm creating a large 60k
pigscript to run (among over half a dozen people adding/modifying scripts)
is that when pig compiles script, it optimizes to give the best performance.
I mean, otherwise we'd actually have to manually create optimized pieces of
scripts that have clearly defined interface in the form of an HDFS file. But
even then, I don't believe it will run as fast as a pig script concatenation
of all of the scripts.
(btw, can somebody confirm this fact for me?)
That's why when I actually run the script on a full set of data, it is a
huge piece of script post concatenation, marco expansion, var substitution,
etc., etc., etc... rendering it even more like assembly language. I mean the
alias names are like
temp_count_group_by_group_b_sales_by_metro_month_1
And then I freak out when I see something like the 2998 error. My eyes glaze
over, and I'm like, okay, it's time to email pig-user group... And to answer
the other question, I eventually got to the duplicate alias after rebuilding
everything yet again and redeploying. My current hypothesis is that maybe an
scp failed and I didn't see the error message and one of our own libraries
was corrupt ?
It might have been an UDF that we're using that crashed it...
if only I had the stacktrace...
On Thu, Sep 30, 2010 at 2:02 PM, Dmitriy Ryaboy <dv...@gmail.com> wrote:
> Yeah, sure, as soon as we all quit our respective jobs and start a support
> company :)
>
> (on a more serious note, improving the error messages is a big item for Pig
> 0.9)
>
> (also, good god, 60k line Pig script. Look into workflow management tools
> that allow you to avid creating such monoliths).
>
> On Wed, Sep 29, 2010 at 8:18 PM, hc busy <hc...@gmail.com> wrote:
>
> > "null" was the error. this 60k PigLatin script that I'm running hasn't
> > changed that much, but suddenly started erroring out. I've rebuilt pig
> > release 7 from scratch, checked java version, err... checked PiggyBank
> and
> > our own libraries, not there changed.
> >
> > You know, some comercial software that has professional support will
> > actually send emails when an "unhandled error" occurs. The email is
> > received
> > by the developer/support and diagnosed. And in that email, it would
> contain
> > all the gory details that the product doesn't want to display to the
> user.
> >
> >
> > I wonder if you guys are up to doing something like that for pig?
> >
> >
> >
> > On Wed, Sep 29, 2010 at 8:13 PM, Jeff Zhang <zj...@gmail.com> wrote:
> >
> > > No other stack trace ? And in what situation does this happen ?
> > >
> > >
> > >
> > > On Thu, Sep 30, 2010 at 11:09 AM, hc busy <hc...@gmail.com> wrote:
> > > > Guys, I'm seeing this one
> > > >
> > > > 2998
> > > >
> > > > Unexpected internal error.
> > > >
> > > >
> > > > Can we be more specific or dump a stack trace when this happens?
> > > >
> > >
> > >
> > >
> > > --
> > > Best Regards
> > >
> > > Jeff Zhang
> > >
> >
>
Re: funny error
Posted by hc busy <hc...@gmail.com>.
Well, that's a great point! The only reason why I'm creating a large 60k
pigscript to run (among over half a dozen people adding/modifying scripts)
is that when pig compiles script, it optimizes to give the best performance.
I mean, otherwise we'd actually have to manually create optimized pieces of
scripts that have clearly defined interface in the form of an HDFS file. But
even then, I don't believe it will run as fast as a pig script concatenation
of all of the scripts.
(btw, can somebody confirm this fact for me?)
That's why when I actually run the script on a full set of data, it is a
huge piece of script post concatenation, marco expansion, var substitution,
etc., etc., etc... rendering it even more like assembly language. I mean the
alias names are like
temp_count_group_by_group_b_sales_by_metro_month_1
And then I freak out when I see something like the 2998 error. My eyes glaze
over, and I'm like, okay, it's time to email pig-user group... And to answer
the other question, I eventually got to the duplicate alias after rebuilding
everything yet again and redeploying. My current hypothesis is that maybe an
scp failed and I didn't see the error message and one of our own libraries
was corrupt ?
It might have been an UDF that we're using that crashed it...
if only I had the stacktrace...
On Thu, Sep 30, 2010 at 2:02 PM, Dmitriy Ryaboy <dv...@gmail.com> wrote:
> Yeah, sure, as soon as we all quit our respective jobs and start a support
> company :)
>
> (on a more serious note, improving the error messages is a big item for Pig
> 0.9)
>
> (also, good god, 60k line Pig script. Look into workflow management tools
> that allow you to avid creating such monoliths).
>
> On Wed, Sep 29, 2010 at 8:18 PM, hc busy <hc...@gmail.com> wrote:
>
> > "null" was the error. this 60k PigLatin script that I'm running hasn't
> > changed that much, but suddenly started erroring out. I've rebuilt pig
> > release 7 from scratch, checked java version, err... checked PiggyBank
> and
> > our own libraries, not there changed.
> >
> > You know, some comercial software that has professional support will
> > actually send emails when an "unhandled error" occurs. The email is
> > received
> > by the developer/support and diagnosed. And in that email, it would
> contain
> > all the gory details that the product doesn't want to display to the
> user.
> >
> >
> > I wonder if you guys are up to doing something like that for pig?
> >
> >
> >
> > On Wed, Sep 29, 2010 at 8:13 PM, Jeff Zhang <zj...@gmail.com> wrote:
> >
> > > No other stack trace ? And in what situation does this happen ?
> > >
> > >
> > >
> > > On Thu, Sep 30, 2010 at 11:09 AM, hc busy <hc...@gmail.com> wrote:
> > > > Guys, I'm seeing this one
> > > >
> > > > 2998
> > > >
> > > > Unexpected internal error.
> > > >
> > > >
> > > > Can we be more specific or dump a stack trace when this happens?
> > > >
> > >
> > >
> > >
> > > --
> > > Best Regards
> > >
> > > Jeff Zhang
> > >
> >
>
Re: funny error
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
Yeah, sure, as soon as we all quit our respective jobs and start a support
company :)
(on a more serious note, improving the error messages is a big item for Pig
0.9)
(also, good god, 60k line Pig script. Look into workflow management tools
that allow you to avid creating such monoliths).
On Wed, Sep 29, 2010 at 8:18 PM, hc busy <hc...@gmail.com> wrote:
> "null" was the error. this 60k PigLatin script that I'm running hasn't
> changed that much, but suddenly started erroring out. I've rebuilt pig
> release 7 from scratch, checked java version, err... checked PiggyBank and
> our own libraries, not there changed.
>
> You know, some comercial software that has professional support will
> actually send emails when an "unhandled error" occurs. The email is
> received
> by the developer/support and diagnosed. And in that email, it would contain
> all the gory details that the product doesn't want to display to the user.
>
>
> I wonder if you guys are up to doing something like that for pig?
>
>
>
> On Wed, Sep 29, 2010 at 8:13 PM, Jeff Zhang <zj...@gmail.com> wrote:
>
> > No other stack trace ? And in what situation does this happen ?
> >
> >
> >
> > On Thu, Sep 30, 2010 at 11:09 AM, hc busy <hc...@gmail.com> wrote:
> > > Guys, I'm seeing this one
> > >
> > > 2998
> > >
> > > Unexpected internal error.
> > >
> > >
> > > Can we be more specific or dump a stack trace when this happens?
> > >
> >
> >
> >
> > --
> > Best Regards
> >
> > Jeff Zhang
> >
>
Re: funny error
Posted by hc busy <hc...@gmail.com>.
"null" was the error. this 60k PigLatin script that I'm running hasn't
changed that much, but suddenly started erroring out. I've rebuilt pig
release 7 from scratch, checked java version, err... checked PiggyBank and
our own libraries, not there changed.
You know, some comercial software that has professional support will
actually send emails when an "unhandled error" occurs. The email is received
by the developer/support and diagnosed. And in that email, it would contain
all the gory details that the product doesn't want to display to the user.
I wonder if you guys are up to doing something like that for pig?
On Wed, Sep 29, 2010 at 8:13 PM, Jeff Zhang <zj...@gmail.com> wrote:
> No other stack trace ? And in what situation does this happen ?
>
>
>
> On Thu, Sep 30, 2010 at 11:09 AM, hc busy <hc...@gmail.com> wrote:
> > Guys, I'm seeing this one
> >
> > 2998
> >
> > Unexpected internal error.
> >
> >
> > Can we be more specific or dump a stack trace when this happens?
> >
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>
Re: funny error
Posted by Jeff Zhang <zj...@gmail.com>.
No other stack trace ? And in what situation does this happen ?
On Thu, Sep 30, 2010 at 11:09 AM, hc busy <hc...@gmail.com> wrote:
> Guys, I'm seeing this one
>
> 2998
>
> Unexpected internal error.
>
>
> Can we be more specific or dump a stack trace when this happens?
>
--
Best Regards
Jeff Zhang