You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Kevin Burton <bu...@spinn3r.com> on 2014/05/19 02:32:40 UTC
Can't get excite demo to work with YARN / Hadoop 2.4 …
I'm really stumped on this one.
I have some experience with Pig from about a year ago so I'm somewhat
familiar with it.
I'm running Hadoop 2.4 and I recompiled pig with:
ant clean jar-withouthadoop -Dhadoopversion=23
I uploaded the excite demo and bzip2 decompressed it…
The file is there. If I 'dump' it,.. I'll get output.
However, I get empty output:
Input(s):
Successfully read 0 records from: "/excite.log"
Output(s):
Successfully stored 0 records in: "/test-output3"
… this is the script I'm running:
raw = LOAD '/excite.log' USING PigStorage('\t') AS (user, time, query);
foo = FOREACH raw GENERATE user, time, query;
STORE foo INTO '/test-output3' ;
… I had the STORE with a USING PigStorage() … but removed it to debug and
that didn't fix the problem..
Any advice here?
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
Skype: *burtonator*
blog: http://burtonator.wordpress.com
… or check out my Google+
profile<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
War is peace. Freedom is slavery. Ignorance is strength. Corporations are
people.
Re: Can't get excite demo to work with YARN / Hadoop 2.4 …
Posted by Cheolsoo Park <pi...@gmail.com>.
Do you mind filing a jira? If counters are not correct with Hadoop 2.4, we
should fix it.
Btw, I am using Pig trunk with Hadoop 2.4 and haven't seen an issue with
counters.
On Sun, May 18, 2014 at 7:55 PM, Kevin Burton <bu...@spinn3r.com> wrote:
> AH!!! I figured it out… it's a bug with Pig not reporting the correct
> number of records emitted.
>
>
> On Sun, May 18, 2014 at 5:32 PM, Kevin Burton <bu...@spinn3r.com> wrote:
>
> > I'm really stumped on this one.
> >
> > I have some experience with Pig from about a year ago so I'm somewhat
> > familiar with it.
> >
> > I'm running Hadoop 2.4 and I recompiled pig with:
> >
> > ant clean jar-withouthadoop -Dhadoopversion=23
> >
> > I uploaded the excite demo and bzip2 decompressed it…
> >
> > The file is there. If I 'dump' it,.. I'll get output.
> >
> > However, I get empty output:
> >
> >
> > Input(s):
> > Successfully read 0 records from: "/excite.log"
> >
> > Output(s):
> > Successfully stored 0 records in: "/test-output3"
> >
> > … this is the script I'm running:
> >
> > raw = LOAD '/excite.log' USING PigStorage('\t') AS (user, time, query);
> >
> > foo = FOREACH raw GENERATE user, time, query;
> > STORE foo INTO '/test-output3' ;
> >
> > … I had the STORE with a USING PigStorage() … but removed it to debug and
> > that didn't fix the problem..
> >
> > Any advice here?
> >
> > --
> >
> > Founder/CEO Spinn3r.com
> > Location: *San Francisco, CA*
> > Skype: *burtonator*
> > blog: http://burtonator.wordpress.com
> > … or check out my Google+ profile<
> https://plus.google.com/102718274791889610666/posts>
> > <http://spinn3r.com>
> > War is peace. Freedom is slavery. Ignorance is strength. Corporations are
> > people.
> >
> >
>
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> Skype: *burtonator*
> blog: http://burtonator.wordpress.com
> … or check out my Google+
> profile<https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
> War is peace. Freedom is slavery. Ignorance is strength. Corporations are
> people.
>
Re: Can't get excite demo to work with YARN / Hadoop 2.4 …
Posted by Kevin Burton <bu...@spinn3r.com>.
AH!!! I figured it out… it's a bug with Pig not reporting the correct
number of records emitted.
On Sun, May 18, 2014 at 5:32 PM, Kevin Burton <bu...@spinn3r.com> wrote:
> I'm really stumped on this one.
>
> I have some experience with Pig from about a year ago so I'm somewhat
> familiar with it.
>
> I'm running Hadoop 2.4 and I recompiled pig with:
>
> ant clean jar-withouthadoop -Dhadoopversion=23
>
> I uploaded the excite demo and bzip2 decompressed it…
>
> The file is there. If I 'dump' it,.. I'll get output.
>
> However, I get empty output:
>
>
> Input(s):
> Successfully read 0 records from: "/excite.log"
>
> Output(s):
> Successfully stored 0 records in: "/test-output3"
>
> … this is the script I'm running:
>
> raw = LOAD '/excite.log' USING PigStorage('\t') AS (user, time, query);
>
> foo = FOREACH raw GENERATE user, time, query;
> STORE foo INTO '/test-output3' ;
>
> … I had the STORE with a USING PigStorage() … but removed it to debug and
> that didn't fix the problem..
>
> Any advice here?
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> Skype: *burtonator*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile<https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
> War is peace. Freedom is slavery. Ignorance is strength. Corporations are
> people.
>
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
Skype: *burtonator*
blog: http://burtonator.wordpress.com
… or check out my Google+
profile<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
War is peace. Freedom is slavery. Ignorance is strength. Corporations are
people.