You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Doug Daniels <dd...@mortardata.com> on 2011/11/10 03:32:32 UTC
pig cookbook
I was just working on a pig script to group some data by a field and then generate percentages for each group. Without windowing functions at my disposal, I wound up using a group by on the field for the numerator, a group all for the denominator, and a cross to pull them together.
Afterward, I found myself wanting to drop that snippet somewhere that other folks could use it, or (even better) to have found a snippet in the first place and not had to write it!
So I was thinking it'd be great to have a "pig cookbook" online with common operations that people do with pig. Does anything like that exist yet? If not, where would be a good place for it?
Best,
Doug
RE: pig cookbook
Posted by Corinne Chandel <ch...@yahoo-inc.com>.
In Pig 9 this material was moved into Performance and Efficiency, section Performance Enhancers:
http://pig.apache.org/docs/r0.9.1/perf.html#performance-enhancers
Thanks/C
-----Original Message-----
From: Ashutosh Chauhan [mailto:hashutosh@apache.org]
Sent: Thursday, November 10, 2011 11:00 AM
To: user@pig.apache.org
Subject: Re: pig cookbook
We used to have a cookbook.
http://pig.apache.org/docs/r0.7.0/cookbook.htmlBut cant find similar
link for 0.8/0.9
Ashutosh
On Thu, Nov 10, 2011 at 00:34, Daniel Dai <da...@hortonworks.com> wrote:
> Sounds like a good candidate for Piggybank for Macro. We don't have it yet
> and hopefully we can have it soon.
>
> Currently you can drop your code to FAQ (
> https://cwiki.apache.org/confluence/display/PIG/FAQ). If we have
> significant number of recipe, we can move them to a separate section in
> cwiki.
>
> Daniel
>
> On Wed, Nov 9, 2011 at 6:32 PM, Doug Daniels <ddaniels@mortardata.com
> >wrote:
>
> > I was just working on a pig script to group some data by a field and then
> > generate percentages for each group. Without windowing functions at my
> > disposal, I wound up using a group by on the field for the numerator, a
> > group all for the denominator, and a cross to pull them together.
> >
> > Afterward, I found myself wanting to drop that snippet somewhere that
> > other folks could use it, or (even better) to have found a snippet in the
> > first place and not had to write it!
> >
> > So I was thinking it'd be great to have a "pig cookbook" online with
> > common operations that people do with pig. Does anything like that exist
> > yet? If not, where would be a good place for it?
> >
> > Best,
> > Doug
> >
>
Re: pig cookbook
Posted by Ashutosh Chauhan <ha...@apache.org>.
We used to have a cookbook.
http://pig.apache.org/docs/r0.7.0/cookbook.htmlBut cant find similar
link for 0.8/0.9
Ashutosh
On Thu, Nov 10, 2011 at 00:34, Daniel Dai <da...@hortonworks.com> wrote:
> Sounds like a good candidate for Piggybank for Macro. We don't have it yet
> and hopefully we can have it soon.
>
> Currently you can drop your code to FAQ (
> https://cwiki.apache.org/confluence/display/PIG/FAQ). If we have
> significant number of recipe, we can move them to a separate section in
> cwiki.
>
> Daniel
>
> On Wed, Nov 9, 2011 at 6:32 PM, Doug Daniels <ddaniels@mortardata.com
> >wrote:
>
> > I was just working on a pig script to group some data by a field and then
> > generate percentages for each group. Without windowing functions at my
> > disposal, I wound up using a group by on the field for the numerator, a
> > group all for the denominator, and a cross to pull them together.
> >
> > Afterward, I found myself wanting to drop that snippet somewhere that
> > other folks could use it, or (even better) to have found a snippet in the
> > first place and not had to write it!
> >
> > So I was thinking it'd be great to have a "pig cookbook" online with
> > common operations that people do with pig. Does anything like that exist
> > yet? If not, where would be a good place for it?
> >
> > Best,
> > Doug
> >
>
Re: pig cookbook
Posted by Daniel Dai <da...@hortonworks.com>.
Sounds like a good candidate for Piggybank for Macro. We don't have it yet
and hopefully we can have it soon.
Currently you can drop your code to FAQ (
https://cwiki.apache.org/confluence/display/PIG/FAQ). If we have
significant number of recipe, we can move them to a separate section in
cwiki.
Daniel
On Wed, Nov 9, 2011 at 6:32 PM, Doug Daniels <dd...@mortardata.com>wrote:
> I was just working on a pig script to group some data by a field and then
> generate percentages for each group. Without windowing functions at my
> disposal, I wound up using a group by on the field for the numerator, a
> group all for the denominator, and a cross to pull them together.
>
> Afterward, I found myself wanting to drop that snippet somewhere that
> other folks could use it, or (even better) to have found a snippet in the
> first place and not had to write it!
>
> So I was thinking it'd be great to have a "pig cookbook" online with
> common operations that people do with pig. Does anything like that exist
> yet? If not, where would be a good place for it?
>
> Best,
> Doug
>
Re: pig cookbook
Posted by Doug Daniels <dd...@mortardata.com>.
Thanks Dmitriy, I didn't know you could do that!
I added an entry to the FAQ that Daniel pointed out: http://bit.ly/sZ8UOL
-Doug
On 11/10/11 6:25 PM, "Dmitriy Ryaboy" <dv...@gmail.com> wrote:
>Wait, a cross? No, don't do a cross. You can just cast the result of
>group
>all as a scalar, this will be much more efficient.
>
>D
>
>On Wed, Nov 9, 2011 at 6:32 PM, Doug Daniels
><dd...@mortardata.com>wrote:
>
>> I was just working on a pig script to group some data by a field and
>>then
>> generate percentages for each group. Without windowing functions at my
>> disposal, I wound up using a group by on the field for the numerator, a
>> group all for the denominator, and a cross to pull them together.
>>
>> Afterward, I found myself wanting to drop that snippet somewhere that
>> other folks could use it, or (even better) to have found a snippet in
>>the
>> first place and not had to write it!
>>
>> So I was thinking it'd be great to have a "pig cookbook" online with
>> common operations that people do with pig. Does anything like that
>>exist
>> yet? If not, where would be a good place for it?
>>
>> Best,
>> Doug
>>
Re: pig cookbook
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
Wait, a cross? No, don't do a cross. You can just cast the result of group
all as a scalar, this will be much more efficient.
D
On Wed, Nov 9, 2011 at 6:32 PM, Doug Daniels <dd...@mortardata.com>wrote:
> I was just working on a pig script to group some data by a field and then
> generate percentages for each group. Without windowing functions at my
> disposal, I wound up using a group by on the field for the numerator, a
> group all for the denominator, and a cross to pull them together.
>
> Afterward, I found myself wanting to drop that snippet somewhere that
> other folks could use it, or (even better) to have found a snippet in the
> first place and not had to write it!
>
> So I was thinking it'd be great to have a "pig cookbook" online with
> common operations that people do with pig. Does anything like that exist
> yet? If not, where would be a good place for it?
>
> Best,
> Doug
>