You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Ia...@barclayscapital.com on 2012/01/19 11:05:50 UTC
Problem with Aggregate Functions
Hello,
I wonder if you guys can help. I'm running Pig 0.9.1 with Hadoop 0.20.203.0, and am having the below problem consistently. I have tried various cast operators, none of which are working, so I must assume I'm making a syntactical error. I just cant see what it is. Can anyone please help?
grunt> describe temp1;
temp1: {group: chararray,temp: {(a::Env_Company: chararray,a::TradeNPV: double)}}
grunt> temp2 = foreach temp1 generate group, SUM(a.TradeNPV);
2012-01-19 09:58:19,243 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1045:
<line 10, column 38> Could not infer the matching function for org.apache.pig.builtin.SUM as multiple or none of them fit. Please use an explicit cast.
Thanks in advance,
Ian Meyers
Barclays Capital
FICC Solutions Architecture
* Direct : +44 (0) 20 777 37437 * Extension: 37437
* ian.meyers@barclayscapital.com<ma...@barclayscapital.com>
_______________________________________________
This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the foregoing. Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP. This email may relate to or be sent from other members of the Barclays Group.
_______________________________________________
RE: Problem with Aggregate Functions
Posted by Ia...@barclayscapital.com.
Very Helpful - thanks.
-----Original Message-----
From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com]
Sent: Thursday, January 19, 2012 5:58 PM
To: user@pig.apache.org
Subject: Re: Problem with Aggregate Functions
Does this help?
http://squarecog.wordpress.com/2010/05/11/group-operator-in-apache-pig/
On Thu, Jan 19, 2012 at 3:38 AM, <Ia...@barclayscapital.com> wrote:
> Right you are - thanks very much. It's still not clear to me when to use
> the parent bag name vs. when to use the internal name, but will try this
> option in the future.
>
> Cheers!
>
> Ian
>
> -----Original Message-----
> From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com]
> Sent: Thursday, January 19, 2012 11:21 AM
> To: user@pig.apache.org
> Subject: Re: Problem with Aggregate Functions
>
> foreach temp1 generate group, SUM(temp.TradeNPV)
>
> (or possibly temp.a::TradeNPV)
>
> "a::" is a prefix generate to deduplicate which of the relations a column
> came from after a join, which Pig knows to ignore if it can determine which
> field you mean without conflicts. It's not an actual field in this schema.
> The actual collection is called "temp".
>
> D
>
> On Thu, Jan 19, 2012 at 2:05 AM, <Ia...@barclayscapital.com> wrote:
>
> > Hello,
> >
> > I wonder if you guys can help. I'm running Pig 0.9.1 with Hadoop
> > 0.20.203.0, and am having the below problem consistently. I have tried
> > various cast operators, none of which are working, so I must assume I'm
> > making a syntactical error. I just cant see what it is. Can anyone please
> > help?
> >
> > grunt> describe temp1;
> > temp1: {group: chararray,temp: {(a::Env_Company: chararray,a::TradeNPV:
> > double)}}
> >
> > grunt> temp2 = foreach temp1 generate group, SUM(a.TradeNPV);
> > 2012-01-19 09:58:19,243 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> > ERROR 1045:
> > <line 10, column 38> Could not infer the matching function for
> > org.apache.pig.builtin.SUM as multiple or none of them fit. Please use an
> > explicit cast.
> >
> > Thanks in advance,
> >
> > Ian Meyers
> >
> > Barclays Capital
> > FICC Solutions Architecture
> >
> > * Direct : +44 (0) 20 777 37437 * Extension: 37437
> > * ian.meyers@barclayscapital.com<ma...@barclayscapital.com>
> >
> >
> > _______________________________________________
> >
> > This e-mail may contain information that is confidential, privileged or
> > otherwise protected from disclosure. If you are not an intended recipient
> > of this e-mail, do not duplicate or redistribute it by any means. Please
> > delete it and any attachments and notify the sender that you have
> received
> > it in error. Unless specifically indicated, this e-mail is not an offer
> to
> > buy or sell or a solicitation to buy or sell any securities, investment
> > products or other financial product or service, an official confirmation
> of
> > any transaction, or an official statement of Barclays. Any views or
> > opinions presented are solely those of the author and do not necessarily
> > represent those of Barclays. This e-mail is subject to terms available at
> > the following link: www.barcap.com/emaildisclaimer. By messaging with
> > Barclays you consent to the foregoing. Barclays Capital is the
> investment
> > banking division of Barclays Bank PLC, a company registered in England
> > (number 1026167) with its registered office at 1 Churchill Place, London,
> > E14 5HP. This email may relate to or be sent from other members of the
> > Barclays Group.
> > _______________________________________________
> >
>
Re: Problem with Aggregate Functions
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
Does this help?
http://squarecog.wordpress.com/2010/05/11/group-operator-in-apache-pig/
On Thu, Jan 19, 2012 at 3:38 AM, <Ia...@barclayscapital.com> wrote:
> Right you are - thanks very much. It's still not clear to me when to use
> the parent bag name vs. when to use the internal name, but will try this
> option in the future.
>
> Cheers!
>
> Ian
>
> -----Original Message-----
> From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com]
> Sent: Thursday, January 19, 2012 11:21 AM
> To: user@pig.apache.org
> Subject: Re: Problem with Aggregate Functions
>
> foreach temp1 generate group, SUM(temp.TradeNPV)
>
> (or possibly temp.a::TradeNPV)
>
> "a::" is a prefix generate to deduplicate which of the relations a column
> came from after a join, which Pig knows to ignore if it can determine which
> field you mean without conflicts. It's not an actual field in this schema.
> The actual collection is called "temp".
>
> D
>
> On Thu, Jan 19, 2012 at 2:05 AM, <Ia...@barclayscapital.com> wrote:
>
> > Hello,
> >
> > I wonder if you guys can help. I'm running Pig 0.9.1 with Hadoop
> > 0.20.203.0, and am having the below problem consistently. I have tried
> > various cast operators, none of which are working, so I must assume I'm
> > making a syntactical error. I just cant see what it is. Can anyone please
> > help?
> >
> > grunt> describe temp1;
> > temp1: {group: chararray,temp: {(a::Env_Company: chararray,a::TradeNPV:
> > double)}}
> >
> > grunt> temp2 = foreach temp1 generate group, SUM(a.TradeNPV);
> > 2012-01-19 09:58:19,243 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> > ERROR 1045:
> > <line 10, column 38> Could not infer the matching function for
> > org.apache.pig.builtin.SUM as multiple or none of them fit. Please use an
> > explicit cast.
> >
> > Thanks in advance,
> >
> > Ian Meyers
> >
> > Barclays Capital
> > FICC Solutions Architecture
> >
> > * Direct : +44 (0) 20 777 37437 * Extension: 37437
> > * ian.meyers@barclayscapital.com<ma...@barclayscapital.com>
> >
> >
> > _______________________________________________
> >
> > This e-mail may contain information that is confidential, privileged or
> > otherwise protected from disclosure. If you are not an intended recipient
> > of this e-mail, do not duplicate or redistribute it by any means. Please
> > delete it and any attachments and notify the sender that you have
> received
> > it in error. Unless specifically indicated, this e-mail is not an offer
> to
> > buy or sell or a solicitation to buy or sell any securities, investment
> > products or other financial product or service, an official confirmation
> of
> > any transaction, or an official statement of Barclays. Any views or
> > opinions presented are solely those of the author and do not necessarily
> > represent those of Barclays. This e-mail is subject to terms available at
> > the following link: www.barcap.com/emaildisclaimer. By messaging with
> > Barclays you consent to the foregoing. Barclays Capital is the
> investment
> > banking division of Barclays Bank PLC, a company registered in England
> > (number 1026167) with its registered office at 1 Churchill Place, London,
> > E14 5HP. This email may relate to or be sent from other members of the
> > Barclays Group.
> > _______________________________________________
> >
>
RE: Problem with Aggregate Functions
Posted by Ia...@barclayscapital.com.
Right you are - thanks very much. It's still not clear to me when to use the parent bag name vs. when to use the internal name, but will try this option in the future.
Cheers!
Ian
-----Original Message-----
From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com]
Sent: Thursday, January 19, 2012 11:21 AM
To: user@pig.apache.org
Subject: Re: Problem with Aggregate Functions
foreach temp1 generate group, SUM(temp.TradeNPV)
(or possibly temp.a::TradeNPV)
"a::" is a prefix generate to deduplicate which of the relations a column
came from after a join, which Pig knows to ignore if it can determine which
field you mean without conflicts. It's not an actual field in this schema.
The actual collection is called "temp".
D
On Thu, Jan 19, 2012 at 2:05 AM, <Ia...@barclayscapital.com> wrote:
> Hello,
>
> I wonder if you guys can help. I'm running Pig 0.9.1 with Hadoop
> 0.20.203.0, and am having the below problem consistently. I have tried
> various cast operators, none of which are working, so I must assume I'm
> making a syntactical error. I just cant see what it is. Can anyone please
> help?
>
> grunt> describe temp1;
> temp1: {group: chararray,temp: {(a::Env_Company: chararray,a::TradeNPV:
> double)}}
>
> grunt> temp2 = foreach temp1 generate group, SUM(a.TradeNPV);
> 2012-01-19 09:58:19,243 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1045:
> <line 10, column 38> Could not infer the matching function for
> org.apache.pig.builtin.SUM as multiple or none of them fit. Please use an
> explicit cast.
>
> Thanks in advance,
>
> Ian Meyers
>
> Barclays Capital
> FICC Solutions Architecture
>
> * Direct : +44 (0) 20 777 37437 * Extension: 37437
> * ian.meyers@barclayscapital.com<ma...@barclayscapital.com>
>
>
> _______________________________________________
>
> This e-mail may contain information that is confidential, privileged or
> otherwise protected from disclosure. If you are not an intended recipient
> of this e-mail, do not duplicate or redistribute it by any means. Please
> delete it and any attachments and notify the sender that you have received
> it in error. Unless specifically indicated, this e-mail is not an offer to
> buy or sell or a solicitation to buy or sell any securities, investment
> products or other financial product or service, an official confirmation of
> any transaction, or an official statement of Barclays. Any views or
> opinions presented are solely those of the author and do not necessarily
> represent those of Barclays. This e-mail is subject to terms available at
> the following link: www.barcap.com/emaildisclaimer. By messaging with
> Barclays you consent to the foregoing. Barclays Capital is the investment
> banking division of Barclays Bank PLC, a company registered in England
> (number 1026167) with its registered office at 1 Churchill Place, London,
> E14 5HP. This email may relate to or be sent from other members of the
> Barclays Group.
> _______________________________________________
>
Re: Problem with Aggregate Functions
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
foreach temp1 generate group, SUM(temp.TradeNPV)
(or possibly temp.a::TradeNPV)
"a::" is a prefix generate to deduplicate which of the relations a column
came from after a join, which Pig knows to ignore if it can determine which
field you mean without conflicts. It's not an actual field in this schema.
The actual collection is called "temp".
D
On Thu, Jan 19, 2012 at 2:05 AM, <Ia...@barclayscapital.com> wrote:
> Hello,
>
> I wonder if you guys can help. I'm running Pig 0.9.1 with Hadoop
> 0.20.203.0, and am having the below problem consistently. I have tried
> various cast operators, none of which are working, so I must assume I'm
> making a syntactical error. I just cant see what it is. Can anyone please
> help?
>
> grunt> describe temp1;
> temp1: {group: chararray,temp: {(a::Env_Company: chararray,a::TradeNPV:
> double)}}
>
> grunt> temp2 = foreach temp1 generate group, SUM(a.TradeNPV);
> 2012-01-19 09:58:19,243 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1045:
> <line 10, column 38> Could not infer the matching function for
> org.apache.pig.builtin.SUM as multiple or none of them fit. Please use an
> explicit cast.
>
> Thanks in advance,
>
> Ian Meyers
>
> Barclays Capital
> FICC Solutions Architecture
>
> * Direct : +44 (0) 20 777 37437 * Extension: 37437
> * ian.meyers@barclayscapital.com<ma...@barclayscapital.com>
>
>
> _______________________________________________
>
> This e-mail may contain information that is confidential, privileged or
> otherwise protected from disclosure. If you are not an intended recipient
> of this e-mail, do not duplicate or redistribute it by any means. Please
> delete it and any attachments and notify the sender that you have received
> it in error. Unless specifically indicated, this e-mail is not an offer to
> buy or sell or a solicitation to buy or sell any securities, investment
> products or other financial product or service, an official confirmation of
> any transaction, or an official statement of Barclays. Any views or
> opinions presented are solely those of the author and do not necessarily
> represent those of Barclays. This e-mail is subject to terms available at
> the following link: www.barcap.com/emaildisclaimer. By messaging with
> Barclays you consent to the foregoing. Barclays Capital is the investment
> banking division of Barclays Bank PLC, a company registered in England
> (number 1026167) with its registered office at 1 Churchill Place, London,
> E14 5HP. This email may relate to or be sent from other members of the
> Barclays Group.
> _______________________________________________
>