You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by John Morrison <jo...@gmail.com> on 2012/05/17 17:24:09 UTC
? ERROR 1070: Could not resolve sum using imports
Hi,
I am new to ping and am unable to use pig builtin functions (please see
details below).
Is this a CLASSPATH issue?
Any ideas on how to resolve?
Thanks,
John
Details
### Line in pig script causing issue
C = foreach B generate flatten($0), sum(lane_nbr) ;
### Error message
2012-05-17 11:15:41,673 [main] ERROR org.apache.pig.tools.grunt.Grunt - \
ERROR 1070: Could not resolve sum using imports: [,
org.apache.pig.builtin., org.apache.pig.impl.builtin.]
pig -version
Apache Pig version 0.8.1-cdh3u3 (rexported)
Re: ? ERROR 1070: Could not resolve sum using imports
Posted by John Meagher <jo...@gmail.com>.
The UDFs are case sensitive. Use SUM and it will work.
On Thu, May 17, 2012 at 11:24 AM, John Morrison
<jo...@gmail.com> wrote:
> Hi,
>
> I am new to ping and am unable to use pig builtin functions (please see
> details below).
>
> Is this a CLASSPATH issue?
>
> Any ideas on how to resolve?
>
> Thanks,
>
> John
>
> Details
>
> ### Line in pig script causing issue
> C = foreach B generate flatten($0), sum(lane_nbr) ;
>
> ### Error message
> 2012-05-17 11:15:41,673 [main] ERROR org.apache.pig.tools.grunt.Grunt - \
> ERROR 1070: Could not resolve sum using imports: [,
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>
> pig -version
> Apache Pig version 0.8.1-cdh3u3 (rexported)
Re: ? ERROR 1070: Could not resolve sum using imports
Posted by Jonathan Coveney <jc...@gmail.com>.
It's ok. The Pig object model is quite powerful, but can be hard to grok at
first. Don't be afraid to ask questions, though it helps to post a full
script that reproduces the problem, and any relevant stack traces. :)
2012/5/17 John Morrison <jo...@gmail.com>
> Thanks Jonathan, that was the key. I have been trying for several days
> to solve this simple problem.
>
> I guess I need to RTFM a little closer. :)
>
> -John
>
> On Thu, May 17, 2012 at 9:07 PM, Jonathan Coveney <jcoveney@gmail.com
> >wrote:
>
> > Ah, this is the most common error for people starting out :)
> >
> > SUM is a function which takes a bag (not a relation or a column) and
> > returns the sum of the elements in that bag.
> >
> > Try:
> >
> > A = LOAD 'v.log' AS (a:double) ;
> > B = foreach (group A all) generate SUM(A) ;
> > dump B;
> >
> > 2012/5/17 John Morrison <jo...@gmail.com>
> >
> > > OK. I have simplified the script and tried 2 different ways without
> > > success:
> > >
> > > 1) B = foreach A generate flatten($0), SUM($0.a) ;
> > > 2) B = foreach A generate flatten($0), SUM(a) ;
> > >
> > > Which both produce different errors(see below)?
> > >
> > > Thanks,
> > >
> > > John
> > >
> > > Data:
> > > cat v.log
> > >
> > > 1
> > > 2
> > > 3
> > > 4
> > >
> > >
> > > Complete script 1 with error:
> > >
> > > A = LOAD 'v.log' AS (a:double) ;
> > > dump A;
> > > B = foreach A generate flatten($0), SUM($0.a) ;
> > > dump B;
> > >
> > > Error for above script:
> > > 2012-05-17 20:54:11,836 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> > > ERROR 1000: Error during parsing. Invalid alias: a in null
> > >
> > >
> > > Complete script 2 with error:
> > >
> > >
> > > A = LOAD 'v.log' AS (a:double) ;
> > > dump A;
> > > B = foreach A generate flatten($0), SUM(a) ;
> > > dump B;
> > >
> > >
> > > 2012-05-17 20:56:11,795 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> > > ERROR 1045: Could not infer the matching function for
> > > org.apache.pig.builtin.SUM as multiple or none of them fit. Please use
> an
> > > explicit cast.
> > >
> > >
> > >
> > >
> > > On Thu, May 17, 2012 at 6:31 PM, Jonathan Coveney <jcoveney@gmail.com
> > > >wrote:
> > >
> > > > I imagine the issue is that you're not giving it a bag... can you
> paste
> > > the
> > > > whole script? That said, my guess is that this comes right after a
> > group
> > > > by, and that the following will fix it:
> > > >
> > > > C = foreach B generate flatten($0), sum($0.lane_nbr) ;
> > > >
> > > > but I'd need the whole script to be sure
> > > >
> > > > 2012/5/17 John Morrison <jo...@gmail.com>
> > > >
> > > > > OK. New problem with needing to use explicit cast?
> > > > >
> > > > > Could you give me syntax for explict cast?
> > > > >
> > > > > 2012-05-17 15:29:13,769 [main] ERROR
> > org.apache.pig.tools.grunt.Grunt -
> > > > > ERROR 1046: Multiple matching functions for
> > org.apache.pig.builtin.SUM
> > > > with
> > > > > input schema: ({{bytearray}}, {{double}}). Please use an explicit
> > cast.
> > > > >
> > > > >
> > > > >
> > > > > On Thu, May 17, 2012 at 11:42 AM, Prashant Kommireddi
> > > > > <pr...@gmail.com>wrote:
> > > > >
> > > > > > UDFs are case-sensitive. It should be all caps - SUM
> > > > > >
> > > > > > Can you please give that a try?
> > > > > >
> > > > > >
> > > > > > On May 17, 2012, at 8:24 AM, John Morrison <
> > > john.h.morrison@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I am new to ping and am unable to use pig builtin functions
> > (please
> > > > see
> > > > > > > details below).
> > > > > > >
> > > > > > > Is this a CLASSPATH issue?
> > > > > > >
> > > > > > > Any ideas on how to resolve?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > John
> > > > > > >
> > > > > > > Details
> > > > > > >
> > > > > > > ### Line in pig script causing issue
> > > > > > > C = foreach B generate flatten($0), sum(lane_nbr) ;
> > > > > > >
> > > > > > > ### Error message
> > > > > > > 2012-05-17 11:15:41,673 [main] ERROR
> > > org.apache.pig.tools.grunt.Grunt
> > > > > - \
> > > > > > > ERROR 1070: Could not resolve sum using imports: [,
> > > > > > > org.apache.pig.builtin., org.apache.pig.impl.builtin.]
> > > > > > >
> > > > > > > pig -version
> > > > > > > Apache Pig version 0.8.1-cdh3u3 (rexported)
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > John Morrison
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > John Morrison
> > >
> >
>
>
>
> --
> John Morrison
>
Re: ? ERROR 1070: Could not resolve sum using imports
Posted by John Morrison <jo...@gmail.com>.
Thanks Jonathan, that was the key. I have been trying for several days
to solve this simple problem.
I guess I need to RTFM a little closer. :)
-John
On Thu, May 17, 2012 at 9:07 PM, Jonathan Coveney <jc...@gmail.com>wrote:
> Ah, this is the most common error for people starting out :)
>
> SUM is a function which takes a bag (not a relation or a column) and
> returns the sum of the elements in that bag.
>
> Try:
>
> A = LOAD 'v.log' AS (a:double) ;
> B = foreach (group A all) generate SUM(A) ;
> dump B;
>
> 2012/5/17 John Morrison <jo...@gmail.com>
>
> > OK. I have simplified the script and tried 2 different ways without
> > success:
> >
> > 1) B = foreach A generate flatten($0), SUM($0.a) ;
> > 2) B = foreach A generate flatten($0), SUM(a) ;
> >
> > Which both produce different errors(see below)?
> >
> > Thanks,
> >
> > John
> >
> > Data:
> > cat v.log
> >
> > 1
> > 2
> > 3
> > 4
> >
> >
> > Complete script 1 with error:
> >
> > A = LOAD 'v.log' AS (a:double) ;
> > dump A;
> > B = foreach A generate flatten($0), SUM($0.a) ;
> > dump B;
> >
> > Error for above script:
> > 2012-05-17 20:54:11,836 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> > ERROR 1000: Error during parsing. Invalid alias: a in null
> >
> >
> > Complete script 2 with error:
> >
> >
> > A = LOAD 'v.log' AS (a:double) ;
> > dump A;
> > B = foreach A generate flatten($0), SUM(a) ;
> > dump B;
> >
> >
> > 2012-05-17 20:56:11,795 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> > ERROR 1045: Could not infer the matching function for
> > org.apache.pig.builtin.SUM as multiple or none of them fit. Please use an
> > explicit cast.
> >
> >
> >
> >
> > On Thu, May 17, 2012 at 6:31 PM, Jonathan Coveney <jcoveney@gmail.com
> > >wrote:
> >
> > > I imagine the issue is that you're not giving it a bag... can you paste
> > the
> > > whole script? That said, my guess is that this comes right after a
> group
> > > by, and that the following will fix it:
> > >
> > > C = foreach B generate flatten($0), sum($0.lane_nbr) ;
> > >
> > > but I'd need the whole script to be sure
> > >
> > > 2012/5/17 John Morrison <jo...@gmail.com>
> > >
> > > > OK. New problem with needing to use explicit cast?
> > > >
> > > > Could you give me syntax for explict cast?
> > > >
> > > > 2012-05-17 15:29:13,769 [main] ERROR
> org.apache.pig.tools.grunt.Grunt -
> > > > ERROR 1046: Multiple matching functions for
> org.apache.pig.builtin.SUM
> > > with
> > > > input schema: ({{bytearray}}, {{double}}). Please use an explicit
> cast.
> > > >
> > > >
> > > >
> > > > On Thu, May 17, 2012 at 11:42 AM, Prashant Kommireddi
> > > > <pr...@gmail.com>wrote:
> > > >
> > > > > UDFs are case-sensitive. It should be all caps - SUM
> > > > >
> > > > > Can you please give that a try?
> > > > >
> > > > >
> > > > > On May 17, 2012, at 8:24 AM, John Morrison <
> > john.h.morrison@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I am new to ping and am unable to use pig builtin functions
> (please
> > > see
> > > > > > details below).
> > > > > >
> > > > > > Is this a CLASSPATH issue?
> > > > > >
> > > > > > Any ideas on how to resolve?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > John
> > > > > >
> > > > > > Details
> > > > > >
> > > > > > ### Line in pig script causing issue
> > > > > > C = foreach B generate flatten($0), sum(lane_nbr) ;
> > > > > >
> > > > > > ### Error message
> > > > > > 2012-05-17 11:15:41,673 [main] ERROR
> > org.apache.pig.tools.grunt.Grunt
> > > > - \
> > > > > > ERROR 1070: Could not resolve sum using imports: [,
> > > > > > org.apache.pig.builtin., org.apache.pig.impl.builtin.]
> > > > > >
> > > > > > pig -version
> > > > > > Apache Pig version 0.8.1-cdh3u3 (rexported)
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > John Morrison
> > > >
> > >
> >
> >
> >
> > --
> > John Morrison
> >
>
--
John Morrison
Re: ? ERROR 1070: Could not resolve sum using imports
Posted by Jonathan Coveney <jc...@gmail.com>.
Ah, this is the most common error for people starting out :)
SUM is a function which takes a bag (not a relation or a column) and
returns the sum of the elements in that bag.
Try:
A = LOAD 'v.log' AS (a:double) ;
B = foreach (group A all) generate SUM(A) ;
dump B;
2012/5/17 John Morrison <jo...@gmail.com>
> OK. I have simplified the script and tried 2 different ways without
> success:
>
> 1) B = foreach A generate flatten($0), SUM($0.a) ;
> 2) B = foreach A generate flatten($0), SUM(a) ;
>
> Which both produce different errors(see below)?
>
> Thanks,
>
> John
>
> Data:
> cat v.log
>
> 1
> 2
> 3
> 4
>
>
> Complete script 1 with error:
>
> A = LOAD 'v.log' AS (a:double) ;
> dump A;
> B = foreach A generate flatten($0), SUM($0.a) ;
> dump B;
>
> Error for above script:
> 2012-05-17 20:54:11,836 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1000: Error during parsing. Invalid alias: a in null
>
>
> Complete script 2 with error:
>
>
> A = LOAD 'v.log' AS (a:double) ;
> dump A;
> B = foreach A generate flatten($0), SUM(a) ;
> dump B;
>
>
> 2012-05-17 20:56:11,795 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1045: Could not infer the matching function for
> org.apache.pig.builtin.SUM as multiple or none of them fit. Please use an
> explicit cast.
>
>
>
>
> On Thu, May 17, 2012 at 6:31 PM, Jonathan Coveney <jcoveney@gmail.com
> >wrote:
>
> > I imagine the issue is that you're not giving it a bag... can you paste
> the
> > whole script? That said, my guess is that this comes right after a group
> > by, and that the following will fix it:
> >
> > C = foreach B generate flatten($0), sum($0.lane_nbr) ;
> >
> > but I'd need the whole script to be sure
> >
> > 2012/5/17 John Morrison <jo...@gmail.com>
> >
> > > OK. New problem with needing to use explicit cast?
> > >
> > > Could you give me syntax for explict cast?
> > >
> > > 2012-05-17 15:29:13,769 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> > > ERROR 1046: Multiple matching functions for org.apache.pig.builtin.SUM
> > with
> > > input schema: ({{bytearray}}, {{double}}). Please use an explicit cast.
> > >
> > >
> > >
> > > On Thu, May 17, 2012 at 11:42 AM, Prashant Kommireddi
> > > <pr...@gmail.com>wrote:
> > >
> > > > UDFs are case-sensitive. It should be all caps - SUM
> > > >
> > > > Can you please give that a try?
> > > >
> > > >
> > > > On May 17, 2012, at 8:24 AM, John Morrison <
> john.h.morrison@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I am new to ping and am unable to use pig builtin functions (please
> > see
> > > > > details below).
> > > > >
> > > > > Is this a CLASSPATH issue?
> > > > >
> > > > > Any ideas on how to resolve?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > John
> > > > >
> > > > > Details
> > > > >
> > > > > ### Line in pig script causing issue
> > > > > C = foreach B generate flatten($0), sum(lane_nbr) ;
> > > > >
> > > > > ### Error message
> > > > > 2012-05-17 11:15:41,673 [main] ERROR
> org.apache.pig.tools.grunt.Grunt
> > > - \
> > > > > ERROR 1070: Could not resolve sum using imports: [,
> > > > > org.apache.pig.builtin., org.apache.pig.impl.builtin.]
> > > > >
> > > > > pig -version
> > > > > Apache Pig version 0.8.1-cdh3u3 (rexported)
> > > >
> > >
> > >
> > >
> > > --
> > > John Morrison
> > >
> >
>
>
>
> --
> John Morrison
>
Re: ? ERROR 1070: Could not resolve sum using imports
Posted by John Morrison <jo...@gmail.com>.
OK. I have simplified the script and tried 2 different ways without
success:
1) B = foreach A generate flatten($0), SUM($0.a) ;
2) B = foreach A generate flatten($0), SUM(a) ;
Which both produce different errors(see below)?
Thanks,
John
Data:
cat v.log
1
2
3
4
Complete script 1 with error:
A = LOAD 'v.log' AS (a:double) ;
dump A;
B = foreach A generate flatten($0), SUM($0.a) ;
dump B;
Error for above script:
2012-05-17 20:54:11,836 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 1000: Error during parsing. Invalid alias: a in null
Complete script 2 with error:
A = LOAD 'v.log' AS (a:double) ;
dump A;
B = foreach A generate flatten($0), SUM(a) ;
dump B;
2012-05-17 20:56:11,795 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 1045: Could not infer the matching function for
org.apache.pig.builtin.SUM as multiple or none of them fit. Please use an
explicit cast.
On Thu, May 17, 2012 at 6:31 PM, Jonathan Coveney <jc...@gmail.com>wrote:
> I imagine the issue is that you're not giving it a bag... can you paste the
> whole script? That said, my guess is that this comes right after a group
> by, and that the following will fix it:
>
> C = foreach B generate flatten($0), sum($0.lane_nbr) ;
>
> but I'd need the whole script to be sure
>
> 2012/5/17 John Morrison <jo...@gmail.com>
>
> > OK. New problem with needing to use explicit cast?
> >
> > Could you give me syntax for explict cast?
> >
> > 2012-05-17 15:29:13,769 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> > ERROR 1046: Multiple matching functions for org.apache.pig.builtin.SUM
> with
> > input schema: ({{bytearray}}, {{double}}). Please use an explicit cast.
> >
> >
> >
> > On Thu, May 17, 2012 at 11:42 AM, Prashant Kommireddi
> > <pr...@gmail.com>wrote:
> >
> > > UDFs are case-sensitive. It should be all caps - SUM
> > >
> > > Can you please give that a try?
> > >
> > >
> > > On May 17, 2012, at 8:24 AM, John Morrison <jo...@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I am new to ping and am unable to use pig builtin functions (please
> see
> > > > details below).
> > > >
> > > > Is this a CLASSPATH issue?
> > > >
> > > > Any ideas on how to resolve?
> > > >
> > > > Thanks,
> > > >
> > > > John
> > > >
> > > > Details
> > > >
> > > > ### Line in pig script causing issue
> > > > C = foreach B generate flatten($0), sum(lane_nbr) ;
> > > >
> > > > ### Error message
> > > > 2012-05-17 11:15:41,673 [main] ERROR org.apache.pig.tools.grunt.Grunt
> > - \
> > > > ERROR 1070: Could not resolve sum using imports: [,
> > > > org.apache.pig.builtin., org.apache.pig.impl.builtin.]
> > > >
> > > > pig -version
> > > > Apache Pig version 0.8.1-cdh3u3 (rexported)
> > >
> >
> >
> >
> > --
> > John Morrison
> >
>
--
John Morrison
Re: ? ERROR 1070: Could not resolve sum using imports
Posted by Jonathan Coveney <jc...@gmail.com>.
I imagine the issue is that you're not giving it a bag... can you paste the
whole script? That said, my guess is that this comes right after a group
by, and that the following will fix it:
C = foreach B generate flatten($0), sum($0.lane_nbr) ;
but I'd need the whole script to be sure
2012/5/17 John Morrison <jo...@gmail.com>
> OK. New problem with needing to use explicit cast?
>
> Could you give me syntax for explict cast?
>
> 2012-05-17 15:29:13,769 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1046: Multiple matching functions for org.apache.pig.builtin.SUM with
> input schema: ({{bytearray}}, {{double}}). Please use an explicit cast.
>
>
>
> On Thu, May 17, 2012 at 11:42 AM, Prashant Kommireddi
> <pr...@gmail.com>wrote:
>
> > UDFs are case-sensitive. It should be all caps - SUM
> >
> > Can you please give that a try?
> >
> >
> > On May 17, 2012, at 8:24 AM, John Morrison <jo...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I am new to ping and am unable to use pig builtin functions (please see
> > > details below).
> > >
> > > Is this a CLASSPATH issue?
> > >
> > > Any ideas on how to resolve?
> > >
> > > Thanks,
> > >
> > > John
> > >
> > > Details
> > >
> > > ### Line in pig script causing issue
> > > C = foreach B generate flatten($0), sum(lane_nbr) ;
> > >
> > > ### Error message
> > > 2012-05-17 11:15:41,673 [main] ERROR org.apache.pig.tools.grunt.Grunt
> - \
> > > ERROR 1070: Could not resolve sum using imports: [,
> > > org.apache.pig.builtin., org.apache.pig.impl.builtin.]
> > >
> > > pig -version
> > > Apache Pig version 0.8.1-cdh3u3 (rexported)
> >
>
>
>
> --
> John Morrison
>
Re: ? ERROR 1070: Could not resolve sum using imports
Posted by John Morrison <jo...@gmail.com>.
OK. New problem with needing to use explicit cast?
Could you give me syntax for explict cast?
2012-05-17 15:29:13,769 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 1046: Multiple matching functions for org.apache.pig.builtin.SUM with
input schema: ({{bytearray}}, {{double}}). Please use an explicit cast.
On Thu, May 17, 2012 at 11:42 AM, Prashant Kommireddi
<pr...@gmail.com>wrote:
> UDFs are case-sensitive. It should be all caps - SUM
>
> Can you please give that a try?
>
>
> On May 17, 2012, at 8:24 AM, John Morrison <jo...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I am new to ping and am unable to use pig builtin functions (please see
> > details below).
> >
> > Is this a CLASSPATH issue?
> >
> > Any ideas on how to resolve?
> >
> > Thanks,
> >
> > John
> >
> > Details
> >
> > ### Line in pig script causing issue
> > C = foreach B generate flatten($0), sum(lane_nbr) ;
> >
> > ### Error message
> > 2012-05-17 11:15:41,673 [main] ERROR org.apache.pig.tools.grunt.Grunt - \
> > ERROR 1070: Could not resolve sum using imports: [,
> > org.apache.pig.builtin., org.apache.pig.impl.builtin.]
> >
> > pig -version
> > Apache Pig version 0.8.1-cdh3u3 (rexported)
>
--
John Morrison
Re: ? ERROR 1070: Could not resolve sum using imports
Posted by Prashant Kommireddi <pr...@gmail.com>.
UDFs are case-sensitive. It should be all caps - SUM
Can you please give that a try?
On May 17, 2012, at 8:24 AM, John Morrison <jo...@gmail.com> wrote:
> Hi,
>
> I am new to ping and am unable to use pig builtin functions (please see
> details below).
>
> Is this a CLASSPATH issue?
>
> Any ideas on how to resolve?
>
> Thanks,
>
> John
>
> Details
>
> ### Line in pig script causing issue
> C = foreach B generate flatten($0), sum(lane_nbr) ;
>
> ### Error message
> 2012-05-17 11:15:41,673 [main] ERROR org.apache.pig.tools.grunt.Grunt - \
> ERROR 1070: Could not resolve sum using imports: [,
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>
> pig -version
> Apache Pig version 0.8.1-cdh3u3 (rexported)