You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Mathias Fryde <ma...@mimesis-republic.com> on 2009/03/17 15:59:18 UTC
Pig / Map data type for keys / String cannot be cast to integer
Hello,
I have a map whit a key that have diffrent values like *: Location, Zone,
35, Place ....
*When I execute my script with a filter on this key I a have a Cast error.*
source = LOAD 'thesource' USING PigStorage('|') AS ( themap: map []);
A = FILTER source BY themap#'key01' == 'Location' ;
DUMP **A**;*
I also tried :
*A = FILTER source BY (chararray)themap#'key01' == 'Location' ;*
*A = FILTER source BY (chararray)themap#'key01' == 'Location' ;
**A = FILTER source BY **themap#'data:Resource:id' matches '.***Location**
.*';
*but still have the same error ... *
*
- *anyone as a solution How do do it? *
- *Why Pig down't use default string? *
- *Do Pig define datatypes alone ?*
org.apache.pig.tools.grunt.Grunt -
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
open iterator for alias A
at org.apache.pig.PigServer.openIterator(PigServer.java:438)
at
org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:359)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:193)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:99)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:82)
at org.apache.pig.Main.main(Main.java:354)
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002:
Unable to store alias A
at org.apache.pig.PigServer.store(PigServer.java:469)
at org.apache.pig.PigServer.openIterator(PigServer.java:426)
... 5 more
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002:
Unable to store alias A
at org.apache.pig.PigServer.store(PigServer.java:502)
at org.apache.pig.PigServer.store(PigServer.java:465)
... 6 more
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
java.lang.String cannot be cast to java.lang.Integer
at
org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execute(LocalExecutionEngine.java:178)
at
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:695)
at org.apache.pig.PigServer.store(PigServer.java:498)
... 7 more
Caused by: *java.lang.ClassCastException: java.lang.String cannot be cast to
java.lang.Integer*
at java.lang.Integer.compareTo(Integer.java:35)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr.doComparison(EqualToExpr.java:127)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr.getNext(EqualToExpr.java:100)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POAnd.getNext(POAnd.java:78)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFilter.getNext(POFilter.java:148)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.store(POStore.java:138)
at
org.apache.pig.backend.local.executionengine.LocalPigLauncher.launchPig(LocalPigLauncher.java:68)
at
org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execute(LocalExecutionEngine.java:166)
regards,
Mathias
RE: Pig / Map data type for keys / String cannot be cast to integer
Posted by Santhosh Srinivasan <sm...@yahoo-inc.com>.
Sure. That works too. Probably, it's a better idea to return a boolean
instead of a String.
Thanks,
Santhosh
-----Original Message-----
From: Mathias Fryde [mailto:mathias.fryde@mimesis-republic.com]
Sent: Thursday, March 19, 2009 1:35 AM
To: pig-user@hadoop.apache.org
Subject: Re: Pig / Map data type for keys / String cannot be cast to
integer
Hi,
Thanks for your answer, I manage to build an UDF to make my Filter work
:
My script :
register './myudf.jar'
source = LOAD 'thesource' USING PigStorage('|') AS ( themap: map []);
A = FILTER source BY org.apache.pig.myudf.ConditionFilter(themap#'key01'
,
'Location' ) ;
DUMP A;
the Udf Code :
package org.apache.pig.myudf;
import java.io.IOException;
import java.lang.*;
import java.util.Date;
import java.text.*;
import org.apache.pig.EvalFunc;
import org.apache.pig.data.Tuple;
import org.apache.pig.impl.util.WrappedIOException;
public class ConditionFilter extends EvalFunc<String>
{
public String exec(Tuple input) throws IOException {
if (input == null || input.size() == 0)
return null;
try{
Object obj = input.get(0);
String ourresult = "FALSE";
if (obj instanceof String) {
String ourinput = (String) obj;
String ourcondition = (String) input.get(1);
if (ourinput.equals(ourcondition))
ourresult = "TRUE";
}
return ourresult ;
}catch(Exception e){
throw WrappedIOException.wrap("Caught exception processing
input
row ", e);
}
}
}
regards,
Mathias.
2009/3/18 Santhosh Srinivasan <sm...@yahoo-inc.com>
> The value 35 for key01 on the second row is being interpreted as an
> integer. The issue is captured in PIG-724
> (https://issues.apache.org/jira/browse/PIG-724). The only workaround I
> can think of right now, is to ensure consistency in the values for the
> keys, i.e., knock of numeric values for key key01 in your data.
>
> Thanks,
> Santhosh
>
> -----Original Message-----
> From: Mathias Fryde [mailto:mathias.fryde@mimesis-republic.com]
> Sent: Wednesday, March 18, 2009 5:47 AM
> To: pig-user@hadoop.apache.org
> Subject: Re: Pig / Map data type for keys / String cannot be cast to
> integer
>
> Hello here is a simple example.
> regards,
>
> [key01#Location,key02#value01]
> [key01#35,key02#value01]
> [key01#Location,key02#value01]
> [key01#Zone,key02#value01]
> [key01#toto,key02#value01]
> [key01#0,key02#value01]
> [key02#value01]
> [key01#Location,key02#value01]
>
>
>
>
> 2009/3/18 Mridul Muralidharan <mr...@yahoo-inc.com>
>
> >
> > I am not sure what the problem is, but my guess would be that the
map
> is
> > getting loaded as (chararray, int) - and the comparison is failing
cos
> of it
> > ?
> > Someone from pig team can confirm if this is a known bug/issue (iirc
> it is
> > not).
> >
> > If possible, can you post a snippet of the input file ?
> >
> > Thanks,
> > Mridul
> >
> >
> >
> > Mathias Fryde wrote:
> >
> >> Hello,
> >> I have a map whit a key that have diffrent values like *:
Location,
> Zone,
> >> 35, Place ....
> >> *When I execute my script with a filter on this key I a have a Cast
> >> error.*
> >>
> >> source = LOAD 'thesource' USING PigStorage('|') AS ( themap: map
> []);
> >> A = FILTER source BY themap#'key01' == 'Location' ;
> >> DUMP **A**;*
> >>
> >> I also tried :
> >> *A = FILTER source BY (chararray)themap#'key01' == 'Location' ;*
> >> *A = FILTER source BY (chararray)themap#'key01' == 'Location' ;
> >> **A = FILTER source BY **themap#'data:Resource:id' matches
> '.***Location**
> >> .*';
> >> *but still have the same error ... *
> >>
> >> *
> >>
> >> - *anyone as a solution How do do it? *
> >> - *Why Pig down't use default string? *
> >> - *Do Pig define datatypes alone ?*
> >>
> >>
> >>
> >> org.apache.pig.tools.grunt.Grunt -
> >> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066:
> Unable to
> >> open iterator for alias A
> >> at org.apache.pig.PigServer.openIterator(PigServer.java:438)
> >> at
> >>
>
org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:359)
> >> at
> >>
> >>
>
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptPar
> ser.java:193)
> >> at
> >>
> >>
>
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java
> :99)
> >> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:82)
> >> at org.apache.pig.Main.main(Main.java:354)
> >> Caused by: org.apache.pig.impl.logicalLayer.FrontendException:
ERROR
> 1002:
> >> Unable to store alias A
> >> at org.apache.pig.PigServer.store(PigServer.java:469)
> >> at org.apache.pig.PigServer.openIterator(PigServer.java:426)
> >> ... 5 more
> >> Caused by: org.apache.pig.impl.logicalLayer.FrontendException:
ERROR
> 1002:
> >> Unable to store alias A
> >> at org.apache.pig.PigServer.store(PigServer.java:502)
> >> at org.apache.pig.PigServer.store(PigServer.java:465)
> >> ... 6 more
> >> Caused by: org.apache.pig.backend.executionengine.ExecException:
> ERROR 0:
> >> java.lang.String cannot be cast to java.lang.Integer
> >> at
> >>
> >>
>
org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execut
> e(LocalExecutionEngine.java:178)
> >> at
> >>
>
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:695)
> >> at org.apache.pig.PigServer.store(PigServer.java:498)
> >> ... 7 more
> >> Caused by: *java.lang.ClassCastException: java.lang.String cannot
be
> cast
> >> to
> >> java.lang.Integer*
> >> at java.lang.Integer.compareTo(Integer.java:35)
> >> at
> >>
> >>
>
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOp
> erators.EqualToExpr.doComparison(EqualToExpr.java:127)
> >> at
> >>
> >>
>
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOp
> erators.EqualToExpr.getNext(EqualToExpr.java:100)
> >> at
> >>
> >>
>
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOp
> erators.POAnd.getNext(POAnd.java:78)
> >> at
> >>
> >>
>
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOp
> erators.POFilter.getNext(POFilter.java:148)
> >> at
> >>
> >>
>
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOper
> ator.processInput(PhysicalOperator.java:231)
> >> at
> >>
> >>
>
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOp
> erators.POStore.store(POStore.java:138)
> >> at
> >>
> >>
>
org.apache.pig.backend.local.executionengine.LocalPigLauncher.launchPig(
> LocalPigLauncher.java:68)
> >> at
> >>
> >>
>
org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execut
> e(LocalExecutionEngine.java:166)
> >>
> >> regards,
> >> Mathias
> >>
> >>
> >
>
Re: Pig / Map data type for keys / String cannot be cast to integer
Posted by Mathias Fryde <ma...@mimesis-republic.com>.
Hi,
Thanks for your answer, I manage to build an UDF to make my Filter work :
My script :
register './myudf.jar'
source = LOAD 'thesource' USING PigStorage('|') AS ( themap: map []);
A = FILTER source BY org.apache.pig.myudf.ConditionFilter(themap#'key01' ,
'Location' ) ;
DUMP A;
the Udf Code :
package org.apache.pig.myudf;
import java.io.IOException;
import java.lang.*;
import java.util.Date;
import java.text.*;
import org.apache.pig.EvalFunc;
import org.apache.pig.data.Tuple;
import org.apache.pig.impl.util.WrappedIOException;
public class ConditionFilter extends EvalFunc<String>
{
public String exec(Tuple input) throws IOException {
if (input == null || input.size() == 0)
return null;
try{
Object obj = input.get(0);
String ourresult = "FALSE";
if (obj instanceof String) {
String ourinput = (String) obj;
String ourcondition = (String) input.get(1);
if (ourinput.equals(ourcondition))
ourresult = "TRUE";
}
return ourresult ;
}catch(Exception e){
throw WrappedIOException.wrap("Caught exception processing input
row ", e);
}
}
}
regards,
Mathias.
2009/3/18 Santhosh Srinivasan <sm...@yahoo-inc.com>
> The value 35 for key01 on the second row is being interpreted as an
> integer. The issue is captured in PIG-724
> (https://issues.apache.org/jira/browse/PIG-724). The only workaround I
> can think of right now, is to ensure consistency in the values for the
> keys, i.e., knock of numeric values for key key01 in your data.
>
> Thanks,
> Santhosh
>
> -----Original Message-----
> From: Mathias Fryde [mailto:mathias.fryde@mimesis-republic.com]
> Sent: Wednesday, March 18, 2009 5:47 AM
> To: pig-user@hadoop.apache.org
> Subject: Re: Pig / Map data type for keys / String cannot be cast to
> integer
>
> Hello here is a simple example.
> regards,
>
> [key01#Location,key02#value01]
> [key01#35,key02#value01]
> [key01#Location,key02#value01]
> [key01#Zone,key02#value01]
> [key01#toto,key02#value01]
> [key01#0,key02#value01]
> [key02#value01]
> [key01#Location,key02#value01]
>
>
>
>
> 2009/3/18 Mridul Muralidharan <mr...@yahoo-inc.com>
>
> >
> > I am not sure what the problem is, but my guess would be that the map
> is
> > getting loaded as (chararray, int) - and the comparison is failing cos
> of it
> > ?
> > Someone from pig team can confirm if this is a known bug/issue (iirc
> it is
> > not).
> >
> > If possible, can you post a snippet of the input file ?
> >
> > Thanks,
> > Mridul
> >
> >
> >
> > Mathias Fryde wrote:
> >
> >> Hello,
> >> I have a map whit a key that have diffrent values like *: Location,
> Zone,
> >> 35, Place ....
> >> *When I execute my script with a filter on this key I a have a Cast
> >> error.*
> >>
> >> source = LOAD 'thesource' USING PigStorage('|') AS ( themap: map
> []);
> >> A = FILTER source BY themap#'key01' == 'Location' ;
> >> DUMP **A**;*
> >>
> >> I also tried :
> >> *A = FILTER source BY (chararray)themap#'key01' == 'Location' ;*
> >> *A = FILTER source BY (chararray)themap#'key01' == 'Location' ;
> >> **A = FILTER source BY **themap#'data:Resource:id' matches
> '.***Location**
> >> .*';
> >> *but still have the same error ... *
> >>
> >> *
> >>
> >> - *anyone as a solution How do do it? *
> >> - *Why Pig down't use default string? *
> >> - *Do Pig define datatypes alone ?*
> >>
> >>
> >>
> >> org.apache.pig.tools.grunt.Grunt -
> >> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066:
> Unable to
> >> open iterator for alias A
> >> at org.apache.pig.PigServer.openIterator(PigServer.java:438)
> >> at
> >>
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:359)
> >> at
> >>
> >>
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptPar
> ser.java:193)
> >> at
> >>
> >>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java
> :99)
> >> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:82)
> >> at org.apache.pig.Main.main(Main.java:354)
> >> Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR
> 1002:
> >> Unable to store alias A
> >> at org.apache.pig.PigServer.store(PigServer.java:469)
> >> at org.apache.pig.PigServer.openIterator(PigServer.java:426)
> >> ... 5 more
> >> Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR
> 1002:
> >> Unable to store alias A
> >> at org.apache.pig.PigServer.store(PigServer.java:502)
> >> at org.apache.pig.PigServer.store(PigServer.java:465)
> >> ... 6 more
> >> Caused by: org.apache.pig.backend.executionengine.ExecException:
> ERROR 0:
> >> java.lang.String cannot be cast to java.lang.Integer
> >> at
> >>
> >>
> org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execut
> e(LocalExecutionEngine.java:178)
> >> at
> >>
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:695)
> >> at org.apache.pig.PigServer.store(PigServer.java:498)
> >> ... 7 more
> >> Caused by: *java.lang.ClassCastException: java.lang.String cannot be
> cast
> >> to
> >> java.lang.Integer*
> >> at java.lang.Integer.compareTo(Integer.java:35)
> >> at
> >>
> >>
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOp
> erators.EqualToExpr.doComparison(EqualToExpr.java:127)
> >> at
> >>
> >>
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOp
> erators.EqualToExpr.getNext(EqualToExpr.java:100)
> >> at
> >>
> >>
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOp
> erators.POAnd.getNext(POAnd.java:78)
> >> at
> >>
> >>
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOp
> erators.POFilter.getNext(POFilter.java:148)
> >> at
> >>
> >>
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOper
> ator.processInput(PhysicalOperator.java:231)
> >> at
> >>
> >>
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOp
> erators.POStore.store(POStore.java:138)
> >> at
> >>
> >>
> org.apache.pig.backend.local.executionengine.LocalPigLauncher.launchPig(
> LocalPigLauncher.java:68)
> >> at
> >>
> >>
> org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execut
> e(LocalExecutionEngine.java:166)
> >>
> >> regards,
> >> Mathias
> >>
> >>
> >
>
RE: Pig / Map data type for keys / String cannot be cast to integer
Posted by Santhosh Srinivasan <sm...@yahoo-inc.com>.
The value 35 for key01 on the second row is being interpreted as an
integer. The issue is captured in PIG-724
(https://issues.apache.org/jira/browse/PIG-724). The only workaround I
can think of right now, is to ensure consistency in the values for the
keys, i.e., knock of numeric values for key key01 in your data.
Thanks,
Santhosh
-----Original Message-----
From: Mathias Fryde [mailto:mathias.fryde@mimesis-republic.com]
Sent: Wednesday, March 18, 2009 5:47 AM
To: pig-user@hadoop.apache.org
Subject: Re: Pig / Map data type for keys / String cannot be cast to
integer
Hello here is a simple example.
regards,
[key01#Location,key02#value01]
[key01#35,key02#value01]
[key01#Location,key02#value01]
[key01#Zone,key02#value01]
[key01#toto,key02#value01]
[key01#0,key02#value01]
[key02#value01]
[key01#Location,key02#value01]
2009/3/18 Mridul Muralidharan <mr...@yahoo-inc.com>
>
> I am not sure what the problem is, but my guess would be that the map
is
> getting loaded as (chararray, int) - and the comparison is failing cos
of it
> ?
> Someone from pig team can confirm if this is a known bug/issue (iirc
it is
> not).
>
> If possible, can you post a snippet of the input file ?
>
> Thanks,
> Mridul
>
>
>
> Mathias Fryde wrote:
>
>> Hello,
>> I have a map whit a key that have diffrent values like *: Location,
Zone,
>> 35, Place ....
>> *When I execute my script with a filter on this key I a have a Cast
>> error.*
>>
>> source = LOAD 'thesource' USING PigStorage('|') AS ( themap: map
[]);
>> A = FILTER source BY themap#'key01' == 'Location' ;
>> DUMP **A**;*
>>
>> I also tried :
>> *A = FILTER source BY (chararray)themap#'key01' == 'Location' ;*
>> *A = FILTER source BY (chararray)themap#'key01' == 'Location' ;
>> **A = FILTER source BY **themap#'data:Resource:id' matches
'.***Location**
>> .*';
>> *but still have the same error ... *
>>
>> *
>>
>> - *anyone as a solution How do do it? *
>> - *Why Pig down't use default string? *
>> - *Do Pig define datatypes alone ?*
>>
>>
>>
>> org.apache.pig.tools.grunt.Grunt -
>> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066:
Unable to
>> open iterator for alias A
>> at org.apache.pig.PigServer.openIterator(PigServer.java:438)
>> at
>>
org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:359)
>> at
>>
>>
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptPar
ser.java:193)
>> at
>>
>>
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java
:99)
>> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:82)
>> at org.apache.pig.Main.main(Main.java:354)
>> Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR
1002:
>> Unable to store alias A
>> at org.apache.pig.PigServer.store(PigServer.java:469)
>> at org.apache.pig.PigServer.openIterator(PigServer.java:426)
>> ... 5 more
>> Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR
1002:
>> Unable to store alias A
>> at org.apache.pig.PigServer.store(PigServer.java:502)
>> at org.apache.pig.PigServer.store(PigServer.java:465)
>> ... 6 more
>> Caused by: org.apache.pig.backend.executionengine.ExecException:
ERROR 0:
>> java.lang.String cannot be cast to java.lang.Integer
>> at
>>
>>
org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execut
e(LocalExecutionEngine.java:178)
>> at
>>
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:695)
>> at org.apache.pig.PigServer.store(PigServer.java:498)
>> ... 7 more
>> Caused by: *java.lang.ClassCastException: java.lang.String cannot be
cast
>> to
>> java.lang.Integer*
>> at java.lang.Integer.compareTo(Integer.java:35)
>> at
>>
>>
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOp
erators.EqualToExpr.doComparison(EqualToExpr.java:127)
>> at
>>
>>
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOp
erators.EqualToExpr.getNext(EqualToExpr.java:100)
>> at
>>
>>
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOp
erators.POAnd.getNext(POAnd.java:78)
>> at
>>
>>
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOp
erators.POFilter.getNext(POFilter.java:148)
>> at
>>
>>
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOper
ator.processInput(PhysicalOperator.java:231)
>> at
>>
>>
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOp
erators.POStore.store(POStore.java:138)
>> at
>>
>>
org.apache.pig.backend.local.executionengine.LocalPigLauncher.launchPig(
LocalPigLauncher.java:68)
>> at
>>
>>
org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execut
e(LocalExecutionEngine.java:166)
>>
>> regards,
>> Mathias
>>
>>
>
Re: Pig / Map data type for keys / String cannot be cast to integer
Posted by Mathias Fryde <ma...@mimesis-republic.com>.
Hello here is a simple example.
regards,
[key01#Location,key02#value01]
[key01#35,key02#value01]
[key01#Location,key02#value01]
[key01#Zone,key02#value01]
[key01#toto,key02#value01]
[key01#0,key02#value01]
[key02#value01]
[key01#Location,key02#value01]
2009/3/18 Mridul Muralidharan <mr...@yahoo-inc.com>
>
> I am not sure what the problem is, but my guess would be that the map is
> getting loaded as (chararray, int) - and the comparison is failing cos of it
> ?
> Someone from pig team can confirm if this is a known bug/issue (iirc it is
> not).
>
> If possible, can you post a snippet of the input file ?
>
> Thanks,
> Mridul
>
>
>
> Mathias Fryde wrote:
>
>> Hello,
>> I have a map whit a key that have diffrent values like *: Location, Zone,
>> 35, Place ....
>> *When I execute my script with a filter on this key I a have a Cast
>> error.*
>>
>> source = LOAD 'thesource' USING PigStorage('|') AS ( themap: map []);
>> A = FILTER source BY themap#'key01' == 'Location' ;
>> DUMP **A**;*
>>
>> I also tried :
>> *A = FILTER source BY (chararray)themap#'key01' == 'Location' ;*
>> *A = FILTER source BY (chararray)themap#'key01' == 'Location' ;
>> **A = FILTER source BY **themap#'data:Resource:id' matches '.***Location**
>> .*';
>> *but still have the same error ... *
>>
>> *
>>
>> - *anyone as a solution How do do it? *
>> - *Why Pig down't use default string? *
>> - *Do Pig define datatypes alone ?*
>>
>>
>>
>> org.apache.pig.tools.grunt.Grunt -
>> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
>> open iterator for alias A
>> at org.apache.pig.PigServer.openIterator(PigServer.java:438)
>> at
>> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:359)
>> at
>>
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:193)
>> at
>>
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:99)
>> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:82)
>> at org.apache.pig.Main.main(Main.java:354)
>> Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002:
>> Unable to store alias A
>> at org.apache.pig.PigServer.store(PigServer.java:469)
>> at org.apache.pig.PigServer.openIterator(PigServer.java:426)
>> ... 5 more
>> Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002:
>> Unable to store alias A
>> at org.apache.pig.PigServer.store(PigServer.java:502)
>> at org.apache.pig.PigServer.store(PigServer.java:465)
>> ... 6 more
>> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
>> java.lang.String cannot be cast to java.lang.Integer
>> at
>>
>> org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execute(LocalExecutionEngine.java:178)
>> at
>> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:695)
>> at org.apache.pig.PigServer.store(PigServer.java:498)
>> ... 7 more
>> Caused by: *java.lang.ClassCastException: java.lang.String cannot be cast
>> to
>> java.lang.Integer*
>> at java.lang.Integer.compareTo(Integer.java:35)
>> at
>>
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr.doComparison(EqualToExpr.java:127)
>> at
>>
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr.getNext(EqualToExpr.java:100)
>> at
>>
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POAnd.getNext(POAnd.java:78)
>> at
>>
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFilter.getNext(POFilter.java:148)
>> at
>>
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
>> at
>>
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.store(POStore.java:138)
>> at
>>
>> org.apache.pig.backend.local.executionengine.LocalPigLauncher.launchPig(LocalPigLauncher.java:68)
>> at
>>
>> org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execute(LocalExecutionEngine.java:166)
>>
>> regards,
>> Mathias
>>
>>
>
Re: Pig / Map data type for keys / String cannot be cast to integer
Posted by Mridul Muralidharan <mr...@yahoo-inc.com>.
I am not sure what the problem is, but my guess would be that the map is
getting loaded as (chararray, int) - and the comparison is failing cos
of it ?
Someone from pig team can confirm if this is a known bug/issue (iirc it
is not).
If possible, can you post a snippet of the input file ?
Thanks,
Mridul
Mathias Fryde wrote:
> Hello,
> I have a map whit a key that have diffrent values like *: Location, Zone,
> 35, Place ....
> *When I execute my script with a filter on this key I a have a Cast error.*
>
> source = LOAD 'thesource' USING PigStorage('|') AS ( themap: map []);
> A = FILTER source BY themap#'key01' == 'Location' ;
> DUMP **A**;*
>
> I also tried :
> *A = FILTER source BY (chararray)themap#'key01' == 'Location' ;*
> *A = FILTER source BY (chararray)themap#'key01' == 'Location' ;
> **A = FILTER source BY **themap#'data:Resource:id' matches '.***Location**
> .*';
> *but still have the same error ... *
>
> *
>
> - *anyone as a solution How do do it? *
> - *Why Pig down't use default string? *
> - *Do Pig define datatypes alone ?*
>
>
>
> org.apache.pig.tools.grunt.Grunt -
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
> open iterator for alias A
> at org.apache.pig.PigServer.openIterator(PigServer.java:438)
> at
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:359)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:193)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:99)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:82)
> at org.apache.pig.Main.main(Main.java:354)
> Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002:
> Unable to store alias A
> at org.apache.pig.PigServer.store(PigServer.java:469)
> at org.apache.pig.PigServer.openIterator(PigServer.java:426)
> ... 5 more
> Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002:
> Unable to store alias A
> at org.apache.pig.PigServer.store(PigServer.java:502)
> at org.apache.pig.PigServer.store(PigServer.java:465)
> ... 6 more
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> java.lang.String cannot be cast to java.lang.Integer
> at
> org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execute(LocalExecutionEngine.java:178)
> at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:695)
> at org.apache.pig.PigServer.store(PigServer.java:498)
> ... 7 more
> Caused by: *java.lang.ClassCastException: java.lang.String cannot be cast to
> java.lang.Integer*
> at java.lang.Integer.compareTo(Integer.java:35)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr.doComparison(EqualToExpr.java:127)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr.getNext(EqualToExpr.java:100)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POAnd.getNext(POAnd.java:78)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFilter.getNext(POFilter.java:148)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.store(POStore.java:138)
> at
> org.apache.pig.backend.local.executionengine.LocalPigLauncher.launchPig(LocalPigLauncher.java:68)
> at
> org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execute(LocalExecutionEngine.java:166)
>
> regards,
> Mathias
>
Fwd: Pig / Map data type for keys / String cannot be cast to integer
Posted by Mathias Fryde <ma...@mimesis-republic.com>.
Hello,
Does anyone have an idea ?
Regards,
Mathias
---------- Forwarded message ----------
From: Mathias Fryde <ma...@mimesis-republic.com>
Date: 2009/3/17
Subject: Pig / Map data type for keys / String cannot be cast to integer
To: pig-user@hadoop.apache.org
Hello,
I have a map whit a key that have diffrent values like *: Location, Zone,
35, Place ....
*When I execute my script with a filter on this key I a have a Cast error.*
source = LOAD 'thesource' USING PigStorage('|') AS ( themap: map []);
A = FILTER source BY themap#'key01' == 'Location' ;
DUMP **A**;*
I also tried :
*A = FILTER source BY (chararray)themap#'key01' == 'Location' ;*
*A = FILTER source BY (chararray)themap#'key01' == 'Location' ;
**A = FILTER source BY **themap#'data:Resource:id' matches '.***Location**
.*';
*but still have the same error ... *
*
- *anyone as a solution How do do it? *
- *Why Pig down't use default string? *
- *Do Pig define datatypes alone ?*
org.apache.pig.tools.grunt.Grunt -
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
open iterator for alias A
at org.apache.pig.PigServer.openIterator(PigServer.java:438)
at
org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:359)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:193)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:99)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:82)
at org.apache.pig.Main.main(Main.java:354)
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002:
Unable to store alias A
at org.apache.pig.PigServer.store(PigServer.java:469)
at org.apache.pig.PigServer.openIterator(PigServer.java:426)
... 5 more
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002:
Unable to store alias A
at org.apache.pig.PigServer.store(PigServer.java:502)
at org.apache.pig.PigServer.store(PigServer.java:465)
... 6 more
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
java.lang.String cannot be cast to java.lang.Integer
at
org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execute(LocalExecutionEngine.java:178)
at
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:695)
at org.apache.pig.PigServer.store(PigServer.java:498)
... 7 more
Caused by: *java.lang.ClassCastException: java.lang.String cannot be cast to
java.lang.Integer*
at java.lang.Integer.compareTo(Integer.java:35)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr.doComparison(EqualToExpr.java:127)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr.getNext(EqualToExpr.java:100)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POAnd.getNext(POAnd.java:78)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFilter.getNext(POFilter.java:148)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.store(POStore.java:138)
at
org.apache.pig.backend.local.executionengine.LocalPigLauncher.launchPig(LocalPigLauncher.java:68)
at
org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execute(LocalExecutionEngine.java:166)
regards,
Mathias