You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by "Uppuluri, Rohini" <ro...@teamaol.com> on 2010/07/29 17:46:20 UTC

Pig Data types issue

Hi all, 

I have a strange issue with data types. We have a custom loader which
loads data from logs similar to apache logs.
It sets the type of a field(var) to be int. When I examine its type, it
shows it to be int(using describe). However, when I do a numeric
comparison (for example: var==200) it throws an error.
 
java.lang.ClassCastException: java.lang.Integer cannot be cast to
java.lang.String
 at java.lang.String.compareTo(String.java:92)
 
And when a string comparison is done (using var == '200'), it again
throws an error.

ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1039: Incompatible types
in EqualTo Operator left hand side:int right hand side:chararray

When I explicitly type cast or use a UDF to convert it from  string to
integer or long and then do the numeric comparison, it works fine.
 
I searched if there are any previously  such issues have been reported
before, but I couldnt find anything related.
It would be great if any one could  throw more light on this if I am
missing something here.
 
Thanks,
-Rohini

Re: Pig Data types issue

Posted by Mridul Muralidharan <mr...@yahoo-inc.com>.
Can you try explicitly using the schema in the load ?
raw = LOAD '$inputPath' using my_custom_loader() AS (..., myint:int, ...);


I think pig will flag a schema mismatch if the declared schema does not 
match what is returned by loader.


Regards,
Mridul


On Friday 30 July 2010 11:21 AM, Uppuluri, Rohini wrote:
> Hi,
>
> The schema is being set as follows:
> s.add(new Schema.FieldSchema("var", DataType.INTEGER));
>
> In the pig script, using describe, I am getting the schema back as
> raw: { ...... var:int, ......}
>
> An excerpt from the pig script is as follows:
> ----
> raw = LOAD '$inputPath' using my_custom_loader();
> describe raw;
> ------
>
>
> Regards,
> -Rohini
>
>
> -----Original Message-----
> From: Mridul Muralidharan [mailto:mridulm@yahoo-inc.com]
> Sent: Friday, July 30, 2010 4:24 AM
> To: pig-user@hadoop.apache.org
> Cc: Uppuluri, Rohini
> Subject: Re: Pig Data types issue
>
>
> Are you returning the appropriate schema and using the correct schema in
> the pig script ?
> More info might help though !
>
> Regards,
> Mridul
>
> On Thursday 29 July 2010 09:16 PM, Uppuluri, Rohini wrote:
>> Hi all,
>>
>> I have a strange issue with data types. We have a custom loader which
>> loads data from logs similar to apache logs.
>> It sets the type of a field(var) to be int. When I examine its type,
>> it shows it to be int(using describe). However, when I do a numeric
>> comparison (for example: var==200) it throws an error.
>>
>> java.lang.ClassCastException: java.lang.Integer cannot be cast to
>> java.lang.String
>>    at java.lang.String.compareTo(String.java:92)
>>
>> And when a string comparison is done (using var == '200'), it again
>> throws an error.
>>
>> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1039: Incompatible
>> types in EqualTo Operator left hand side:int right hand side:chararray
>>
>> When I explicitly type cast or use a UDF to convert it from  string to
>
>> integer or long and then do the numeric comparison, it works fine.
>>
>> I searched if there are any previously  such issues have been reported
>
>> before, but I couldnt find anything related.
>> It would be great if any one could  throw more light on this if I am
>> missing something here.
>>
>> Thanks,
>> -Rohini
>


Re: Pig Data types issue

Posted by Mridul Muralidharan <mr...@yahoo-inc.com>.
Are you returning the appropriate schema and using the correct schema in 
the pig script ?
More info might help though !

Regards,
Mridul

On Thursday 29 July 2010 09:16 PM, Uppuluri, Rohini wrote:
> Hi all,
>
> I have a strange issue with data types. We have a custom loader which
> loads data from logs similar to apache logs.
> It sets the type of a field(var) to be int. When I examine its type, it
> shows it to be int(using describe). However, when I do a numeric
> comparison (for example: var==200) it throws an error.
>
> java.lang.ClassCastException: java.lang.Integer cannot be cast to
> java.lang.String
>   at java.lang.String.compareTo(String.java:92)
>
> And when a string comparison is done (using var == '200'), it again
> throws an error.
>
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1039: Incompatible types
> in EqualTo Operator left hand side:int right hand side:chararray
>
> When I explicitly type cast or use a UDF to convert it from  string to
> integer or long and then do the numeric comparison, it works fine.
>
> I searched if there are any previously  such issues have been reported
> before, but I couldnt find anything related.
> It would be great if any one could  throw more light on this if I am
> missing something here.
>
> Thanks,
> -Rohini