You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Neil Conway <ne...@gmail.com> on 2009/05/03 08:15:06 UTC
Re: more ClassCastExceptions with joins, group by and non-strings
On Sat, Apr 25, 2009 at 4:47 PM, Peter Alvaro <pa...@gmail.com> wrote:
> create table bar (
> bas string--,
> --bat double
> )
> ROW FORMAT DELIMITED FIELDS TERMINATED BY '\174';
"bat" shouldn't be commented-out, if you want to use the provided test data.
> It appears that queries with all three of (join, group by, non-string
> datatype) cause a crash in the serde code run at the reducer:
I can reproduce this issue with the trunk Hive code (+ Hadoop 0.19.1).
Has anyone from the Hive team had a chance to investigate this?
Thanks,
Neil
Re: more ClassCastExceptions with joins, group by and non-strings
Posted by Neil Conway <ne...@gmail.com>.
Prasad,
I've confirmed that I don't see the issue with Hive 0.3 (and Hadoop
0.19.1), but I can still reproduce it with trunk. I can take a look at
finding the exact revision that caused the problem later this week...
Neil
On Sun, May 3, 2009 at 4:06 PM, Prasad Chakka <pc...@facebook.com> wrote:
> I am not able to reproduce the error. I just tried it on our cluster (hadoop-0.17.2.1 & hive-0.3).
>
>
> ________________________________
> From: Neil Conway <ne...@gmail.com>
> Reply-To: <hi...@hadoop.apache.org>
> Date: Sun, 3 May 2009 15:38:48 -0700
> To: <hi...@hadoop.apache.org>
> Subject: Re: more ClassCastExceptions with joins, group by and non-strings
>
> Prasad,
>
> Thanks for the response. However, as far as I can tell, in the example
> query Peter gave we are already casting the non-string columns to
> string, but we're still seeing a ClassCastException. Any other
> suggestions?
>
> Thanks,
>
> Neil
>
> On Sun, May 3, 2009 at 7:29 AM, Prasad Chakka <pc...@facebook.com> wrote:
>> I think this is a known bug. casting the non-string column should work.
>>
>> See the below email thread (and HIVE-405)
>> http://mail-archives.apache.org/mod_mbox/hadoop-hive-user/200904.mbox/browser
>>
>>
>> ________________________________
>> From: Neil Conway <ne...@gmail.com>
>> Reply-To: <hi...@hadoop.apache.org>
>> Date: Sat, 2 May 2009 23:15:06 -0700
>> To: <hi...@hadoop.apache.org>
>> Subject: Re: more ClassCastExceptions with joins, group by and non-strings
>>
>> On Sat, Apr 25, 2009 at 4:47 PM, Peter Alvaro <pa...@gmail.com> wrote:
>>> create table bar (
>>> bas string--,
>>> --bat double
>>> )
>>> ROW FORMAT DELIMITED FIELDS TERMINATED BY '\174';
>>
>> "bat" shouldn't be commented-out, if you want to use the provided test data.
>>
>>> It appears that queries with all three of (join, group by, non-string
>>> datatype) cause a crash in the serde code run at the reducer:
>>
>> I can reproduce this issue with the trunk Hive code (+ Hadoop 0.19.1).
>> Has anyone from the Hive team had a chance to investigate this?
>>
>> Thanks,
>>
>> Neil
>>
>>
>
>
Re: more ClassCastExceptions with joins, group by and non-strings
Posted by Prasad Chakka <pc...@facebook.com>.
I am not able to reproduce the error. I just tried it on our cluster (hadoop-0.17.2.1 & hive-0.3).
________________________________
From: Neil Conway <ne...@gmail.com>
Reply-To: <hi...@hadoop.apache.org>
Date: Sun, 3 May 2009 15:38:48 -0700
To: <hi...@hadoop.apache.org>
Subject: Re: more ClassCastExceptions with joins, group by and non-strings
Prasad,
Thanks for the response. However, as far as I can tell, in the example
query Peter gave we are already casting the non-string columns to
string, but we're still seeing a ClassCastException. Any other
suggestions?
Thanks,
Neil
On Sun, May 3, 2009 at 7:29 AM, Prasad Chakka <pc...@facebook.com> wrote:
> I think this is a known bug. casting the non-string column should work.
>
> See the below email thread (and HIVE-405)
> http://mail-archives.apache.org/mod_mbox/hadoop-hive-user/200904.mbox/browser
>
>
> ________________________________
> From: Neil Conway <ne...@gmail.com>
> Reply-To: <hi...@hadoop.apache.org>
> Date: Sat, 2 May 2009 23:15:06 -0700
> To: <hi...@hadoop.apache.org>
> Subject: Re: more ClassCastExceptions with joins, group by and non-strings
>
> On Sat, Apr 25, 2009 at 4:47 PM, Peter Alvaro <pa...@gmail.com> wrote:
>> create table bar (
>> bas string--,
>> --bat double
>> )
>> ROW FORMAT DELIMITED FIELDS TERMINATED BY '\174';
>
> "bat" shouldn't be commented-out, if you want to use the provided test data.
>
>> It appears that queries with all three of (join, group by, non-string
>> datatype) cause a crash in the serde code run at the reducer:
>
> I can reproduce this issue with the trunk Hive code (+ Hadoop 0.19.1).
> Has anyone from the Hive team had a chance to investigate this?
>
> Thanks,
>
> Neil
>
>
Re: more ClassCastExceptions with joins, group by and non-strings
Posted by Neil Conway <ne...@gmail.com>.
Prasad,
Thanks for the response. However, as far as I can tell, in the example
query Peter gave we are already casting the non-string columns to
string, but we're still seeing a ClassCastException. Any other
suggestions?
Thanks,
Neil
On Sun, May 3, 2009 at 7:29 AM, Prasad Chakka <pc...@facebook.com> wrote:
> I think this is a known bug. casting the non-string column should work.
>
> See the below email thread (and HIVE-405)
> http://mail-archives.apache.org/mod_mbox/hadoop-hive-user/200904.mbox/browser
>
>
> ________________________________
> From: Neil Conway <ne...@gmail.com>
> Reply-To: <hi...@hadoop.apache.org>
> Date: Sat, 2 May 2009 23:15:06 -0700
> To: <hi...@hadoop.apache.org>
> Subject: Re: more ClassCastExceptions with joins, group by and non-strings
>
> On Sat, Apr 25, 2009 at 4:47 PM, Peter Alvaro <pa...@gmail.com> wrote:
>> create table bar (
>> bas string--,
>> --bat double
>> )
>> ROW FORMAT DELIMITED FIELDS TERMINATED BY '\174';
>
> "bat" shouldn't be commented-out, if you want to use the provided test data.
>
>> It appears that queries with all three of (join, group by, non-string
>> datatype) cause a crash in the serde code run at the reducer:
>
> I can reproduce this issue with the trunk Hive code (+ Hadoop 0.19.1).
> Has anyone from the Hive team had a chance to investigate this?
>
> Thanks,
>
> Neil
>
>
Re: more ClassCastExceptions with joins, group by and non-strings
Posted by Prasad Chakka <pc...@facebook.com>.
I think this is a known bug. casting the non-string column should work.
See the below email thread (and HIVE-405)
http://mail-archives.apache.org/mod_mbox/hadoop-hive-user/200904.mbox/browser
________________________________
From: Neil Conway <ne...@gmail.com>
Reply-To: <hi...@hadoop.apache.org>
Date: Sat, 2 May 2009 23:15:06 -0700
To: <hi...@hadoop.apache.org>
Subject: Re: more ClassCastExceptions with joins, group by and non-strings
On Sat, Apr 25, 2009 at 4:47 PM, Peter Alvaro <pa...@gmail.com> wrote:
> create table bar (
> bas string--,
> --bat double
> )
> ROW FORMAT DELIMITED FIELDS TERMINATED BY '\174';
"bat" shouldn't be commented-out, if you want to use the provided test data.
> It appears that queries with all three of (join, group by, non-string
> datatype) cause a crash in the serde code run at the reducer:
I can reproduce this issue with the trunk Hive code (+ Hadoop 0.19.1).
Has anyone from the Hive team had a chance to investigate this?
Thanks,
Neil