You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Vijay <te...@gmail.com> on 2009/10/13 01:46:13 UTC
Problem of queries hanging with DynamicSerDe
Hi,
I have this very strange problem of queries hanging when I use the
org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe with the
serialization format
org.apache.hadoop.hive.serde2.thrift.TCTLSeparatedProtocol. I'm basically
loading a very standard apache log file (pretty small) using this example
from one of the examples on the net.
CREATE TABLE apachelog (
ipaddress STRING, identd STRING, user STRING,finishtime STRING,
requestline string, returncode INT, size INT)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe'
WITH SERDEPROPERTIES (
'serialization.format'='org.apache.hadoop.hive.serde2.thrift.TCTLSeparatedProtocol',
'quote.delim'='("|\\[|\\])',
'field.delim'=' ',
'serialization.null.format'='-')
STORED AS TEXTFILE;
After that, I simply load a small log file. Everything is good except, when
I run any simple query like SELECT count(1) from apachelog, the job "hangs."
It basically sits there forever and the job gets killed after 10 minutes. I
must be missing something very basic.
I can get the same setup working well using the RegexSerDe but I'd rather
use the DynamicSerDe like above. I'm using hive-0.4 branch but I'm pretty
sure I saw the same behavior using trunk as well. There is not a thing I can
find in the /tmp/<>/hive.log file.
Thanks for your help!
Vijay
Re: Problem of queries hanging with DynamicSerDe
Posted by Vijay <te...@gmail.com>.
Thanks for pointing that out Zheng! I thought DynamicSerDe might be more
efficient than RegexSerDe. RegexSerDe ofcourse, works great for me. The
example I followed was from a blog outside the Hive wiki so it must be
pretty old.
On Mon, Oct 12, 2009 at 6:49 PM, Zheng Shao <zs...@gmail.com> wrote:
> Hi Vijay,
>
> DynamicSerDe is deprecated.
>
> Please use the following SerDe instead:
> https://issues.apache.org/jira/browse/HIVE-662
>
> Can you point us to where you see this example? We should update it with
> RegexSerDe.
>
> Zheng
>
>
> On Mon, Oct 12, 2009 at 4:46 PM, Vijay <te...@gmail.com> wrote:
>
>> Hi,
>>
>> I have this very strange problem of queries hanging when I use the
>> org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe with the
>> serialization format
>> org.apache.hadoop.hive.serde2.thrift.TCTLSeparatedProtocol. I'm basically
>> loading a very standard apache log file (pretty small) using this example
>> from one of the examples on the net.
>>
>> CREATE TABLE apachelog (
>> ipaddress STRING, identd STRING, user STRING,finishtime STRING,
>> requestline string, returncode INT, size INT)
>> ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe'
>> WITH SERDEPROPERTIES (
>>
>> 'serialization.format'='org.apache.hadoop.hive.serde2.thrift.TCTLSeparatedProtocol',
>> 'quote.delim'='("|\\[|\\])',
>> 'field.delim'=' ',
>> 'serialization.null.format'='-')
>> STORED AS TEXTFILE;
>>
>> After that, I simply load a small log file. Everything is good except,
>> when I run any simple query like SELECT count(1) from apachelog, the job
>> "hangs." It basically sits there forever and the job gets killed after 10
>> minutes. I must be missing something very basic.
>>
>> I can get the same setup working well using the RegexSerDe but I'd rather
>> use the DynamicSerDe like above. I'm using hive-0.4 branch but I'm pretty
>> sure I saw the same behavior using trunk as well. There is not a thing I can
>> find in the /tmp/<>/hive.log file.
>>
>> Thanks for your help!
>> Vijay
>>
>
>
>
> --
> Yours,
> Zheng
>
Re: 回复: How can I make hive available for Hadoop 0.20.1
Posted by Arijit Mukherjee <ar...@gmail.com>.
Hi
I've checked out the 0.4.0 release, and tried to build it with "ant
package" - but it tries to download different hadoop versions and
ultimately fails with this type of message:
[ivy:retrieve] :::: WARNINGS
[ivy:retrieve] [FAILED ]
hadoop#core;0.17.2.1!hadoop.tar.gz(source): Downloaded file size
doesn't match expected Content Length for
http://archive.apache.org/dist/hadoop/core/hadoop-0.17.2.1/hadoop-0.17.2.1.tar.gz.
Please retry. (231662ms)
[ivy:retrieve] [FAILED ]
hadoop#core;0.17.2.1!hadoop.tar.gz(source): Downloaded file size
doesn't match expected Content Length for
http://archive.apache.org/dist/hadoop/core/hadoop-0.17.2.1/hadoop-0.17.2.1.tar.gz.
Please retry. (231662ms)
[ivy:retrieve] ==== hadoop-resolver: tried
I'm connected to the net, and the downloads are working properly
(because I was able to build the cloudera version of Hadoop). My
question is - why is the build process tries to download hadoop if
I've already got it? Should HADOOP_HOME be used somehow in the build
process?
Regards
Arijit
2009/10/13 杨卓荦 <cl...@yahoo.com.cn>
>
> Thank you, Zheng, You are a nice guy.
> 谢谢邵铮大哥,您真是好人。其实我在facebook上加了您了。
>
> ________________________________
> 发件人: Zheng Shao <zs...@gmail.com>
> 收件人: hive-user@hadoop.apache.org; clarkyzl-hive@yahoo.com.cn
> 已发送: 周二, 10 13, 2009 12:01:03 下午
> 主 题: Re: How can I make hive available for Hadoop 0.20.1
>
> Hi Clark,
>
> hive release 0.4.0 is just out. It's compatible with hadoop 0.17 to 0.20
>
> Please download it from https://svn.apache.org/viewvc/hadoop/hive/tags/release-0.4.0/
>
> Zheng
>
> On Mon, Oct 12, 2009 at 7:46 PM, 杨卓荦 <cl...@yahoo.com.cn> wrote:
>>
>> I access to the
>> http://www.apache.org/dyn/closer.cgi/hadoop/hive/
>> only Hive 0.3.0 available it works with Hadoop 0.19.0
>>
>> but mine is
>>
>> $ hadoop version
>> Hadoop 0.20.2-dev
>> Subversion -r
>> Compiled by root on Fri Oct 9 11:25:48 CST 2009
>>
>> how can I make my hive work with it?
>>
>> ________________________________
>> 好玩贺卡等你发,邮箱贺卡全新上线!
>
>
> --
> Yours,
> Zheng
>
> ________________________________
> 好玩贺卡等你发,邮箱贺卡全新上线!
--
"And when the night is cloudy,
There is still a light that shines on me,
Shine on until tomorrow, let it be."
回复: How can I make hive available for Hadoop 0.20.1
Posted by 杨卓荦 <cl...@yahoo.com.cn>.
Thank you, Zheng, You are a nice guy.
谢谢邵铮大哥,您真是好人。其实我在facebook上加了您了。
________________________________
发件人: Zheng Shao <zs...@gmail.com>
收件人: hive-user@hadoop.apache.org; clarkyzl-hive@yahoo.com.cn
已发送: 周二, 10 13, 2009 12:01:03 下午
主 题: Re: How can I make hive available for Hadoop 0.20.1
Hi Clark,
hive release 0.4.0 is just out. It's compatible with hadoop 0.17 to 0.20
Please download it from https://svn.apache.org/viewvc/hadoop/hive/tags/release-0.4.0/
Zheng
On Mon, Oct 12, 2009 at 7:46 PM, 杨卓荦 <cl...@yahoo.com.cn> wrote:
>I access to the
> http://www.apache.org/dyn/closer.cgi/hadoop/hive/
>only Hive 0.3.0 available it works with Hadoop 0.19.0
>
>but mine is
>
>$ hadoop version
>Hadoop 0.20.2-dev
>Subversion -r
>Compiled by root on Fri Oct 9 11:25:48 CST 2009
>
>how can I make my hive work with it?
>
>________________________________
好玩贺卡等你发,邮箱贺卡全新上线!
--
Yours,
Zheng
___________________________________________________________
好玩贺卡等你发,邮箱贺卡全新上线!
http://card.mail.cn.yahoo.com/
RE: How can I make hive available for Hadoop 0.20.1
Posted by Ashish Thusoo <at...@facebook.com>.
It almost seems like the file is getting corrupted in transfer. You could try another mirror for downloads. In order to change the hadoop.mirror parameter in build.properties
Ashish
________________________________
From: Arijit Mukherjee [mailto:arijit72@gmail.com]
Sent: Wednesday, October 14, 2009 2:36 AM
To: hive-user@hadoop.apache.org
Subject: Re: How can I make hive available for Hadoop 0.20.1
Thank you Zheng. But what could be the reason I'm getting all those downloaded file size errors while building? Other downloads are working fine - except this one while building Hive. Unless I can resolve that, I'll be stuck with the Hive distribution within Hadoop 0.19.2 (contrib folder).
Regards
Arijit
2009/10/13 Edward Capriolo <ed...@gmail.com>>
2009/10/13 Zheng Shao <zs...@gmail.com>>
I think 0.4.0-rc2 is the same as 0.4.0-release.
Ashish can probably confirm that.
Zheng
2009/10/13 Ryan LeCompte <le...@gmail.com>>
Hi Zheng,
Can you please list changes between 0.4.0-rc2 and 0.4.0-release?
Thanks!
Ryan
2009/10/13 Zheng Shao <zs...@gmail.com>>
Hi Clark,
hive release 0.4.0 is just out. It's compatible with hadoop 0.17 to 0.20
Please download it from https://svn.apache.org/viewvc/hadoop/hive/tags/release-0.4.0/
Zheng
On Mon, Oct 12, 2009 at 7:46 PM, 杨卓荦 <cl...@yahoo.com.cn>> wrote:
I access to the
http://www.apache.org/dyn/closer.cgi/hadoop/hive/
only Hive 0.3.0 available it works with Hadoop 0.19.0
but mine is
$ hadoop version
Hadoop 0.20.2-dev
Subversion -r
Compiled by root on Fri Oct 9 11:25:48 CST 2009
how can I make my hive work with it?
________________________________
好玩贺卡等你发,邮箱贺卡全新上线!<http://cn.rd.yahoo.com/mail_cn/tagline/card/*http://card.mail.cn.yahoo.com/>
--
Yours,
Zheng
--
Yours,
Zheng
My
question is - why is the build process tries to download hadoop if
I've already got it? Should HADOOP_HOME be used somehow in the build
process?
Thanks to the shims Todd cooked up one branch of hive supports several hadoop branches. 17-20. hive downloads those versions so it can build the shims against them.
--
"And when the night is cloudy,
There is still a light that shines on me,
Shine on until tomorrow, let it be."
Re: How can I make hive available for Hadoop 0.20.1
Posted by Arijit Mukherjee <ar...@gmail.com>.
Thank you Zheng. But what could be the reason I'm getting all those
downloaded file size errors while building? Other downloads are working fine
- except this one while building Hive. Unless I can resolve that, I'll be
stuck with the Hive distribution within Hadoop 0.19.2 (contrib folder).
Regards
Arijit
2009/10/13 Edward Capriolo <ed...@gmail.com>
>
> 2009/10/13 Zheng Shao <zs...@gmail.com>
>
>> I think 0.4.0-rc2 is the same as 0.4.0-release.
>> Ashish can probably confirm that.
>>
>> Zheng
>>
>> 2009/10/13 Ryan LeCompte <le...@gmail.com>
>>
>> Hi Zheng,
>>>
>>> Can you please list changes between 0.4.0-rc2 and 0.4.0-release?
>>>
>>> Thanks!
>>>
>>> Ryan
>>>
>>> 2009/10/13 Zheng Shao <zs...@gmail.com>
>>>
>>> Hi Clark,
>>>>
>>>> hive release 0.4.0 is just out. It's compatible with hadoop 0.17 to
>>>> 0.20
>>>>
>>>> Please download it from
>>>> https://svn.apache.org/viewvc/hadoop/hive/tags/release-0.4.0/
>>>>
>>>> Zheng
>>>>
>>>>
>>>> On Mon, Oct 12, 2009 at 7:46 PM, 杨卓荦 <cl...@yahoo.com.cn>wrote:
>>>>
>>>>> I access to the
>>>>> http://www.apache.org/dyn/closer.cgi/hadoop/hive/
>>>>> only Hive 0.3.0 available it works with Hadoop 0.19.0
>>>>>
>>>>> but mine is
>>>>>
>>>>> $ hadoop version
>>>>> Hadoop 0.20.2-dev
>>>>> Subversion -r
>>>>> Compiled by root on Fri Oct 9 11:25:48 CST 2009
>>>>>
>>>>> how can I make my hive work with it?
>>>>>
>>>>> ------------------------------
>>>>> 好玩贺卡等你发,邮箱贺卡全新上线!<http://cn.rd.yahoo.com/mail_cn/tagline/card/*http://card.mail.cn.yahoo.com/>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Yours,
>>>> Zheng
>>>>
>>>
>>>
>>
>>
>> --
>> Yours,
>> Zheng
>>
>
>
>
>
> My
> question is - why is the build process tries to download hadoop if
> I've already got it? Should HADOOP_HOME be used somehow in the build
> process?
>
> Thanks to the shims Todd cooked up one branch of hive supports several
> hadoop branches. 17-20. hive downloads those versions so it can build the
> shims against them.
>
--
"And when the night is cloudy,
There is still a light that shines on me,
Shine on until tomorrow, let it be."
回复: How can I make hive available for Hadoop 0.20.1
Posted by "Clark Yang (杨卓荦)" <cl...@yahoo.com.cn>.
Ashish
When can I download it from mirrors?
________________________________
发件人: Ashish Thusoo <at...@facebook.com>
收件人: "hive-user@hadoop.apache.org" <hi...@hadoop.apache.org>
已发送: 周三, 10 14, 2009 1:54:32 上午
主 题: RE: How can I make hive available for Hadoop 0.20.1
It is the same. I have pushed out the release today morning and it should be available by tomorrow. Will send out a more formal announcement later once it has been pushed out to the apache mirrors.
Ashish
________________________________
From: Edward Capriolo [mailto:edlinuxguru@gmail.com]
Sent: Tuesday, October 13, 2009 10:39 AM
To: hive-user@hadoop.apache.org
Subject: Re: How can I make hive available for Hadoop 0.20.1
2009/10/13 Zheng Shao <zs...@gmail.com>
I think 0.4.0-rc2 is the same as 0.4.0-release.
>Ashish can probably confirm that.
>
>Zheng
>
>
>2009/10/13 Ryan LeCompte <le...@gmail.com>
>
>
>Hi Zheng,
>>
>>Can you please list changes between 0.4.0-rc2 and 0.4.0-release?
>>
>>Thanks!
>>
>>Ryan
>>
>>
>>2009/10/13 Zheng Shao <zs...@gmail.com>
>>
>>
>>Hi Clark,
>>>
>>>hive release 0.4.0 is just out. It's compatible with hadoop 0.17 to 0.20
>>>
>>>Please download it from https://svn.apache.org/viewvc/hadoop/hive/tags/release-0.4.0/
>>>
>>>Zheng
>>>
>>>
>>>
>>>On Mon, Oct 12, 2009 at 7:46 PM, 杨卓荦 <cl...@yahoo.com.cn> wrote:
>>>
>>>I access to the
>>>> http://www.apache.org/dyn/closer.cgi/hadoop/hive/
>>>>only Hive 0.3.0 available it works with Hadoop 0.19.0
>>>>
>>>>but mine is
>>>>
>>>>$ hadoop version
>>>>Hadoop 0.20.2-dev
>>>>Subversion -r
>>>>Compiled by root on Fri Oct 9 11:25:48 CST 2009
>>>>
>>>>how can I make my hive work with it?
>>>>
>>>>________________________________
好玩贺卡等你发,邮箱贺卡全新上线!
>>>
>>>
>>>--
>>>Yours,
>>>Zheng
>>>
>>
>
>
>--
>Yours,
>Zheng
>
My
question is - why is the build process tries to download hadoop if
I've already got it? Should HADOOP_HOME be used somehow in the build
process?
Thanks to the shims Todd cooked up one branch of hive supports several hadoop branches. 17-20. hive downloads those versions so it can build the shims against them.
___________________________________________________________
好玩贺卡等你发,邮箱贺卡全新上线!
http://card.mail.cn.yahoo.com/
RE: How can I make hive available for Hadoop 0.20.1
Posted by Ashish Thusoo <at...@facebook.com>.
It is the same. I have pushed out the release today morning and it should be available by tomorrow. Will send out a more formal announcement later once it has been pushed out to the apache mirrors.
Ashish
________________________________
From: Edward Capriolo [mailto:edlinuxguru@gmail.com]
Sent: Tuesday, October 13, 2009 10:39 AM
To: hive-user@hadoop.apache.org
Subject: Re: How can I make hive available for Hadoop 0.20.1
2009/10/13 Zheng Shao <zs...@gmail.com>>
I think 0.4.0-rc2 is the same as 0.4.0-release.
Ashish can probably confirm that.
Zheng
2009/10/13 Ryan LeCompte <le...@gmail.com>>
Hi Zheng,
Can you please list changes between 0.4.0-rc2 and 0.4.0-release?
Thanks!
Ryan
2009/10/13 Zheng Shao <zs...@gmail.com>>
Hi Clark,
hive release 0.4.0 is just out. It's compatible with hadoop 0.17 to 0.20
Please download it from https://svn.apache.org/viewvc/hadoop/hive/tags/release-0.4.0/
Zheng
On Mon, Oct 12, 2009 at 7:46 PM, 杨卓荦 <cl...@yahoo.com.cn>> wrote:
I access to the
http://www.apache.org/dyn/closer.cgi/hadoop/hive/
only Hive 0.3.0 available it works with Hadoop 0.19.0
but mine is
$ hadoop version
Hadoop 0.20.2-dev
Subversion -r
Compiled by root on Fri Oct 9 11:25:48 CST 2009
how can I make my hive work with it?
________________________________
好玩贺卡等你发,邮箱贺卡全新上线!<http://cn.rd.yahoo.com/mail_cn/tagline/card/*http://card.mail.cn.yahoo.com/>
--
Yours,
Zheng
--
Yours,
Zheng
My
question is - why is the build process tries to download hadoop if
I've already got it? Should HADOOP_HOME be used somehow in the build
process?
Thanks to the shims Todd cooked up one branch of hive supports several hadoop branches. 17-20. hive downloads those versions so it can build the shims against them.
Re: How can I make hive available for Hadoop 0.20.1
Posted by Edward Capriolo <ed...@gmail.com>.
2009/10/13 Zheng Shao <zs...@gmail.com>
> I think 0.4.0-rc2 is the same as 0.4.0-release.
> Ashish can probably confirm that.
>
> Zheng
>
> 2009/10/13 Ryan LeCompte <le...@gmail.com>
>
> Hi Zheng,
>>
>> Can you please list changes between 0.4.0-rc2 and 0.4.0-release?
>>
>> Thanks!
>>
>> Ryan
>>
>> 2009/10/13 Zheng Shao <zs...@gmail.com>
>>
>> Hi Clark,
>>>
>>> hive release 0.4.0 is just out. It's compatible with hadoop 0.17 to 0.20
>>>
>>> Please download it from
>>> https://svn.apache.org/viewvc/hadoop/hive/tags/release-0.4.0/
>>>
>>> Zheng
>>>
>>>
>>> On Mon, Oct 12, 2009 at 7:46 PM, 杨卓荦 <cl...@yahoo.com.cn> wrote:
>>>
>>>> I access to the
>>>> http://www.apache.org/dyn/closer.cgi/hadoop/hive/
>>>> only Hive 0.3.0 available it works with Hadoop 0.19.0
>>>>
>>>> but mine is
>>>>
>>>> $ hadoop version
>>>> Hadoop 0.20.2-dev
>>>> Subversion -r
>>>> Compiled by root on Fri Oct 9 11:25:48 CST 2009
>>>>
>>>> how can I make my hive work with it?
>>>>
>>>> ------------------------------
>>>> 好玩贺卡等你发,邮箱贺卡全新上线!<http://cn.rd.yahoo.com/mail_cn/tagline/card/*http://card.mail.cn.yahoo.com/>
>>>
>>>
>>>
>>>
>>> --
>>> Yours,
>>> Zheng
>>>
>>
>>
>
>
> --
> Yours,
> Zheng
>
My
question is - why is the build process tries to download hadoop if
I've already got it? Should HADOOP_HOME be used somehow in the build
process?
Thanks to the shims Todd cooked up one branch of hive supports several
hadoop branches. 17-20. hive downloads those versions so it can build the
shims against them.
Re: How can I make hive available for Hadoop 0.20.1
Posted by Zheng Shao <zs...@gmail.com>.
I think 0.4.0-rc2 is the same as 0.4.0-release.
Ashish can probably confirm that.
Zheng
2009/10/13 Ryan LeCompte <le...@gmail.com>
> Hi Zheng,
>
> Can you please list changes between 0.4.0-rc2 and 0.4.0-release?
>
> Thanks!
>
> Ryan
>
> 2009/10/13 Zheng Shao <zs...@gmail.com>
>
> Hi Clark,
>>
>> hive release 0.4.0 is just out. It's compatible with hadoop 0.17 to 0.20
>>
>> Please download it from
>> https://svn.apache.org/viewvc/hadoop/hive/tags/release-0.4.0/
>>
>> Zheng
>>
>>
>> On Mon, Oct 12, 2009 at 7:46 PM, 杨卓荦 <cl...@yahoo.com.cn> wrote:
>>
>>> I access to the
>>> http://www.apache.org/dyn/closer.cgi/hadoop/hive/
>>> only Hive 0.3.0 available it works with Hadoop 0.19.0
>>>
>>> but mine is
>>>
>>> $ hadoop version
>>> Hadoop 0.20.2-dev
>>> Subversion -r
>>> Compiled by root on Fri Oct 9 11:25:48 CST 2009
>>>
>>> how can I make my hive work with it?
>>>
>>> ------------------------------
>>> 好玩贺卡等你发,邮箱贺卡全新上线!<http://cn.rd.yahoo.com/mail_cn/tagline/card/*http://card.mail.cn.yahoo.com/>
>>
>>
>>
>>
>> --
>> Yours,
>> Zheng
>>
>
>
--
Yours,
Zheng
Re: How can I make hive available for Hadoop 0.20.1
Posted by Ryan LeCompte <le...@gmail.com>.
Hi Zheng,
Can you please list changes between 0.4.0-rc2 and 0.4.0-release?
Thanks!
Ryan
2009/10/13 Zheng Shao <zs...@gmail.com>
> Hi Clark,
>
> hive release 0.4.0 is just out. It's compatible with hadoop 0.17 to 0.20
>
> Please download it from
> https://svn.apache.org/viewvc/hadoop/hive/tags/release-0.4.0/
>
> Zheng
>
>
> On Mon, Oct 12, 2009 at 7:46 PM, 杨卓荦 <cl...@yahoo.com.cn> wrote:
>
>> I access to the
>> http://www.apache.org/dyn/closer.cgi/hadoop/hive/
>> only Hive 0.3.0 available it works with Hadoop 0.19.0
>>
>> but mine is
>>
>> $ hadoop version
>> Hadoop 0.20.2-dev
>> Subversion -r
>> Compiled by root on Fri Oct 9 11:25:48 CST 2009
>>
>> how can I make my hive work with it?
>>
>> ------------------------------
>> 好玩贺卡等你发,邮箱贺卡全新上线!<http://cn.rd.yahoo.com/mail_cn/tagline/card/*http://card.mail.cn.yahoo.com/>
>
>
>
>
> --
> Yours,
> Zheng
>
Re: How can I make hive available for Hadoop 0.20.1
Posted by Zheng Shao <zs...@gmail.com>.
Hi Clark,
hive release 0.4.0 is just out. It's compatible with hadoop 0.17 to 0.20
Please download it from
https://svn.apache.org/viewvc/hadoop/hive/tags/release-0.4.0/
Zheng
On Mon, Oct 12, 2009 at 7:46 PM, 杨卓荦 <cl...@yahoo.com.cn> wrote:
> I access to the
> http://www.apache.org/dyn/closer.cgi/hadoop/hive/
> only Hive 0.3.0 available it works with Hadoop 0.19.0
>
> but mine is
>
> $ hadoop version
> Hadoop 0.20.2-dev
> Subversion -r
> Compiled by root on Fri Oct 9 11:25:48 CST 2009
>
> how can I make my hive work with it?
>
> ------------------------------
> 好玩贺卡等你发,邮箱贺卡全新上线!<http://cn.rd.yahoo.com/mail_cn/tagline/card/*http://card.mail.cn.yahoo.com/>
--
Yours,
Zheng
How can I make hive available for Hadoop 0.20.1
Posted by 杨卓荦 <cl...@yahoo.com.cn>.
I access to the
http://www.apache.org/dyn/closer.cgi/hadoop/hive/
only Hive 0.3.0 available it works with Hadoop 0.19.0
but mine is
$ hadoop version
Hadoop 0.20.2-dev
Subversion -r
Compiled by root on Fri Oct 9 11:25:48 CST 2009
how can I make my hive work with it?
___________________________________________________________
好玩贺卡等你发,邮箱贺卡全新上线!
http://card.mail.cn.yahoo.com/
Re: Problem of queries hanging with DynamicSerDe
Posted by Zheng Shao <zs...@gmail.com>.
Hi Vijay,
DynamicSerDe is deprecated.
Please use the following SerDe instead:
https://issues.apache.org/jira/browse/HIVE-662
Can you point us to where you see this example? We should update it with
RegexSerDe.
Zheng
On Mon, Oct 12, 2009 at 4:46 PM, Vijay <te...@gmail.com> wrote:
> Hi,
>
> I have this very strange problem of queries hanging when I use the
> org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe with the
> serialization format
> org.apache.hadoop.hive.serde2.thrift.TCTLSeparatedProtocol. I'm basically
> loading a very standard apache log file (pretty small) using this example
> from one of the examples on the net.
>
> CREATE TABLE apachelog (
> ipaddress STRING, identd STRING, user STRING,finishtime STRING,
> requestline string, returncode INT, size INT)
> ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe'
> WITH SERDEPROPERTIES (
>
> 'serialization.format'='org.apache.hadoop.hive.serde2.thrift.TCTLSeparatedProtocol',
> 'quote.delim'='("|\\[|\\])',
> 'field.delim'=' ',
> 'serialization.null.format'='-')
> STORED AS TEXTFILE;
>
> After that, I simply load a small log file. Everything is good except, when
> I run any simple query like SELECT count(1) from apachelog, the job "hangs."
> It basically sits there forever and the job gets killed after 10 minutes. I
> must be missing something very basic.
>
> I can get the same setup working well using the RegexSerDe but I'd rather
> use the DynamicSerDe like above. I'm using hive-0.4 branch but I'm pretty
> sure I saw the same behavior using trunk as well. There is not a thing I can
> find in the /tmp/<>/hive.log file.
>
> Thanks for your help!
> Vijay
>
--
Yours,
Zheng