You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by "Lu, Wei" <wl...@microstrategy.com> on 2012/01/11 08:13:57 UTC
RE:
BTW, I am using hive 0.7
From: Lu, Wei
Sent: Wednesday, January 11, 2012 3:13 PM
To: 'user@hive.apache.org'
Subject:
Hi,
I am using ThriftHive.Client to access a pretty large table.
SQL Statement:
select a11.asin asin,
max(a11.title) title,
a11.salesrank salesrank,
a11.category category,
avg(a11.avg_rating) WJXBFS1,
sum(a11.total_num_reviews) WJXBFS2,
sum(a11.num_subcategories) WJXBFS3
from table_details a11
group by a11.asin,
a11.salesrank,
a11.category.
The statement will select a pretty large result set (1,000,000+ rows). When I use ThriftHive.Client fetchAll() to get all the row strings, an exception returns like below:
Exception in thread "main" org.apache.thrift.TApplicationException: Internal error processing fetchAll
at org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
at org.apache.hadoop.hive.service.ThriftHive$Client.recv_fetchAll(ThriftHive.java:224)
at org.apache.hadoop.hive.service.ThriftHive$Client.fetchAll(ThriftHive.java:208)
... ...
Why does that happen? How could I deal with it? It seem that incremental fetch is not supported by hive.
Regards,
Wei
RE:
Posted by Steven Wong <sw...@netflix.com>.
Try fetchOne or fetchN.
From: Lu, Wei [mailto:wlu@microstrategy.com]
Sent: Tuesday, January 10, 2012 11:14 PM
To: Lu, Wei; user@hive.apache.org
Subject: RE:
BTW, I am using hive 0.7
From: Lu, Wei
Sent: Wednesday, January 11, 2012 3:13 PM
To: 'user@hive.apache.org'
Subject:
Hi,
I am using ThriftHive.Client to access a pretty large table.
SQL Statement:
select a11.asin asin,
max(a11.title) title,
a11.salesrank salesrank,
a11.category category,
avg(a11.avg_rating) WJXBFS1,
sum(a11.total_num_reviews) WJXBFS2,
sum(a11.num_subcategories) WJXBFS3
from table_details a11
group by a11.asin,
a11.salesrank,
a11.category.
The statement will select a pretty large result set (1,000,000+ rows). When I use ThriftHive.Client fetchAll() to get all the row strings, an exception returns like below:
Exception in thread "main" org.apache.thrift.TApplicationException: Internal error processing fetchAll
at org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
at org.apache.hadoop.hive.service.ThriftHive$Client.recv_fetchAll(ThriftHive.java:224)
at org.apache.hadoop.hive.service.ThriftHive$Client.fetchAll(ThriftHive.java:208)
... ...
Why does that happen? How could I deal with it? It seem that incremental fetch is not supported by hive.
Regards,
Wei