You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Keith <hi...@gmail.com> on 2014/04/29 17:18:01 UTC
Bug in Hive Partition windowing functions?
Hi,
we have an issue with windowing function query never completed when
running against the large dataset > 25,000 rows. That is the reducer
(only one) never exit and it appears stuck in an infinite loop.
I looked at the Reducer counter and it never changes over the 6 hours when it
gets stuck in a loop.
When the data set is small < 25K rows, it runs fine.
Is there any work around this issue? We tested
against Hive 0.11/0.12/0.13 and the same result is the same.
create table window_function_fail
as
select a.*,
sum(case when bprice is not null then 1 else 0 end) over (partition by
date,name order by otime,bprice,aprice desc ROWS BETWEEN UNBOUNDED
PRECEDING AND CURRENT ROW) bidpid
from
large_table a;
create table large_table(
date string ,
name string ,
stime string ,
bprice decimal ,
aprice decimal ,
otime double
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' stored as textfile;
Thanks in advance.