You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Igor Tatarinov <ig...@decide.com> on 2012/07/26 20:40:50 UTC

Nested SELECT DISTINCT runs out of memory

I have this query that consistently fails with out-of-memory errors. I know
it can be re-written without a nested subquery (using count distinct) and
then it runs fine.

Why does this query fail though? Is this is a known Hive issue? The
subquery returns 5M records.

SELECT x, COUNT(1) AS num
FROM (SELECT DISTINCT x, y) t
GROUP BY x;

I am using EMR Hive 0.8.1