CDH4.4 - 0 progress on select queries

[Keith Wright <kw...@nanigans.com>]

Hi all,

   We have happily been using hive on our CDH install for some time without issues.  We recently upgraded from CDH4.2.0 to CDH 4.4.0 and now select queries against one table fails with the syslog for the job showing what's below.  Couple of things to note:

 *   This is a partitioned table
 *   An example failure query:  select count(*) from <table> where <partition conditions>
 *   other tables do not appear to be effected however many/most of those have small files.  This table's directories have only 1 file which is around 60 GB in size
 *   Queries using OOTB hive 0.9.0 pointing at the centralized meta store work without issue
 *   It looks like an infinite loop issue where this query usually completes in ~2 min but we let it run for 45 with the job tracker showing 0% progress on mapping (perhaps related to https://issues.apache.org/jira/browse/HIVE-5235)

Have others seen this behavior?

Thanks!!!

2013-11-07 13:58:02,102 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Processing alias organic_events for file hdfs://nameservice1/events/organic/2013/11/05
2013-11-07 13:58:02,102 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7 forwarding 1 rows
2013-11-07 13:58:02,102 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows
2013-11-07 13:58:02,102 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 forwarding 1 rows
2013-11-07 13:58:02,103 INFO ExecMapper: ExecMapper: processing 1 rows: used memory = 178407888
2013-11-07 13:58:02,104 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7 forwarding 10 rows
2013-11-07 13:58:02,104 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 10 rows
2013-11-07 13:58:02,104 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 forwarding 10 rows
2013-11-07 13:58:02,104 INFO ExecMapper: ExecMapper: processing 10 rows: used memory = 178407888
….

Where processing row counts and used memory keeps increasing