You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by XIAOBING ZHOU <xz...@gmail.com> on 2014/10/17 21:10:08 UTC

Re: Review Request 26887: StatsNoJobTask doesn't close RecordReader, FSDataInputStream of which keeps open to prevent stale data clean

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26887/
-----------------------------------------------------------

(Updated Oct. 17, 2014, 7:10 p.m.)


Review request for hive.


Repository: hive-git


Description
-------

run the test

mvn -Phadoop-2  test -Dtest=TestCliDriver -Dqfile=alter_merge_stats_orc.q

to reproduce it. Simply, this query does three data loads which generates three base orc files.
ANALYZE TABLE...COMPUTE STATISTICS NOSCAN will execute StatsNoJobTask to get stats, where file handle is held so as not able to clean base file. As a result, after running ALTER TABLE..CONCATENATE, follow-up queries go to stale base file and merged file.


Diffs (updated)
-----

  ql/src/java/org/apache/hadoop/hive/ql/exec/StatsNoJobTask.java 07efb8f 

Diff: https://reviews.apache.org/r/26887/diff/


Testing
-------

passed alter_merge_stats_orc.q


Thanks,

XIAOBING ZHOU