You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Sergey Shelukhin <se...@hortonworks.com> on 2018/07/18 23:19:20 UTC

Review Request 67969: HIVE-20115 Acid tables should not use footer scan for analyze

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67969/
-----------------------------------------------------------

Review request for hive and Eugene Koifman.


Repository: hive-git


Description
-------

see jira


Diffs
-----

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRTableScan1.java 64f9c70f05 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ProcessAnalyzeTable.java 03cceace40 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 49709e596e 
  ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkProcessAnalyzeTable.java 28d4de7f7b 
  ql/src/test/queries/clientpositive/acid_no_buckets.q bcf9e0634b 
  ql/src/test/results/clientpositive/llap/acid_no_buckets.q.out 36a6a5d5d1 


Diff: https://reviews.apache.org/r/67969/diff/1/


Testing
-------


Thanks,

Sergey Shelukhin


Re: Review Request 67969: HIVE-20115 Acid tables should not use footer scan for analyze

Posted by Sergey Shelukhin <se...@hortonworks.com>.

> On July 19, 2018, 7:29 p.m., Eugene Koifman wrote:
> > ql/src/test/queries/clientpositive/acid_no_buckets.q
> > Lines 37 (patched)
> > <https://reviews.apache.org/r/67969/diff/1/?file=2061551#file2061551line37>
> >
> >     I don't understand this comment.  There was update/insert done (line 25) since last analyze at line 22-23.  Shouldn't analyze at 34-35 change stats?
> >     Or are they auto updated after each statement?

Yes. Updated the comment


- Sergey


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67969/#review206245
-----------------------------------------------------------


On July 18, 2018, 11:19 p.m., Sergey Shelukhin wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67969/
> -----------------------------------------------------------
> 
> (Updated July 18, 2018, 11:19 p.m.)
> 
> 
> Review request for hive and Eugene Koifman.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> see jira
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRTableScan1.java 64f9c70f05 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/ProcessAnalyzeTable.java 03cceace40 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 49709e596e 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkProcessAnalyzeTable.java 28d4de7f7b 
>   ql/src/test/queries/clientpositive/acid_no_buckets.q bcf9e0634b 
>   ql/src/test/results/clientpositive/llap/acid_no_buckets.q.out 36a6a5d5d1 
> 
> 
> Diff: https://reviews.apache.org/r/67969/diff/1/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>


Re: Review Request 67969: HIVE-20115 Acid tables should not use footer scan for analyze

Posted by Eugene Koifman <ek...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67969/#review206245
-----------------------------------------------------------




ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRTableScan1.java
Line 87 (original), 88 (patched)
<https://reviews.apache.org/r/67969/#comment289150>

    This is not introduced in this patch, but maybe this "if" should be a method since the same condition is checked in 3 places - to keep it in sync.



ql/src/test/queries/clientpositive/acid_no_buckets.q
Lines 37 (patched)
<https://reviews.apache.org/r/67969/#comment289149>

    I don't understand this comment.  There was update/insert done (line 25) since last analyze at line 22-23.  Shouldn't analyze at 34-35 change stats?
    Or are they auto updated after each statement?


- Eugene Koifman


On July 18, 2018, 4:19 p.m., Sergey Shelukhin wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67969/
> -----------------------------------------------------------
> 
> (Updated July 18, 2018, 4:19 p.m.)
> 
> 
> Review request for hive and Eugene Koifman.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> see jira
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRTableScan1.java 64f9c70f05 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/ProcessAnalyzeTable.java 03cceace40 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 49709e596e 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkProcessAnalyzeTable.java 28d4de7f7b 
>   ql/src/test/queries/clientpositive/acid_no_buckets.q bcf9e0634b 
>   ql/src/test/results/clientpositive/llap/acid_no_buckets.q.out 36a6a5d5d1 
> 
> 
> Diff: https://reviews.apache.org/r/67969/diff/1/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>


Re: Review Request 67969: HIVE-20115 Acid tables should not use footer scan for analyze

Posted by Sergey Shelukhin <se...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67969/
-----------------------------------------------------------

(Updated July 19, 2018, 8:10 p.m.)


Review request for hive and Eugene Koifman.


Repository: hive-git


Description
-------

see jira


Diffs (updated)
-----

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRTableScan1.java 64f9c70f05 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ProcessAnalyzeTable.java 03cceace40 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 49709e596e 
  ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkProcessAnalyzeTable.java 28d4de7f7b 
  ql/src/java/org/apache/hadoop/hive/ql/stats/BasicStatsNoJobTask.java 3128ee8200 
  ql/src/test/queries/clientpositive/acid_no_buckets.q bcf9e0634b 
  ql/src/test/results/clientpositive/acid_table_stats.q.out 841a5a42ae 
  ql/src/test/results/clientpositive/llap/acid_no_buckets.q.out 36a6a5d5d1 
  ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_4.q.out eda3985d0a 
  ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_5.q.out 99832ff847 


Diff: https://reviews.apache.org/r/67969/diff/3/

Changes: https://reviews.apache.org/r/67969/diff/2-3/


Testing
-------


Thanks,

Sergey Shelukhin


Re: Review Request 67969: HIVE-20115 Acid tables should not use footer scan for analyze

Posted by Sergey Shelukhin <se...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67969/
-----------------------------------------------------------

(Updated July 19, 2018, 8:04 p.m.)


Review request for hive and Eugene Koifman.


Repository: hive-git


Description
-------

see jira


Diffs (updated)
-----

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRTableScan1.java 64f9c70f05 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ProcessAnalyzeTable.java 03cceace40 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 49709e596e 
  ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkProcessAnalyzeTable.java 28d4de7f7b 
  ql/src/java/org/apache/hadoop/hive/ql/stats/BasicStatsNoJobTask.java 3128ee8200 
  ql/src/test/queries/clientpositive/acid_no_buckets.q bcf9e0634b 
  ql/src/test/results/clientpositive/acid_table_stats.q.out 841a5a42ae 
  ql/src/test/results/clientpositive/llap/acid_no_buckets.q.out 36a6a5d5d1 
  ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_4.q.out eda3985d0a 
  ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_5.q.out 99832ff847 


Diff: https://reviews.apache.org/r/67969/diff/2/

Changes: https://reviews.apache.org/r/67969/diff/1-2/


Testing
-------


Thanks,

Sergey Shelukhin