You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Alexander Kolbasov <ak...@gmail.com> on 2018/03/04 09:11:13 UTC
Re: Review Request 65745: HIVE-18743 CREATE TABLE on S3 data can be
extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65745/
-----------------------------------------------------------
(Updated March 4, 2018, 9:11 a.m.)
Review request for hive, Andrew Sherman, Janaki Lahorani, Zoltan Haindrich, Sahil Takiar, Thejas Nair, and Vihang Karajgaonkar.
Changes
-------
Added unit test.
Summary (updated)
-----------------
HIVE-18743 CREATE TABLE on S3 data can be extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
Bugs: HIVE-18743
https://issues.apache.org/jira/browse/HIVE-18743
Repository: hive-git
Description (updated)
-------
HIVE-18743 CREATE TABLE on S3 data can be extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
Diffs (updated)
-----
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 89354a2d34249903a9ff13c4ed913a68de93057e
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java ac71d0882f985a2b475eb197a4852cc943a96a1f
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreUtils.java 50f873a013a9aa3cea0a2af8146484b9387c08f2
standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/utils/TestMetaStoreUtils.java d6c13d3f2a4dc54dab978f891b51d623a4d29762
Diff: https://reviews.apache.org/r/65745/diff/4/
Changes: https://reviews.apache.org/r/65745/diff/3-4/
Testing
-------
Thanks,
Alexander Kolbasov
Re: Review Request 65745: HIVE-18743: CREATE TABLE on S3 data can be
extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
Posted by Alexander Kolbasov <ak...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65745/
-----------------------------------------------------------
(Updated April 19, 2018, 3:51 p.m.)
Review request for hive, Andrew Sherman, Janaki Lahorani, Zoltan Haindrich, Sahil Takiar, Thejas Nair, and Vihang Karajgaonkar.
Changes
-------
Merged with latest master.
Summary (updated)
-----------------
HIVE-18743: CREATE TABLE on S3 data can be extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
Bugs: HIVE-18743
https://issues.apache.org/jira/browse/HIVE-18743
Repository: hive-git
Description (updated)
-------
HIVE-18743: CREATE TABLE on S3 data can be extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
Diffs (updated)
-----
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 60bed9841f65fd6ef74a14be3f2723c1825c7adc
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java ae9ec5cad812d49ee30ebb52e0dba5c0325ca78e
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreUtils.java d022bc0343901a588722b49d476a5eb6ac1f8104
standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/utils/TestMetaStoreUtils.java d6c13d3f2a4dc54dab978f891b51d623a4d29762
Diff: https://reviews.apache.org/r/65745/diff/8/
Changes: https://reviews.apache.org/r/65745/diff/7-8/
Testing
-------
Added both positive unit test verifying that stats are updated and negative test verifying that stats are not updated when they shouldn't be.
Thanks,
Alexander Kolbasov
Re: Review Request 65745: HIVE-18743 CREATE TABLE on S3 data can be
extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
Posted by Alexander Kolbasov <ak...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65745/
-----------------------------------------------------------
(Updated March 4, 2018, 7:08 p.m.)
Review request for hive, Andrew Sherman, Janaki Lahorani, Zoltan Haindrich, Sahil Takiar, Thejas Nair, and Vihang Karajgaonkar.
Changes
-------
Added test for the case where environment context has STATS_GENERATED set.
Bugs: HIVE-18743
https://issues.apache.org/jira/browse/HIVE-18743
Repository: hive-git
Description
-------
HIVE-18743 CREATE TABLE on S3 data can be extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
Diffs (updated)
-----
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 89354a2d34249903a9ff13c4ed913a68de93057e
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java ac71d0882f985a2b475eb197a4852cc943a96a1f
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreUtils.java 50f873a013a9aa3cea0a2af8146484b9387c08f2
standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/utils/TestMetaStoreUtils.java d6c13d3f2a4dc54dab978f891b51d623a4d29762
Diff: https://reviews.apache.org/r/65745/diff/7/
Changes: https://reviews.apache.org/r/65745/diff/6-7/
Testing
-------
Added both positive unit test verifying that stats are updated and negative test verifying that stats are not updated when they shouldn't be.
Thanks,
Alexander Kolbasov
Re: Review Request 65745: HIVE-18743 CREATE TABLE on S3 data can be
extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
Posted by Alexander Kolbasov <ak...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65745/
-----------------------------------------------------------
(Updated March 4, 2018, 6:14 p.m.)
Review request for hive, Andrew Sherman, Janaki Lahorani, Zoltan Haindrich, Sahil Takiar, Thejas Nair, and Vihang Karajgaonkar.
Changes
-------
- Simplified tests
- Simplified tests for trimMapNulls as well
Bugs: HIVE-18743
https://issues.apache.org/jira/browse/HIVE-18743
Repository: hive-git
Description
-------
HIVE-18743 CREATE TABLE on S3 data can be extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
Diffs (updated)
-----
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 89354a2d34249903a9ff13c4ed913a68de93057e
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java ac71d0882f985a2b475eb197a4852cc943a96a1f
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreUtils.java 50f873a013a9aa3cea0a2af8146484b9387c08f2
standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/utils/TestMetaStoreUtils.java d6c13d3f2a4dc54dab978f891b51d623a4d29762
Diff: https://reviews.apache.org/r/65745/diff/6/
Changes: https://reviews.apache.org/r/65745/diff/5-6/
Testing
-------
Added both positive unit test verifying that stats are updated and negative test verifying that stats are not updated when they shouldn't be.
Thanks,
Alexander Kolbasov
Re: Review Request 65745: HIVE-18743 CREATE TABLE on S3 data can be
extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
Posted by Alexander Kolbasov <ak...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65745/
-----------------------------------------------------------
(Updated March 4, 2018, 9:16 a.m.)
Review request for hive, Andrew Sherman, Janaki Lahorani, Zoltan Haindrich, Sahil Takiar, Thejas Nair, and Vihang Karajgaonkar.
Changes
-------
Fixed log message.
Bugs: HIVE-18743
https://issues.apache.org/jira/browse/HIVE-18743
Repository: hive-git
Description
-------
HIVE-18743 CREATE TABLE on S3 data can be extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
Diffs (updated)
-----
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 89354a2d34249903a9ff13c4ed913a68de93057e
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java ac71d0882f985a2b475eb197a4852cc943a96a1f
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreUtils.java 50f873a013a9aa3cea0a2af8146484b9387c08f2
standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/utils/TestMetaStoreUtils.java d6c13d3f2a4dc54dab978f891b51d623a4d29762
Diff: https://reviews.apache.org/r/65745/diff/5/
Changes: https://reviews.apache.org/r/65745/diff/4-5/
Testing
-------
Added both positive unit test verifying that stats are updated and negative test verifying that stats are not updated when they shouldn't be.
Thanks,
Alexander Kolbasov
Re: Review Request 65745: HIVE-18743 CREATE TABLE on S3 data can be
extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
Posted by Alexander Kolbasov <ak...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65745/
-----------------------------------------------------------
(Updated March 4, 2018, 9:12 a.m.)
Review request for hive, Andrew Sherman, Janaki Lahorani, Zoltan Haindrich, Sahil Takiar, Thejas Nair, and Vihang Karajgaonkar.
Bugs: HIVE-18743
https://issues.apache.org/jira/browse/HIVE-18743
Repository: hive-git
Description
-------
HIVE-18743 CREATE TABLE on S3 data can be extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
Diffs
-----
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 89354a2d34249903a9ff13c4ed913a68de93057e
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java ac71d0882f985a2b475eb197a4852cc943a96a1f
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreUtils.java 50f873a013a9aa3cea0a2af8146484b9387c08f2
standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/utils/TestMetaStoreUtils.java d6c13d3f2a4dc54dab978f891b51d623a4d29762
Diff: https://reviews.apache.org/r/65745/diff/4/
Testing (updated)
-------
Added both positive unit test verifying that stats are updated and negative test verifying that stats are not updated when they shouldn't be.
Thanks,
Alexander Kolbasov