You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Jesús Camacho Rodríguez <jc...@hortonworks.com> on 2017/11/16 23:22:42 UTC
Review Request 63893: HIVE-14487
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63893/
-----------------------------------------------------------
Review request for hive and Ashutosh Chauhan.
Bugs: HIVE-14487
https://issues.apache.org/jira/browse/HIVE-14487
Repository: hive-git
Description
-------
HIVE-14487
Diffs
-----
itests/src/test/resources/testconfiguration.properties 3b1005f85c419b1fa5646bc5a4b96788250e7c4b
ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java e7af5e004fb560b574b82f6d1b60517511802f37
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 67739a1db9fc52a67f4f5ea7dba80fe0e95750c8
ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java e9ae590f4c291ad00e385899822ef944d50cdcde
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 55b1da90300bdc864e9258d1c49c3c21791f3b64
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g f3f1a7e8da52332d93aa57865617befd6c07a2fc
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1de3dd7230c5c3708fd7dd6baf9c6913e91cab3c
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java d139155a1d1433ca9186994cef2616f007a66fe9
ql/src/java/org/apache/hadoop/hive/ql/plan/BasicStatsWork.java 0621bd413ec578f30a236de0be5774fc83457aa5
ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java 9425f6e89113afd82d1cdca30c75da6101b63b4a
ql/src/java/org/apache/hadoop/hive/ql/plan/ImportTableDesc.java 1770046869170c30ebb27a84db7566d477bd9877
ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java c09589c5f23825ea39b85737e04cc0e365e61fea
ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 3c1e92af12185bb95e27fbf0af3038ca27566197
ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_no_grant.q PRE-CREATION
ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_other.q PRE-CREATION
ql/src/test/queries/clientpositive/materialized_view_create_rewrite_3.q PRE-CREATION
ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_no_grant.q.out PRE-CREATION
ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_other.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_create.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_2.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_3.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_multi_db.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_describe.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_drop.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb_2.q.out PRE-CREATION
ql/src/test/results/clientpositive/materialized_view_create.q.out 0fca69f4fa94a24482d09a1b30a72611eec6c643
ql/src/test/results/clientpositive/materialized_view_create_rewrite.q.out 81a7950773983e28a0f36ce806b6cda394993ac0
ql/src/test/results/clientpositive/materialized_view_create_rewrite_3.q.out PRE-CREATION
ql/src/test/results/clientpositive/materialized_view_create_rewrite_multi_db.q.out f715803438d100599fc669cb9b1086523cd44099
ql/src/test/results/clientpositive/materialized_view_describe.q.out 8b7691279c0b24c599656c372de22ef9cf83f8f7
Diff: https://reviews.apache.org/r/63893/diff/1/
Testing
-------
Thanks,
Jesús Camacho Rodríguez
Re: Review Request 63893: HIVE-14487
Posted by Jesús Camacho Rodríguez <jc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63893/
-----------------------------------------------------------
(Updated Nov. 18, 2017, 12:33 a.m.)
Review request for hive and Ashutosh Chauhan.
Bugs: HIVE-14487
https://issues.apache.org/jira/browse/HIVE-14487
Repository: hive-git
Description (updated)
-------
HIVE-14487
Diffs (updated)
-----
itests/src/test/resources/testconfiguration.properties 3b1005f85c419b1fa5646bc5a4b96788250e7c4b
ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java e7af5e004fb560b574b82f6d1b60517511802f37
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 67739a1db9fc52a67f4f5ea7dba80fe0e95750c8
ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java e9ae590f4c291ad00e385899822ef944d50cdcde
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 55b1da90300bdc864e9258d1c49c3c21791f3b64
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g f3f1a7e8da52332d93aa57865617befd6c07a2fc
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1de3dd7230c5c3708fd7dd6baf9c6913e91cab3c
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java d139155a1d1433ca9186994cef2616f007a66fe9
ql/src/java/org/apache/hadoop/hive/ql/plan/BasicStatsWork.java 0621bd413ec578f30a236de0be5774fc83457aa5
ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java 9425f6e89113afd82d1cdca30c75da6101b63b4a
ql/src/java/org/apache/hadoop/hive/ql/plan/ImportTableDesc.java 1770046869170c30ebb27a84db7566d477bd9877
ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java c09589c5f23825ea39b85737e04cc0e365e61fea
ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 3c1e92af12185bb95e27fbf0af3038ca27566197
ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_no_grant.q PRE-CREATION
ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_other.q PRE-CREATION
ql/src/test/queries/clientpositive/materialized_view_create_rewrite.q 1749cb023ae2e4476ac386b667a708f50f8db462
ql/src/test/queries/clientpositive/materialized_view_create_rewrite_3.q PRE-CREATION
ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_no_grant.q.out PRE-CREATION
ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_other.q.out PRE-CREATION
ql/src/test/results/clientpositive/beeline/materialized_view_create_rewrite.q.out 81a7950773983e28a0f36ce806b6cda394993ac0
ql/src/test/results/clientpositive/create_view.q.out 8763fdb75fc910f75c53346c9541703e800c128c
ql/src/test/results/clientpositive/create_view_translate.q.out e5748976bdb271bef386e6820be5236a36d4c941
ql/src/test/results/clientpositive/explain_ddl.q.out f0e54c515377e7e6e002b7cc9ae986340a93acce
ql/src/test/results/clientpositive/llap/materialized_view_create.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_2.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_3.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_multi_db.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_describe.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_drop.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb_2.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/selectDistinctStar.q.out befb6e79929ca7fffc9ee56544c25a27c2a84cbd
ql/src/test/results/clientpositive/llap/union_top_level.q.out a0102321711de82100a5b117b853a364109a2622
ql/src/test/results/clientpositive/llap/vector_windowing.q.out 776ae3e085840dcaa298c143966ff1f12e508c3e
ql/src/test/results/clientpositive/materialized_view_create.q.out 0fca69f4fa94a24482d09a1b30a72611eec6c643
ql/src/test/results/clientpositive/materialized_view_create_rewrite.q.out 81a7950773983e28a0f36ce806b6cda394993ac0
ql/src/test/results/clientpositive/materialized_view_create_rewrite_3.q.out PRE-CREATION
ql/src/test/results/clientpositive/materialized_view_create_rewrite_multi_db.q.out f715803438d100599fc669cb9b1086523cd44099
ql/src/test/results/clientpositive/materialized_view_describe.q.out 8b7691279c0b24c599656c372de22ef9cf83f8f7
Diff: https://reviews.apache.org/r/63893/diff/4/
Changes: https://reviews.apache.org/r/63893/diff/3-4/
Testing
-------
Thanks,
Jesús Camacho Rodríguez
Re: Review Request 63893: HIVE-14487
Posted by Jesús Camacho Rodríguez <jc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63893/
-----------------------------------------------------------
(Updated Nov. 18, 2017, 12:31 a.m.)
Review request for hive and Ashutosh Chauhan.
Bugs: HIVE-14487
https://issues.apache.org/jira/browse/HIVE-14487
Repository: hive-git
Description (updated)
-------
wip
Diffs (updated)
-----
itests/src/test/resources/testconfiguration.properties 3b1005f85c419b1fa5646bc5a4b96788250e7c4b
ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java e7af5e004fb560b574b82f6d1b60517511802f37
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveMaterializedViewsRegistry.java 51b6ef58fc196be716c4b07287fbc83503d1df50
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 67739a1db9fc52a67f4f5ea7dba80fe0e95750c8
ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java e9ae590f4c291ad00e385899822ef944d50cdcde
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 55b1da90300bdc864e9258d1c49c3c21791f3b64
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g f3f1a7e8da52332d93aa57865617befd6c07a2fc
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1de3dd7230c5c3708fd7dd6baf9c6913e91cab3c
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java d139155a1d1433ca9186994cef2616f007a66fe9
ql/src/java/org/apache/hadoop/hive/ql/plan/BasicStatsWork.java 0621bd413ec578f30a236de0be5774fc83457aa5
ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java 9425f6e89113afd82d1cdca30c75da6101b63b4a
ql/src/java/org/apache/hadoop/hive/ql/plan/ImportTableDesc.java 1770046869170c30ebb27a84db7566d477bd9877
ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java c09589c5f23825ea39b85737e04cc0e365e61fea
ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 3c1e92af12185bb95e27fbf0af3038ca27566197
ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_no_grant.q PRE-CREATION
ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_other.q PRE-CREATION
ql/src/test/queries/clientpositive/materialized_view_create_rewrite.q 1749cb023ae2e4476ac386b667a708f50f8db462
ql/src/test/queries/clientpositive/materialized_view_create_rewrite_3.q PRE-CREATION
ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_no_grant.q.out PRE-CREATION
ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_other.q.out PRE-CREATION
ql/src/test/results/clientpositive/beeline/materialized_view_create_rewrite.q.out 81a7950773983e28a0f36ce806b6cda394993ac0
ql/src/test/results/clientpositive/create_view.q.out 8763fdb75fc910f75c53346c9541703e800c128c
ql/src/test/results/clientpositive/create_view_translate.q.out e5748976bdb271bef386e6820be5236a36d4c941
ql/src/test/results/clientpositive/explain_ddl.q.out f0e54c515377e7e6e002b7cc9ae986340a93acce
ql/src/test/results/clientpositive/llap/materialized_view_create.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_2.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_3.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_multi_db.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_describe.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_drop.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb_2.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/selectDistinctStar.q.out befb6e79929ca7fffc9ee56544c25a27c2a84cbd
ql/src/test/results/clientpositive/llap/union_top_level.q.out a0102321711de82100a5b117b853a364109a2622
ql/src/test/results/clientpositive/llap/vector_windowing.q.out 776ae3e085840dcaa298c143966ff1f12e508c3e
ql/src/test/results/clientpositive/materialized_view_create.q.out 0fca69f4fa94a24482d09a1b30a72611eec6c643
ql/src/test/results/clientpositive/materialized_view_create_rewrite.q.out 81a7950773983e28a0f36ce806b6cda394993ac0
ql/src/test/results/clientpositive/materialized_view_create_rewrite_3.q.out PRE-CREATION
ql/src/test/results/clientpositive/materialized_view_create_rewrite_multi_db.q.out f715803438d100599fc669cb9b1086523cd44099
ql/src/test/results/clientpositive/materialized_view_describe.q.out 8b7691279c0b24c599656c372de22ef9cf83f8f7
Diff: https://reviews.apache.org/r/63893/diff/3/
Changes: https://reviews.apache.org/r/63893/diff/2-3/
Testing
-------
Thanks,
Jesús Camacho Rodríguez
Re: Review Request 63893: HIVE-14487
Posted by Jesús Camacho Rodríguez <jc...@hortonworks.com>.
> On Nov. 17, 2017, 11:22 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
> > Line 1488 (original), 1488 (patched)
> > <https://reviews.apache.org/r/63893/diff/2/?file=1896941#file1896941line1488>
> >
> > This table object is never read from. Any reason for its existence.
It is passed to StatsWork in L1533. I think we do not need the full table, as StatsWork will only use some of the information. I can tackle in a follow-up?
> On Nov. 17, 2017, 11:22 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
> > Lines 1515 (patched)
> > <https://reviews.apache.org/r/63893/diff/2/?file=1896941#file1896941line1517>
> >
> > Is it necessary to make metastore call to get Table? Can this be avoided.
Observe this is the same as L1495. As mentioned above, I think we do not need the complete table for StatsWork, just a few properties. However, to make sure we have everything needed, I took it from metastore as it is already an existing one. I can take care of this in the follow-up to check whether we need to pass the complete table.
> On Nov. 17, 2017, 11:22 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java
> > Lines 341 (patched)
> > <https://reviews.apache.org/r/63893/diff/2/?file=1896948#file1896948line341>
> >
> > We don't store comments for regular tables in property? Is this special for (materialized) views?
In fact, apparently we do: L779 in CreateTableDesc.java. The code was originally based on that one, thus they should be quite similar.
- Jesús
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63893/#review191406
-----------------------------------------------------------
On Nov. 17, 2017, 7:26 p.m., Jesús Camacho Rodríguez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63893/
> -----------------------------------------------------------
>
> (Updated Nov. 17, 2017, 7:26 p.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-14487
> https://issues.apache.org/jira/browse/HIVE-14487
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-14487
>
>
> Diffs
> -----
>
> itests/src/test/resources/testconfiguration.properties 3b1005f85c419b1fa5646bc5a4b96788250e7c4b
> ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java e7af5e004fb560b574b82f6d1b60517511802f37
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 67739a1db9fc52a67f4f5ea7dba80fe0e95750c8
> ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java e9ae590f4c291ad00e385899822ef944d50cdcde
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 55b1da90300bdc864e9258d1c49c3c21791f3b64
> ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g f3f1a7e8da52332d93aa57865617befd6c07a2fc
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1de3dd7230c5c3708fd7dd6baf9c6913e91cab3c
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java d139155a1d1433ca9186994cef2616f007a66fe9
> ql/src/java/org/apache/hadoop/hive/ql/plan/BasicStatsWork.java 0621bd413ec578f30a236de0be5774fc83457aa5
> ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java 9425f6e89113afd82d1cdca30c75da6101b63b4a
> ql/src/java/org/apache/hadoop/hive/ql/plan/ImportTableDesc.java 1770046869170c30ebb27a84db7566d477bd9877
> ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java c09589c5f23825ea39b85737e04cc0e365e61fea
> ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 3c1e92af12185bb95e27fbf0af3038ca27566197
> ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_no_grant.q PRE-CREATION
> ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_other.q PRE-CREATION
> ql/src/test/queries/clientpositive/materialized_view_create_rewrite.q 1749cb023ae2e4476ac386b667a708f50f8db462
> ql/src/test/queries/clientpositive/materialized_view_create_rewrite_3.q PRE-CREATION
> ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_no_grant.q.out PRE-CREATION
> ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_other.q.out PRE-CREATION
> ql/src/test/results/clientpositive/beeline/materialized_view_create_rewrite.q.out 81a7950773983e28a0f36ce806b6cda394993ac0
> ql/src/test/results/clientpositive/create_view.q.out 8763fdb75fc910f75c53346c9541703e800c128c
> ql/src/test/results/clientpositive/create_view_translate.q.out e5748976bdb271bef386e6820be5236a36d4c941
> ql/src/test/results/clientpositive/explain_ddl.q.out f0e54c515377e7e6e002b7cc9ae986340a93acce
> ql/src/test/results/clientpositive/llap/materialized_view_create.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_3.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_multi_db.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_describe.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_drop.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb_2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/selectDistinctStar.q.out befb6e79929ca7fffc9ee56544c25a27c2a84cbd
> ql/src/test/results/clientpositive/llap/union_top_level.q.out a0102321711de82100a5b117b853a364109a2622
> ql/src/test/results/clientpositive/llap/vector_windowing.q.out 776ae3e085840dcaa298c143966ff1f12e508c3e
> ql/src/test/results/clientpositive/materialized_view_create.q.out 0fca69f4fa94a24482d09a1b30a72611eec6c643
> ql/src/test/results/clientpositive/materialized_view_create_rewrite.q.out 81a7950773983e28a0f36ce806b6cda394993ac0
> ql/src/test/results/clientpositive/materialized_view_create_rewrite_3.q.out PRE-CREATION
> ql/src/test/results/clientpositive/materialized_view_create_rewrite_multi_db.q.out f715803438d100599fc669cb9b1086523cd44099
> ql/src/test/results/clientpositive/materialized_view_describe.q.out 8b7691279c0b24c599656c372de22ef9cf83f8f7
>
>
> Diff: https://reviews.apache.org/r/63893/diff/2/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jesús Camacho Rodríguez
>
>
Re: Review Request 63893: HIVE-14487
Posted by Ashutosh Chauhan <ha...@apache.org>.
> On Nov. 17, 2017, 11:22 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
> > Line 1488 (original), 1488 (patched)
> > <https://reviews.apache.org/r/63893/diff/2/?file=1896941#file1896941line1488>
> >
> > This table object is never read from. Any reason for its existence.
>
> Jesús Camacho Rodríguez wrote:
> It is passed to StatsWork in L1533. I think we do not need the full table, as StatsWork will only use some of the information. I can tackle in a follow-up?
Ok, lets take this as a follow-up.
> On Nov. 17, 2017, 11:22 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java
> > Lines 341 (patched)
> > <https://reviews.apache.org/r/63893/diff/2/?file=1896948#file1896948line341>
> >
> > We don't store comments for regular tables in property? Is this special for (materialized) views?
>
> Jesús Camacho Rodríguez wrote:
> In fact, apparently we do: L779 in CreateTableDesc.java. The code was originally based on that one, thus they should be quite similar.
Right. I got confused with column comments which are stored in a seperate field in metastore table.
- Ashutosh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63893/#review191406
-----------------------------------------------------------
On Nov. 18, 2017, 12:33 a.m., Jesús Camacho Rodríguez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63893/
> -----------------------------------------------------------
>
> (Updated Nov. 18, 2017, 12:33 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-14487
> https://issues.apache.org/jira/browse/HIVE-14487
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-14487
>
>
> Diffs
> -----
>
> itests/src/test/resources/testconfiguration.properties 3b1005f85c419b1fa5646bc5a4b96788250e7c4b
> ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java e7af5e004fb560b574b82f6d1b60517511802f37
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 67739a1db9fc52a67f4f5ea7dba80fe0e95750c8
> ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java e9ae590f4c291ad00e385899822ef944d50cdcde
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 55b1da90300bdc864e9258d1c49c3c21791f3b64
> ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g f3f1a7e8da52332d93aa57865617befd6c07a2fc
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1de3dd7230c5c3708fd7dd6baf9c6913e91cab3c
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java d139155a1d1433ca9186994cef2616f007a66fe9
> ql/src/java/org/apache/hadoop/hive/ql/plan/BasicStatsWork.java 0621bd413ec578f30a236de0be5774fc83457aa5
> ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java 9425f6e89113afd82d1cdca30c75da6101b63b4a
> ql/src/java/org/apache/hadoop/hive/ql/plan/ImportTableDesc.java 1770046869170c30ebb27a84db7566d477bd9877
> ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java c09589c5f23825ea39b85737e04cc0e365e61fea
> ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 3c1e92af12185bb95e27fbf0af3038ca27566197
> ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_no_grant.q PRE-CREATION
> ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_other.q PRE-CREATION
> ql/src/test/queries/clientpositive/materialized_view_create_rewrite.q 1749cb023ae2e4476ac386b667a708f50f8db462
> ql/src/test/queries/clientpositive/materialized_view_create_rewrite_3.q PRE-CREATION
> ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_no_grant.q.out PRE-CREATION
> ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_other.q.out PRE-CREATION
> ql/src/test/results/clientpositive/beeline/materialized_view_create_rewrite.q.out 81a7950773983e28a0f36ce806b6cda394993ac0
> ql/src/test/results/clientpositive/create_view.q.out 8763fdb75fc910f75c53346c9541703e800c128c
> ql/src/test/results/clientpositive/create_view_translate.q.out e5748976bdb271bef386e6820be5236a36d4c941
> ql/src/test/results/clientpositive/explain_ddl.q.out f0e54c515377e7e6e002b7cc9ae986340a93acce
> ql/src/test/results/clientpositive/llap/materialized_view_create.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_3.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_multi_db.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_describe.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_drop.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb_2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/selectDistinctStar.q.out befb6e79929ca7fffc9ee56544c25a27c2a84cbd
> ql/src/test/results/clientpositive/llap/union_top_level.q.out a0102321711de82100a5b117b853a364109a2622
> ql/src/test/results/clientpositive/llap/vector_windowing.q.out 776ae3e085840dcaa298c143966ff1f12e508c3e
> ql/src/test/results/clientpositive/materialized_view_create.q.out 0fca69f4fa94a24482d09a1b30a72611eec6c643
> ql/src/test/results/clientpositive/materialized_view_create_rewrite.q.out 81a7950773983e28a0f36ce806b6cda394993ac0
> ql/src/test/results/clientpositive/materialized_view_create_rewrite_3.q.out PRE-CREATION
> ql/src/test/results/clientpositive/materialized_view_create_rewrite_multi_db.q.out f715803438d100599fc669cb9b1086523cd44099
> ql/src/test/results/clientpositive/materialized_view_describe.q.out 8b7691279c0b24c599656c372de22ef9cf83f8f7
>
>
> Diff: https://reviews.apache.org/r/63893/diff/4/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jesús Camacho Rodríguez
>
>
Re: Review Request 63893: HIVE-14487
Posted by Ashutosh Chauhan <ha...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63893/#review191406
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
Line 1488 (original), 1488 (patched)
<https://reviews.apache.org/r/63893/#comment269216>
This table object is never read from. Any reason for its existence.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
Lines 1515 (patched)
<https://reviews.apache.org/r/63893/#comment269217>
Is it necessary to make metastore call to get Table? Can this be avoided.
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
Lines 12708 (patched)
<https://reviews.apache.org/r/63893/#comment269219>
May use getTableObjectByName() to cache table object.
ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java
Lines 341 (patched)
<https://reviews.apache.org/r/63893/#comment269225>
We don't store comments for regular tables in property? Is this special for (materialized) views?
- Ashutosh Chauhan
On Nov. 17, 2017, 7:26 p.m., Jesús Camacho Rodríguez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63893/
> -----------------------------------------------------------
>
> (Updated Nov. 17, 2017, 7:26 p.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-14487
> https://issues.apache.org/jira/browse/HIVE-14487
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-14487
>
>
> Diffs
> -----
>
> itests/src/test/resources/testconfiguration.properties 3b1005f85c419b1fa5646bc5a4b96788250e7c4b
> ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java e7af5e004fb560b574b82f6d1b60517511802f37
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 67739a1db9fc52a67f4f5ea7dba80fe0e95750c8
> ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java e9ae590f4c291ad00e385899822ef944d50cdcde
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 55b1da90300bdc864e9258d1c49c3c21791f3b64
> ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g f3f1a7e8da52332d93aa57865617befd6c07a2fc
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1de3dd7230c5c3708fd7dd6baf9c6913e91cab3c
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java d139155a1d1433ca9186994cef2616f007a66fe9
> ql/src/java/org/apache/hadoop/hive/ql/plan/BasicStatsWork.java 0621bd413ec578f30a236de0be5774fc83457aa5
> ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java 9425f6e89113afd82d1cdca30c75da6101b63b4a
> ql/src/java/org/apache/hadoop/hive/ql/plan/ImportTableDesc.java 1770046869170c30ebb27a84db7566d477bd9877
> ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java c09589c5f23825ea39b85737e04cc0e365e61fea
> ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 3c1e92af12185bb95e27fbf0af3038ca27566197
> ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_no_grant.q PRE-CREATION
> ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_other.q PRE-CREATION
> ql/src/test/queries/clientpositive/materialized_view_create_rewrite.q 1749cb023ae2e4476ac386b667a708f50f8db462
> ql/src/test/queries/clientpositive/materialized_view_create_rewrite_3.q PRE-CREATION
> ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_no_grant.q.out PRE-CREATION
> ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_other.q.out PRE-CREATION
> ql/src/test/results/clientpositive/beeline/materialized_view_create_rewrite.q.out 81a7950773983e28a0f36ce806b6cda394993ac0
> ql/src/test/results/clientpositive/create_view.q.out 8763fdb75fc910f75c53346c9541703e800c128c
> ql/src/test/results/clientpositive/create_view_translate.q.out e5748976bdb271bef386e6820be5236a36d4c941
> ql/src/test/results/clientpositive/explain_ddl.q.out f0e54c515377e7e6e002b7cc9ae986340a93acce
> ql/src/test/results/clientpositive/llap/materialized_view_create.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_3.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_multi_db.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_describe.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_drop.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb_2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/selectDistinctStar.q.out befb6e79929ca7fffc9ee56544c25a27c2a84cbd
> ql/src/test/results/clientpositive/llap/union_top_level.q.out a0102321711de82100a5b117b853a364109a2622
> ql/src/test/results/clientpositive/llap/vector_windowing.q.out 776ae3e085840dcaa298c143966ff1f12e508c3e
> ql/src/test/results/clientpositive/materialized_view_create.q.out 0fca69f4fa94a24482d09a1b30a72611eec6c643
> ql/src/test/results/clientpositive/materialized_view_create_rewrite.q.out 81a7950773983e28a0f36ce806b6cda394993ac0
> ql/src/test/results/clientpositive/materialized_view_create_rewrite_3.q.out PRE-CREATION
> ql/src/test/results/clientpositive/materialized_view_create_rewrite_multi_db.q.out f715803438d100599fc669cb9b1086523cd44099
> ql/src/test/results/clientpositive/materialized_view_describe.q.out 8b7691279c0b24c599656c372de22ef9cf83f8f7
>
>
> Diff: https://reviews.apache.org/r/63893/diff/2/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jesús Camacho Rodríguez
>
>
Re: Review Request 63893: HIVE-14487
Posted by Jesús Camacho Rodríguez <jc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63893/
-----------------------------------------------------------
(Updated Nov. 17, 2017, 7:26 p.m.)
Review request for hive and Ashutosh Chauhan.
Changes
-------
Updating q files
Bugs: HIVE-14487
https://issues.apache.org/jira/browse/HIVE-14487
Repository: hive-git
Description
-------
HIVE-14487
Diffs (updated)
-----
itests/src/test/resources/testconfiguration.properties 3b1005f85c419b1fa5646bc5a4b96788250e7c4b
ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java e7af5e004fb560b574b82f6d1b60517511802f37
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 67739a1db9fc52a67f4f5ea7dba80fe0e95750c8
ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java e9ae590f4c291ad00e385899822ef944d50cdcde
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 55b1da90300bdc864e9258d1c49c3c21791f3b64
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g f3f1a7e8da52332d93aa57865617befd6c07a2fc
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1de3dd7230c5c3708fd7dd6baf9c6913e91cab3c
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java d139155a1d1433ca9186994cef2616f007a66fe9
ql/src/java/org/apache/hadoop/hive/ql/plan/BasicStatsWork.java 0621bd413ec578f30a236de0be5774fc83457aa5
ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java 9425f6e89113afd82d1cdca30c75da6101b63b4a
ql/src/java/org/apache/hadoop/hive/ql/plan/ImportTableDesc.java 1770046869170c30ebb27a84db7566d477bd9877
ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java c09589c5f23825ea39b85737e04cc0e365e61fea
ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 3c1e92af12185bb95e27fbf0af3038ca27566197
ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_no_grant.q PRE-CREATION
ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_other.q PRE-CREATION
ql/src/test/queries/clientpositive/materialized_view_create_rewrite.q 1749cb023ae2e4476ac386b667a708f50f8db462
ql/src/test/queries/clientpositive/materialized_view_create_rewrite_3.q PRE-CREATION
ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_no_grant.q.out PRE-CREATION
ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_other.q.out PRE-CREATION
ql/src/test/results/clientpositive/beeline/materialized_view_create_rewrite.q.out 81a7950773983e28a0f36ce806b6cda394993ac0
ql/src/test/results/clientpositive/create_view.q.out 8763fdb75fc910f75c53346c9541703e800c128c
ql/src/test/results/clientpositive/create_view_translate.q.out e5748976bdb271bef386e6820be5236a36d4c941
ql/src/test/results/clientpositive/explain_ddl.q.out f0e54c515377e7e6e002b7cc9ae986340a93acce
ql/src/test/results/clientpositive/llap/materialized_view_create.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_2.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_3.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_multi_db.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_describe.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_drop.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb_2.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/selectDistinctStar.q.out befb6e79929ca7fffc9ee56544c25a27c2a84cbd
ql/src/test/results/clientpositive/llap/union_top_level.q.out a0102321711de82100a5b117b853a364109a2622
ql/src/test/results/clientpositive/llap/vector_windowing.q.out 776ae3e085840dcaa298c143966ff1f12e508c3e
ql/src/test/results/clientpositive/materialized_view_create.q.out 0fca69f4fa94a24482d09a1b30a72611eec6c643
ql/src/test/results/clientpositive/materialized_view_create_rewrite.q.out 81a7950773983e28a0f36ce806b6cda394993ac0
ql/src/test/results/clientpositive/materialized_view_create_rewrite_3.q.out PRE-CREATION
ql/src/test/results/clientpositive/materialized_view_create_rewrite_multi_db.q.out f715803438d100599fc669cb9b1086523cd44099
ql/src/test/results/clientpositive/materialized_view_describe.q.out 8b7691279c0b24c599656c372de22ef9cf83f8f7
Diff: https://reviews.apache.org/r/63893/diff/2/
Changes: https://reviews.apache.org/r/63893/diff/1-2/
Testing
-------
Thanks,
Jesús Camacho Rodríguez