You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by j....@gmail.com on 2018/01/03 20:09:00 UTC
Review Request 64925: HIVE-18349: Misc metastore changes for
debuggability
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64925/
-----------------------------------------------------------
Review request for hive and Thejas Nair.
Bugs: HIVE-18349
https://issues.apache.org/jira/browse/HIVE-18349
Repository: hive-git
Description
-------
HIVE-18349: Misc metastore changes for debuggability
Diffs
-----
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java f1b58c526db15cfdf9098af603549643c0f4dd13
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java 2e43dc85eae2490683a52444c9456e201697a3b3
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/BoneCPDataSourceProvider.java 4ff2bb77d387bd2182f8cdd248303edd15dd12bd
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/HikariCPDataSourceProvider.java 6ffc24a27a452f9a063831a2a1d39a7b1be5d6b5
Diff: https://reviews.apache.org/r/64925/diff/1/
Testing
-------
Thanks,
Prasanth_J
Re: Review Request 64925: HIVE-18349: Misc metastore changes for
debuggability
Posted by Thejas Nair <th...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64925/#review194713
-----------------------------------------------------------
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 1635 (patched)
<https://reviews.apache.org/r/64925/#comment273644>
looks like this success boolean is not tracking success of commit.
Exception is already being thrown from above stack.
- Thejas Nair
On Jan. 3, 2018, 8:09 p.m., Prasanth_J wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64925/
> -----------------------------------------------------------
>
> (Updated Jan. 3, 2018, 8:09 p.m.)
>
>
> Review request for hive and Thejas Nair.
>
>
> Bugs: HIVE-18349
> https://issues.apache.org/jira/browse/HIVE-18349
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-18349: Misc metastore changes for debuggability
>
>
> Diffs
> -----
>
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java f1b58c526db15cfdf9098af603549643c0f4dd13
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java 2e43dc85eae2490683a52444c9456e201697a3b3
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/BoneCPDataSourceProvider.java 4ff2bb77d387bd2182f8cdd248303edd15dd12bd
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/HikariCPDataSourceProvider.java 6ffc24a27a452f9a063831a2a1d39a7b1be5d6b5
>
>
> Diff: https://reviews.apache.org/r/64925/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Prasanth_J
>
>
Re: Review Request 64925: HIVE-18349: Misc metastore changes for
debuggability
Posted by j....@gmail.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64925/
-----------------------------------------------------------
(Updated Jan. 5, 2018, 9:44 p.m.)
Review request for hive and Thejas Nair.
Changes
-------
stats related metastore events did not pass the success state correctly that was causing these test failures.
Bugs: HIVE-18349
https://issues.apache.org/jira/browse/HIVE-18349
Repository: hive-git
Description
-------
HIVE-18349: Misc metastore changes for debuggability
Diffs (updated)
-----
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreEndFunctionListener.java 1ca18b9
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 89354a2
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java f1b58c5
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreEndFunctionContext.java 998531f
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java 2e43dc8
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/BoneCPDataSourceProvider.java 4ff2bb7
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/HikariCPDataSourceProvider.java 6ffc24a
Diff: https://reviews.apache.org/r/64925/diff/7/
Changes: https://reviews.apache.org/r/64925/diff/6-7/
Testing
-------
Thanks,
Prasanth_J
Re: Review Request 64925: HIVE-18349: Misc metastore changes for
debuggability
Posted by j....@gmail.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64925/
-----------------------------------------------------------
(Updated Jan. 5, 2018, 12:03 a.m.)
Review request for hive and Thejas Nair.
Bugs: HIVE-18349
https://issues.apache.org/jira/browse/HIVE-18349
Repository: hive-git
Description
-------
HIVE-18349: Misc metastore changes for debuggability
Diffs (updated)
-----
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreEndFunctionListener.java 1ca18b9
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 89354a2
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java f1b58c5
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreEndFunctionContext.java 998531f
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java 2e43dc8
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/BoneCPDataSourceProvider.java 4ff2bb7
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/HikariCPDataSourceProvider.java 6ffc24a
Diff: https://reviews.apache.org/r/64925/diff/6/
Changes: https://reviews.apache.org/r/64925/diff/5-6/
Testing
-------
Thanks,
Prasanth_J
Re: Review Request 64925: HIVE-18349: Misc metastore changes for
debuggability
Posted by j....@gmail.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64925/
-----------------------------------------------------------
(Updated Jan. 4, 2018, 11:59 p.m.)
Review request for hive and Thejas Nair.
Changes
-------
Minor fix
Bugs: HIVE-18349
https://issues.apache.org/jira/browse/HIVE-18349
Repository: hive-git
Description
-------
HIVE-18349: Misc metastore changes for debuggability
Diffs (updated)
-----
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreEndFunctionListener.java 1ca18b9
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 89354a2
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java f1b58c5
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreEndFunctionContext.java 998531f
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java 2e43dc8
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/BoneCPDataSourceProvider.java 4ff2bb7
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/HikariCPDataSourceProvider.java 6ffc24a
Diff: https://reviews.apache.org/r/64925/diff/5/
Changes: https://reviews.apache.org/r/64925/diff/4-5/
Testing
-------
Thanks,
Prasanth_J
Re: Review Request 64925: HIVE-18349: Misc metastore changes for
debuggability
Posted by j....@gmail.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64925/
-----------------------------------------------------------
(Updated Jan. 4, 2018, 10:48 p.m.)
Review request for hive and Thejas Nair.
Changes
-------
Handle some more cases where failure to commit will throw exception.
Bugs: HIVE-18349
https://issues.apache.org/jira/browse/HIVE-18349
Repository: hive-git
Description
-------
HIVE-18349: Misc metastore changes for debuggability
Diffs (updated)
-----
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreEndFunctionListener.java 1ca18b9
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 89354a2
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java f1b58c5
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreEndFunctionContext.java 998531f
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java 2e43dc8
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/BoneCPDataSourceProvider.java 4ff2bb7
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/HikariCPDataSourceProvider.java 6ffc24a
Diff: https://reviews.apache.org/r/64925/diff/4/
Changes: https://reviews.apache.org/r/64925/diff/3-4/
Testing
-------
Thanks,
Prasanth_J
Re: Review Request 64925: HIVE-18349: Misc metastore changes for
debuggability
Posted by j....@gmail.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64925/
-----------------------------------------------------------
(Updated Jan. 4, 2018, 8:22 p.m.)
Review request for hive and Thejas Nair.
Changes
-------
fixed a test
Bugs: HIVE-18349
https://issues.apache.org/jira/browse/HIVE-18349
Repository: hive-git
Description
-------
HIVE-18349: Misc metastore changes for debuggability
Diffs (updated)
-----
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreEndFunctionListener.java 1ca18b9
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java f1b58c5
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreEndFunctionContext.java 998531f
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java 2e43dc8
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/BoneCPDataSourceProvider.java 4ff2bb7
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/HikariCPDataSourceProvider.java 6ffc24a
Diff: https://reviews.apache.org/r/64925/diff/3/
Changes: https://reviews.apache.org/r/64925/diff/2-3/
Testing
-------
Thanks,
Prasanth_J
Re: Review Request 64925: HIVE-18349: Misc metastore changes for
debuggability
Posted by j....@gmail.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64925/
-----------------------------------------------------------
(Updated Jan. 4, 2018, 9:10 a.m.)
Review request for hive and Thejas Nair.
Changes
-------
Addressed review comments
Bugs: HIVE-18349
https://issues.apache.org/jira/browse/HIVE-18349
Repository: hive-git
Description
-------
HIVE-18349: Misc metastore changes for debuggability
Diffs (updated)
-----
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreEndFunctionListener.java 1ca18b9
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java f1b58c5
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreEndFunctionContext.java 998531f
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java 2e43dc8
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/BoneCPDataSourceProvider.java 4ff2bb7
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/HikariCPDataSourceProvider.java 6ffc24a
Diff: https://reviews.apache.org/r/64925/diff/2/
Changes: https://reviews.apache.org/r/64925/diff/1-2/
Testing
-------
Thanks,
Prasanth_J
Re: Review Request 64925: HIVE-18349: Misc metastore changes for
debuggability
Posted by j....@gmail.com.
> On Jan. 3, 2018, 10:14 p.m., Alexander Kolbasov wrote:
> > standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
> > Lines 884 (patched)
> > <https://reviews.apache.org/r/64925/diff/1/?file=1929986#file1929986line884>
> >
> > Is it ok to leak table name names in the log?
Table names are not sensitive info.
> On Jan. 3, 2018, 10:14 p.m., Alexander Kolbasov wrote:
> > standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
> > Lines 1670 (patched)
> > <https://reviews.apache.org/r/64925/diff/1/?file=1929986#file1929986line1670>
> >
> > Same here.
This is different case. If commit transaction fails the failure is not captured in logs.
- Prasanth_J
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64925/#review194722
-----------------------------------------------------------
On Jan. 4, 2018, 9:10 a.m., Prasanth_J wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64925/
> -----------------------------------------------------------
>
> (Updated Jan. 4, 2018, 9:10 a.m.)
>
>
> Review request for hive and Thejas Nair.
>
>
> Bugs: HIVE-18349
> https://issues.apache.org/jira/browse/HIVE-18349
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-18349: Misc metastore changes for debuggability
>
>
> Diffs
> -----
>
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreEndFunctionListener.java 1ca18b9
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java f1b58c5
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreEndFunctionContext.java 998531f
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java 2e43dc8
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/BoneCPDataSourceProvider.java 4ff2bb7
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/HikariCPDataSourceProvider.java 6ffc24a
>
>
> Diff: https://reviews.apache.org/r/64925/diff/2/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Prasanth_J
>
>
Re: Review Request 64925: HIVE-18349: Misc metastore changes for
debuggability
Posted by Alexander Kolbasov <ak...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64925/#review194722
-----------------------------------------------------------
I think it is better to separate changes to BoneCP/Hikari configs from changes to HiveMetastore into separate JIRAs since they deal with completely different issues so bundling them together doesn't help.
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 884 (patched)
<https://reviews.apache.org/r/64925/#comment273657>
Is it ok to leak table name names in the log?
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 885 (patched)
<https://reviews.apache.org/r/64925/#comment273656>
Can context.getException() be null here?
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 1635 (patched)
<https://reviews.apache.org/r/64925/#comment273658>
How can this ever happen? The only way to fail is to throw exception, so this isn't going to be executed ever.
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 1670 (patched)
<https://reviews.apache.org/r/64925/#comment273659>
Same here.
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 1718 (patched)
<https://reviews.apache.org/r/64925/#comment273660>
Same here
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 1773 (patched)
<https://reviews.apache.org/r/64925/#comment273661>
Same here
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 1828 (patched)
<https://reviews.apache.org/r/64925/#comment273662>
Same here
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 1883 (patched)
<https://reviews.apache.org/r/64925/#comment273663>
Same here
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 1938 (patched)
<https://reviews.apache.org/r/64925/#comment273664>
Same here
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Line 2178 (original), 2203 (patched)
<https://reviews.apache.org/r/64925/#comment273665>
Same here
- Alexander Kolbasov
On Jan. 3, 2018, 8:09 p.m., Prasanth_J wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64925/
> -----------------------------------------------------------
>
> (Updated Jan. 3, 2018, 8:09 p.m.)
>
>
> Review request for hive and Thejas Nair.
>
>
> Bugs: HIVE-18349
> https://issues.apache.org/jira/browse/HIVE-18349
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-18349: Misc metastore changes for debuggability
>
>
> Diffs
> -----
>
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java f1b58c526db15cfdf9098af603549643c0f4dd13
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java 2e43dc85eae2490683a52444c9456e201697a3b3
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/BoneCPDataSourceProvider.java 4ff2bb77d387bd2182f8cdd248303edd15dd12bd
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/HikariCPDataSourceProvider.java 6ffc24a27a452f9a063831a2a1d39a7b1be5d6b5
>
>
> Diff: https://reviews.apache.org/r/64925/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Prasanth_J
>
>
Re: Review Request 64925: HIVE-18349: Misc metastore changes for
debuggability
Posted by Thejas Nair <th...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64925/#review194712
-----------------------------------------------------------
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 882 (patched)
<https://reviews.apache.org/r/64925/#comment273643>
passing table info as argument seems like the better option, since endFunction can be called from methods that don't have table info.
(similar to the way startTableFunction is used from drop_table_..)
- Thejas Nair
On Jan. 3, 2018, 8:09 p.m., Prasanth_J wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64925/
> -----------------------------------------------------------
>
> (Updated Jan. 3, 2018, 8:09 p.m.)
>
>
> Review request for hive and Thejas Nair.
>
>
> Bugs: HIVE-18349
> https://issues.apache.org/jira/browse/HIVE-18349
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-18349: Misc metastore changes for debuggability
>
>
> Diffs
> -----
>
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java f1b58c526db15cfdf9098af603549643c0f4dd13
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java 2e43dc85eae2490683a52444c9456e201697a3b3
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/BoneCPDataSourceProvider.java 4ff2bb77d387bd2182f8cdd248303edd15dd12bd
> standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/datasource/HikariCPDataSourceProvider.java 6ffc24a27a452f9a063831a2a1d39a7b1be5d6b5
>
>
> Diff: https://reviews.apache.org/r/64925/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Prasanth_J
>
>