You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Sohan Jain <so...@fb.com> on 2011/06/03 22:20:03 UTC
Review Request: HIVE-2194: Add actions to MetaStoreEventListener to be
performed on alter table and alter partition operations.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/853/
-----------------------------------------------------------
Review request for hive and Paul Yang.
Summary
-------
HIVE-2038 introduced the MetaStoreEventListener abstract class that defines actions to be performed after particular events on a metastore. Improve upon that class by adding events to be performed on "alter table" and "alter partition" actions. Also, update the hive metastore to call the appropriate functions of the listeners when the events happen.
This addresses bug HIVE-2194.
https://issues.apache.org/jira/browse/HIVE-2194
Diffs
-----
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/AlterHandler.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AlterPartitionEvent.java PRE-CREATION
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AlterTableEvent.java PRE-CREATION
trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java 1130342
trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java 1130342
Diff: https://reviews.apache.org/r/853/diff
Testing
-------
Added test cases to TestMetaStoreEventListener.
Thanks,
Sohan
Re: Review Request: HIVE-2194: Add actions to MetaStoreEventListener to be
performed on alter table and alter partition operations.
Posted by Sohan Jain <so...@fb.com>.
> On 2011-06-03 20:55:01, Carl Steinbach wrote:
> > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java, line 54
> > <https://reviews.apache.org/r/853/diff/1/?file=20442#file20442line54>
> >
> > This method should either succeed or throw an exception. Since it already throws checked exceptions, adding a boolean return value makes things more complicated for the caller, and signals the same type of failure as an exception but without the ability to include information about what caused the failure. I think it would be better to modify the code so that an exception is always thrown when success==FALSE.
> >
> > This comment also applies to the various alterTable and alterPartition methods located in other classes.
>
> Sohan Jain wrote:
> Thanks for your feedback, Carl. I understand your point about the return value of those functions. Is throwing a MetaException in the case of failure appropriate?
>
> Furthermore, I noticed that a few other RawStore operations return boolean values, such as dropDatabase, alterDatabase, dropTable, addPartition, etc. The return value in each case corresponds to successfully committing the transaction. Should we adopt a uniform interface across these functions?
Hi, is there any update on this? Thanks for looking!
- Sohan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/853/#review758
-----------------------------------------------------------
On 2011-06-06 23:23:50, Sohan Jain wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/853/
> -----------------------------------------------------------
>
> (Updated 2011-06-06 23:23:50)
>
>
> Review request for hive and Paul Yang.
>
>
> Summary
> -------
>
> HIVE-2038 introduced the MetaStoreEventListener abstract class that defines actions to be performed after particular events on a metastore. Improve upon that class by adding events to be performed on "alter table" and "alter partition" actions. Also, update the hive metastore to call the appropriate functions of the listeners when the events happen.
>
>
> This addresses bug HIVE-2194.
> https://issues.apache.org/jira/browse/HIVE-2194
>
>
> Diffs
> -----
>
> trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java 1130342
> trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java 1130342
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AlterTableEvent.java PRE-CREATION
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/AlterHandler.java 1130342
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 1130342
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1130342
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java 1130342
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1130342
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AlterPartitionEvent.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/853/diff
>
>
> Testing
> -------
>
> Added test cases to TestMetaStoreEventListener.
>
>
> Thanks,
>
> Sohan
>
>
Re: Review Request: HIVE-2194: Add actions to MetaStoreEventListener to be
performed on alter table and alter partition operations.
Posted by Sohan Jain <so...@fb.com>.
> On 2011-06-03 20:55:01, Carl Steinbach wrote:
> > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java, line 54
> > <https://reviews.apache.org/r/853/diff/1/?file=20442#file20442line54>
> >
> > This method should either succeed or throw an exception. Since it already throws checked exceptions, adding a boolean return value makes things more complicated for the caller, and signals the same type of failure as an exception but without the ability to include information about what caused the failure. I think it would be better to modify the code so that an exception is always thrown when success==FALSE.
> >
> > This comment also applies to the various alterTable and alterPartition methods located in other classes.
Thanks for your feedback, Carl. I understand your point about the return value of those functions. Is throwing a MetaException in the case of failure appropriate?
Furthermore, I noticed that a few other RawStore operations return boolean values, such as dropDatabase, alterDatabase, dropTable, addPartition, etc. The return value in each case corresponds to successfully committing the transaction. Should we adopt a uniform interface across these functions?
- Sohan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/853/#review758
-----------------------------------------------------------
On 2011-06-03 20:20:03, Sohan Jain wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/853/
> -----------------------------------------------------------
>
> (Updated 2011-06-03 20:20:03)
>
>
> Review request for hive and Paul Yang.
>
>
> Summary
> -------
>
> HIVE-2038 introduced the MetaStoreEventListener abstract class that defines actions to be performed after particular events on a metastore. Improve upon that class by adding events to be performed on "alter table" and "alter partition" actions. Also, update the hive metastore to call the appropriate functions of the listeners when the events happen.
>
>
> This addresses bug HIVE-2194.
> https://issues.apache.org/jira/browse/HIVE-2194
>
>
> Diffs
> -----
>
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/AlterHandler.java 1130342
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 1130342
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1130342
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java 1130342
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1130342
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1130342
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AlterPartitionEvent.java PRE-CREATION
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AlterTableEvent.java PRE-CREATION
> trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java 1130342
> trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java 1130342
>
> Diff: https://reviews.apache.org/r/853/diff
>
>
> Testing
> -------
>
> Added test cases to TestMetaStoreEventListener.
>
>
> Thanks,
>
> Sohan
>
>
Re: Review Request: HIVE-2194: Add actions to MetaStoreEventListener to be
performed on alter table and alter partition operations.
Posted by Carl Steinbach <ca...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/853/#review758
-----------------------------------------------------------
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
<https://reviews.apache.org/r/853/#comment1598>
This method should either succeed or throw an exception. Since it already throws checked exceptions, adding a boolean return value makes things more complicated for the caller, and signals the same type of failure as an exception but without the ability to include information about what caused the failure. I think it would be better to modify the code so that an exception is always thrown when success==FALSE.
This comment also applies to the various alterTable and alterPartition methods located in other classes.
- Carl
On 2011-06-03 20:20:03, Sohan Jain wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/853/
> -----------------------------------------------------------
>
> (Updated 2011-06-03 20:20:03)
>
>
> Review request for hive and Paul Yang.
>
>
> Summary
> -------
>
> HIVE-2038 introduced the MetaStoreEventListener abstract class that defines actions to be performed after particular events on a metastore. Improve upon that class by adding events to be performed on "alter table" and "alter partition" actions. Also, update the hive metastore to call the appropriate functions of the listeners when the events happen.
>
>
> This addresses bug HIVE-2194.
> https://issues.apache.org/jira/browse/HIVE-2194
>
>
> Diffs
> -----
>
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/AlterHandler.java 1130342
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 1130342
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1130342
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java 1130342
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1130342
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1130342
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AlterPartitionEvent.java PRE-CREATION
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AlterTableEvent.java PRE-CREATION
> trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java 1130342
> trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java 1130342
>
> Diff: https://reviews.apache.org/r/853/diff
>
>
> Testing
> -------
>
> Added test cases to TestMetaStoreEventListener.
>
>
> Thanks,
>
> Sohan
>
>
Re: Review Request: HIVE-2194: Add actions to MetaStoreEventListener to be
performed on alter table and alter partition operations.
Posted by Sohan Jain <so...@fb.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/853/
-----------------------------------------------------------
(Updated 2011-06-29 20:48:43.096053)
Review request for hive and Paul Yang.
Changes
-------
-fixed Paul's comments
Summary
-------
HIVE-2038 introduced the MetaStoreEventListener abstract class that defines actions to be performed after particular events on a metastore. Improve upon that class by adding events to be performed on "alter table" and "alter partition" actions. Also, update the hive metastore to call the appropriate functions of the listeners when the events happen.
This addresses bug HIVE-2194.
https://issues.apache.org/jira/browse/HIVE-2194
Diffs (updated)
-----
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 1140399
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1140399
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java 1140399
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1140399
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AlterPartitionEvent.java PRE-CREATION
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AlterTableEvent.java PRE-CREATION
trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java 1140399
trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java 1140399
Diff: https://reviews.apache.org/r/853/diff
Testing
-------
Added test cases to TestMetaStoreEventListener.
Thanks,
Sohan
Re: Review Request: HIVE-2194: Add actions to MetaStoreEventListener to be
performed on alter table and alter partition operations.
Posted by Sohan Jain <so...@fb.com>.
> On 2011-06-27 22:47:07, Paul Yang wrote:
> > trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java, lines 142-144
> > <https://reviews.apache.org/r/853/diff/3/?file=21427#file21427line142>
> >
> > Can we check for equality between origP and the old partition instead of comparing fields?
The equals() method fails, because the parameter "transient_lastDdlTime" is sometimes off by one. Otherwise, the two objects are entirely equal. I think the db name, table name, and list of values also ensures equality.
> On 2011-06-27 22:47:07, Paul Yang wrote:
> > trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java, lines 183-194
> > <https://reviews.apache.org/r/853/diff/3/?file=21427#file21427line183>
> >
> > origCols.equals(oldCols)?
Fixed
> On 2011-06-27 22:47:07, Paul Yang wrote:
> > trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java, lines 159-161
> > <https://reviews.apache.org/r/853/diff/3/?file=21427#file21427line159>
> >
> > Same here
The equals() method also fails here, even though the tables are logically equal. In the case above, "tbl" does not have a PrincipalPrivilegeSet. "oldTable" has a PrincipalPrivilegeSet, but all of it's fields are set to null. This discrepancy comes from how we get the tables, but I think they should still be considered equal.
- Sohan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/853/#review920
-----------------------------------------------------------
On 2011-06-21 20:25:04, Sohan Jain wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/853/
> -----------------------------------------------------------
>
> (Updated 2011-06-21 20:25:04)
>
>
> Review request for hive and Paul Yang.
>
>
> Summary
> -------
>
> HIVE-2038 introduced the MetaStoreEventListener abstract class that defines actions to be performed after particular events on a metastore. Improve upon that class by adding events to be performed on "alter table" and "alter partition" actions. Also, update the hive metastore to call the appropriate functions of the listeners when the events happen.
>
>
> This addresses bug HIVE-2194.
> https://issues.apache.org/jira/browse/HIVE-2194
>
>
> Diffs
> -----
>
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 1138144
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138144
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java 1138144
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138144
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AlterPartitionEvent.java PRE-CREATION
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AlterTableEvent.java PRE-CREATION
> trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java 1138144
> trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java 1138144
>
> Diff: https://reviews.apache.org/r/853/diff
>
>
> Testing
> -------
>
> Added test cases to TestMetaStoreEventListener.
>
>
> Thanks,
>
> Sohan
>
>
Re: Review Request: HIVE-2194: Add actions to MetaStoreEventListener to be
performed on alter table and alter partition operations.
Posted by Paul Yang <py...@fb.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/853/#review920
-----------------------------------------------------------
trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java
<https://reviews.apache.org/r/853/#comment1983>
Can we check for equality between origP and the old partition instead of comparing fields?
trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java
<https://reviews.apache.org/r/853/#comment1982>
Same here
trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java
<https://reviews.apache.org/r/853/#comment1981>
origCols.equals(oldCols)?
- Paul
On 2011-06-21 20:25:04, Sohan Jain wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/853/
> -----------------------------------------------------------
>
> (Updated 2011-06-21 20:25:04)
>
>
> Review request for hive and Paul Yang.
>
>
> Summary
> -------
>
> HIVE-2038 introduced the MetaStoreEventListener abstract class that defines actions to be performed after particular events on a metastore. Improve upon that class by adding events to be performed on "alter table" and "alter partition" actions. Also, update the hive metastore to call the appropriate functions of the listeners when the events happen.
>
>
> This addresses bug HIVE-2194.
> https://issues.apache.org/jira/browse/HIVE-2194
>
>
> Diffs
> -----
>
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 1138144
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138144
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java 1138144
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138144
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AlterPartitionEvent.java PRE-CREATION
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AlterTableEvent.java PRE-CREATION
> trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java 1138144
> trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java 1138144
>
> Diff: https://reviews.apache.org/r/853/diff
>
>
> Testing
> -------
>
> Added test cases to TestMetaStoreEventListener.
>
>
> Thanks,
>
> Sohan
>
>
Re: Review Request: HIVE-2194: Add actions to MetaStoreEventListener to be
performed on alter table and alter partition operations.
Posted by Sohan Jain <so...@fb.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/853/
-----------------------------------------------------------
(Updated 2011-06-21 20:25:04.600512)
Review request for hive and Paul Yang.
Changes
-------
-updated test cases to resolve conflicts with new commits to trunk.
Summary
-------
HIVE-2038 introduced the MetaStoreEventListener abstract class that defines actions to be performed after particular events on a metastore. Improve upon that class by adding events to be performed on "alter table" and "alter partition" actions. Also, update the hive metastore to call the appropriate functions of the listeners when the events happen.
This addresses bug HIVE-2194.
https://issues.apache.org/jira/browse/HIVE-2194
Diffs (updated)
-----
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 1138144
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138144
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java 1138144
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138144
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AlterPartitionEvent.java PRE-CREATION
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AlterTableEvent.java PRE-CREATION
trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java 1138144
trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java 1138144
Diff: https://reviews.apache.org/r/853/diff
Testing
-------
Added test cases to TestMetaStoreEventListener.
Thanks,
Sohan
Re: Review Request: HIVE-2194: Add actions to MetaStoreEventListener to be
performed on alter table and alter partition operations.
Posted by Sohan Jain <so...@fb.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/853/
-----------------------------------------------------------
(Updated 2011-06-06 23:23:50.520460)
Review request for hive and Paul Yang.
Changes
-------
- changed interface for alterTable and alterPartition back to void return type
Summary
-------
HIVE-2038 introduced the MetaStoreEventListener abstract class that defines actions to be performed after particular events on a metastore. Improve upon that class by adding events to be performed on "alter table" and "alter partition" actions. Also, update the hive metastore to call the appropriate functions of the listeners when the events happen.
This addresses bug HIVE-2194.
https://issues.apache.org/jira/browse/HIVE-2194
Diffs (updated)
-----
trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java 1130342
trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AlterTableEvent.java PRE-CREATION
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/AlterHandler.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AlterPartitionEvent.java PRE-CREATION
Diff: https://reviews.apache.org/r/853/diff
Testing
-------
Added test cases to TestMetaStoreEventListener.
Thanks,
Sohan