You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Ashutosh Chauhan (JIRA)" <ji...@apache.org> on 2011/05/03 19:45:03 UTC
[jira] [Created] (HIVE-2147) Add api to send / receive message to
metastore
Add api to send / receive message to metastore
----------------------------------------------
Key: HIVE-2147
URL: https://issues.apache.org/jira/browse/HIVE-2147
Project: Hive
Issue Type: Improvement
Components: Metastore
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
Fix For: 0.8.0
This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13032573#comment-13032573 ]
Ashutosh Chauhan commented on HIVE-2147:
----------------------------------------
@Carl, Looks like RB is not coming back up. I am not sure if I am the only one not able to create new review request. Don't know with whom to followup for it? In the meantime, Carl if you have comments can you please post them here.
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038907#comment-13038907 ]
Carl Steinbach commented on HIVE-2147:
--------------------------------------
Right, but I think it's policy to also attach the patch to the JIRA ticket (we reference the JIRA ID in the commit message, not the reviewboard request ID). Thanks.
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038946#comment-13038946 ]
jiraposter@reviews.apache.org commented on HIVE-2147:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/738/#review713
-----------------------------------------------------------
trunk/metastore/if/hive_metastore.thrift
<https://reviews.apache.org/r/738/#comment1428>
Having separate calls for sending request and response messages looks unnecessary. A sendMessage() function with separate request and response message types should work just as well, and will help to avoid confusion -- otherwise I think people will assume that receiveMessage is a polling call.
This is starting to look like a general purpose messaging/rpc framework. Is that the intent?
trunk/metastore/if/hive_metastore.thrift
<https://reviews.apache.org/r/738/#comment1429>
Identifying the message type using an integer seems brittle. This won't work if you have more than one application that is firing events at the metastore.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
<https://reviews.apache.org/r/738/#comment1430>
So the event model is that each event may be handled by at most one event handler?
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
<https://reviews.apache.org/r/738/#comment1431>
Please add some DEBUG or TRACE level logging here that indicates which handler consumed a particular event, or if an event was unserviceable.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
<https://reviews.apache.org/r/738/#comment1432>
Semantically this function looks more like "sendRequest" than "receiveMessage" (and "sendMessage" looks more like "fireEvent").
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
<https://reviews.apache.org/r/738/#comment1433>
Checkstyle: you need a space between control flow tokens and open parens.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
<https://reviews.apache.org/r/738/#comment1434>
Nice to have: javadoc.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java
<https://reviews.apache.org/r/738/#comment1435>
canProcessSendMessage() looks like a redundant call. Is there any reason that this can't be be rolled into processSendMessage()?
- Carl
On 2011-05-12 21:03:29, Ashutosh Chauhan wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/738/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-05-12 21:03:29)
bq.
bq.
bq. Review request for hive and Carl Steinbach.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Updated patch to include missing ASF license and generated thrift code.
bq.
bq.
bq. This addresses bug HIVE-2147.
bq. https://issues.apache.org/jira/browse/HIVE-2147
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/metastore/if/hive_metastore.thrift 1102450
bq. trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 1102450
bq. trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1102450
bq. trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 1102450
bq. trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 1102450
bq. trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 1102450
bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1102450
bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1102450
bq. trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1102450
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1102450
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1102450
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 1102450
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java 1102450
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/MessageEvent.java PRE-CREATION
bq. trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java 1102450
bq. trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java 1102450
bq.
bq. Diff: https://reviews.apache.org/r/738/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Updated TestMetaStoreEventListener to test new api.
bq.
bq.
bq. Thanks,
bq.
bq. Ashutosh
bq.
bq.
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040414#comment-13040414 ]
Ashutosh Chauhan commented on HIVE-2147:
----------------------------------------
Carl,
Whats the difference between 'event framework' and 'rpc framework'? From your comments, it seems, you are defining them as follows:
{code}
Event Framework = sendMsg()
RPC Framework = sendMsg() + recvMsg()
{code}
If this is correct, then are you suggesting to remove recvMsg() api from this patch and propose it in another jira ?
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047468#comment-13047468 ]
Paul Yang commented on HIVE-2147:
---------------------------------
I agree with John's suggestion for PARTITION_EVENTS. For this event table, when will rows be dropped? Also, for when partitions are represented using a string, we've followed the convention that they are called partition names. Can we use that for MPartitionSet?
Since MPartitionSet.partVals is a string, we should make it indexed, much like partitionName for the PARTITION table.
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.8.0
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch, hive_2147-2.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13032628#comment-13032628 ]
jiraposter@reviews.apache.org commented on HIVE-2147:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/737/#review666
-----------------------------------------------------------
metastore/src/java/org/apache/hadoop/hive/metastore/events/MessageEvent.java
<https://reviews.apache.org/r/737/#comment1328>
Missing ASF license.
- Carl
On 2011-05-12 19:56:03, Carl Steinbach wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/737/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-05-12 19:56:03)
bq.
bq.
bq. Review request for hive.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Posted for Ashutosh: https://issues.apache.org/jira/secure/attachment/12478073/api-without-thrift.patch
bq.
bq.
bq. This addresses bug HIVE-2147.
bq. https://issues.apache.org/jira/browse/HIVE-2147
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. metastore/if/hive_metastore.thrift 6be3f93
bq. metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1247887
bq. metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 5562ffc
bq. metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 2ce02bb
bq. metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java 75e01ec
bq. metastore/src/java/org/apache/hadoop/hive/metastore/events/MessageEvent.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/737/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Carl
bq.
bq.
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046731#comment-13046731 ]
John Sichi commented on HIVE-2147:
----------------------------------
I think we should close out this JIRA and create a new one specific to the latest approach.
Regarding the JDO change, instead of calling the table DONE_PARTITION_SET, I'd recommend naming it PARTITION_EVENTS, and add an EVENT_TYPE field containing values such as 'LOAD_DONE'. This would allow for later adding new events such as 'LOAD_BEGIN'. Then parameterize the API's accordingly. Also, use EVENT_TIME instead of CREATE_TIME.
It seems like we're going to end up with sequential scans against this table? Or is it going to be possible to index the searches on PARTITION_VALS?
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.8.0
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch, hive_2147-2.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13032624#comment-13032624 ]
jiraposter@reviews.apache.org commented on HIVE-2147:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/737/
-----------------------------------------------------------
Review request for hive.
Summary
-------
Posted for Ashutosh: https://issues.apache.org/jira/secure/attachment/12478073/api-without-thrift.patch
This addresses bug HIVE-2147.
https://issues.apache.org/jira/browse/HIVE-2147
Diffs
-----
metastore/if/hive_metastore.thrift 6be3f93
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1247887
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 5562ffc
metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 2ce02bb
metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java 75e01ec
metastore/src/java/org/apache/hadoop/hive/metastore/events/MessageEvent.java PRE-CREATION
Diff: https://reviews.apache.org/r/737/diff
Testing
-------
Thanks,
Carl
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13032677#comment-13032677 ]
jiraposter@reviews.apache.org commented on HIVE-2147:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/738/
-----------------------------------------------------------
Review request for hive and Carl Steinbach.
Summary
-------
Updated patch to include missing ASF license and generated thrift code.
This addresses bug HIVE-2147.
https://issues.apache.org/jira/browse/HIVE-2147
Diffs
-----
trunk/metastore/if/hive_metastore.thrift 1102450
trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 1102450
trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1102450
trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 1102450
trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 1102450
trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 1102450
trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1102450
trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1102450
trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1102450
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1102450
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1102450
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 1102450
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java 1102450
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/MessageEvent.java PRE-CREATION
trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java 1102450
trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java 1102450
Diff: https://reviews.apache.org/r/738/diff
Testing
-------
Updated TestMetaStoreEventListener to test new api.
Thanks,
Ashutosh
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2147:
-----------------------------------
Attachment: hive_2147-2.patch
Patch which adds specific apis instead of generic ones.
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch, hive_2147-2.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2147:
-----------------------------------
Affects Version/s: 0.8.0
Status: Patch Available (was: Open)
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.8.0
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch, hive_2147-2.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038893#comment-13038893 ]
Ashutosh Chauhan commented on HIVE-2147:
----------------------------------------
Patch on Reviewboard is latest and applies fine. https://reviews.apache.org/r/738/diff/#index_header
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2147:
---------------------------------
Status: Open (was: Patch Available)
@Ashutosh: I left some comments on RB. Thanks.
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041008#comment-13041008 ]
Carl Steinbach commented on HIVE-2147:
--------------------------------------
Yes, that's pretty much my definition of events vs. RPCs. How do you define it?
One advantage of this definition is that it's explicit that event generators don't have to block waiting for a response from a listener. Another advantage is that it's compatible with the model where multiple listeners can handle each event since there's no need to bundle a response from each of them (which also helps to cut down on latency).
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13039249#comment-13039249 ]
jiraposter@reviews.apache.org commented on HIVE-2147:
-----------------------------------------------------
bq. On 2011-05-25 03:43:30, Carl Steinbach wrote:
bq. > trunk/metastore/if/hive_metastore.thrift, line 347
bq. > <https://reviews.apache.org/r/738/diff/1/?file=18685#file18685line347>
bq. >
bq. > Having separate calls for sending request and response messages looks unnecessary. A sendMessage() function with separate request and response message types should work just as well, and will help to avoid confusion -- otherwise I think people will assume that receiveMessage is a polling call.
bq. >
bq. > This is starting to look like a general purpose messaging/rpc framework. Is that the intent?
bq. >
bq. > A sendMessage() function with separate request and response message types should work just as well.
That is correct. But semantically they are different. In sendMessage() user is just notifying Metastore of an event and is not bothered of return value. recvMessage() user is asking for a response for his message. This distinction is further enforced by return types. We could just have one api sendMessage() for both as you suggested, but having distinct apis for sending and receiving makes it easier for client to understand the semantics.
bq. > This is starting to look like a general purpose messaging/rpc framework.
Well general purpose rpc framework would be much more sophisticated. I am not aiming for that.
bq. On 2011-05-25 03:43:30, Carl Steinbach wrote:
bq. > trunk/metastore/if/hive_metastore.thrift, line 348
bq. > <https://reviews.apache.org/r/738/diff/1/?file=18685#file18685line348>
bq. >
bq. > Identifying the message type using an integer seems brittle. This won't work if you have more than one application that is firing events at the metastore.
There are two other alternatives that I thought of before settling on this one.
1) Add specific apis for different message types. This would have made doing this generic api redundant but then this will result in application specific apis in the metastore. E.g., in HCatalog we want to send a message for "set of partitions" telling Metastore to mark them as done. What does finalizePartition() mean in metastore api when Metastore itself is not aware of this concept as this is application specific. This would be confusing.
2) Use enums instead of integer. This will result in similar problem as above though on a lower scale. Enums give compile time safety so we have to define them in Metastore code. Defining application specific enums doesnt look like a good idea because of similar reasons.
bq. On 2011-05-25 03:43:30, Carl Steinbach wrote:
bq. > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 3126
bq. > <https://reviews.apache.org/r/738/diff/1/?file=18694#file18694line3126>
bq. >
bq. > So the event model is that each event may be handled by at most one event handler?
Yes.
bq. On 2011-05-25 03:43:30, Carl Steinbach wrote:
bq. > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 3134
bq. > <https://reviews.apache.org/r/738/diff/1/?file=18694#file18694line3134>
bq. >
bq. > Please add some DEBUG or TRACE level logging here that indicates which handler consumed a particular event, or if an event was unserviceable.
Will add logging.
bq. On 2011-05-25 03:43:30, Carl Steinbach wrote:
bq. > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 3149
bq. > <https://reviews.apache.org/r/738/diff/1/?file=18694#file18694line3149>
bq. >
bq. > Semantically this function looks more like "sendRequest" than "receiveMessage" (and "sendMessage" looks more like "fireEvent").
Same as my very first comment.
bq. On 2011-05-25 03:43:30, Carl Steinbach wrote:
bq. > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 3151
bq. > <https://reviews.apache.org/r/738/diff/1/?file=18694#file18694line3151>
bq. >
bq. > Checkstyle: you need a space between control flow tokens and open parens.
bq. >
will roll this in.
bq. On 2011-05-25 03:43:30, Carl Steinbach wrote:
bq. > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java, line 640
bq. > <https://reviews.apache.org/r/738/diff/1/?file=18696#file18696line640>
bq. >
bq. > Nice to have: javadoc.
Will add.
bq. On 2011-05-25 03:43:30, Carl Steinbach wrote:
bq. > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java, line 86
bq. > <https://reviews.apache.org/r/738/diff/1/?file=18697#file18697line86>
bq. >
bq. > canProcessSendMessage() looks like a redundant call. Is there any reason that this can't be be rolled into processSendMessage()?
bq. >
Event model is every event is handled by atmost one handler. If we roll this in processSendMsg() then we have to make this method return boolean which will tell whether this event got serviced by this handler or not. Then how will it communicate back the actual return value. In case of sendMsg() this is fine, but recvMsg() returns a valid value which is then need to be returned to a client. So, we first ask handler if it can handle the message and then expect a valid return value in processRecvMsg() call.
- Ashutosh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/738/#review713
-----------------------------------------------------------
On 2011-05-12 21:03:29, Ashutosh Chauhan wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/738/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-05-12 21:03:29)
bq.
bq.
bq. Review request for hive and Carl Steinbach.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Updated patch to include missing ASF license and generated thrift code.
bq.
bq.
bq. This addresses bug HIVE-2147.
bq. https://issues.apache.org/jira/browse/HIVE-2147
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/metastore/if/hive_metastore.thrift 1102450
bq. trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 1102450
bq. trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1102450
bq. trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 1102450
bq. trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 1102450
bq. trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 1102450
bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1102450
bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1102450
bq. trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1102450
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1102450
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1102450
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 1102450
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java 1102450
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/MessageEvent.java PRE-CREATION
bq. trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java 1102450
bq. trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java 1102450
bq.
bq. Diff: https://reviews.apache.org/r/738/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Updated TestMetaStoreEventListener to test new api.
bq.
bq.
bq. Thanks,
bq.
bq. Ashutosh
bq.
bq.
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13031476#comment-13031476 ]
Carl Steinbach commented on HIVE-2147:
--------------------------------------
@Ashutosh: can you please post a reviewboard request? Thanks.
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13036282#comment-13036282 ]
Ashutosh Chauhan commented on HIVE-2147:
----------------------------------------
Can someone take a look at this one?
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2147:
-----------------------------------
Status: Patch Available (was: Open)
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047509#comment-13047509 ]
Ashutosh Chauhan commented on HIVE-2147:
----------------------------------------
Paul,
I replied on HIVE-2215. Lets continue the discussion there since this I marked as closed.
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.8.0
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch, hive_2147-2.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2147:
-----------------------------------
Resolution: Won't Fix
Status: Resolved (was: Patch Available)
As suggested, HIVE-2215 has been opened for this.
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 0.8.0
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch, hive_2147-2.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038890#comment-13038890 ]
Carl Steinbach commented on HIVE-2147:
--------------------------------------
@Ashutosh: can you please attach the most recent version of the patch? Thanks.
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2147:
-----------------------------------
Attachment: api-without-thrift.patch
This is a git diff on top of HIVE-2038. As discussed in April contributor meeting, the best way to do this is by using recently introduced hiveobjectRef. This patch doesnt contain thrift generated code to make it easier to review.
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038912#comment-13038912 ]
Ashutosh Chauhan commented on HIVE-2147:
----------------------------------------
I was thinking that if you have some comments and if i need to update the patch then I will generate new patch and attach here. So, was waiting for +1. If you have already reviewed the one on RB then I can attach that here.
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040006#comment-13040006 ]
jiraposter@reviews.apache.org commented on HIVE-2147:
-----------------------------------------------------
bq. On 2011-05-25 03:43:30, Carl Steinbach wrote:
bq. > trunk/metastore/if/hive_metastore.thrift, line 348
bq. > <https://reviews.apache.org/r/738/diff/1/?file=18685#file18685line348>
bq. >
bq. > Identifying the message type using an integer seems brittle. This won't work if you have more than one application that is firing events at the metastore.
bq.
bq. Ashutosh Chauhan wrote:
bq. There are two other alternatives that I thought of before settling on this one.
bq. 1) Add specific apis for different message types. This would have made doing this generic api redundant but then this will result in application specific apis in the metastore. E.g., in HCatalog we want to send a message for "set of partitions" telling Metastore to mark them as done. What does finalizePartition() mean in metastore api when Metastore itself is not aware of this concept as this is application specific. This would be confusing.
bq. 2) Use enums instead of integer. This will result in similar problem as above though on a lower scale. Enums give compile time safety so we have to define them in Metastore code. Defining application specific enums doesnt look like a good idea because of similar reasons.
Another approach would be to serialize the events, and then wrap the serialized event object in a Thrift struct that also contains a field with the fully qualified class name of the event. When the server receives the event struct it would pass the class name to each of the listeners, which then have the option of deserializing the event if they are capable of handling it.
Using qualified event class names would help to eliminate the burden of coordinating the event id namespace, and it would also make it much easier to emit meaningful logging information on the server side.
bq. On 2011-05-25 03:43:30, Carl Steinbach wrote:
bq. > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 3126
bq. > <https://reviews.apache.org/r/738/diff/1/?file=18694#file18694line3126>
bq. >
bq. > So the event model is that each event may be handled by at most one event handler?
bq.
bq. Ashutosh Chauhan wrote:
bq. Yes.
Why not give each listener the opportunity to handle every event? It seems like a pretty conventional use case that you would want to have more than one listener operate on some events. This also seems like a more conventional policy for a pluggable listener framework.
bq. On 2011-05-25 03:43:30, Carl Steinbach wrote:
bq. > trunk/metastore/if/hive_metastore.thrift, line 347
bq. > <https://reviews.apache.org/r/738/diff/1/?file=18685#file18685line347>
bq. >
bq. > Having separate calls for sending request and response messages looks unnecessary. A sendMessage() function with separate request and response message types should work just as well, and will help to avoid confusion -- otherwise I think people will assume that receiveMessage is a polling call.
bq. >
bq. > This is starting to look like a general purpose messaging/rpc framework. Is that the intent?
bq. >
bq.
bq. Ashutosh Chauhan wrote:
bq. >> A sendMessage() function with separate request and response message types should work just as well.
bq. That is correct. But semantically they are different. In sendMessage() user is just notifying Metastore of an event and is not bothered of return value. recvMessage() user is asking for a response for his message. This distinction is further enforced by return types. We could just have one api sendMessage() for both as you suggested, but having distinct apis for sending and receiving makes it easier for client to understand the semantics.
bq.
bq. >> This is starting to look like a general purpose messaging/rpc framework.
bq. Well general purpose rpc framework would be much more sophisticated. I am not aiming for that.
bq. > Well general purpose rpc framework would be much more sophisticated. I am not aiming for that.
Sorry, I shouldn't have said "general purpose", but it does sound like adding an extensible RPC mechanism is one of the goals of this patch. This seems like a significant departure from the goal of HIVE-2038, which I think was more narrowly focused on adding an event framework, or at least that's what I remember discussing at the contrib meeting. RPCs are quite different from events, and if if we add this feature it shouldn't be conflated with events. I think the RPC modifications should be proposed in a separate ticket, and we should keep the focus in this ticket on adding an event framework.
bq. On 2011-05-25 03:43:30, Carl Steinbach wrote:
bq. > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java, line 86
bq. > <https://reviews.apache.org/r/738/diff/1/?file=18697#file18697line86>
bq. >
bq. > canProcessSendMessage() looks like a redundant call. Is there any reason that this can't be be rolled into processSendMessage()?
bq. >
bq.
bq. Ashutosh Chauhan wrote:
bq. Event model is every event is handled by atmost one handler. If we roll this in processSendMsg() then we have to make this method return boolean which will tell whether this event got serviced by this handler or not. Then how will it communicate back the actual return value. In case of sendMsg() this is fine, but recvMsg() returns a valid value which is then need to be returned to a client. So, we first ask handler if it can handle the message and then expect a valid return value in processRecvMsg() call.
I wasn't sure before if these changes were supposed to enable RPCs. Now that that's clear, I don't think this code belongs in MetaStoreEventListener. It belongs in MetaStoreGenericRPCHandler, or something like that. Can you please remove the RPC code, and change the names of the event code (e.g. one way communication from client to server) to make it clear that these are events, and not something more generic like RPCs?
- Carl
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/738/#review713
-----------------------------------------------------------
On 2011-05-12 21:03:29, Ashutosh Chauhan wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/738/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-05-12 21:03:29)
bq.
bq.
bq. Review request for hive and Carl Steinbach.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Updated patch to include missing ASF license and generated thrift code.
bq.
bq.
bq. This addresses bug HIVE-2147.
bq. https://issues.apache.org/jira/browse/HIVE-2147
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/metastore/if/hive_metastore.thrift 1102450
bq. trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 1102450
bq. trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1102450
bq. trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 1102450
bq. trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 1102450
bq. trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 1102450
bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1102450
bq. trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1102450
bq. trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1102450
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1102450
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1102450
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 1102450
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java 1102450
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/MessageEvent.java PRE-CREATION
bq. trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java 1102450
bq. trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java 1102450
bq.
bq. Diff: https://reviews.apache.org/r/738/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Updated TestMetaStoreEventListener to test new api.
bq.
bq.
bq. Thanks,
bq.
bq. Ashutosh
bq.
bq.
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2147) Add api to send / receive message to
metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13031774#comment-13031774 ]
Ashutosh Chauhan commented on HIVE-2147:
----------------------------------------
Looks like RB is down. Trying to create review request since yesterday. No luck.
> Add api to send / receive message to metastore
> ----------------------------------------------
>
> Key: HIVE-2147
> URL: https://issues.apache.org/jira/browse/HIVE-2147
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: api-without-thrift.patch
>
>
> This is follow-up work on HIVE-2038.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira