You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Sanjay Radia (JIRA)" <ji...@apache.org> on 2009/09/14 02:20:57 UTC
[jira] Commented: (HADOOP-5640) Allow ServicePlugins to hook
callbacks into key service events
[ https://issues.apache.org/jira/browse/HADOOP-5640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754794#action_12754794 ]
Sanjay Radia commented on HADOOP-5640:
--------------------------------------
>From what I understand, if Thrift was used to do the DN-NN communications then this feature would not be need.
I don't know the scope of doing that, but that would be my first choice in addressing this issue. Since Thrift RPC has replaced client-NN rpc, can't you do the same for the DN to NN rpc?
# Strongly consider using Thrift rpc for NN-DN heartbeats if that is reasonably easy to do.
# Look at the lifecycle Jira (HDFS-326) and explain if there is any overlap on the lifecycle events.
# Explain how you are planning to deliver the events.
** I assume that you do not need any cross machine events (eg NN-DN); if you do please explain why.
** Have you considered using JMX notifications? Hadoop does use JMX so adding JMX notifications should be easy.
# Mark the APIs as Unstable for now.
PS, sorry for the late reply to you jira.
> Allow ServicePlugins to hook callbacks into key service events
> --------------------------------------------------------------
>
> Key: HADOOP-5640
> URL: https://issues.apache.org/jira/browse/HADOOP-5640
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hadoop-5640.txt, hadoop-5640.txt, HADOOP-5640.v2.txt, hadoop-5640.v3.txt
>
>
> HADOOP-5257 added the ability for NameNode and DataNode to start and stop ServicePlugin implementations at NN/DN start/stop. However, this is insufficient integration for some common use cases.
> We should add some functionality for Plugins to subscribe to events generated by the service they're plugging into. Some potential hook points are:
> NameNode:
> - new datanode registered
> - datanode has died
> - exception caught
> - etc?
> DataNode:
> - startup
> - initial registration with NN complete (this is important for HADOOP-4707 to sync up datanode.dnRegistration.name with the NN-side registration)
> - namenode reconnect
> - some block transfer hooks?
> - exception caught
> I see two potential routes for implementation:
> 1) We make an enum for the types of hookpoints and have a general function in the ServicePlugin interface. Something like:
> {code:java}
> enum HookPoint {
> DN_STARTUP,
> DN_RECEIVED_NEW_BLOCK,
> DN_CAUGHT_EXCEPTION,
> ...
> }
> void runHook(HookPoint hp, Object value);
> {code}
> 2) We make classes specific to each "pluggable" as was originally suggested in HADDOP-5257. Something like:
> {code:java}
> class DataNodePlugin {
> void datanodeStarted() {}
> void receivedNewBlock(block info, etc) {}
> void caughtException(Exception e) {}
> ...
> }
> {code}
> I personally prefer option (2) since we can ensure plugin API compatibility at compile-time, and we avoid an ugly switch statement in a runHook() function.
> Interested to hear what people's thoughts are here.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.