You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by "Parth Brahmbhatt (JIRA)" <ji...@apache.org> on 2015/12/09 03:41:10 UTC
[jira] [Created] (STORM-1381) Client side topology submission hook.
Parth Brahmbhatt created STORM-1381:
---------------------------------------
Summary: Client side topology submission hook.
Key: STORM-1381
URL: https://issues.apache.org/jira/browse/STORM-1381
Project: Apache Storm
Issue Type: New Feature
Components: storm-core
Affects Versions: 0.11.0
Reporter: Parth Brahmbhatt
Assignee: Parth Brahmbhatt
Priority: Trivial
Fix For: 0.11.0
A client side hook is suppose to be invoked when a user submits the topology using TopologySubmitter. We already have nimbus side hook for all the topology actions however those are good if users don't want to actually inspect the topology being submitted or the classes that makes up the topology (spouts and bolts) as on nimbus side these classes are not available in class path.
As a concrete example, in hortonworks we wanted to integrate storm with atlas to provide complete lineage of data even when it passes through a storm topology. Atlas needed to actually look inside the topology components (i.e. kafka spout to figure out what topic the data is being pulled from, or hbase bolt to figure out which cluster and what table data is being pushed into.) to give a meaningful lineage. We originally proposed that they use the server side hook but with that they had to download the user uploaded jar and add it to the class path dynamically or spin a new jvn whose output will then be read by the atlas integration hook.
The client side hook is suppose to make it easy when the topology itself needs to be examined. We are using this in our internal repo for atlas integration.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)