[jira] [Updated] (STORM-54) New topology config to specify an additional path to include on the classpath for workers

["Rick Kellogg (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/STORM-54?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rick Kellogg updated STORM-54:
------------------------------
    Component/s: storm-core

> New topology config to specify an additional path to include on the classpath for workers
> -----------------------------------------------------------------------------------------
>
>                 Key: STORM-54
>                 URL: https://issues.apache.org/jira/browse/STORM-54
>             Project: Apache Storm
>          Issue Type: New Feature
>          Components: storm-core
>            Reporter: James Xu
>            Assignee: Kyle Nusbaum
>             Fix For: 0.9.3
>
>
> The use case for this is to avoid having to copy dependencies into the cluster for every deploy. Instead, you deploy them once to the topology-specific path on every worker machine and Storm will include those jars on the classpath of the workers.
> --------------------
> b4hand: I personally like this feature better than https://github.com/nathanmarz/storm/issues/281 as I think it handles large third-party dependencies like HBase better than a JAR specific approach. HBase relies on specific configuration files being in specific locations relative to its root directory. Likewise, it has native libraries that it needs to link against. I think it would be simpler to just add the output of hbase classpath (which contains wildcards) to the storm.yaml config and this should just work. Whereas the addPlatformJar approach doesn't take into account wildcards, config files, or native libraries. Also, since storm already allows adding to the java.libary.path via the storm.yaml, it seems like this approach would be more similar to the existing functionality rather than having two entirely different approaches to handling 3rd party resources.
> I wouldn't mind taking a stab at this issue myself.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)