You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2014/06/18 20:16:25 UTC
[jira] [Commented] (STORM-54) New topology config to specify an
additional path to include on the classpath for workers
[ https://issues.apache.org/jira/browse/STORM-54?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14036069#comment-14036069 ]
ASF GitHub Bot commented on STORM-54:
-------------------------------------
GitHub user knusbaum opened a pull request:
https://github.com/apache/incubator-storm/pull/149
STORM-54: Per-Topology Classpath and Environment for Workers
Storm-54 only mentions a classpath, but I've also included per-topology environment variables here.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/knusbaum/incubator-storm STORM-54
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-storm/pull/149.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #149
----
commit 718255a9201193ca7bc3e43fa191a09949bc482e
Author: Kyle Nusbaum <kn...@yahoo-inc.com>
Date: 2014-06-17T21:36:01Z
Added support for per-topology classpaths
commit 57b5e1a399373449f2fd50f4e503e2a8f91a700a
Author: Kyle Nusbaum <kn...@yahoo-inc.com>
Date: 2014-06-17T22:24:22Z
Merge remote-tracking branch 'community/master' into community-master
commit e1735d2760a7e66d0459968d066cfafb0a9ba1cf
Author: Kyle Nusbaum <kn...@yahoo-inc.com>
Date: 2014-06-17T23:00:43Z
Added support for per-topology environment-variables
commit d00f966098eb1e422d9507334c8d1d475112c9ac
Author: Kyle Nusbaum <kn...@yahoo-inc.com>
Date: 2014-06-18T17:57:00Z
Added tests for per-topology classpath and per-topology environment-variables.
----
> New topology config to specify an additional path to include on the classpath for workers
> -----------------------------------------------------------------------------------------
>
> Key: STORM-54
> URL: https://issues.apache.org/jira/browse/STORM-54
> Project: Apache Storm (Incubating)
> Issue Type: New Feature
> Reporter: James Xu
>
> The use case for this is to avoid having to copy dependencies into the cluster for every deploy. Instead, you deploy them once to the topology-specific path on every worker machine and Storm will include those jars on the classpath of the workers.
> --------------------
> b4hand: I personally like this feature better than https://github.com/nathanmarz/storm/issues/281 as I think it handles large third-party dependencies like HBase better than a JAR specific approach. HBase relies on specific configuration files being in specific locations relative to its root directory. Likewise, it has native libraries that it needs to link against. I think it would be simpler to just add the output of hbase classpath (which contains wildcards) to the storm.yaml config and this should just work. Whereas the addPlatformJar approach doesn't take into account wildcards, config files, or native libraries. Also, since storm already allows adding to the java.libary.path via the storm.yaml, it seems like this approach would be more similar to the existing functionality rather than having two entirely different approaches to handling 3rd party resources.
> I wouldn't mind taking a stab at this issue myself.
--
This message was sent by Atlassian JIRA
(v6.2#6252)