You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Ryan Chan <ry...@gmail.com> on 2014/01/25 17:31:01 UTC
No way to specify config file (storm.yml) location
There is an old post also talking about this and no useful reply (1+ year
ago):
https://groups.google.com/forum/#!msg/storm-user/51wualp1xUk/ywkWaMWiLCcJ
It mentioned to use the following option if we want to launch the
supervisor, e.g.
java -client -Dstorm.options= -Dstorm.conf.file=/app/shared/
storm/storm.yaml supervisor
However, this option require the storm.yaml must be in the classpath,
however, even I specified the classpath, I still not able to use that
config file, like the OP, so this issue is not doable right?
Re: No way to specify config file (storm.yml) location
Posted by Ryan Chan <ry...@gmail.com>.
Jeff,
Thanks. This script is useful, please make it merge into the official
release.
Just a nitpick, the workers are still using the default logback
configuration so they still run into default logs folder.
On Mon, Jan 27, 2014 at 10:48 AM, Jeff Potter
<jp...@codepuppy.com>wrote:
>
> Hi Ryan --
>
> We ended up patching both storm 0.8 and storm 0.9 for this -- here's the
> patch I wrote for 0.9. (This has been on my to-do list to get to Nathan, so
> thanks for the reminder.)
>
> This adds a "--conf-dir" option to ./bin/storm that you can then point at
> a dir that contains storm.yaml; and also adds a --logback-file option that
> you can then point at your log back file.
>
> In our setup, the storm src code is read-only to the runtime, and the dir
> paths that are writeable (for logging, or for generating the storm.yaml
> from a template) aren't known at install time.
>
> best,
> Jeff
>
>
> 25a26
> > LOGBACK_FILE = STORM_DIR + "/logback/cluster.xml"
> 254c255
> < "-Dlogback.configurationFile=" + STORM_DIR +
> "/logback/cluster.xml",
> ---
> > "-Dlogback.configurationFile=" + LOGBACK_FILE
> 274c275
> < "-Dlogback.configurationFile=" + STORM_DIR +
> "/logback/cluster.xml",
> ---
> > "-Dlogback.configurationFile=" + LOGBACK_FILE
> 295c296
> < "-Dlogback.configurationFile=" + STORM_DIR +
> "/logback/cluster.xml",
> ---
> > "-Dlogback.configurationFile=" + LOGBACK_FILE
> 316c317
> < "-Dlogback.configurationFile=" + STORM_DIR +
> "/logback/cluster.xml",
> ---
> > "-Dlogback.configurationFile=" + LOGBACK_FILE
> 336c337
> < "-Dlogback.configurationFile=" + STORM_DIR +
> "/logback/cluster.xml"
> ---
> > "-Dlogback.configurationFile=" + LOGBACK_FILE
> 421a423,428
> > elif token == "--logback-file":
> > global LOGBACK_FILE
> > LOGBACK_FILE = curr.pop()
> > elif token == "--conf-dir":
> > global USER_CONF_DIR, CLUSTER_CONF_DIR
> > USER_CONF_DIR = CLUSTER_CONF_DIR = curr.pop()
>
>
> On Jan 25, 2014, at 11:31 AM, Ryan Chan <ry...@gmail.com> wrote:
>
> There is an old post also talking about this and no useful reply (1+ year
> ago):
> https://groups.google.com/forum/#!msg/storm-user/51wualp1xUk/ywkWaMWiLCcJ
>
> It mentioned to use the following option if we want to launch the
> supervisor, e.g.
>
> java -client -Dstorm.options= -Dstorm.conf.file=/app/shared/
> storm/storm.yaml supervisor
>
> However, this option require the storm.yaml must be in the classpath,
> however, even I specified the classpath, I still not able to use that
> config file, like the OP, so this issue is not doable right?
>
>
>
>
>
>
>
Re: No way to specify config file (storm.yml) location
Posted by Jeff Potter <jp...@codepuppy.com>.
Hi Ryan —
We ended up patching both storm 0.8 and storm 0.9 for this — here’s the patch I wrote for 0.9. (This has been on my to-do list to get to Nathan, so thanks for the reminder.)
This adds a “--conf-dir” option to ./bin/storm that you can then point at a dir that contains storm.yaml; and also adds a --logback-file option that you can then point at your log back file.
In our setup, the storm src code is read-only to the runtime, and the dir paths that are writeable (for logging, or for generating the storm.yaml from a template) aren’t known at install time.
best,
Jeff
25a26
> LOGBACK_FILE = STORM_DIR + "/logback/cluster.xml"
254c255
< "-Dlogback.configurationFile=" + STORM_DIR + "/logback/cluster.xml",
---
> "-Dlogback.configurationFile=" + LOGBACK_FILE
274c275
< "-Dlogback.configurationFile=" + STORM_DIR + "/logback/cluster.xml",
---
> "-Dlogback.configurationFile=" + LOGBACK_FILE
295c296
< "-Dlogback.configurationFile=" + STORM_DIR + "/logback/cluster.xml",
---
> "-Dlogback.configurationFile=" + LOGBACK_FILE
316c317
< "-Dlogback.configurationFile=" + STORM_DIR + "/logback/cluster.xml",
---
> "-Dlogback.configurationFile=" + LOGBACK_FILE
336c337
< "-Dlogback.configurationFile=" + STORM_DIR + "/logback/cluster.xml"
---
> "-Dlogback.configurationFile=" + LOGBACK_FILE
421a423,428
> elif token == "--logback-file":
> global LOGBACK_FILE
> LOGBACK_FILE = curr.pop()
> elif token == "--conf-dir":
> global USER_CONF_DIR, CLUSTER_CONF_DIR
> USER_CONF_DIR = CLUSTER_CONF_DIR = curr.pop()
On Jan 25, 2014, at 11:31 AM, Ryan Chan <ry...@gmail.com> wrote:
> There is an old post also talking about this and no useful reply (1+ year ago): https://groups.google.com/forum/#!msg/storm-user/51wualp1xUk/ywkWaMWiLCcJ
>
> It mentioned to use the following option if we want to launch the supervisor, e.g.
>
> java -client -Dstorm.options= -Dstorm.conf.file=/app/shared/storm/storm.yaml supervisor
>
> However, this option require the storm.yaml must be in the classpath, however, even I specified the classpath, I still not able to use that config file, like the OP, so this issue is not doable right?
>
>
>
>
>