Is there a way to register metrics in flux yaml

[Marc Zbyszynski <mm...@gmail.com>]

Hello Everyone,

I am using flux to deploy multilang topologies with a lot of FluxShellBolt
instances. Do you know if there is any way to register a metric directly in
the flux yaml without writing any Java code? I know that the multilang
protocol supports submitting data to metrics that have been registered, but
I don't see any way to register the metrics in Flux or by invoking a method
on FluxShellBolt. Is that right, or am I missing something?

Thank you for your help!

-Marc

Re: Is there a way to register metrics in flux yaml

[Romeo Nocon <ro...@gmail.com>]

Hi,

I'm using an older version of storm (0.9.4) and flux so I'm not sure this
will work on newer versions.  I also don't know anythign about
FluxShellBolts.  That said I put "topology.metrics.consumer.register" in
the config section of flux file. Yes it's a hack.

config
  ...

  topology.metrics.consumer.register:

  - class: "com.company.GraphiteMetricsConsumer"

    parallelism.hint: ${topo.metrics.graphite.parallelism}


Then I let the code do the rest.  Below is a quick grep of some relevant
files.

grep -ir topology.metrics.consumer.register *

storm-core/src/jvm/org/apache/storm/Config.java:    public static final
String TOPOLOGY_METRICS_CONSUMER_REGISTER =
"topology.metrics.consumer.register";

storm-core/src/jvm/org/apache/storm/Config.java:        List l =
(List)conf.get(TOPOLOGY_METRICS_CONSUMER_REGISTER);

storm-core/src/jvm/org/apache/storm/Config.java:
conf.put(TOPOLOGY_METRICS_CONSUMER_REGISTER, l);

storm-core/src/jvm/org/apache/storm/daemon/StormCommon.java:
List<Map<String, Object>> registerInfo = (List<Map<String, Object>>)
conf.get(Config.TOPOLOGY_METRICS_CONSUMER_REGISTER);

On Wed, Feb 15, 2017 at 11:24 AM, P. Taylor Goetz <pt...@gmail.com> wrote:

> Hi Marc,
>
> Currently there is not a way to directly register metrics using Flux. It
> is something that is probably doable, though. Feel free to file a JIRA to
> add this functionality.
>
> -Taylor
>
> > On Feb 15, 2017, at 1:49 PM, Marc Zbyszynski <mm...@gmail.com> wrote:
> >
> > Hello Everyone,
> >
> > I am using flux to deploy multilang topologies with a lot of
> FluxShellBolt instances. Do you know if there is any way to register a
> metric directly in the flux yaml without writing any Java code? I know that
> the multilang protocol supports submitting data to metrics that have been
> registered, but I don't see any way to register the metrics in Flux or by
> invoking a method on FluxShellBolt. Is that right, or am I missing
> something?
> >
> > Thank you for your help!
> >
> > -Marc
>
>

Re: Is there a way to register metrics in flux yaml

[Jungtaek Lim <ka...@gmail.com>]

That seems not possible even using ShellSpout/ShellBolt directly, since
there's no way to pass metrics to register. I tracked the history, and
registering method was there but got removed long time ago.

If we can define the way correctly, Flux might be able to register metrics
too.

Seems like we didn't file this issue, so I'll file one.

- Jungtaek Lim (HeartSaVioR)

2017년 2월 16일 (목) 오전 4:24, P. Taylor Goetz <pt...@gmail.com>님이 작성:

> Hi Marc,
>
> Currently there is not a way to directly register metrics using Flux. It
> is something that is probably doable, though. Feel free to file a JIRA to
> add this functionality.
>
> -Taylor
>
> > On Feb 15, 2017, at 1:49 PM, Marc Zbyszynski <mm...@gmail.com> wrote:
> >
> > Hello Everyone,
> >
> > I am using flux to deploy multilang topologies with a lot of
> FluxShellBolt instances. Do you know if there is any way to register a
> metric directly in the flux yaml without writing any Java code? I know that
> the multilang protocol supports submitting data to metrics that have been
> registered, but I don't see any way to register the metrics in Flux or by
> invoking a method on FluxShellBolt. Is that right, or am I missing
> something?
> >
> > Thank you for your help!
> >
> > -Marc
>
>

Re: Is there a way to register metrics in flux yaml

["P. Taylor Goetz" <pt...@gmail.com>]

Hi Marc,

Currently there is not a way to directly register metrics using Flux. It is something that is probably doable, though. Feel free to file a JIRA to add this functionality.

-Taylor

> On Feb 15, 2017, at 1:49 PM, Marc Zbyszynski <mm...@gmail.com> wrote:
> 
> Hello Everyone,
> 
> I am using flux to deploy multilang topologies with a lot of FluxShellBolt instances. Do you know if there is any way to register a metric directly in the flux yaml without writing any Java code? I know that the multilang protocol supports submitting data to metrics that have been registered, but I don't see any way to register the metrics in Flux or by invoking a method on FluxShellBolt. Is that right, or am I missing something?
> 
> Thank you for your help!
> 
> -Marc