You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@storm.apache.org by Rajeev <ra...@gmail.com> on 2018/01/28 04:35:56 UTC

addConfiguration in flux

Hi,

How can we specify the addConfiguration values that can be used in custom
scheduling under flux ?

For example:
I have a custom scheduler that implements IScheduler.
I have the below under my Topology.java:

TopologyBuilder builder = new TopologyBuilder();
builder.setSpout( "Source_data_amqtest", new MessagingQueueSource()
).addConfiguration( "group", 4 );


How can i specify this addConfiguration( "group", 4 ) for a spout in Flux ?

Regards,
Rajeev.

Re: addConfiguration in flux

Posted by Rajeev <ra...@gmail.com>.

Hi Priyank,

This works great.. thanks..

Regards,
Rajeev.

On Tue, 6 Feb 2018 at 08:37, Priyank Shah <ps...@hortonworks.com> wrote:

> Hi Rajeev,
>
>
>
> In that case may be you can add a method called setGroup on your
> spout/bolt and set the value to 4 in flux using configMethods and then
> override getComponentConfiguration method in your spout/bolt to return a
> map that has the configuration that you would add in first place using
> TopologyBuilder approach. An example is at
> https://github.com/apache/storm/blob/master/external/storm-kafka/src/jvm/org/apache/storm/kafka/KafkaSpout.java#L214
>
>
>
> Can you give it a try and see if it works?
>
>
>
> *From: *Rajeev <ra...@gmail.com>
> *Reply-To: *"user@storm.apache.org" <us...@storm.apache.org>
> *Date: *Monday, February 5, 2018 at 3:45 PM
> *To: *"user@storm.apache.org" <us...@storm.apache.org>
> *Subject: *Re: addConfiguration in flux
>
>
>
> Hi Priyank,
>
>
>
> Thanks for the reply.
>
> I think the configMethods can be used within the spout or bolt codes.. but
> i need to use the configuration for my custom scheduler like how we can do
> it from java topology builder.
>
> Can we do the same in flux ?
>
>
>
> Regards,
>
> Rajeev.
>
>
>
> On Tue, 30 Jan 2018 at 03:32, Priyank Shah <ps...@hortonworks.com> wrote:
>
> Check out http://storm.apache.org/releases/2.0.0-SNAPSHOT/flux.html and
> search for configMethods on that page. It has an example for how to call a
> withFoo method on the bolt instance. You can do the same on
> MessagingQueueSource instance that you can create using component
> definition.
>
>
>
> *From: *Rajeev <ra...@gmail.com>
> *Reply-To: *"user@storm.apache.org" <us...@storm.apache.org>
> *Date: *Saturday, January 27, 2018 at 8:36 PM
> *To: *"user@storm.apache.org" <us...@storm.apache.org>
> *Subject: *addConfiguration in flux
>
>
>
> Hi,
>
>
>
> How can we specify the addConfiguration values that can be used in custom
> scheduling under flux ?
>
>
>
> For example:
> I have a custom scheduler that implements IScheduler.
>
> I have the below under my Topology.java:
>
> TopologyBuilder builder = new TopologyBuilder();
>
> builder.setSpout( "Source_data_amqtest", new MessagingQueueSource()
> ).addConfiguration( "group", 4 );
>
>
>
>
>
> How can i specify this addConfiguration( "group", 4 ) for a spout in Flux ?
>
>
>
> Regards,
>
> Rajeev.
>
>

Re: addConfiguration in flux

Posted by Priyank Shah <ps...@hortonworks.com>.

Hi Rajeev,

In that case may be you can add a method called setGroup on your spout/bolt and set the value to 4 in flux using configMethods and then override getComponentConfiguration method in your spout/bolt to return a map that has the configuration that you would add in first place using TopologyBuilder approach. An example is at https://github.com/apache/storm/blob/master/external/storm-kafka/src/jvm/org/apache/storm/kafka/KafkaSpout.java#L214

Can you give it a try and see if it works?

From: Rajeev <ra...@gmail.com>
Reply-To: "user@storm.apache.org" <us...@storm.apache.org>
Date: Monday, February 5, 2018 at 3:45 PM
To: "user@storm.apache.org" <us...@storm.apache.org>
Subject: Re: addConfiguration in flux

Hi Priyank,

Thanks for the reply.
I think the configMethods can be used within the spout or bolt codes.. but i need to use the configuration for my custom scheduler like how we can do it from java topology builder.
Can we do the same in flux ?

Regards,
Rajeev.

On Tue, 30 Jan 2018 at 03:32, Priyank Shah <ps...@hortonworks.com>> wrote:
Check out http://storm.apache.org/releases/2.0.0-SNAPSHOT/flux.html and search for configMethods on that page. It has an example for how to call a withFoo method on the bolt instance. You can do the same on MessagingQueueSource instance that you can create using component definition.

From: Rajeev <ra...@gmail.com>>
Reply-To: "user@storm.apache.org<ma...@storm.apache.org>" <us...@storm.apache.org>>
Date: Saturday, January 27, 2018 at 8:36 PM
To: "user@storm.apache.org<ma...@storm.apache.org>" <us...@storm.apache.org>>
Subject: addConfiguration in flux

Hi,

How can we specify the addConfiguration values that can be used in custom scheduling under flux ?

For example:
I have a custom scheduler that implements IScheduler.
I have the below under my Topology.java:
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout( "Source_data_amqtest", new MessagingQueueSource() ).addConfiguration( "group", 4 );

How can i specify this addConfiguration( "group", 4 ) for a spout in Flux ?

Regards,
Rajeev.

Re: addConfiguration in flux

Posted by Rajeev <ra...@gmail.com>.

Hi Priyank,

Thanks for the reply.
I think the configMethods can be used within the spout or bolt codes.. but
i need to use the configuration for my custom scheduler like how we can do
it from java topology builder.
Can we do the same in flux ?

Regards,
Rajeev.

On Tue, 30 Jan 2018 at 03:32, Priyank Shah <ps...@hortonworks.com> wrote:

> Check out http://storm.apache.org/releases/2.0.0-SNAPSHOT/flux.html and
> search for configMethods on that page. It has an example for how to call a
> withFoo method on the bolt instance. You can do the same on
> MessagingQueueSource instance that you can create using component
> definition.
>
>
>
> *From: *Rajeev <ra...@gmail.com>
> *Reply-To: *"user@storm.apache.org" <us...@storm.apache.org>
> *Date: *Saturday, January 27, 2018 at 8:36 PM
> *To: *"user@storm.apache.org" <us...@storm.apache.org>
> *Subject: *addConfiguration in flux
>
>
>
> Hi,
>
>
>
> How can we specify the addConfiguration values that can be used in custom
> scheduling under flux ?
>
>
>
> For example:
> I have a custom scheduler that implements IScheduler.
>
> I have the below under my Topology.java:
>
> TopologyBuilder builder = new TopologyBuilder();
>
> builder.setSpout( "Source_data_amqtest", new MessagingQueueSource()
> ).addConfiguration( "group", 4 );
>
>
>
>
>
> How can i specify this addConfiguration( "group", 4 ) for a spout in Flux ?
>
>
>
> Regards,
>
> Rajeev.
>

Re: addConfiguration in flux

Posted by Priyank Shah <ps...@hortonworks.com>.

Check out http://storm.apache.org/releases/2.0.0-SNAPSHOT/flux.html and search for configMethods on that page. It has an example for how to call a withFoo method on the bolt instance. You can do the same on MessagingQueueSource instance that you can create using component definition.

From: Rajeev <ra...@gmail.com>
Reply-To: "user@storm.apache.org" <us...@storm.apache.org>
Date: Saturday, January 27, 2018 at 8:36 PM
To: "user@storm.apache.org" <us...@storm.apache.org>
Subject: addConfiguration in flux

Hi,

How can we specify the addConfiguration values that can be used in custom scheduling under flux ?

For example:
I have a custom scheduler that implements IScheduler.
I have the below under my Topology.java:
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout( "Source_data_amqtest", new MessagingQueueSource() ).addConfiguration( "group", 4 );

How can i specify this addConfiguration( "group", 4 ) for a spout in Flux ?

Regards,
Rajeev.