You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by I PVP <ip...@hotmail.com> on 2016/04/04 18:45:10 UTC
Topology Scheduling
Is it possible to schedule a topology to run every X hours ? How?
Thanks
--
IPVP
Re: Topology Scheduling
Posted by Spico Florin <sp...@gmail.com>.
Hi!
Please have a look at storm signals. Your topology runs forever, but you
can use storm signals to activate/deactivate the spout.
https://github.com/ptgoetz/storm-signals
I hope that it helps.
Florin
On Thu, Apr 7, 2016 at 10:13 AM, jinhong lu <lu...@gmail.com> wrote:
> Although the scene is wired, but I think you can use crontab to run ‘storm
> jar’ and ‘storm kill’ every several hours.
> But why storm? I believe MR and spark is a better choice for this.
>
> 在 2016年4月7日，14:47，Siddartha Guthikonda <si...@gmail.com> 写道：
>
> Maybe trident batch processing would help. You need to keep the batch open
> for x hours and then process it.
>
> On Wed, Apr 6, 2016 at 6:20 PM, wu lihu <ro...@gmail.com> wrote:
>
>> I don't think it's possible ? Since topology will keeping running
>> until you kill it. I think you could do it through your Spout ??
>>
>> 2016-04-06 22:21 GMT+08:00 Andrey Dudin <do...@gmail.com>:
>> > Try Quartz library.
>> >
>> > 2016-04-04 19:45 GMT+03:00 I PVP <ip...@hotmail.com>:
>> >>
>> >> Is it possible to schedule a topology to run every X hours ? How?
>> >>
>> >> Thanks
>> >>
>> >> --
>> >> IPVP
>> >>
>> >
>> >
>> >
>> > --
>> > С уважением Дудин Андрей
>>
>
>
>
Re: Topology Scheduling
Posted by jinhong lu <lu...@gmail.com>.
Although the scene is wired, but I think you can use crontab to run ‘storm jar’ and ‘storm kill’ every several hours.
But why storm? I believe MR and spark is a better choice for this.
> 在 2016年4月7日，14:47，Siddartha Guthikonda <si...@gmail.com> 写道：
>
> Maybe trident batch processing would help. You need to keep the batch open for x hours and then process it.
>
> On Wed, Apr 6, 2016 at 6:20 PM, wu lihu <routermanwulihu@gmail.com <ma...@gmail.com>> wrote:
> I don't think it's possible ? Since topology will keeping running
> until you kill it. I think you could do it through your Spout ??
>
> 2016-04-06 22:21 GMT+08:00 Andrey Dudin <doodin201@gmail.com <ma...@gmail.com>>:
> > Try Quartz library.
> >
> > 2016-04-04 19:45 GMT+03:00 I PVP <ipvp@hotmail.com <ma...@hotmail.com>>:
> >>
> >> Is it possible to schedule a topology to run every X hours ? How?
> >>
> >> Thanks
> >>
> >> --
> >> IPVP
> >>
> >
> >
> >
> > --
> > С уважением Дудин Андрей
>
Re: Topology Scheduling
Posted by Siddartha Guthikonda <si...@gmail.com>.
Maybe trident batch processing would help. You need to keep the batch open
for x hours and then process it.
On Wed, Apr 6, 2016 at 6:20 PM, wu lihu <ro...@gmail.com> wrote:
> I don't think it's possible ? Since topology will keeping running
> until you kill it. I think you could do it through your Spout ??
>
> 2016-04-06 22:21 GMT+08:00 Andrey Dudin <do...@gmail.com>:
> > Try Quartz library.
> >
> > 2016-04-04 19:45 GMT+03:00 I PVP <ip...@hotmail.com>:
> >>
> >> Is it possible to schedule a topology to run every X hours ? How?
> >>
> >> Thanks
> >>
> >> --
> >> IPVP
> >>
> >
> >
> >
> > --
> > С уважением Дудин Андрей
>
Re: Topology Scheduling
Posted by wu lihu <ro...@gmail.com>.
I don't think it's possible ? Since topology will keeping running
until you kill it. I think you could do it through your Spout ??
2016-04-06 22:21 GMT+08:00 Andrey Dudin <do...@gmail.com>:
> Try Quartz library.
>
> 2016-04-04 19:45 GMT+03:00 I PVP <ip...@hotmail.com>:
>>
>> Is it possible to schedule a topology to run every X hours ? How?
>>
>> Thanks
>>
>> --
>> IPVP
>>
>
>
>
> --
> С уважением Дудин Андрей
Re: Topology Scheduling
Posted by Andrey Dudin <do...@gmail.com>.
Try Quartz library.
2016-04-04 19:45 GMT+03:00 I PVP <ip...@hotmail.com>:
> Is it possible to schedule a topology to run every X hours ? How?
>
> Thanks
>
> --
> IPVP
>
>
--
С уважением Дудин Андрей
Re: Topology Scheduling
Posted by Matthew Lowe <gi...@gmail.com>.
Configure tick tuples?
What's are you trying to do?
> On 04 Apr 2016, at 18:57, Aurelien Violette <au...@webgroup-limited.com> wrote:
>
> Hi,
>
> What is your use case ? Because this seems to be the contrary of the goal of Storm and sounds more like Map Reduce alike job.
>
> Anyway, I've never seen such thing in Storm.
>
> Le 04/04/2016 18:45, I PVP a écrit :
>> Is it possible to schedule a topology to run every X hours ? How?
>>
>> Thanks
>>
>> --
>> IPVP
>
> --
> BR,
> Aurelien Violette
Re: Topology Scheduling
Posted by Aurelien Violette <au...@webgroup-limited.com>.
Hi,
What is your use case ? Because this seems to be the contrary of the
goal of Storm and sounds more like Map Reduce alike job.
Anyway, I've never seen such thing in Storm.
Le 04/04/2016 18:45, I PVP a écrit :
> Is it possible to schedule a topology to run every X hours ? How?
>
> Thanks
>
> --
> IPVP
>
--
BR,
Aurelien Violette