You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by "P. Taylor Goetz" <pt...@gmail.com> on 2014/11/19 05:27:15 UTC
Re: Survey: dynamic topology construction
+ dev@
I think this is a good idea, and I would support it. It seems there’s a lot of “wheel reinvention” going on. I’ve been thinking about implementing something along these lines as well.
I’d propose opening a JIRA and corresponding feature branch to enable collaboration.
One thing to keep in mind is that what’s being proposed (and what I have in mind) has a lot of similarities to what Yelp has done with pyleus [1]. So we should consider reaching out to Yelp devs to see if they’d be interested in collaborating. Licensing/IP should also be kept in mind.
I envision something a little more “API agnostic” — supporting JVM, multi-lang, and Trident.
I’d be interested in hearing others’ ideas.
-Taylor
[1] https://github.com/Yelp/pyleus
On Nov 18, 2014, at 11:37 AM, c_inconnu <c_...@yahoo.fr> wrote:
> Hi,
> I've also developed for my company a small framework to create a topology from JSON.
> I'm not sure if I'm allowed to open source it but it seems a common requirement.
> Maybe we can join forces.
> David
>
> 2014-11-18 16:42 GMT+01:00 Nick Katsipoulakis <ni...@gmail.com>:
> Hello,
>
> I am also doing something similar at the moment. Nathan, I agree that something like that would be extremely useful.Therefore, to answer your questions:
>
> 1) Yes
>
> 2) Can not find any additional supported features at this point.
>
> Thanks,
> Nikos
>
> 2014-11-18 10:15 GMT-05:00 Supun Kamburugamuva <su...@gmail.com>:
>
> Hi Nathan,
>
> I meant the algorithmic part of the middle bolts. We create the spouts and bolts that receive/send the messages using the configuration. We can create the full topology if the middle bolts are also specified but not the processing part (of course). Still we haven't implemented connecting the middle part because it seems our requirements are changing and we haven't finalized all the requirements etc.
>
> Overall I think this is a very useful feature.
>
> Thanks,
> Supun..
>
>
>
> On Tue, Nov 18, 2014 at 9:41 AM, Nathan Leung <nc...@gmail.com> wrote:
> Hi Supun, what kind of problems have you had with your middle bolts? I have been creating production topologies fully from json but maybe there's a feature I'm not using.
>
>
>
>
> --
> Supun Kamburugamuva
> Member, Apache Software Foundation; http://www.apache.org
> E-mail: supun06@gmail.com; Mobile: +1 812 369 6762
> Blog: http://supunk.blogspot.com
>
>
>
>
> --
> Nikolaos Romanos Katsipoulakis,
> University of Pittsburgh, PhD candidate
>
Re: Survey: dynamic topology construction
Posted by Nathan Leung <nc...@gmail.com>.
Hi everyone,
Thanks for your feedback. I've created a jira:
https://issues.apache.org/jira/browse/STORM-561. I have yet to create a
branch; I need to get a dev environment suitable for storm setup first.
-Nathan
On Tue, Nov 18, 2014 at 11:27 PM, P. Taylor Goetz <pt...@gmail.com> wrote:
> + dev@
>
> I think this is a good idea, and I would support it. It seems there’s a
> lot of “wheel reinvention” going on. I’ve been thinking about implementing
> something along these lines as well.
>
> I’d propose opening a JIRA and corresponding feature branch to enable
> collaboration.
>
> One thing to keep in mind is that what’s being proposed (and what I have
> in mind) has a lot of similarities to what Yelp has done with pyleus [1].
> So we should consider reaching out to Yelp devs to see if they’d be
> interested in collaborating. Licensing/IP should also be kept in mind.
>
> I envision something a little more “API agnostic” — supporting JVM,
> multi-lang, and Trident.
>
> I’d be interested in hearing others’ ideas.
>
> -Taylor
>
> [1] https://github.com/Yelp/pyleus
>
> On Nov 18, 2014, at 11:37 AM, c_inconnu <c_...@yahoo.fr> wrote:
>
> Hi,
> I've also developed for my company a small framework to create a topology
> from JSON.
> I'm not sure if I'm allowed to open source it but it seems a common
> requirement.
> Maybe we can join forces.
> David
>
> 2014-11-18 16:42 GMT+01:00 Nick Katsipoulakis <ni...@gmail.com>:
>
>> Hello,
>>
>> I am also doing something similar at the moment. Nathan, I agree that
>> something like that would be extremely useful.Therefore, to answer your
>> questions:
>>
>> 1) Yes
>>
>> 2) Can not find any additional supported features at this point.
>>
>> Thanks,
>> Nikos
>>
>> 2014-11-18 10:15 GMT-05:00 Supun Kamburugamuva <su...@gmail.com>:
>>
>> Hi Nathan,
>>>
>>> I meant the algorithmic part of the middle bolts. We create the spouts
>>> and bolts that receive/send the messages using the configuration. We can
>>> create the full topology if the middle bolts are also specified but not the
>>> processing part (of course). Still we haven't implemented connecting the
>>> middle part because it seems our requirements are changing and we haven't
>>> finalized all the requirements etc.
>>>
>>> Overall I think this is a very useful feature.
>>>
>>> Thanks,
>>> Supun..
>>>
>>>
>>>
>>> On Tue, Nov 18, 2014 at 9:41 AM, Nathan Leung <nc...@gmail.com> wrote:
>>>
>>>> Hi Supun, what kind of problems have you had with your middle bolts? I
>>>> have been creating production topologies fully from json but maybe there's
>>>> a feature I'm not using.
>>>>
>>>
>>>
>>>
>>> --
>>> Supun Kamburugamuva
>>> Member, Apache Software Foundation; http://www.apache.org
>>> E-mail: supun06@gmail.com; Mobile: +1 812 369 6762
>>> Blog: http://supunk.blogspot.com
>>>
>>>
>>
>>
>> --
>> Nikolaos Romanos Katsipoulakis,
>> University of Pittsburgh, PhD candidate
>>
>
>
>