You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by amir bahmanyari <am...@yahoo.com> on 2016/05/22 21:56:36 UTC
Convert a Storm Topology into a Apache Beam Model
Hi ColleaguesI was looking for information on the above topic & I thought I found it!!! But,
I got a page not found :-(http://stackoverflow.com/questions/36640458/are-all-the-trident-operations-supported-by-apache-beam-model
Is there a guideline on how to convert an existing Storm topology (Spout/Bolt) to its equivalant implemented in Beam?Thanks for your valuable timeAmir-
Re: Convert a Storm Topology into a Apache Beam Model
Posted by Tyler Akidau <ta...@google.com>.
Kaniska is right that essentially anything you can describe in Storm is
describable in Beam.
One subtle difference to be aware of: a couple of the more esoteric stream
grouping options in Storm don't have direct analogs in Beam. E.g. the "all"
grouping, used to broadcast a value to all tasks of a downstream bolt,
doesn't have an identical feature in Beam. For broadcasting data to all
instances of a DoFn, side inputs are typically used. But the semantics are
slightly different, in that side inputs are only accessible when a main
input record arrives. Whereas in Storm, a record sent with "all" grouping
will immediately trigger processing of that record in the downstream bolt
tasks.
-Tyler
On Mon, May 23, 2016 at 4:15 PM kaniska Mandal <ka...@gmail.com>
wrote:
> Filtering, Aggregation, Grouping, Merging, Joining, and all other
> operations which are possible by Storm are supported by Apache Beam.
>
> following tutorial provides all the details -
> https://cloud.google.com/dataflow/model/programming-model
>
> just in case you want to run storm topologies in flink -
> https://flink.apache.org/news/2015/12/11/storm-compatibility.html
>
> On Sun, May 22, 2016 at 2:56 PM, amir bahmanyari <am...@yahoo.com>
> wrote:
>
>> Hi Colleagues
>> I was looking for information on the above topic & I thought I found
>> it!!! But,
>>
>> I got a page not found :-(
>>
>> http://stackoverflow.com/questions/36640458/are-all-the-trident-operations-supported-by-apache-beam-model
>>
>> Is there a guideline on how to convert an existing Storm topology
>> (Spout/Bolt) to its equivalant implemented in Beam?
>> Thanks for your valuable time
>> Amir-
>>
>
>
Re: Convert a Storm Topology into a Apache Beam Model
Posted by kaniska Mandal <ka...@gmail.com>.
Filtering, Aggregation, Grouping, Merging, Joining, and all other
operations which are possible by Storm are supported by Apache Beam.
following tutorial provides all the details -
https://cloud.google.com/dataflow/model/programming-model
just in case you want to run storm topologies in flink -
https://flink.apache.org/news/2015/12/11/storm-compatibility.html
On Sun, May 22, 2016 at 2:56 PM, amir bahmanyari <am...@yahoo.com>
wrote:
> Hi Colleagues
> I was looking for information on the above topic & I thought I found it!!!
> But,
>
> I got a page not found :-(
>
> http://stackoverflow.com/questions/36640458/are-all-the-trident-operations-supported-by-apache-beam-model
>
> Is there a guideline on how to convert an existing Storm topology
> (Spout/Bolt) to its equivalant implemented in Beam?
> Thanks for your valuable time
> Amir-
>