Storm 2 features?

["Hannum, Daniel" <Da...@PremierInc.com>]

Is there somewhere that summarizes what Storm 2 will bring to the table? I’ve heard vague things like it will be X % faster, but in terms of new capabilities, I don’t know what it has. There are things I wish Storm could do, like dynamically scaling bolts under load, but I don’t know if Storm 2 will include any of that.

Thanks
Dan

Re: Storm 2 features?

[Jungtaek Lim <ka...@gmail.com>]

Sorry I visit too late. Sadly there're lots of things for me to handle.

To summarize the changes of Storm 2.0.0:

- Clojure to Java translation
This is origin goal of Storm 2.0.0, though it is taking too much time than
expected. UI and tests are not covered yet, and I hope we could translate
UI in 2.0.0. (We may be able to postpone to 2.1.0 there's valid reason.)

- Splitting storm-core to storm-client / storm-server / storm-webapp / etc.
We split storm-core (huge one) to several modules to reduce dependencies to
the user side. We don't address shading some dependencies, so it should be
discussed and handled before releasing Storm 2.0.0.

- Streams API
Storm 2.0.0 will support typed declarative API based on record to record
processing. (We will continue supporting compositional API too.)
Typically, Trident is also declarative API, but it is not typed and based
on micro-batch processing.
https://github.com/apache/storm/blob/master/docs/Stream-API.md

Please note that this API will be released under 'experimental', and
hopefully we could add missing features and make it stable sooner.

- Improvements on Resource Aware Scheduler
I'm not expert on RAS, so someone in Oath can explain the improvements it
better.

- Improvements on LoadAwareShuffleGrouping
While it only considered loads betwen tasks while distributing tuples, now
it also considers locality (worker, host, rack, other) too.

- Messaging subsystem redesign (WIP)
Rationalization, design doc, and performance analysis is available:
https://issues.apache.org/jira/browse/STORM-2306

- Metrics V2 (WIP)
We are about to introduce metrics V2 API which is based on DropWizard
metrics. For now it is not a drop-in replacement of Metrics V1 because
built-in metrics and UI still refer to Metrics V1, but it will eventually
replace Metrics V1.

Thanks,
Jungtaek Lim (HeartSaVioR)

2017년 12월 13일 (수) 오전 9:34, Stephen Powis <sp...@salesforce.com>님이 작성:

> Not a core contributor, so maybe there's a better source for this info,
> but you can always see what tickets are set against the 2.0.0 release in
> jira
>
>
> https://issues.apache.org/jira/browse/STORM-2850?jql=project%20%3D%20STORM%20AND%20fixVersion%20%3D%202.0.0
>
> On Wed, Dec 13, 2017 at 3:13 AM, Hannum, Daniel <
> Daniel_Hannum@premierinc.com> wrote:
>
>> Is there somewhere that summarizes what Storm 2 will bring to the table?
>> I’ve heard vague things like it will be X % faster, but in terms of new
>> capabilities, I don’t know what it has. There are things I wish Storm could
>> do, like dynamically scaling bolts under load, but I don’t know if Storm 2
>> will include any of that.
>>
>>
>>
>> Thanks
>>
>> Dan
>>
>
>

Re: Storm 2 features?

[Stephen Powis <sp...@salesforce.com>]

Not a core contributor, so maybe there's a better source for this info, but
you can always see what tickets are set against the 2.0.0 release in jira

https://issues.apache.org/jira/browse/STORM-2850?jql=project%20%3D%20STORM%20AND%20fixVersion%20%3D%202.0.0

On Wed, Dec 13, 2017 at 3:13 AM, Hannum, Daniel <
Daniel_Hannum@premierinc.com> wrote:

> Is there somewhere that summarizes what Storm 2 will bring to the table?
> I’ve heard vague things like it will be X % faster, but in terms of new
> capabilities, I don’t know what it has. There are things I wish Storm could
> do, like dynamically scaling bolts under load, but I don’t know if Storm 2
> will include any of that.
>
>
>
> Thanks
>
> Dan
>