[jira] [Created] (STORM-815) Auto tuning topologies based on user submitted SLA

["Read Sprabery (JIRA)" <ji...@apache.org>]

Read Sprabery created STORM-815:
-----------------------------------

             Summary: Auto tuning topologies based on user submitted SLA
                 Key: STORM-815
                 URL: https://issues.apache.org/jira/browse/STORM-815
             Project: Apache Storm
          Issue Type: New Feature
            Reporter: Read Sprabery
            Priority: Blocker


Hi all – not sure if this is the place for this or not, so please feel free to move this ticket around.

I have been working on making Storm SLA aware and wanted to create a ticket focused on auto-tuning of submitted topologies. Having talked with [~revans2] , I think this feature will go a long way in improving the usability of Storm.

This will be particularly useful as more abstractions are developed on top of Storm and developers move further away from the internals of topologies with systems like Summingbird (https://github.com/twitter/summingbird). Storm already has the majority of the metrics required to made decisions regarding the stability of a given stream in Zookeeper; these can be used to decide if a given SLA is being met.

The system I'm working on functions externally to Storm, getting measurements from the Thrift interface and then performing parallelism level tuning through the “rebalance” command. If the community is interested, I can upload a document in the upcoming weeks highlighting the challenges and benefits of an auto-tuning system in Storm and that can serve as a first start for developing an auto-tuning scheduler.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)