You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@storm.apache.org by "Roshan Naik (JIRA)" <ji...@apache.org> on 2018/02/17 09:07:00 UTC

[jira] [Closed] (STORM-2310) Handling Back Pressure

     [ https://issues.apache.org/jira/browse/STORM-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Roshan Naik closed STORM-2310.
------------------------------
    Resolution: Fixed
      Assignee: Roshan Naik

The basic backpressure model is implemented as part of STORM-2306. 

The design doc here proposes an extension to the basic model which allows for buffering a much larger backlog of messages during BP situations. My experience with STORM-2306 suggests the basic BP model with shorter queues is preferable to this extended model due to the following reasons:
 * Having more messages waiting in deeper queues only worsens amount of time it takes for the messages to make it through all the bolts and thus increases latency (spout ->...-> terminal bolt) with no added throughput benefit.
 * When worker crashes: 
 ** If ACKing is disabled, leads to a bigger loss of messages.
 ** If ACKing is enabled, the amount of reprocessing required is quite high.

Consequently changing my mind and closing this JIRA.

> Handling Back Pressure
> ----------------------
>
>                 Key: STORM-2310
>                 URL: https://issues.apache.org/jira/browse/STORM-2310
>             Project: Apache Storm
>          Issue Type: Sub-task
>          Components: storm-core
>    Affects Versions: 2.0.0
>            Reporter: Roshan Naik
>            Assignee: Roshan Naik
>            Priority: Major
>
> Design Doc:
> https://docs.google.com/document/d/1btmpBpFeEl-bh1uhQ_W4Ao-cQK7muNBYbXXMwuc4YaU/edit?usp=sharing



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)