[jira] [Commented] (STORM-329) Add Option to Config Message handling strategy when connection timeout

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/STORM-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14146278#comment-14146278 ] 

ASF GitHub Bot commented on STORM-329:
--------------------------------------

GitHub user tedxia opened a pull request:

    https://github.com/apache/storm/pull/268

    STORM-329 : buffer message in client and reconnect remote server async

    

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/tedxia/incubator-storm master

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/storm/pull/268.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #268
    
----
commit 8c52a5b518021a6beff372acbeb66a963a1d4f74
Author: xiajun <xi...@xiaomi.com>
Date:   2014-09-24T12:39:18Z

    STORM-329 : buffer message in client and reconnect remote server async

----


> Add Option to Config Message handling strategy when connection timeout
> ----------------------------------------------------------------------
>
>                 Key: STORM-329
>                 URL: https://issues.apache.org/jira/browse/STORM-329
>             Project: Apache Storm
>          Issue Type: Improvement
>    Affects Versions: 0.9.2-incubating
>            Reporter: Sean Zhong
>            Priority: Minor
>              Labels: Netty
>             Fix For: 0.9.2-incubating
>
>
> This is to address a [concern brought up|https://github.com/apache/incubator-storm/pull/103#issuecomment-43632986] during the work at STORM-297:
> {quote}
> [~revans2] wrote: Your logic makes since to me on why these calls are blocking. My biggest concern around the blocking is in the case of a worker crashing. If a single worker crashes this can block the entire topology from executing until that worker comes back up. In some cases I can see that being something that you would want. In other cases I can see speed being the primary concern and some users would like to get partial data fast, rather then accurate data later.
> Could we make it configurable on a follow up JIRA where we can have a max limit to the buffering that is allowed, before we block, or throw data away (which is what zeromq does)?
> {quote}
> If some worker crash suddenly, how to handle the message which was supposed to be delivered to the worker?
> 1. Should we buffer all message infinitely?
> 2. Should we block the message sending until the connection is resumed?
> 3. Should we config a buffer limit, try to buffer the message first, if the limit is met, then block?
> 4. Should we neither block, nor buffer too much, but choose to drop the messages, and use the built-in storm failover mechanism? 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)