[jira] [Updated] (STORM-157) ack/fail should not require keeping input values in scope.

["Rick Kellogg (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/STORM-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rick Kellogg updated STORM-157:
-------------------------------
    Component/s: storm-core

> ack/fail should not require keeping input values in scope.
> ----------------------------------------------------------
>
>                 Key: STORM-157
>                 URL: https://issues.apache.org/jira/browse/STORM-157
>             Project: Apache Storm
>          Issue Type: Improvement
>          Components: storm-core
>            Reporter: James Xu
>            Priority: Minor
>
> https://github.com/nathanmarz/storm/issues/752
> ack/fail takes a Tuple, but it appears the values are not needed to ack. If we aggregate many things locally before we commit, we keep refs to many Tuples. We think this could be keeping more in memory than we need and pushing some topologies to the breaking point.
> We are not 100% that this is the issue, but it would be good to have something like, getToken on Tuple. And ack/fail might take that Token which is an opaque object that includes the minimal refs to ack.
> ----------
> nathanmarz: +1
> This should be pretty easy to implement. Tuples already have a MessageID which is the primary object used for acking. There's also the "ack val" (the xors of tuples anchored onto this tuple) which should be moved into the MessageID. Then we can use the MessageID as the token, and update the APIs to accept the MessageID for acking and MessageID for anchoring as part of execute. IOutputCollector should be changed to only accept MessageID for acking/failing/anchoring, and then OutputCollector can add the convenience methods for accepting Tuple acking/anchoring/failing.
> ----------
> jmlogan: We had this similar issue when we had very large tuples...at time time I "worked around it" by using Reflection to get ahold of the internal list storing the values, and clearing it.
> I ended up mitigating this problem for good by having smaller tuples, and passing the large payloads through Redis.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)