You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Hari Shreedharan (JIRA)" <ji...@apache.org> on 2012/09/25 05:19:07 UTC
[jira] [Created] (FLUME-1606) Rollbacks of Put transactions does
not clear the transaction from inflight puts.
Hari Shreedharan created FLUME-1606:
---------------------------------------
Summary: Rollbacks of Put transactions does not clear the transaction from inflight puts.
Key: FLUME-1606
URL: https://issues.apache.org/jira/browse/FLUME-1606
Project: Flume
Issue Type: Bug
Reporter: Hari Shreedharan
Assignee: Hari Shreedharan
doRollback() calls queue.completeTransaction() only for take transactions. It should also be called for put transactions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-1606) Rollbacks of Put transactions does
not clear the transaction from inflight puts.
Posted by "Hari Shreedharan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hari Shreedharan updated FLUME-1606:
------------------------------------
Description: doRollback() calls queue.completeTransaction() only for take transactions. It should also be called for put transactions. A side effect of this is that the channel ends up thinking the data file which contains the rolled back transaction still has pending events, and will not delete any files which have ids after that one. Restarting the channel will cause files to be deleted if their ref count is zero, even if they are in inflight puts list. The reason the replay fixes this is that the inflight puts are not inserted into the queue because a commit for the puts is never seen. So this issue can be fixed by simply restarting the agent. (was: doRollback() calls queue.completeTransaction() only for take transactions. It should also be called for put transactions. A side effect of this is that the channel ends up thinking the data file which contains the rolled back transaction still has pending events, and will not delete any files which have ids after that one.)
> Rollbacks of Put transactions does not clear the transaction from inflight puts.
> --------------------------------------------------------------------------------
>
> Key: FLUME-1606
> URL: https://issues.apache.org/jira/browse/FLUME-1606
> Project: Flume
> Issue Type: Bug
> Affects Versions: v1.3.0
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Fix For: v1.3.0
>
> Attachments: FLUME-1606.patch
>
>
> doRollback() calls queue.completeTransaction() only for take transactions. It should also be called for put transactions. A side effect of this is that the channel ends up thinking the data file which contains the rolled back transaction still has pending events, and will not delete any files which have ids after that one. Restarting the channel will cause files to be deleted if their ref count is zero, even if they are in inflight puts list. The reason the replay fixes this is that the inflight puts are not inserted into the queue because a commit for the puts is never seen. So this issue can be fixed by simply restarting the agent.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1606) Rollbacks of Put transactions does
not clear the transaction from inflight puts.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463439#comment-13463439 ]
Hudson commented on FLUME-1606:
-------------------------------
Integrated in flume-trunk #309 (See [https://builds.apache.org/job/flume-trunk/309/])
FLUME-1606. Rollbacks of Put transactions does not clear the transaction from inflight puts. (Revision 79ff3adf0dfcd75b1a80261e6559a25eed1f3e08)
Result = SUCCESS
mpercy : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=79ff3adf0dfcd75b1a80261e6559a25eed1f3e08
Files :
* flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannel.java
* flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FlumeEventQueue.java
* flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java
> Rollbacks of Put transactions does not clear the transaction from inflight puts.
> --------------------------------------------------------------------------------
>
> Key: FLUME-1606
> URL: https://issues.apache.org/jira/browse/FLUME-1606
> Project: Flume
> Issue Type: Bug
> Affects Versions: v1.3.0
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Fix For: v1.3.0
>
> Attachments: FLUME-1606.patch
>
>
> doRollback() calls queue.completeTransaction() only for take transactions. It should also be called for put transactions. A side effect of this is that the channel ends up thinking the data file which contains the rolled back transaction still has pending events, and will not delete any files which have ids after that one. Restarting the channel will cause files to be deleted if their ref count is zero, even if they are in inflight puts list. The reason the replay fixes this is that the inflight puts are not inserted into the queue because a commit for the puts is never seen. So this issue can be fixed by simply restarting the agent, at least until another put transaction is rolled back - which will result in the same issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-1606) Rollbacks of Put transactions does
not clear the transaction from inflight puts.
Posted by "Hari Shreedharan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hari Shreedharan updated FLUME-1606:
------------------------------------
Description: doRollback() calls queue.completeTransaction() only for take transactions. It should also be called for put transactions. A side effect of this is that the channel ends up thinking the data file which contains the rolled back transaction still has pending events, and will not delete any files which have ids after that one. (was: doRollback() calls queue.completeTransaction() only for take transactions. It should also be called for put transactions.)
> Rollbacks of Put transactions does not clear the transaction from inflight puts.
> --------------------------------------------------------------------------------
>
> Key: FLUME-1606
> URL: https://issues.apache.org/jira/browse/FLUME-1606
> Project: Flume
> Issue Type: Bug
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
>
> doRollback() calls queue.completeTransaction() only for take transactions. It should also be called for put transactions. A side effect of this is that the channel ends up thinking the data file which contains the rolled back transaction still has pending events, and will not delete any files which have ids after that one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-1606) Rollbacks of Put transactions does
not clear the transaction from inflight puts.
Posted by "Hari Shreedharan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hari Shreedharan updated FLUME-1606:
------------------------------------
Attachment: FLUME-1606.patch
> Rollbacks of Put transactions does not clear the transaction from inflight puts.
> --------------------------------------------------------------------------------
>
> Key: FLUME-1606
> URL: https://issues.apache.org/jira/browse/FLUME-1606
> Project: Flume
> Issue Type: Bug
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Attachments: FLUME-1606.patch
>
>
> doRollback() calls queue.completeTransaction() only for take transactions. It should also be called for put transactions. A side effect of this is that the channel ends up thinking the data file which contains the rolled back transaction still has pending events, and will not delete any files which have ids after that one.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-1606) Rollbacks of Put transactions does
not clear the transaction from inflight puts.
Posted by "Hari Shreedharan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hari Shreedharan updated FLUME-1606:
------------------------------------
Description: doRollback() calls queue.completeTransaction() only for take transactions. It should also be called for put transactions. A side effect of this is that the channel ends up thinking the data file which contains the rolled back transaction still has pending events, and will not delete any files which have ids after that one. Restarting the channel will cause files to be deleted if their ref count is zero, even if they are in inflight puts list. The reason the replay fixes this is that the inflight puts are not inserted into the queue because a commit for the puts is never seen. So this issue can be fixed by simply restarting the agent, at least until another put transaction is rolled back - which will result in the same issue. (was: doRollback() calls queue.completeTransaction() only for take transactions. It should also be called for put transactions. A side effect of this is that the channel ends up thinking the data file which contains the rolled back transaction still has pending events, and will not delete any files which have ids after that one. Restarting the channel will cause files to be deleted if their ref count is zero, even if they are in inflight puts list. The reason the replay fixes this is that the inflight puts are not inserted into the queue because a commit for the puts is never seen. So this issue can be fixed by simply restarting the agent.)
> Rollbacks of Put transactions does not clear the transaction from inflight puts.
> --------------------------------------------------------------------------------
>
> Key: FLUME-1606
> URL: https://issues.apache.org/jira/browse/FLUME-1606
> Project: Flume
> Issue Type: Bug
> Affects Versions: v1.3.0
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Fix For: v1.3.0
>
> Attachments: FLUME-1606.patch
>
>
> doRollback() calls queue.completeTransaction() only for take transactions. It should also be called for put transactions. A side effect of this is that the channel ends up thinking the data file which contains the rolled back transaction still has pending events, and will not delete any files which have ids after that one. Restarting the channel will cause files to be deleted if their ref count is zero, even if they are in inflight puts list. The reason the replay fixes this is that the inflight puts are not inserted into the queue because a commit for the puts is never seen. So this issue can be fixed by simply restarting the agent, at least until another put transaction is rolled back - which will result in the same issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira