You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2013/04/07 05:45:18 UTC
[jira] [Commented] (FLUME-1516) FileChannel Write Dual Checkpoints
to avoid replays
[ https://issues.apache.org/jira/browse/FLUME-1516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624615#comment-13624615 ]
Hudson commented on FLUME-1516:
-------------------------------
Integrated in flume-trunk #388 (See [https://builds.apache.org/job/flume-trunk/388/])
FLUME-1516: FileChannel Write Dual Checkpoints to avoid replays (Revision 6ca616800ec897551fbb14959ce3a5f0c1d69aed)
Result = FAILURE
brock : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=6ca616800ec897551fbb14959ce3a5f0c1d69aed
Files :
* flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStore.java
* flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FlumeEventQueue.java
* flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannelConfiguration.java
* flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFile.java
* flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/ReplayHandler.java
* flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Log.java
* flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Serialization.java
* flume-ng-channels/flume-file-channel/src/main/proto/filechannel.proto
* flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/LogFileV3.java
* flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannelRestart.java
* flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/LogFile.java
* flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestUtils.java
* flume-ng-doc/sphinx/FlumeUserGuide.rst
* flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannelBase.java
* flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/proto/ProtosFactory.java
* flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java
* flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFactory.java
* flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFileV3.java
> FileChannel Write Dual Checkpoints to avoid replays
> ---------------------------------------------------
>
> Key: FLUME-1516
> URL: https://issues.apache.org/jira/browse/FLUME-1516
> Project: Flume
> Issue Type: New Feature
> Components: Channel, File Channel
> Affects Versions: v1.3.0
> Reporter: Brock Noland
> Assignee: Hari Shreedharan
> Fix For: v1.4.0
>
> Attachments: DualCheckpoints.pdf, DualCheckpointsv2.pdf, DualCheckpointsv3.pdf, FLUME-1516-1.patch, FLUME-1516-2.patch, FLUME-1516-3.patch, FLUME-1516-4.patch, FLUME-1516-5.patch, FLUME-1516-6.patch, FLUME-1516-7.patch, FLUME-1516-8.patch, FLUME-1516.patch
>
>
> Per the LFS paper (http://www.cs.berkeley.edu/~brewer/cs262/LFS.pdf) we can write two checkpoints to avoid replaying the logs in the case we crash/shutdown while writing a checkpoint.
> Section 4:
> "In order to handle a crash during a checkpoint operation there are actually two checkpoint regions, and checkpoint operations alternate between them. The checkpoint time is in the last block of the checkpoint so if the checkpoint fails the time will not be updated. During reboot, the system reads both checkpoint regions and uses the one with the most recent time."
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira