You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Arvind Prabhakar <ar...@apache.org> on 2012/05/23 01:06:26 UTC
Review Request: FLUME-1205. NPE related to checkpointing in File Channel
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5200/
-----------------------------------------------------------
Review request for Flume and Brock Noland.
Summary
-------
The NPE reported in FLUME-1205 is due to a race condition during the initialization of the File Channel where the checkpoint reference is not yet initialized but the Log is declared open. As a result the background worker runs into the given NPE.
The fix is to delay the declaration of open to the very last step of replay method by which time all the needed state will be initialized. Doing this required removing precondition checks from Log.writeCheckpoint() and Log.roll() methods, both of which are private and their access paths for normal operations do apply these checks.
This addresses bug FLUME-1205.
https://issues.apache.org/jira/browse/FLUME-1205
Diffs
-----
/trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java 1341666
/trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Log.java 1341666
Diff: https://reviews.apache.org/r/5200/diff
Testing
-------
Ran all unit tests.
Thanks,
Arvind
Re: Review Request: FLUME-1205. NPE related to checkpointing in File Channel
Posted by Brock Noland <br...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5200/#review8048
-----------------------------------------------------------
Ship it!
lgtm
- Brock
On 2012-05-22 23:06:26, Arvind Prabhakar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/5200/
> -----------------------------------------------------------
>
> (Updated 2012-05-22 23:06:26)
>
>
> Review request for Flume and Brock Noland.
>
>
> Summary
> -------
>
> The NPE reported in FLUME-1205 is due to a race condition during the initialization of the File Channel where the checkpoint reference is not yet initialized but the Log is declared open. As a result the background worker runs into the given NPE.
>
> The fix is to delay the declaration of open to the very last step of replay method by which time all the needed state will be initialized. Doing this required removing precondition checks from Log.writeCheckpoint() and Log.roll() methods, both of which are private and their access paths for normal operations do apply these checks.
>
>
> This addresses bug FLUME-1205.
> https://issues.apache.org/jira/browse/FLUME-1205
>
>
> Diffs
> -----
>
> /trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java 1341666
> /trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Log.java 1341666
>
> Diff: https://reviews.apache.org/r/5200/diff
>
>
> Testing
> -------
>
> Ran all unit tests.
>
>
> Thanks,
>
> Arvind
>
>