You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Hari Shreedharan (JIRA)" <ji...@apache.org> on 2012/08/04 18:58:02 UTC
[jira] [Comment Edited] (FLUME-1417) File Channel checkpoint can be
bad leading to the channel being unable to start.
[ https://issues.apache.org/jira/browse/FLUME-1417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13428632#comment-13428632 ]
Hari Shreedharan edited comment on FLUME-1417 at 8/4/12 4:56 PM:
-----------------------------------------------------------------
Brock - Does that make sense? Or is there some logic to make sure the files are removed only if all puts into that file were committed? To me it looks like this series of events would be problematic:
File ( i )
n puts
commit
n takes (file i is no longer in flume event queue fileIDCounts)
---
file roll -> current active file in directory (File i+1)
background worker kicks in -> removes file i.
Is this a possibility or is it something that is not a concern.
Thanks.
was (Author: hshreedharan):
Brock - Does that make sense? Or is there some logic to make sure the files are removed only if all puts into that file were committed? To me it looks like this series of events would be problematic:
File (i)
n puts
commit
n takes (file i is no longer in flume event queue fileIDCounts)
---
file roll -> current active file in directory (File i+1)
background worker kicks in -> removes file i.
Is this a possibility or is it something that is not a concern.
Thanks.
> File Channel checkpoint can be bad leading to the channel being unable to start.
> --------------------------------------------------------------------------------
>
> Key: FLUME-1417
> URL: https://issues.apache.org/jira/browse/FLUME-1417
> Project: Flume
> Issue Type: Bug
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
>
> ERROR file.Log: Failed to initialize Log on [channel=file-channel]
> java.lang.NullPointerException
> at org.apache.flume.channel.file.Log.writeCheckpoint(Log.java:739)
> at org.apache.flume.channel.file.Log.replay(Log.java:261)
> at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:228)
> at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:228)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> ERROR file.FileChannel: Failed to start the file channel [channel=file-channel]
> java.lang.NullPointerException
> at org.apache.flume.channel.file.Log.writeCheckpoint(Log.java:739)
> at org.apache.flume.channel.file.Log.replay(Log.java:261)
> at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:228)
> at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:228)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira