You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Yongkun Wang (JIRA)" <ji...@apache.org> on 2013/08/23 10:12:52 UTC
[jira] [Resolved] (FLUME-2176) SpoolDir Source, get 'File has
changed' exception but actually there is no change on the file
[ https://issues.apache.org/jira/browse/FLUME-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yongkun Wang resolved FLUME-2176.
---------------------------------
Resolution: Fixed
Assignee: Yongkun Wang
> SpoolDir Source, get 'File has changed' exception but actually there is no change on the file
> ---------------------------------------------------------------------------------------------
>
> Key: FLUME-2176
> URL: https://issues.apache.org/jira/browse/FLUME-2176
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.4.0
> Reporter: Yongkun Wang
> Assignee: Yongkun Wang
>
> I am using a script to generate files and then copy them one by one to the spooling directory. I got 'File has changed size' exception, but I am pretty sure the file wasn't changed.
> {code}
> 23 Aug 2013 10:37:02,704 ERROR [pool-5-thread-1] (org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run:173) - Uncaught exception in Runnable
> java.lang.IllegalStateException: File has changed size since being read: /log/flume-ng/agent1/spooldir/spd1/log.00000029.20130822-121450171+0900.8729556570223344.seq
> at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.retireCurrentFile(ReliableSpoolingFileEventReader.java:286)
> at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:226)
> at org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:160)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {code}
> My configuration:
> {code}
> agent1.sources = spd1
> agent1.sources.spd1.type = spooldir
> agent1.sources.spd1.spoolDir = /log/flume-ng/agent1/spooldir/spd1
> agent1.sources.spd1.deserializer.maxLineLength = 8192
> agent1.sources.spd1.channels = file1
> agent1.channels = file1
> agent1.channels.file1.type = file
> agent1.channels.file1.checkpointDir = /log/flume-ng/agent1/checkpoint
> agent1.channels.file1.dataDirs = /log/flume-ng/agent1/data
> agent1.channels.file1.capacity = 2000000
> agent1.channels.file1.transactionCapacity = 100
> agent1.sinks = avro1
> agent1.sinks.avro1.type = avro
> agent1.sinks.avro1.channel = file1
> agent1.sinks.avro1.hostname = remote_host
> agent1.sinks.avro1.port = 33333
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira