Caught IOException writing to HDFSWriter . . . File exists

[Christopher Shannon <cs...@gmail.com>]

Flume version 1.3.0. Upgrade not possible because of vendor bundling.

I'm having trouble with this nasty exception, which is filling up our logs
bigtime. So I'm soliciting recommendations.

I shall be obliged

- Chris Shannon

Exception:
12 Mar 2014 10:36:42,689 WARN  [hdfs-k11-call-runner-13]
(org.apache.flume.sink.hdfs.BucketWriter.append:378)  - Caught IOException
writing to HDFSWriter (File exists). Closing file
(/abc/def/ghi/jkl/mno/pqr/pdate=2014-03-12/req_11.1394602505072.bz2.tmp)
and rethrowing exception.
12 Mar 2014 10:36:42,689 WARN  [hdfs-k11-call-runner-13]
(org.apache.flume.sink.hdfs.BucketWriter.append:384)  - Caught IOException
while closing file
(/abc/def/ghi/jkl/mno/pqr/pdate=2014-03-12/req_11.1394602505072.bz2.tmp).
Exception follows.
java.io.IOException: File exists
    at java.io.FileOutputStream.writeBytes(Native Method)
    at java.io.FileOutputStream.write(FileOutputStream.java:276)
    at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:76)
    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:134)
    at java.io.FilterOutputStream.flush(FilterOutputStream.java:134)
    at java.io.DataOutputStream.flush(DataOutputStream.java:117)
    at
org.apache.flume.sink.hdfs.HDFSCompressedDataStream.sync(HDFSCompressedDataStream.java:95)
    at
org.apache.flume.sink.hdfs.BucketWriter.doFlush(BucketWriter.java:345)
    at
org.apache.flume.sink.hdfs.BucketWriter.access$500(BucketWriter.java:53)
    at org.apache.flume.sink.hdfs.BucketWriter$4.run(BucketWriter.java:310)
    at org.apache.flume.sink.hdfs.BucketWriter$4.run(BucketWriter.java:308)
    at
org.apache.flume.sink.hdfs.BucketWriter.runPrivileged(BucketWriter.java:143)
    at org.apache.flume.sink.hdfs.BucketWriter.flush(BucketWriter.java:308)
    at org.apache.flume.sink.hdfs.BucketWriter.close(BucketWriter.java:257)
    at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:382)
    at
org.apache.flume.sink.hdfs.HDFSEventSink$2.call(HDFSEventSink.java:729)
    at
org.apache.flume.sink.hdfs.HDFSEventSink$2.call(HDFSEventSink.java:727)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314)
    at java.util.concurrent.FutureTask.run(FutureTask.java:149)
    at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
    at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
    at java.lang.Thread.run(Thread.java:738)