[jira] [Created] (FLUME-2419) Bucket writer was closed due to idling exception

["ashok Kumar (JIRA)" <ji...@apache.org>]

ashok Kumar created FLUME-2419:
----------------------------------

             Summary: Bucket writer was closed due to idling exception
                 Key: FLUME-2419
                 URL: https://issues.apache.org/jira/browse/FLUME-2419
             Project: Flume
          Issue Type: Bug
    Affects Versions: v1.4.0
            Reporter: ashok Kumar
            Priority: Minor


We have flume collecting the daily log and sinks into hdfs with the configuration, 

activity.sinks.eventhdfssink_1.hdfs.rollSize = 66060288
activity.sinks.eventhdfssink_1.hdfs.rollCount = 0
activity.sinks.eventhdfssink_1.hdfs.rollInterval = 0
activity.sinks.eventhdfssink_1.hdfs.idleTimeout = 180

We have the default configuration for maxopenfile and the writing data went into endless loop with the following exception:

"java.io.IOException: This bucket writer was closed due to idling and this handle is thus no longer valid"


I looked over the source code (BucketWriter.java) and I establish the exceptions occurs only at the time the file is closed and also due to idle time out. 

if (!isOpen) {
      if(idleClosed) {
        throw new IOException("This bucket writer was closed due to idling and this handle " +
            "is thus no longer valid");
      }
      open();
    }

However the idlecallback has been called out during the flush() from BucketWriter.java,

                if(onIdleCallback != null)
                  onIdleCallback.run(onIdleCallbackPath);

the idlecallback from HDFSEventSink.java,

WriterCallback idleCallback = null;
          if(idleTimeout != 0) {
            idleCallback = new WriterCallback() {
              @Override
              public void run(String bucketPath) {
                sfWriters.remove(bucketPath);
              }
            };

Here, we remove the entry from sfWriters map, thus forcing a recreation of the writer during subsequent calls. Thus, I do not see a case where we can have the file handle closed as well as idleClosed be true without removing the necessary references. FWIW, we are seeing logs that indicate that onIdleCallback has been triggered. Are there any other cases in which this exception can hit?

If it is of help, we saw this exception when our application (wrongly) started pumping old events (older than 2 weeks). Is there a possibility of writer handles created earlier being closed, but not being visible to the BucketWriter.java code?

Any pointers to potential configuration issues is also appreciated.




--
This message was sent by Atlassian JIRA
(v6.2#6252)