[jira] [Created] (FLUME-3084) HDFS Sink - Provide ability to roll the file beyond interval, size and count.

["Kallam Reddy (JIRA)" <ji...@apache.org>]

Kallam Reddy created FLUME-3084:
-----------------------------------

             Summary: HDFS Sink - Provide ability to roll the file beyond interval, size and count.
                 Key: FLUME-3084
                 URL: https://issues.apache.org/jira/browse/FLUME-3084
             Project: Flume
          Issue Type: Bug
          Components: Sinks+Sources
            Reporter: Kallam Reddy


Currently we are creating Avro containers using HDFS Sink, and we don't have a fine grained control over when can we roll a file and create new file, 
since we are using avro records and containers, whenever the avro version changes we need to roll a new file (close the current file and create new file to roll over to new avro schema). 

Suggested changes: 
1) Make BucketWriter.java public and make required fields and mthods protected (like, shouldRotate and pass Event to shouldRotate method, so that we can determine based on the event we are parsing currently), so that we can extend that class and modify required changes.
2) In HDFSEventSink.java, make initializeBucketWriter method protected and provide getters to the private properties.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)