[jira] [Commented] (FLUME-1856) HDFS sink idleTimeout does not close removed BucketWriters

["Hari Shreedharan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLUME-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932740#comment-13932740 ] 

Hari Shreedharan commented on FLUME-1856:
-----------------------------------------

This should be fixed, can you verify?

> HDFS sink idleTimeout does not close removed BucketWriters
> ----------------------------------------------------------
>
>                 Key: FLUME-1856
>                 URL: https://issues.apache.org/jira/browse/FLUME-1856
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: v1.4.0, v1.3.1
>            Reporter: Connor Woodson
>            Assignee: Gopinathan A
>         Attachments: FLUME-1856-0.patch
>
>
> When a BucketWriter idles out, it is removed from the bucket list but it is never closed. Optimal solution will be to override LinkedHashMap.remove similar to the removeEldestEntry code to close the bucket.
> The effects of this not happening is that if you rely on idleTimeout for closing buckets (such as setting no roll period and attempting to roll via timestamped path), as the buckets never get closed the data never goes to s3 and so the heap gets very full.



--
This message was sent by Atlassian JIRA
(v6.2#6252)