[jira] [Commented] (FLUME-2795) Sinks with hdfs path with escape sequence do not close current .tmp file when changit to new directory

["Umesh Chaudhary (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLUME-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528863#comment-15528863 ] 

Umesh Chaudhary commented on FLUME-2795:
----------------------------------------

Hi [~dscarlat], do you still see this issue? If so, would you mind explaining the scenario bit more and did you try setting hdfs.idleTimeout	 ?

> Sinks with hdfs path with escape sequence do not close current .tmp file when changit to new directory
> ------------------------------------------------------------------------------------------------------
>
>                 Key: FLUME-2795
>                 URL: https://issues.apache.org/jira/browse/FLUME-2795
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: v1.5.0
>         Environment: cdh5.4.4
>  over ubuntu
>            Reporter: David Scarlatti
>
> I have a hdfs sink with this config:
> tier1.sinks.sink1.type         = hdfs
> tier1.sinks.sink1.channel      = channel1
> tier1.sinks.sink1.hdfs.path    = /user/bla/%y-%m-%d
> tier1.sinks.sink1.hdfs.filePrefix =bla
> tier1.sinks.sink1.hdfs.rollSize = 0
> tier1.sinks.sink1.hdfs.rollInterval = 0
> tier1.sinks.sink1.hdfs.rollCount = 150000
> tier1.sinks.sink1.hdfs.useLocalTimeStamp = true
> tier1.sinks.sink1.hdfs.fileType = DataStream
> tier1.sinks.sink1.hdfs.batchSize = 100
> every night at 23:59:59 a new folder is created in the HDFS and the folder for the previous day has a last file with .tmp extension, the file is incomplete and only when the flume agent is restarted this .tmp file is completed and closed an renamed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)