[jira] [Commented] (FLUME-2637) flume 1.5 doesnt update flume .tmp file in HDFS until roll

["Hari Shreedharan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLUME-2637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14348145#comment-14348145 ] 

Hari Shreedharan commented on FLUME-2637:
-----------------------------------------

That is expected behavior. Flume rolls the file and then renames them - that is how most file writes to Hadoop happen, so incomplete files can be ignored.

> flume 1.5 doesnt update flume .tmp file in HDFS until roll
> ----------------------------------------------------------
>
>                 Key: FLUME-2637
>                 URL: https://issues.apache.org/jira/browse/FLUME-2637
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: v1.5.0
>            Reporter: sutanu das
>            Priority: Critical
>
> When flume write to a HDFS Sync, it doesnt update the size (bytes0 and timstamp of the HDFS file (.tmp file), until the .tmp file is rolled
> But when we get the file in ufs (via hadoop fs -get) , we see the correct size and timestemp
> see below:
> [hdfs@ham-dal-d001 ~]$ hdfs dfs -ls -R /prod/hadoop/smallsite/flume_ingest_hak_1/ClearPass_flume.1425066465833.tmp 
> -rw-r--r-- 3 root hdfs 3598 2015-02-27 19:47 /prod/hadoop/smallsite/flume_ingest_hak_1/ClearPass_flume.1425066465833.tmp 
> [hdfs@ham-dal-d001 ~]$
> hadoop fs -get /prod/hadoop/smallsite/flume_ingest_hak_1/ClearPass_flume.1425066465833.tmp /tmp/s.out



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)