[jira] [Commented] (FLUME-2113) Flume: No errors thrown but flume fails to transfer the file completely

["Alexander Alten-Lorenz (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLUME-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13693825#comment-13693825 ] 

Alexander Alten-Lorenz commented on FLUME-2113:
-----------------------------------------------

I dropped this into question. Please note, Flume is NOT designed to transfer large files. Flume is a log collection aggregator, and can handle multiple sources at once. To transfer large files please use HDFS related tools like hadoop fs -put, or use the WebHDFS API (http://hadoop.apache.org/docs/r1.0.4/webhdfs.html).
                
> Flume: No errors thrown but flume fails to transfer the file completely
> -----------------------------------------------------------------------
>
>                 Key: FLUME-2113
>                 URL: https://issues.apache.org/jira/browse/FLUME-2113
>             Project: Flume
>          Issue Type: Question
>            Reporter: kasa v varun
>            Priority: Minor
>              Labels: hdfs, sink
>
> Flow: using a basic flow, spoolDir -> fileChannel -> HDFS
> No extra parameters set in the .conf file
> File size that i'm trying to transfer: 1.4GB
> Situation: the agents starts fine, the file transfer starts fine, the file in source gets renamed to .COMPLETED, the complete file is not getting transfered to HDFS, no errors/exceptions are being thrown. I ran the same adhoc several times i found that out of the 1.4 gigs only ~169Mb is getting transfered. Seems weird !
> Any suggestions? Any solutions? any hypotheses?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira