[jira] [Commented] (CAMEL-12472) Downloading a large file with streamDownload and stepwise hangs

["Claus Ibsen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CAMEL-12472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588846#comment-16588846 ] 

Claus Ibsen commented on CAMEL-12472:
-------------------------------------

Okay thanks for reporting. I think this is a corner-case or limitations when you use streamDownload as then as you say cannot potentially do other FTP opertaions afterwards, until you have read the data stream. 

Anyone got thoughts if there is something we can do, or if not we can add some notes in the documentation about this issue

> Downloading a large file with streamDownload and stepwise hangs 
> ----------------------------------------------------------------
>
>                 Key: CAMEL-12472
>                 URL: https://issues.apache.org/jira/browse/CAMEL-12472
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-ftp
>    Affects Versions: 2.21.0
>         Environment: * Camel versions: 2.17.0 and 2.21.0
>  * Ftp servers: plain vsftpd server, org.apache.ftpserver as a mock, Camel test environment ftp.
>  * Files: depending on configuration from 1mb to 5mb file is vital for error to happen. 
>            Reporter: Karol Koltun
>            Priority: Major
>         Attachments: 0001-FtpSimpleConsumeStreamingStepwiseTest.patch
>
>
> *Downloading a file exceeding certain, system-dependent size with streamDownload and stepwise options turned on hangs and causes timeout.*
> I prepared a test which triggers the error. The patch file is pretty big, as I had to make a file enough big to make the timeout happen. Basing on my predictions, the size of the file triggering the error depends on FTP configuration and Java caching policy (no proof available yet). Working with plain Vsftpd server even 1mb files triggered timeouts. In the test environment the limit on my desktop is 5 mb. If the test passes, please make the file bigger.
> My intepretation of the problem:
>  # Start downloading a file with size exceeding InputStream cache (on my pc approx. 1mb is the limit).
>  [FtpOperations.java:373|https://github.com/apache/camel/blob/dc6caa696255240a2a27c3bf229fc3aac9014401/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/FtpOperations.java#L423]
> {code:java}
> InputStream is = this.client.retrieveFileStream(remoteName);
> {code}
>  #  The server responds 150 and opens data connection.
> {code:java}
> [user_ftp] FTP response: Client "127.0.0.1", "150 Opening BINARY mode data connection for x (1048576 bytes)."
> {code}
>  # The data connection does not end because InputStream is waiting for reads and it has not cached whole file. No "226 Transfer complete" response from server.
>  # Try to change directory as stepwise is turned on.
>  [FtpOperations.java:387|https://github.com/apache/camel/blob/dc6caa696255240a2a27c3bf229fc3aac9014401/components/camel-ftp/src/main/java/org/apache/camel/component/file/remote/FtpOperations.java#L443]
> {code:java}
> this.changeCurrentDirectory(currentDir);
> {code}
>  # Camel hangs as the server is still in the data connection and we are waiting for response from CWD command.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)