[net] FTPClient doesn't download full images

[Julia Ruzicka <ju...@simutech.at>]

Hi everyone!

 

This is my first time using a mailing list, so sorry in advance if I mess
something up!

 

I'm using version 3.6 of the "Net" Library, in combination with Java 8,
Android Studio 3.6.1 (min SDK=24, target SDK=27) and I'm experiencing a
weird issue with file download:

 

Downloading small images (jpg, around 12kb each) or text files from my FTP
server always succeeds but I've also got bigger images (png, 2000x2000,
around 8-9mb each) and they only download partially: I pulled them from the
emulator/device and they end up with a black, horizontal bar at the bottom
and they're smaller than they should be (see below for examples).

 

I tested my code with 'ftpClient.retrieveFile()' (version 1) and
'ftpClient.retrieveFileStream()' (version 2, InputStream or
BufferedInputStream) but I get the same result with both:

 

public void downloadFiles(String remoteFolder, String localFolder,
ArrayList<String> filenames) { //This runs inside an 'AsyncTask'!

    //login here

    ftpClient.setConnectTimeout(10000);

    ftpClient.setDefaultTimeout(30000);

    OutputStream out = null;

 

    try {

        ftpClient.enterLocalPassiveMode();

        ftpClient.setFileType(FTP.BINARY_FILE_TYPE); 

 

        if(ftpClient.changeWorkingDirectory(remoteFolder)) {

            for(String filename : filenames) {

                FTPFile[] singleFile = ftpClient.listFiles(filename);

 

                if(singleFile.length > 0) { //check if file exists

                    String localPath = localFolder + File.separator +
filename;

                    out = new FileOutputStream(localPath);

                    

                    //Version 1:

                    /*if(!ftpClient.retrieveFile(filename, out)) {

                        //Set error message here

                        out.close();

                        break;

                    }

                    out.close();*/

 

 

                    //Version 2:

                    InputStream is = ftpClient.retrieveFileStream(filename);


                    BufferedInputStream bis = new BufferedInputStream(is);

                    byte[] buf = new byte[10000000];

                    int len;

 

                    while ((len = bis.read(buf)) > 0) { //Same result using
only 'is'

                        out.write(buf, 0, len);

                    }

                    

                    if(ftpClient.completePendingCommand()) {

                        Log.d(TAG,"DONE"); //This is always printed, even
when the files aren't complete

                    } else {

                        Log.d(TAG,"NOT DONE");

                    }

 

                    out.close();

                    bis.close();

                } else {

                    //Another error message here

                    break;

                }

            }

        }

    } catch (IOException e) {

        //And another error message here

    } finally {

        try {

            if(out!=null) { out.close(); }

        } catch(IOException e) {

            //Doesn't matter

        }

    }

 

    //logout here

}

 

I printed the full length (added up 'len') and only very rarely it manages
to download everything, usually there are a couple of bytes missing (always
less than 10 so far) - example:

 

File 1 is 8965167 bytes (according to Windows' file explorer) but the added
up 'len' is only 8965165 or 8965161 or .

File 2 is 8972047 bytes but it only downloads 8972044 or 8972038 or .

 

Did I miss anything, some type of setting for bigger files? How do I make it
download the full images?

 

Thanks in advance for any tip!

Re: [net] FTPClient doesn't download full images

[Robert Paasche <r....@pripares.com>]

The following line is wrong:
     while ((len = bis.read(buf)) > 0) { //Same result using only 'is'

read == -1 marks the end of the stream. 0 shows only that the stream is
still waiting for the next chunk of data.

Please read the JavaDocs:

> Returns:
> the number of bytes read, or -1 if the end of the stream has been reached.


  while ((len = bis.read(buf)) >= 0) { //Same result using only 'is'
if (len >0)
       out.write(buf, 0, len);
else
        // prevent high CPU load
        LockSupport.parkNanos(TimeUnit.MILLISECONDS.toNanos(500));
 }

AW: [net] FTPClient doesn't download full images

[Julia Ruzicka <ju...@simutech.at>]

Hi everyone!

I created a small test app that uses my "download" class on a button press
(but doesn't do anything else) and did some more testing with it (with
"FTP.BINARY_FILE_TYPE"):

- Downloading a 14mb txt file (and its copies) always finishes with a
complete file (even with a long file name)
- Downloading a 12kb jpg (and its copies) always finishes with a complete
file
- Downloading the 8mb (png) or 6mb (jpg version) images sometimes completes
but most of the time it doesn't - the file size is now always the same but
there's still a black bar at the bottom of the images (height differs)
- I uploaded a couple of random google images to my FTP server and most of
them are affected by this problem too.

You can find all the tested google images (including the sources) and the
results in this imgur album:

https://imgur.com/a/Mzngp11

I uploaded a couple of the images to a public FTP test server
(https://dlptest.com/ftp-test/) and I get the same results with it.

Has anyone got an idea what's wrong and how do fix this?




---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org