[NET][FTP] - Exception while storing large file stream to mainframe

[Co...@courts.mo.gov]

Hello

I have a file that is sent weekly to a FTP server on a mainframe. The
transfer has worked in the past with smaller file sizes but now the file
size is up to 32,227,500 bytes
and an IOException is thrown while using the Util.copyStream() method.

When I point the program at our test microsoft ftp server it works without
any errors.

If I load microsoft's command-line ftp program and send the same file to
the mainframe ftp by hand it works without any errors.

There seems to be a problem with the combination of large files, mainframe
servers and commons:net.

Any ideas would be appreciated.

CODE -
      outputStream = ftpClient.storeFileStream(remoteFileName);
      this.logReplyMessages(ftpClient.getReplyStrings());
      if(!FTPReply.isPositivePreliminary(ftpClient.getReplyCode())) {
        throw new Exception(this.buildExceptionMessage("ERROR: UNABLE TO
OPEN DATA CONNECTION TO FTP SERVER -", ftpClient.getReplyStrings()));
      }

      inputStream = new FileInputStream(file);

      Util.copyStream(inputStream, outputStream, 5000, file.length(), this,
true);

      inputStream.close();
      outputStream.close();

      ftpClient.completePendingCommand();
      this.logReplyMessages(ftpClient.getReplyStrings());
      if(!FTPReply.isPositiveCompletion(ftpClient.getReplyCode())) {
        throw new Exception(this.buildExceptionMessage("ERROR: UNABLE TO
COMPLETE FILE TRANSFER TO FTP SERVER -", ftpClient.getReplyStrings()));
      }


LOG -
220-SDFTP151 IBM FTP CS V1R7 at HOSTNAME, 09:51:41 on 2006-12-21.
220 Connection will close if idle for more than 40 minutes.
230 USER is logged on.  Working directory is "USER.".
125 Storing data set DT.T27OSC.DO.FILE
org.apache.commons.net.io.CopyStreamException: IOException caught while
copying.
      at org.apache.commons.net.io.Util.copyStream(Util.java:127)
      at gov.mo.courts.DataCollectionFtp.sendFile(
DataCollectionFtp.java:369)
      at gov.mo.courts.DataCollectionFtp.run(DataCollectionFtp.java:150)
      at gov.mo.courts.DataCollectionFtp.main(DataCollectionFtp.java:70)

SERVER INFO -
ftp> literal syst
215 MVS is the operating system of this server. FTP Server is running on
z/OS.
ftp> literal stat
211-Server FTP talking to host IP, port 2203
211-User: USER  Working directory: USER.
211-The control connection has transferred 289 bytes
211-There is no current data connection.
211-The next data connection will be actively opened
211-to host IP, port 2203,
211-using Mode Stream, Structure File, type ASCII, byte-size 8
211-Automatic recall of migrated data sets.
211-Automatic mount of direct access volumes.
211-Auto tape mount is allowed.
211-Inactivity timer is set to 2400
211-Timer FTPKEEPALIVE is set to 0
211-VCOUNT is 59
211-ASA control characters in ASA files opened for text processing
211-will be transferred as ASA control characters.
211-Trailing blanks are removed from a fixed format
211-data set when it is retrieved.
211-Data set mode.  (Do not treat each qualifier as a directory.)
211-ISPFSTATS is set to FALSE
211-Primary allocation 1 track.  Secondary allocation 1 track.
211-Partitioned data sets will be created with 27 directory blocks.
211-FileType SEQ (Sequential - default).
211-Number of access method buffers is 5
211-RDWs from variable format data sets are discarded.
211-Records on input tape are unspecified format
211-SITE DB2 subsystem name is DB2
211-Wrapping data into next record.
211-Tape write is not allowed to use BSAM I/O
211-Truncated records will not be treated as an error
211-JESLRECL is 80
211-JESRECFM is Fixed
211-JESINTERFACELEVEL is 1
211-Server site variable JESTRAILINGBLANKS is set to TRUE
211-Confidence level of data transfers is neither checked nor reported
211-ENcoding is set to SBCS
211-SBDataconn codeset names: IBM-1047,IBM-850
211-Outbound SBCS ASCII data uses CRLF line terminator
211-Outbound MBCS ASCII data uses CRLF line terminator
211-DBSUB is set to FALSE
211-SBSUB is set to FALSE
211-SBSUBCHAR is set to SPACE
211-SMS is active.
211-New data sets will be catalogued if a store operation ends abnormally
211-Single quotes will override the current working directory.
211-UMASK value is 027
211-Process id is 67112046
211-Checkpoint interval is 0
211-Authentication type: None
211-Port of Entry resource class for IPv4 clients is: TERMINAL
211-Record format VB, Lrecl: 256, Blocksize: 6233
211-Server site variable LISTSUBDIR is set to TRUE
211 *** end of status ***

Cory Bestgen
Information Technology Division
Office of State Courts Administrator
(573) 522 - 5455
cory.bestgen@courts.mo.gov


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org