You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ftpserver-users@mina.apache.org by Aidan Diffey <ai...@gmail.com> on 2010/11/29 12:50:31 UTC
Re: FTP server active / passive mode UPDATE
Hello All (Niklas, John).
Firstly I would like to thank you for your ongoing help.
I have taken John's advise and implemented my own
FTPFileSystemFactory, View and file.
I have implemented my FtpFile createInputStream to read the data from
the database and return a new ByteArrayOutputStream containing the
byte[] data read from the database.
This has not solved the problem, BUT it has allowed me to view a stack
trace which might be helpful in diagnosing the problem. See stack
trace and some wire shark trace as well. From looking at this and the
code, it would appear that something has set the inputstream inside
the BufferedInputStream to null.
-- STACK TRACE --
11:21:23,515 WARN [IODataConnection] Exception during data transfer,
closing data connection socket
java.io.IOException: Stream closed
at java.io.BufferedInputStream.getInIfOpen(BufferedInputStream.java:134)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
at java.io.FilterInputStream.read(FilterInputStream.java:90)
at org.apache.ftpserver.impl.IODataConnection.transfer(IODataConnection.java:243)
at org.apache.ftpserver.impl.IODataConnection.transferToClient(IODataConnection.java:161)
at org.apache.ftpserver.command.impl.RETR.execute(RETR.java:166)
at org.apache.ftpserver.impl.DefaultFtpHandler.messageReceived(DefaultFtpHandler.java:210)
at org.apache.ftpserver.listener.nio.FtpHandlerAdapter.messageReceived(FtpHandlerAdapter.java:61)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:721)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:433)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:47)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:801)
at org.apache.ftpserver.listener.nio.FtpLoggingFilter.messageReceived(FtpLoggingFilter.java:85)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:433)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:47)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:801)
at org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:71)
at org.apache.mina.filter.logging.MdcInjectionFilter.filter(MdcInjectionFilter.java:136)
at org.apache.mina.filter.util.CommonEventFilter.messageReceived(CommonEventFilter.java:70)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:433)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:47)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:801)
at org.apache.mina.filter.codec.ProtocolCodecFilter$ProtocolDecoderOutputImpl.flush(ProtocolCodecFilter.java:375)
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:229)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:433)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:47)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:801)
at org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:71)
at org.apache.mina.core.session.IoEvent.run(IoEvent.java:63)
at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTask(OrderedThreadPoolExecutor.java:801)
at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTasks(OrderedThreadPoolExecutor.java:793)
at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.run(OrderedThreadPoolExecutor.java:735)
at java.lang.Thread.run(Thread.java:595)
11:21:23,609 INFO [FtpLoggingFilter] SENT: 150 File status okay;
about to open data connection.
11:21:23,609 INFO [FtpLoggingFilter] SENT: 551 <FILENAME>: Error on input file.
--- WIRESHARK ----
TYPE I
200 Command TYPE okay.
CWD
250 Ok
PASV
227 Entering Passive Mode (10,20,0,111,17,163)
RETR <FILENAME>
150 File status okay; about to open data connection.
551 <FILENAME>: Error on input file.
Re: FTP server active / passive mode UPDATE
Posted by Niklas Gustavsson <ni...@protocol7.com>.
On Thu, Dec 2, 2010 at 10:12 AM, Aidan Diffey <ai...@gmail.com> wrote:
> The problem has been fixed.. My code that was reading the database
> was throwing a NullPointerException which was causing the stream close
> problem.
>
> Also, moving the code from the FTPLet into a FileSystemView / Factory
> and FTPFile appears to have solved the Data Connection Issue.
Good to hear :-)
/niklas
Re: FTP server active / passive mode UPDATE
Posted by Aidan Diffey <ai...@gmail.com>.
Hello All.
The problem has been fixed.. My code that was reading the database
was throwing a NullPointerException which was causing the stream close
problem.
Also, moving the code from the FTPLet into a FileSystemView / Factory
and FTPFile appears to have solved the Data Connection Issue.
Thanks for the (massive) help.
Cheers
Re: FTP server active / passive mode UPDATE
Posted by Aidan Diffey <ai...@gmail.com>.
Hello.
That is interesting.
When I change the createOutputStream method to the following it
appears to work and doesn't throw the exception.
private static byte[] staticBytes = "I am a static byte string".getBytes();
/**
*
* @param offset
* @return
* @throws IOException
*/
public InputStream createInputStream(long offset) throws IOException
{
//Code in here to open the database and get stream,
try
{
return new ByteArrayInputStream(staticBytes);
}
catch (Exception e)
{
}
return null;
}
What I am doing is trying to transfer 300k of data from upto 40
virtual IP addresses. This may occur at the same time.
I have put my code back to the original code and it appears to be
working now. I will let it run for a bit and see what happens. I have
noticed that sometime when I restart the server on the windows machine
it will work, and other times it will not.
Cheers for the ongoing help
On 29 November 2010 19:50, Niklas Gustavsson <ni...@protocol7.com> wrote:
>> I have a single FTPLet coded that uses the beforeCommand and afterCommand.
>> The afterCommand on PASS sets the client IP address and the
>> MessageEndpointFactory.
>>
>>
>> When the NIOListener receives an RETR (from the server on any of the Virtual
>> IP addresses), it goes through the normal RETR class in Apache FTP server
>> and calls the MySystemFactory and classes in order to get the
>> ByteArrayOutputStream.
>
> To reduce the test case further, could you remove the IP handling
> stuff, the DAOs and the zipping and simple return a constant byte
> array. Are you still seeing the same problem? If so, please provide
> that as a use case.
>
> /niklas
>
Re: FTP server active / passive mode UPDATE
Posted by Niklas Gustavsson <ni...@protocol7.com>.
> I have a single FTPLet coded that uses the beforeCommand and afterCommand.
> The afterCommand on PASS sets the client IP address and the
> MessageEndpointFactory.
>
>
> When the NIOListener receives an RETR (from the server on any of the Virtual
> IP addresses), it goes through the normal RETR class in Apache FTP server
> and calls the MySystemFactory and classes in order to get the
> ByteArrayOutputStream.
To reduce the test case further, could you remove the IP handling
stuff, the DAOs and the zipping and simple return a constant byte
array. Are you still seeing the same problem? If so, please provide
that as a use case.
/niklas
Re: FTP server active / passive mode UPDATE
Posted by Aidan Diffey <ai...@gmail.com>.
On 29/11/2010 16:42, Niklas Gustavsson wrote:
> On Mon, Nov 29, 2010 at 12:50 PM, Aidan Diffey<ai...@gmail.com> wrote:
>> I have taken John's advise and implemented my own
>> FTPFileSystemFactory, View and file.
>>
>> I have implemented my FtpFile createInputStream to read the data from
>> the database and return a new ByteArrayOutputStream containing the
>> byte[] data read from the database.
>>
>> This has not solved the problem, BUT it has allowed me to view a stack
>> trace which might be helpful in diagnosing the problem. See stack
>> trace and some wire shark trace as well. From looking at this and the
>> code, it would appear that something has set the inputstream inside
>> the BufferedInputStream to null.
> Could you provide a test case (including some minimal code for your
> file system implementation independent on a DB) for this?
>
> /niklas
Hello.
Used case:
I have multiple Virtual IP addresses configured each with their own
Listener configured. My server is a FTP client (using Apache Commons
FTPClient) running on a Linux machine and my server is a Windows machine
running Apache FTPServer.
I have a single FTPLet coded that uses the beforeCommand and
afterCommand. The afterCommand on PASS sets the client IP address and
the MessageEndpointFactory.
When the NIOListener receives an RETR (from the server on any of the
Virtual IP addresses), it goes through the normal RETR class in Apache
FTP server and calls the MySystemFactory and classes in order to get the
ByteArrayOutputStream.
On the suggestion from John, I moved the code out of the FTPLet and into
these classes.
I am wondering if this is a problem with Windows rather than any
implementation.
---------------------------------
FileSystemFacotry:
public class MySystemFactory implements FileSystemFactory
/**
*
* @param user
* @return
* @throws FtpException
*/
public FileSystemView createFileSystemView(User user) throws
FtpException
{
synchronized (user)
{
return new MyFileSystemView(user);
}
}
---------------------------------
MyFileSystemView
public class MyFileSystemView implements FileSystemView
{
private String serverIPAddress;
private String clientIPAddress;
private MessageEndpointFactory factory = null;
private String cwd = "";
//Getters and setters for server and client IP Addresses and MessageEndpoint
/**
*
* @param user
*/
public MyFileSystemView (User user)
{
}
public void dispose()
{
// TODO Auto-generated method stub
}
/**
*
* @param fileName
* @return
* @throws FtpException
*/
public FtpFile getFile(String fileName) throws FtpException
{
MyFTPFile result = null;
try
{
fileName = fileName.toLowerCase();
if (fileName.contains("xml"))
{
result = new MyFTPFile(fileName, clientIPAddress,
ercuIPAddress);
result.setFactory(factory);
}
else
{
Record record = Log.getInstance().getRecord(fileName);
if (record != null)
{
result = new MyFTPFile(fileName, clientIPAddress,
ercuIPAddress);
}
}
}
catch (Exception e)
{
throw new FtpException(e.getMessage());
}
return result;
}
-----------------------
MyFTPFile
public class MyFTPFile implements FtpFile
private String fileName = "";
private String serverIPAddress = "";
private String clientIPAddress = "";
private MessageEndpointFactory factory = null;
/**
*
* @param directory
*/
public MyFTPFile(String fileName, String clientIPAddress, String
serverIPAddress )
{
this.fileName = fileName;
this.serverIPAddress = serverIPAddress ;
this.clientIPAddress = clientIPAddress;
}
/**
*
* @param offset
* @return
* @throws IOException
*/
public InputStream createInputStream(long offset) throws IOException
{
//Code in here to open the database and get stream,
try
{
//Check name
Record record = Log.getInstance().getRecord(fileName);
//DAO calling SQL
String deviceID =
Device.getInstance().getDeviceIdForIpAddress(clientIPAddress);
//DAO calling SQL
DeviceRecord device =
DeviceeEquipment.getInstance().getDeviceRecord(deviceID ); //DAO
calling SQL
byte[] zip = null;
zip = Encryption.getInstance().generateMessage(DeviceRecord)
zip = Encryption.compress(zip);
return new ByteArrayInputStream(zip);
}
catch (Exception e)
{
}
return null;
}
-------------------------------------
Re: FTP server active / passive mode UPDATE
Posted by Aidan Diffey <ai...@gmail.com>.
On 29/11/2010 16:42, Niklas Gustavsson wrote:
> On Mon, Nov 29, 2010 at 12:50 PM, Aidan Diffey<ai...@gmail.com> wrote:
>> I have taken John's advise and implemented my own
>> FTPFileSystemFactory, View and file.
>>
>> I have implemented my FtpFile createInputStream to read the data from
>> the database and return a new ByteArrayOutputStream containing the
>> byte[] data read from the database.
>>
>> This has not solved the problem, BUT it has allowed me to view a stack
>> trace which might be helpful in diagnosing the problem. See stack
>> trace and some wire shark trace as well. From looking at this and the
>> code, it would appear that something has set the inputstream inside
>> the BufferedInputStream to null.
> Could you provide a test case (including some minimal code for your
> file system implementation independent on a DB) for this?
>
> /niklas
Hello.
Not too sure if this sent first time. If it did, sorry.
Used case:
I have multiple Virtual IP addresses configured each with their own
Listener configured. My server is a FTP client (using Apache Commons
FTPClient) running on a Linux machine and my server is a Windows machine
running Apache FTPServer.
I have a single FTPLet coded that uses the beforeCommand and
afterCommand. The afterCommand on PASS sets the client IP address and
the MessageEndpointFactory.
When the NIOListener receives an RETR (from the server on any of the
Virtual IP addresses), it goes through the normal RETR class in Apache
FTP server and calls the MySystemFactory and classes in order to get the
ByteArrayOutputStream.
On the suggestion from John, I moved the code out of the FTPLet and into
these classes.
I am wondering if this is a problem with Windows rather than any
implementation.
---------------------------------
FileSystemFacotry:
public class MySystemFactory implements FileSystemFactory
/**
*
* @param user
* @return
* @throws FtpException
*/
public FileSystemView createFileSystemView(User user) throws
FtpException
{
synchronized (user)
{
return new MyFileSystemView(user);
}
}
---------------------------------
MyFileSystemView
public class MyFileSystemView implements FileSystemView
{
private String serverIPAddress;
private String clientIPAddress;
private MessageEndpointFactory factory = null;
private String cwd = "";
//Getters and setters for server and client IP Addresses and
MessageEndpoint
/**
*
* @param user
*/
public MyFileSystemView (User user)
{
}
public void dispose()
{
// TODO Auto-generated method stub
}
/**
*
* @param fileName
* @return
* @throws FtpException
*/
public FtpFile getFile(String fileName) throws FtpException
{
MyFTPFile result = null;
try
{
fileName = fileName.toLowerCase();
if (fileName.contains("xml"))
{
result = new MyFTPFile(fileName, clientIPAddress,
ercuIPAddress);
result.setFactory(factory);
}
else
{
Record record = Log.getInstance().getRecord(fileName);
if (record != null)
{
result = new MyFTPFile(fileName, clientIPAddress,
ercuIPAddress);
}
}
}
catch (Exception e)
{
throw new FtpException(e.getMessage());
}
return result;
}
-----------------------
MyFTPFile
public class MyFTPFile implements FtpFile
private String fileName = "";
private String serverIPAddress = "";
private String clientIPAddress = "";
private MessageEndpointFactory factory = null;
/**
*
* @param directory
*/
public MyFTPFile(String fileName, String clientIPAddress, String
serverIPAddress )
{
this.fileName = fileName;
this.serverIPAddress = serverIPAddress ;
this.clientIPAddress = clientIPAddress;
}
/**
*
* @param offset
* @return
* @throws IOException
*/
public InputStream createInputStream(long offset) throws IOException
{
//Code in here to open the database and get stream,
try
{
//Check name
Record record = Log.getInstance().getRecord(fileName);
//DAO calling SQL
String deviceID =
Device.getInstance().getDeviceIdForIpAddress(clientIPAddress);
//DAO calling SQL
DeviceRecord device =
DeviceeEquipment.getInstance().getDeviceRecord(deviceID ); //DAO
calling SQL
byte[] zip = null;
zip = Encryption.getInstance().generateMessage(DeviceRecord)
zip = Encryption.compress(zip);
return new ByteArrayInputStream(zip);
}
catch (Exception e)
{
}
return null;
}
-------------------------------------
Re: FTP server active / passive mode UPDATE
Posted by Niklas Gustavsson <ni...@protocol7.com>.
On Mon, Nov 29, 2010 at 12:50 PM, Aidan Diffey <ai...@gmail.com> wrote:
> I have taken John's advise and implemented my own
> FTPFileSystemFactory, View and file.
>
> I have implemented my FtpFile createInputStream to read the data from
> the database and return a new ByteArrayOutputStream containing the
> byte[] data read from the database.
>
> This has not solved the problem, BUT it has allowed me to view a stack
> trace which might be helpful in diagnosing the problem. See stack
> trace and some wire shark trace as well. From looking at this and the
> code, it would appear that something has set the inputstream inside
> the BufferedInputStream to null.
Could you provide a test case (including some minimal code for your
file system implementation independent on a DB) for this?
/niklas