You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@commons.apache.org by Jens Viebig <js...@gmx.de> on 2005/09/01 18:09:59 UTC
[VFS] FTPClientWrapper Broken Pipe
Hello List,
i'm using VFS to regulary check some files on an FTP server. This works fine for a long time, but after a while the check crashes with the following stacktrace:
org.apache.commons.vfs.FileSystemException: Could not determine the type of file ftp://user:password@myserver.com/myfile.txt.
at org.apache.commons.vfs.provider.AbstractFileObject.attach()V(Optimized Method)
at org.apache.commons.vfs.provider.AbstractFileObject.exists()Z(AbstractFileObject.java:391)
at my.class.Something.toDo(Unknown Source)
at java.lang.Thread.run()V(Unknown Source)
at java.lang.Thread.startThreadFromVM(Ljava.lang.Thread;)V(Unknown Source)
Caused by: java.net.SocketException: Broken pipe
at jrockit.net.SocketNativeIO.write(IIIZ)I(Unknown Source)
at jrockit.net.SocketNativeIO.write(III)I(Unknown Source)
at jrockit.io.NativeIO.write(Ljava.io.FileDescriptor;II)I(Optimized Method)
at java.net.AbstractSocketImpl$2.write(II)V(Optimized Method)
at jrockit.io.NativeIOOutputStream.write([BII)V(Optimized Method)
at java.io.BufferedOutputStream.flushBuffer()V(Optimized Method)
at java.io.BufferedOutputStream.write(I)V(Optimized Method)
at org.apache.commons.net.telnet.Telnet._sendByte(I)V(Optimized Method)
at org.apache.commons.net.telnet.TelnetOutputStream.write(I)V(Optimized Method)
at org.apache.commons.net.io.ToNetASCIIOutputStream.write([BII)V(Optimized Method)
at sun.nio.cs.StreamEncoder$CharsetSE.writeBytes()V(StreamEncoder.java:325)
at sun.nio.cs.StreamEncoder$CharsetSE.implWrite([CII)V(Optimized Method)
at sun.nio.cs.StreamEncoder.write([CII)V(StreamEncoder.java:136)
at java.io.OutputStreamWriter.write([CII)V(OutputStreamWriter.java:191)
at java.io.BufferedWriter.flushBuffer()V(BufferedWriter.java:107)
at java.io.BufferedWriter.flush()V(BufferedWriter.java:228)
at org.apache.commons.net.ftp.FTP.sendCommand(Ljava.lang.String;Ljava.lang.String;)I(Optimized Method)
at org.apache.commons.net.ftp.FTP.sendCommand(ILjava.lang.String;)I(Optimized Method)
at org.apache.commons.net.ftp.FTP.port(Ljava.net.InetAddress;I)I(Optimized Method)
at org.apache.commons.vfs.provider.ftp.FTPClientWrapper.listFiles(Ljava.lang.String;Ljava.lang.String;)[Lorg.apache.commons.net.ftp.FTPFile;(Optimized Method)
at org.apache.commons.vfs.provider.ftp.FtpFileObject.doGetChildren()V(Optimized Method)
at org.apache.commons.vfs.provider.ftp.FtpFileObject.getChildFile(Ljava.lang.String;Z)Lorg.apache.commons.net.ftp.FTPFile;(Optimized Method)
at org.apache.commons.vfs.provider.ftp.FtpFileObject.getInfo(Z)V(Optimized Method)
at org.apache.commons.vfs.provider.ftp.FtpFileObject.doAttach()V(Optimized Method)
... 6 more
When it comes to that error, the filesystem provider never recovers from that crash and all actions on files on the same ftp server fail. The ftp server works fine, i can connect fine from the commandline. it seems that the VFS ftp provider tries to reuse that old damaged connection, i don't see a new attempt to login in my ftp server log. There is no connection from VFS to the ftp server.
I looked in the source code of FTPClientWrapper, and tracked the error down to the listFiles function:
...
public FTPFile[] listFiles(String key, String relPath) throws IOException
{
try
{
return getFtpClient().listFiles(key, relPath);
}
catch (FTPConnectionClosedException e)
{
disconnect();
return getFtpClient().listFiles(key, relPath);
}
}
...
the FTPClient of commons.net throws FTPConnectionClosedException and IOException. if a FTPConnectionClosedException is thrown everything is fine, the old connection is closed and a new one is created. If a more general IOException is thrown the connection is not closed and not set to null, so the FTPClientWrapper tries to use it again. Normally isConnected() should fail then, but it seems that it still returns true und is a weak test. I suggest that the connection should also be disconnected and set to null on more general errors, if there is an IOException there is a big chance that the connection is not usable anymore.
Maybe like this:
public FTPFile[] listFiles(String key, String relPath) throws IOException
{
try
{
return getFtpClient().listFiles(key, relPath);
}
catch (FTPConnectionClosedException e)
{
disconnect();
return getFtpClient().listFiles(key, relPath);
}
catch (IOException ie)
{
disconnect();
throw ie;
}
}
This fix should be applied to all functions in FTPClientWrapper
Maybe you have a better approach on fixing this issue ?
Re: [VFS] FTPClientWrapper Broken Pipe
Posted by Mario Ivankovits <ma...@ops.co.at>.
Hi Jens!
> With the newest svn checkout i have a new issue. VFS can't create a
> new connection to the ftp server, i get an error:
> On my ftp server log it seems that the client sends an CWD command
> without a directory path, which causes the server to send a syntax error
Thanks for the detailed report.
Yes, there is some testing code in the current head too, I havent had
the time to run the tests.
Sorry for the inconvenience, it should be fixed now.
Ciao,
Mario
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org
Re: [VFS] FTPClientWrapper Broken Pipe
Posted by Jens Viebig <js...@gmx.de>.
Thanks for the quick patch. I'm testing the patched FTPClientWrapper with
the sourcecode of commons-VFS-RC2 and it seems to work fine.
With the newest svn checkout i have a new issue. VFS can't create a new
connection to the ftp server, i get an error:
Caused by:
org.apache.commons.vfs.org.apache.commons.vfs.FileSystemException: Could not
connect to FTP server on "10.1.1.42".
at
org.apache.commons.vfs.provider.ftp.FtpClientFactory.createConnection(Ljava.lang.String;ILjava.lang.String;Ljava.lang.String;Ljava.lang.String;Lorg.apache.commons.vfs.FileSystemOptions;)Lorg.apache.commons.net.ftp.FTPClient;(FtpClientFactory.java:120)
at
org.apache.commons.vfs.provider.ftp.FTPClientWrapper.createClient()Lorg.apache.commons.net.ftp.FTPClient;(FTPClientWrapper.java:60)
at
org.apache.commons.vfs.provider.ftp.FTPClientWrapper.getFtpClient()Lorg.apache.commons.net.ftp.FTPClient;(FTPClientWrapper.java:72)
at
org.apache.commons.vfs.provider.ftp.FTPClientWrapper.<init>(Lorg.apache.commons.vfs.provider.GenericFileName;Lorg.apache.commons.vfs.FileSystemOptions;)V(FTPClientWrapper.java:43)
at
org.apache.commons.vfs.provider.ftp.FtpFileProvider.doCreateFileSystem(Lorg.apache.commons.vfs.FileName;Lorg.apache.commons.vfs.FileSystemOptions;)Lorg.apache.commons.vfs.FileSystem;(FtpFileProvider.java:74)
at
org.apache.commons.vfs.provider.AbstractOriginatingFileProvider.findFile(Lorg.apache.commons.vfs.FileName;Lorg.apache.commons.vfs.FileSystemOptions;)Lorg.apache.commons.vfs.FileObject;(AbstractOriginatingFileProvider.java:76)
at
org.apache.commons.vfs.provider.AbstractOriginatingFileProvider.findFile(Lorg.apache.commons.vfs.FileObject;Ljava.lang.String;Lorg.apache.commons.vfs.FileSystemOptions;)Lorg.apache.commons.vfs.FileObject;(AbstractOriginatingFileProvider.java:61)
at
org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(Lorg.apache.commons.vfs.FileObject;Ljava.lang.String;Lorg.apache.commons.vfs.FileSystemOptions;)Lorg.apache.commons.vfs.FileObject;(DefaultFileSystemManager.java:524)
at
org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(Lorg.apache.commons.vfs.FileObject;Ljava.lang.String;)Lorg.apache.commons.vfs.FileObject;(DefaultFileSystemManager.java:488)
at
org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(Ljava.lang.String;)Lorg.apache.commons.vfs.FileObject;(DefaultFileSystemManager.java:458)
at
com.como.commonwatchfolder.CommonWatchfolderService.run()V(Unknown Source)
at java.lang.Thread.run()V(Unknown Source)
at java.lang.Thread.startThreadFromVM(Ljava.lang.Thread;)V(Unknown
Source)
Caused by: org.apache.commons.vfs.FileSystemException: Could not change to
work directory "/".
at
org.apache.commons.vfs.provider.ftp.FtpClientFactory.createConnection(Ljava.lang.String;ILjava.lang.String;Ljava.lang.String;Ljava.lang.String;Lorg.apache.commons.vfs.FileSystemOptions;)Lorg.apache.commons.net.ftp.FTPClient;(FtpClientFactory.java:97)
On my ftp server log it seems that the client sends an CWD command without a
directory path, which causes the server to send a syntax error
Regards
Jens Viebig
----- Original Message -----
From: "Mario Ivankovits" <ma...@ops.co.at>
To: "Jakarta Commons Users List" <co...@jakarta.apache.org>
Sent: Friday, September 02, 2005 7:52 AM
Subject: Re: [VFS] FTPClientWrapper Broken Pipe
> Jens Viebig wrote:
>> Hope this will be fixed in official future releases of commons-VFS
> I've changed it now, maybe you can check it.
>
>
> Thanks!
> Mario
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org
Re: [VFS] FTPClientWrapper Broken Pipe
Posted by Mario Ivankovits <ma...@ops.co.at>.
Jens Viebig wrote:
> Hope this will be fixed in official future releases of commons-VFS
I've changed it now, maybe you can check it.
Thanks!
Mario
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org
Re: [VFS] FTPClientWrapper Broken Pipe
Posted by Jens Viebig <js...@gmx.de>.
Hello, how do you set a timeout on the client side ?
My ftp server times out after 5 minutes, normally the connection
restarts as expected.
My code runs inside a background service that is running 24/7 so there
are a lot of things that can cause an 'unexpected' IOException. A
physical network is never perfect.
Im using bea jrockit jvm that is 1.4 compatible
Im using commons-net 1.4
Yes it seems that isConnected() that is used in FTPClientWrapper does
not do a physical test. I think i will try to replace
all catch blocks
catch (FTPConnectionClosedException e)
with
catch(IOException e)
in FTPClientWrapper
FTPConnectionClosedException is a subclass of IOException so there is
no need to have two catch blocks.
Hope this will be fixed in official future releases of commons-VFS
Regards
Andrew Elcock wrote:
> What I did to get round this is to apply a timeout and after a certain
> time I "restart" the connection. I hope this works for you to...
>
> Mario Ivankovits wrote:
>
>> Hi!
>>
>>> Caused by: java.net.SocketException: Broken pipe
>>>
>>
>>
>> Do you know why this happens? Some sort of ftp timeout?
>>
>>> catch (IOException ie)
>>> {
>>> disconnect();
>>> throw ie;
>>> }
>>>
>>
>>
>> I ask as maybe we should retry the operation here too?
>>
>>
>>> Normally isConnected() should fail then, but it seems that it still
>>> returns true
>>
>>
>> isConnected() in ftp.java is a logical operation if the connection
>> "should be" established, not a physical check.
>> There is also a socketIsConnected which tries to check if the
>> connections is really alive but it works only with jdk1.4 and up.
>>
>> Which jdk do you use?
>> If its 1.4 or up, which commons-net version do you use?
>>
>> ---
>> Mario
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org
Re: [VFS] FTPClientWrapper Broken Pipe
Posted by Andrew Elcock <an...@elcock.org>.
What I did to get round this is to apply a timeout and after a certain
time I "restart" the connection. I hope this works for you to...
Mario Ivankovits wrote:
> Hi!
>
>> Caused by: java.net.SocketException: Broken pipe
>>
>
> Do you know why this happens? Some sort of ftp timeout?
>
>> catch (IOException ie)
>> {
>> disconnect();
>> throw ie;
>> }
>>
>
> I ask as maybe we should retry the operation here too?
>
>
>> Normally isConnected() should fail then, but it seems that it still
>> returns true
>
> isConnected() in ftp.java is a logical operation if the connection
> "should be" established, not a physical check.
> There is also a socketIsConnected which tries to check if the
> connections is really alive but it works only with jdk1.4 and up.
>
> Which jdk do you use?
> If its 1.4 or up, which commons-net version do you use?
>
> ---
> Mario
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org
Re: [VFS] FTPClientWrapper Broken Pipe
Posted by Mario Ivankovits <ma...@ops.co.at>.
Hi!
>Caused by: java.net.SocketException: Broken pipe
>
Do you know why this happens? Some sort of ftp timeout?
> catch (IOException ie)
> {
> disconnect();
> throw ie;
> }
>
I ask as maybe we should retry the operation here too?
>Normally isConnected() should fail then, but it seems that it still returns true
isConnected() in ftp.java is a logical operation if the connection
"should be" established, not a physical check.
There is also a socketIsConnected which tries to check if the
connections is really alive but it works only with jdk1.4 and up.
Which jdk do you use?
If its 1.4 or up, which commons-net version do you use?
---
Mario
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org