You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@commons.apache.org by sebb <se...@gmail.com> on 2017/01/22 14:37:29 UTC
Re: [NET] Errorhandling of "bad" FTP Servers
Please prefix the subject line with the Commons Component in future (as above)
On 22 January 2017 at 09:12, Oliver Zemann <ol...@gmail.com> wrote:
> Hi, i would like to know how to handle FTP Servers that do not behave like
> they should (or behave different than most others).
> For example, currently i would like to connect to one of those servers which
> sends " MODE Z" directly after a login (USER) which is interpreted wrong by
> apache commons ftp, as its assumed that there should be \d\d\d \w* like "220
> all good my friend"
> But its simply: " MODE Z"
>
> Here is a log from filezilla which can handle this behaviour:
> 09:58:24 Trace: CFtpControlSocket::SendNextCommand()
> 09:58:24 Befehl: USER someUser
> 09:58:24 Trace: CFtpControlSocket::OnReceive()
> 09:58:24 Antwort: 331 [33m ~wait~ user ok, password?
> 09:58:24 Trace: CFtpControlSocket::SendNextCommand()
> 09:58:24 Befehl: PASS *******
> 09:58:25 Trace: CFtpControlSocket::OnReceive()
> 09:58:25 Antwort: 230 [32m.::WAIT::. welcome!.
The login has completed here
> 09:58:25 Trace: CFtpControlSocket::SendNextCommand()
> 09:58:25 Befehl: SYST
> 09:58:25 Trace: CFtpControlSocket::OnReceive()
> 09:58:25 Antwort: 215 Java (SE) Platform SE 6.0
> 09:58:25 Trace: CFtpControlSocket::SendNextCommand()
> 09:58:25 Befehl: FEAT
> 09:58:25 Trace: CFtpControlSocket::OnReceive()
> 09:58:25 Antwort: 211-Extension supported
> 09:58:25 Antwort: MDTM
> 09:58:25 Antwort: MDTM
> 09:58:25 Antwort: MDTM YYYYMMDDHHMMSS[+-TZ];filename
> 09:58:25 Antwort: SIZE
> 09:58:25 Antwort: SITE PSWD;EXEC;SET;INDEX;ZONE;CHMOD;MSG
> 09:58:25 Antwort: XCRC filename;start;end
> 09:58:25 Antwort: 211 End
> 09:58:25 Status: Der Server unterstützt keine Nicht-ASCII-Zeichen.
> 09:58:25 Status: Angemeldet
> 09:58:25 Antwort: MODE Z
> 09:58:25 Trace: Unexpected reply, no reply was pending. <----
> something like this in apache commons ftp???
Note that FileZilla detects this as an error, so clearly this is not
expected behaviour.
Commons NET only looks for a reply when one is expected, so it cannot
detect this.
AFAICS changing this would require a major rewrite.
Since the server appears to be broken, it's not worth doing this.
> 09:58:25 Antwort: 211 End
> 09:58:25 Trace: Unexpected reply, no reply was pending.
> 09:58:25 Status: Empfange Verzeichnisinhalt...
> 09:58:25 Trace: CFtpControlSocket::SendNextCommand()
> 09:58:25 Trace: CFtpControlSocket::ChangeDirSend()
> 09:58:25 Befehl: PWD
> 09:58:25 Trace: CFtpControlSocket::OnReceive()
> 09:58:25 Antwort: 257 "/" [34m<<<>>>OK<<<>>> Logged in ;).
>
> That "MODE Z" is leading to an exception when i use FTPClient::login()
> In __getReply(boolean reportReply) i can see that its trying to convert the
> first 3 chars to a digit, which is of course impossible if its "Z MODE" and
> not a number.
>
> So i am wondering whats the best way to handle such servers? Or is this a
> bug in apache commons ftp? Do i have to implement all calls myself and dont
> rely on login() logout() etc. from apache commons to be able to handle such
> behaviours?
This seems to be a bug in the server.
It is not supposed to send unsolicited data on the command channel.
You might be able to subclass one of the FTP classes and over-ride one
of the methods so it ignores MODE Z if it's not expected.
> Some information: i dont have control over that server. I dont know what
> software the server is running.
Try contacting the server maintainers to raise the issue of the
server's bad behaviour
> Regards
> Oli
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [NET] Errorhandling of "bad" FTP Servers
Posted by sebb <se...@gmail.com>.
On 23 January 2017 at 07:31, Oliver Zemann <ol...@gmail.com> wrote:
>
>
>>> Yes, but apache commons sends first FEAT when i use autodetectUTF8(),
>>> which
>>> is imo wrong.
>>> The javadoc states that autodetectUTF8() has to be used before connect(),
>>> but i dont understand why.
>>
>> setAutodetectUTF8 only sets a flag which is consulted during connect;
>> it does not send anything to the server.
>
> When autodetectUTF8() is set to true, after the connect, apache commons net
> sends a FEAT to check if the server supports UTF8. This is what FileZilla
> does not. Therefore, with FileZilla its working.
> When i remove autodetectUTF8(true) in my code, it works. But thats no
> solution to me because i would like to have UTF8 support.
>>
>>
>>> FileZilla first connects and does the login() and after that it uses the
>>> FEAT command to check if the server supports UTF8.
>>> The server behaves "bad" of that FEAT command, because it sends 211 END
>>> but
>>> then outputs MODE Z (which should before 211 END). And FEAT is issued
>>> because of autodetectUTF8()
>>
>> That is perfectly reasonable behaviour for FileZilla and for Commons NET.
>
> commons net sends the FEAT _before_ the USER command is send. Thats a big
> difference in FileZilla and commons net.
But it does so only if setAutodetectUTF8 is true.
So another workround is not to set that.
But in any case, the server would still send the MODE Z incorrectly,
as proven by the FileZilla session log.
>>
>> No, there is no such method.
>>
>> You could try calling getReply() and catching IO errors.
>> However if there is no data to read that will likely hang.
>> It so, try setting a timeout.
>>
>> You could also try subclassing FTPClient and then you can access the
>> _controlInput_ field.
>> This is where the replies are read from.
>>
>> However reading from the control channel may cause NET to break in subtle
>> ways.
>
> Thanks, guess i will try it that way.
>>
>> AFAIK this is the first time this problem has been reported.
>> Which seems odd if there really are thousands of servers with this
>> behaviour.
>
> I did not mean "thousands of servers with this behaviour" i meant: there are
> thousands of servers i access, i can contact them and tell them they have
> broken ftp server software, but they probably simply don't care.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [NET] Errorhandling of "bad" FTP Servers
Posted by Oliver Zemann <ol...@gmail.com>.
>> Yes, but apache commons sends first FEAT when i use autodetectUTF8(), which
>> is imo wrong.
>> The javadoc states that autodetectUTF8() has to be used before connect(),
>> but i dont understand why.
> setAutodetectUTF8 only sets a flag which is consulted during connect;
> it does not send anything to the server.
When autodetectUTF8() is set to true, after the connect, apache commons
net sends a FEAT to check if the server supports UTF8. This is what
FileZilla does not. Therefore, with FileZilla its working.
When i remove autodetectUTF8(true) in my code, it works. But thats no
solution to me because i would like to have UTF8 support.
>
>> FileZilla first connects and does the login() and after that it uses the
>> FEAT command to check if the server supports UTF8.
>> The server behaves "bad" of that FEAT command, because it sends 211 END but
>> then outputs MODE Z (which should before 211 END). And FEAT is issued
>> because of autodetectUTF8()
> That is perfectly reasonable behaviour for FileZilla and for Commons NET.
commons net sends the FEAT _before_ the USER command is send. Thats a
big difference in FileZilla and commons net.
> No, there is no such method.
>
> You could try calling getReply() and catching IO errors.
> However if there is no data to read that will likely hang.
> It so, try setting a timeout.
>
> You could also try subclassing FTPClient and then you can access the
> _controlInput_ field.
> This is where the replies are read from.
>
> However reading from the control channel may cause NET to break in subtle ways.
Thanks, guess i will try it that way.
> AFAIK this is the first time this problem has been reported.
> Which seems odd if there really are thousands of servers with this behaviour.
I did not mean "thousands of servers with this behaviour" i meant: there
are thousands of servers i access, i can contact them and tell them they
have broken ftp server software, but they probably simply don't care.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [NET] Errorhandling of "bad" FTP Servers
Posted by sebb <se...@gmail.com>.
On 22 January 2017 at 16:37, Oliver Zemann <ol...@gmail.com> wrote:
>
>> Please prefix the subject line with the Commons Component in future (as
>> above)
>
> Sorry, next time i will do that.
>>
>>
>> On 22 January 2017 at 09:12, Oliver Zemann <ol...@gmail.com>
>> wrote:
>>>
>>> Hi, i would like to know how to handle FTP Servers that do not behave
>>> like
>>> they should (or behave different than most others).
>>> For example, currently i would like to connect to one of those servers
>>> which
>>> sends " MODE Z" directly after a login (USER) which is interpreted wrong
>>> by
>>> apache commons ftp, as its assumed that there should be \d\d\d \w* like
>>> "220
>>> all good my friend"
>>> But its simply: " MODE Z"
>>>
>>> Here is a log from filezilla which can handle this behaviour:
>>> 09:58:24 Trace: CFtpControlSocket::SendNextCommand()
>>> 09:58:24 Befehl: USER someUser
>>> 09:58:24 Trace: CFtpControlSocket::OnReceive()
>>> 09:58:24 Antwort: 331 [33m ~wait~ user ok, password?
>>> 09:58:24 Trace: CFtpControlSocket::SendNextCommand()
>>> 09:58:24 Befehl: PASS *******
>>> 09:58:25 Trace: CFtpControlSocket::OnReceive()
>>> 09:58:25 Antwort: 230 [32m.::WAIT::. welcome!.
>>
>> The login has completed here
>
> Yes, but apache commons sends first FEAT when i use autodetectUTF8(), which
> is imo wrong.
> The javadoc states that autodetectUTF8() has to be used before connect(),
> but i dont understand why.
setAutodetectUTF8 only sets a flag which is consulted during connect;
it does not send anything to the server.
> FileZilla first connects and does the login() and after that it uses the
> FEAT command to check if the server supports UTF8.
> The server behaves "bad" of that FEAT command, because it sends 211 END but
> then outputs MODE Z (which should before 211 END). And FEAT is issued
> because of autodetectUTF8()
That is perfectly reasonable behaviour for FileZilla and for Commons NET.
> Would it be possible to read from _controlInput_ until null is returned?
> Seems like someone else encountered something with another server, and
> therefore __ lenientCheck() exists?
>>>
>>> 09:58:25 Befehl: FEAT
>>> 09:58:25 Trace: CFtpControlSocket::OnReceive()
>>> 09:58:25 Antwort: 211-Extension supported
>>> 09:58:25 Antwort: MDTM
>>> 09:58:25 Antwort: MDTM
>>> 09:58:25 Antwort: MDTM YYYYMMDDHHMMSS[+-TZ];filename
>>> 09:58:25 Antwort: SIZE
>>> 09:58:25 Antwort: SITE PSWD;EXEC;SET;INDEX;ZONE;CHMOD;MSG
>>> 09:58:25 Antwort: XCRC filename;start;end
>>> 09:58:25 Antwort: 211 End
>>> 09:58:25 Status: Der Server unterstützt keine Nicht-ASCII-Zeichen.
>>> 09:58:25 Status: Angemeldet
>>> 09:58:25 Antwort: MODE Z
>>> 09:58:25 Trace: Unexpected reply, no reply was pending. <----
>>> something like this in apache commons ftp???
>>
>> Note that FileZilla detects this as an error, so clearly this is not
>> expected behaviour.
>>
>> Commons NET only looks for a reply when one is expected, so it cannot
>> detect this.
>> AFAICS changing this would require a major rewrite.
>> Since the server appears to be broken, it's not worth doing this.
>
> That means, if a server answers a request wrong (like on that server with
> FEAT, because MODE Z should be before 211 END), all further communication
> with apache commons net ftp is impossible to that server because
> _controlInput_ is not "cleared"?
> Is there some kind of "clearControlInput()" method the user can invoke to
> "clear" all (possible) invalid data to make the ftp client more "stable"
> when it has to deal with such ftp servers? Because then i could simply call
> "clearControlInput()" each time before i send my commands.
No, there is no such method.
You could try calling getReply() and catching IO errors.
However if there is no data to read that will likely hang.
It so, try setting a timeout.
You could also try subclassing FTPClient and then you can access the
_controlInput_ field.
This is where the replies are read from.
However reading from the control channel may cause NET to break in subtle ways.
>>
>>> Some information: i dont have control over that server. I dont know what
>>> software the server is running.
>>
>> Try contacting the server maintainers to raise the issue of the
>> server's bad behaviour
>
> Unfortunately this is one of many thousands of servers, located somewhere...
> china, russia, usa etc.
AFAIK this is the first time this problem has been reported.
Which seems odd if there really are thousands of servers with this behaviour.
> I guess some of those servers are even not maintained anymore. So this is
> not a solution. I have to somehow handle those servers :(
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [NET] Errorhandling of "bad" FTP Servers
Posted by Oliver Zemann <ol...@gmail.com>.
> Please prefix the subject line with the Commons Component in future (as above)
Sorry, next time i will do that.
>
> On 22 January 2017 at 09:12, Oliver Zemann <ol...@gmail.com> wrote:
>> Hi, i would like to know how to handle FTP Servers that do not behave like
>> they should (or behave different than most others).
>> For example, currently i would like to connect to one of those servers which
>> sends " MODE Z" directly after a login (USER) which is interpreted wrong by
>> apache commons ftp, as its assumed that there should be \d\d\d \w* like "220
>> all good my friend"
>> But its simply: " MODE Z"
>>
>> Here is a log from filezilla which can handle this behaviour:
>> 09:58:24 Trace: CFtpControlSocket::SendNextCommand()
>> 09:58:24 Befehl: USER someUser
>> 09:58:24 Trace: CFtpControlSocket::OnReceive()
>> 09:58:24 Antwort: 331 [33m ~wait~ user ok, password?
>> 09:58:24 Trace: CFtpControlSocket::SendNextCommand()
>> 09:58:24 Befehl: PASS *******
>> 09:58:25 Trace: CFtpControlSocket::OnReceive()
>> 09:58:25 Antwort: 230 [32m.::WAIT::. welcome!.
> The login has completed here
Yes, but apache commons sends first FEAT when i use autodetectUTF8(),
which is imo wrong.
The javadoc states that autodetectUTF8() has to be used before
connect(), but i dont understand why.
FileZilla first connects and does the login() and after that it uses the
FEAT command to check if the server supports UTF8.
The server behaves "bad" of that FEAT command, because it sends 211 END
but then outputs MODE Z (which should before 211 END). And FEAT is
issued because of autodetectUTF8()
Would it be possible to read from _controlInput_ until null is returned?
Seems like someone else encountered something with another server, and
therefore __ lenientCheck() exists?
>> 09:58:25 Befehl: FEAT
>> 09:58:25 Trace: CFtpControlSocket::OnReceive()
>> 09:58:25 Antwort: 211-Extension supported
>> 09:58:25 Antwort: MDTM
>> 09:58:25 Antwort: MDTM
>> 09:58:25 Antwort: MDTM YYYYMMDDHHMMSS[+-TZ];filename
>> 09:58:25 Antwort: SIZE
>> 09:58:25 Antwort: SITE PSWD;EXEC;SET;INDEX;ZONE;CHMOD;MSG
>> 09:58:25 Antwort: XCRC filename;start;end
>> 09:58:25 Antwort: 211 End
>> 09:58:25 Status: Der Server unterst�tzt keine Nicht-ASCII-Zeichen.
>> 09:58:25 Status: Angemeldet
>> 09:58:25 Antwort: MODE Z
>> 09:58:25 Trace: Unexpected reply, no reply was pending. <----
>> something like this in apache commons ftp???
> Note that FileZilla detects this as an error, so clearly this is not
> expected behaviour.
>
> Commons NET only looks for a reply when one is expected, so it cannot
> detect this.
> AFAICS changing this would require a major rewrite.
> Since the server appears to be broken, it's not worth doing this.
That means, if a server answers a request wrong (like on that server
with FEAT, because MODE Z should be before 211 END), all further
communication with apache commons net ftp is impossible to that server
because _controlInput_ is not "cleared"?
Is there some kind of "clearControlInput()" method the user can invoke
to "clear" all (possible) invalid data to make the ftp client more
"stable" when it has to deal with such ftp servers? Because then i could
simply call "clearControlInput()" each time before i send my commands.
>
>> Some information: i dont have control over that server. I dont know what
>> software the server is running.
> Try contacting the server maintainers to raise the issue of the
> server's bad behaviour
Unfortunately this is one of many thousands of servers, located
somewhere... china, russia, usa etc.
I guess some of those servers are even not maintained anymore. So this
is not a solution. I have to somehow handle those servers :(
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org