You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Bengt Rodehav <be...@rodehav.com> on 2010/06/16 14:44:04 UTC
"Unconnected sockets not implemented" exception in camel-ftp when
using ftps
I have mentioned this problem in a previous conversation but after
investigating the subject further I decided to start a new thread for this.
If some kind of problem is encountered while using ftps (I tested this using
Filezilla server by copying a file that already existed which Filezilla does
not allow), then at the next attempt to connect to the ftps server, I get
the following exception in my logfile:
13:03:27,606 | ERROR | %7Bfile%3Aext%7D | GenericFileOnCompletion |
rg.apache.camel.processor.Logger 248 | Caused by:
[org.apache.camel.component.file.GenericFileOperationFailedException - File
operation failed: null Unconnected sockets not implemented. Code: 221]
org.apache.camel.component.file.GenericFileOperationFailedException: File
operation failed: null Unconnected sockets not implemented. Code: 221
at
org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:108)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
at
org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
at
org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
at
org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:123)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
at
org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
at
org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
at
org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:91)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
I added some logging to see the actual stack trace which is:
java.net.SocketException: Unconnected sockets not implemented
at javax.net.SocketFactory.createSocket(SocketFactory.java:104)
at
org.apache.commons.net.SocketClient.connect(SocketClient.java:175)
at
org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:91)
at
org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)
at
org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)
at
org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:99)
at
org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)
at
org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)
commons-net SocketClient class tries to call the socket factory's
createSocket() method (with no parameters):
_socket_= _socketFactory_.createSocket();
_socket_.connect(new InetSocketAddress(hostname, port),
connectTimeout);
Then intention is to create an "unconnected socket" (hence the error
message) and then subsequently connect it. In my case, the connection
factory being used is an FTPSSocketFactory
(package org.apache.commons.net.ftp) that inherits from
SocketFactory(package javax.net). The SocketFactory class implements the
createSocket() method but throws the exception ("Unconnected sockets not
implemented").
The reason why this happens is that after having executed the
FTPSClient.execPROT() method (package org.apache.commons.net.ftp), with any
other parameter than "C" (in my case I used "P"), the socket factory is set
to an FTPSSocketFactory as follows:
setSocketFactory(new FTPSSocketFactory(context));
After this, no connection attempt will succeed. This problem has been
introduced recently when Claus (on my initiative) added support for secure
data channel in ftps. I'm thus to blame...
I'm not sure how this should be fixed.
It's a bit strange that all the connect methods
in org.apache.commons.net.SocketClient (which is the ultimate base class of
FTPSClient) always try to create an unconnected socket first and then
connects it. If commons-net uses this pattern, then it should make sure that
all its connection factor's support unconnected sockets but the
FTPSSocketFactory dosn't. This sounds like a bug in commons-net.
Knowing that commons-net doesn't release very frequently I think we need a
workaround for this. I guess it would be possible to subclass the FTPClient
class and override all the connect methods to make sure that no attempt is
made to create unconnected sockets. We would then change the
createFtpClient() method in the FtpsEndpoint class to instantiate our own
subclass instead of an FTPSClient. Doesn't sound like a nice clean solution
though.
Any ideas?
/Bengt
Re: "Unconnected sockets not implemented" exception in camel-ftp when
using ftps
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Jun 21, 2010 at 10:36 AM, Claus Ibsen <cl...@gmail.com> wrote:
> On Mon, Jun 21, 2010 at 10:22 AM, Bengt Rodehav <be...@rodehav.com> wrote:
>> Claus,
>>
>> Just tried and it works. Very simple way of achieving our goals I must say.
>> Two questions:
>>
>
> It was just a quick test to see if that resolved it here and now as a
> workaround.
>
>
>> 1. How does the consumer side work? I haven't tried it yet but wouldn't it
>> have the same problems?
>>
>
> Yes there is only one instance of the consumer and its not pooled. So
> it may have the issue of not being able to re-connect due Apache FTP
> issue.
> So we may have to implement a better re-connect logic in camel-ftp.
> Maybe re-creating the FTPClient which works
>
The only issue I can see with creating a new FTPClient is the fact
that the end user can configure the endpoint to use a specific custom
client he have chosen. So in that case Camel should not trash his
client and re-create a new default. This problem would be the same if
you do not pool the producer. As it would be the same instance of the
client being given on the next time the producer is created. Only
fixing the root problem in the FTPClient can fix this.
But this will fix both the consumer and producer issue. I recon this
is the best solution so far?
And we should try to avoid leaks etc. so I think we at first should
ask the client to disconnect itself, before we re-create the client.
>> 2. Regarding the option you would introduce to control this behaviour, will
>> it default to "no-caching" for ftps? I guess it must, given the problems in
>> FTPSClient.
>>
>
> Its a bit confusing if some is pooled and others are not. But I do
> also think its best that it just works out of the box. We should add a
> WARN / INFO etc to the wiki page so its easy to spot. But maybe we
> should fix the re-connect logic and then the producer could still be
> pooled.
>
>
>> /Bengt
>>
>> 2010/6/21 Claus Ibsen <cl...@gmail.com>
>>
>>> On Mon, Jun 21, 2010 at 9:11 AM, Bengt Rodehav <be...@rodehav.com> wrote:
>>> > OK
>>> >
>>>
>>> Bengt could you try changing camel-ftp in the RemoteFileProducer
>>> public class RemoteFileProducer<T> extends GenericFileProducer<T>
>>> implements ServicePoolAware {
>>> And remove the implements ServicePoolAware which should cause Camel to
>>> not pool it and thus you get a new fresh instance each time.
>>>
>>> Then try testing with that to see if that fixes the re-connect issue.
>>>
>>> What we can then do is to add some nice option in the endpoint URI so
>>> you can configure the pooling behavior.
>>>
>>>
>>> > 2010/6/21 Claus Ibsen <cl...@gmail.com>
>>> >
>>> >> On Mon, Jun 21, 2010 at 8:52 AM, Bengt Rodehav <be...@rodehav.com>
>>> wrote:
>>> >> > I have now created a JIRA in commons-net for this issue:
>>> >> >
>>> >> > https://issues.apache.org/jira/browse/NET-327
>>> >> >
>>> >>
>>> >> Good
>>> >>
>>> >> > <https://issues.apache.org/jira/browse/NET-327>However, I still think
>>> we
>>> >> > must fix this in camel-ftp by some kind of work-around (maybe similar
>>> to
>>> >> the
>>> >> > one I proposed in
>>> https://issues.apache.org/activemq/browse/CAMEL-2829).
>>> >> > Have you had a chance to look at it Claus?
>>> >> >
>>> >>
>>> >> No sorry I am to busy with other issues but we got the ticket so we
>>> >> will look into it before 2.4 is cut.
>>> >>
>>> >>
>>> >> > /Bengt
>>> >> >
>>> >> > 2010/6/18 Bengt Rodehav <be...@rodehav.com>
>>> >> >
>>> >> >> JIRA created:
>>> >> >>
>>> >> >> https://issues.apache.org/activemq/browse/CAMEL-2829
>>> >> >>
>>> >> >> <https://issues.apache.org/activemq/browse/CAMEL-2829>/Bengt
>>> >> >>
>>> >> >> 2010/6/18 Bengt Rodehav <be...@rodehav.com>
>>> >> >>
>>> >> >> Claus,
>>> >> >>>
>>> >> >>> I'm not sure what you mean by:
>>> >> >>>
>>> >> >>> "I think we should add an option on the endpoint to allow people to
>>> >> >>> turn this on/off.
>>> >> >>> Some would like to reuse existing connections to avoid the connect
>>> ->
>>> >> >>> upload -> disconnect cycle when they upload many files etc."
>>> >> >>>
>>> >> >>> IMO we would not disconnect more often than today. We would just
>>> change
>>> >> >>> the connect() method in FtpsOperations so that it creates a new
>>> >> instance of
>>> >> >>> FTPSClient prior to a connection attempt. I assume that since the
>>> >> connect()
>>> >> >>> method is called in the first place, we are not currently connected
>>> and
>>> >> >>> there is no way to reuse an existing connection. Am I right?
>>> >> >>>
>>> >> >>> I have made some modifications in camel-ftp along these lines. I
>>> will
>>> >> >>> create a JIRA ticket for this bug and attach my changes as diff
>>> files
>>> >> to the
>>> >> >>> issue. I just tested my changes and they do solve the problem I have
>>> >> >>> encountered but I haven't tested whether I have introduced any new
>>> >> problems
>>> >> >>> or not.
>>> >> >>>
>>> >> >>> Anyway, this is what I did:
>>> >> >>>
>>> >> >>> *FtpsEndpoint*
>>> >> >>> I added the following members to keep track of the configuration of
>>> the
>>> >> >>> FTPSClient:
>>> >> >>>
>>> >> >>> private boolean isNeedClientAuth = false;
>>> >> >>> private KeyManager keyManager;
>>> >> >>> private TrustManager trustManager;
>>> >> >>>
>>> >> >>> I modified the createFtpClient() method so that it doesn't actually
>>> >> create
>>> >> >>> a FTPSClient but only calculates and stores the configuration in the
>>> >> above
>>> >> >>> members. I also renamed it to createFtpsConfiguration() to reflect
>>> >> that.
>>> >> >>>
>>> >> >>> The createRemoteFileOperations() method was also changed
>>> accordingly.
>>> >> It
>>> >> >>> no longer interacts directly with an FTPSClient (since it is not
>>> >> created
>>> >> >>> yet).
>>> >> >>>
>>> >> >>> *FtpsOperations*
>>> >> >>> I added the private createFtpsClient() method that takes care of
>>> >> creating
>>> >> >>> an FTPSClient from configuration stored in the FtpsEndpoint. The
>>> >> >>> createFtpsClient() is called in the connect() method just before the
>>> >> call to
>>> >> >>> the "real" connect() method in the superclass (FtpOperations).
>>> >> >>>
>>> >> >>> I changed the constructor so that it no longer takes a client as an
>>> >> >>> argument. I send null as client to the super class constructor. This
>>> is
>>> >> of
>>> >> >>> course a major change in behavior that you need to take a look at
>>> it.
>>> >> >>> Neither FtpsOperations nor its super class FtpOperations can no
>>> longer
>>> >> >>> assume that a client exists until the first connection attempt has
>>> been
>>> >> >>> made.
>>> >> >>>
>>> >> >>> *FtpOperations*
>>> >> >>> I had to make the "client" member non-final since I now set it on
>>> every
>>> >> >>> call to the connect() method in FtpsOperations.
>>> >> >>>
>>> >> >>>
>>> >> >>> BTW, I agree with you that the SFTPClient in commons-net is flawed.
>>> It
>>> >> >>> should provide a way to re-initialize itself so that a new connect()
>>> >> could
>>> >> >>> be made. The FTPClient seems to work that way and it also provides a
>>> >> >>> disconnect() method. If they do not intend to support this, then
>>> they
>>> >> should
>>> >> >>> at least state, as part of the documentation, that the SFTPClient
>>> >> cannot be
>>> >> >>> reused across multiple connections. I've posted about this on their
>>> >> mailing
>>> >> >>> list but hasn't got any reply yet. Meanwhile the above can be
>>> regarded
>>> >> as a
>>> >> >>> (complicated...) work-around.
>>> >> >>>
>>> >> >>> /Bengt
>>> >> >>>
>>> >> >>>
>>> >> >>> 2010/6/18 Claus Ibsen <cl...@gmail.com>
>>> >> >>>
>>> >> >>> On Thu, Jun 17, 2010 at 11:56 PM, Bengt Rodehav <be...@rodehav.com>
>>> >> >>>> wrote:
>>> >> >>>> > Claus,
>>> >> >>>> >
>>> >> >>>> > Having looked a bit more at the commons-net code it's becoming
>>> clear
>>> >> to
>>> >> >>>> me
>>> >> >>>> > that you probably can't just reconnect an FTPSClient once it has
>>> >> >>>> entered
>>> >> >>>> > secure communication mode. I can't find a way to re-initialise an
>>> >> >>>> > FTPSClient. I tried the disconnect() method (in the super class
>>> >> >>>> FTPClient)
>>> >> >>>> > but it didn't get the FTPSClient out of secure communication
>>> mode.
>>> >> >>>> >
>>> >> >>>> > To be safe I think we should always start with a newly
>>> instantiated
>>> >> >>>> > FTPSClient when we try to connect. Today the FTPSClient is
>>> created
>>> >> by
>>> >> >>>> the
>>> >> >>>> > FtpsEndpoint. I don't think it has to. It could just get all the
>>> >> righ
>>> >> >>>> > configuration parameters and pass it to FtpsOperations which
>>> would
>>> >> then
>>> >> >>>> > instantiate a new FTPSClient at every connect attempt.
>>> >> >>>> >
>>> >> >>>> > If we do it this way for ftps I guess it would make sense with
>>> the
>>> >> same
>>> >> >>>> > setup for plain ftp.
>>> >> >>>> >
>>> >> >>>> > What do you think?
>>> >> >>>> >
>>> >> >>>>
>>> >> >>>> I think we should add an option on the endpoint to allow people to
>>> >> >>>> turn this on/off.
>>> >> >>>> Some would like to reuse existing connections to avoid the connect
>>> ->
>>> >> >>>> upload -> disconnect cycle when they upload many files etc.
>>> >> >>>>
>>> >> >>>>
>>> >> >>>>
>>> >> >>>>
>>> >> >>>> And frankly I can't see the problem why a secure connection cannot
>>> >> >>>> self heal and be able to re-establish itself. After all FTPSClient
>>> got
>>> >> >>>> all the connection information. And hence I would consider it a
>>> >> >>>> missing feature / glitch in the FTP library.
>>> >> >>>>
>>> >> >>>> Imagine if using secure connections for JMS brokers which couldn't
>>> >> >>>> self heal just because a network outage. That would cause a lot of
>>> >> >>>> pain for clients having to re-start.
>>> >> >>>>
>>> >> >>>>
>>> >> >>>> > /Bengt
>>> >> >>>> >
>>> >> >>>> > 2010/6/17 Bengt Rodehav <be...@rodehav.com>
>>> >> >>>> >
>>> >> >>>> >> Claus,
>>> >> >>>> >>
>>> >> >>>> >> Yeah, maybe this is a commons-net issue. As always I find it
>>> >> >>>> surprising
>>> >> >>>> >> that no one else has encountered this problem.
>>> >> >>>> >>
>>> >> >>>> >> As I started writing an email on the commons user list, I took
>>> >> another
>>> >> >>>> look
>>> >> >>>> >> at the code in commons-net. It turns out that the
>>> FTPSSocketFactory
>>> >> >>>> class
>>> >> >>>> >> actually does override the createSocket() methods in
>>> SocketFactory.
>>> >> A
>>> >> >>>> >> similar work-around that I was talking about but on the factory
>>> >> class
>>> >> >>>> >> instead of in the SocketClient class. FTPSSocketFactory
>>> delegates
>>> >> the
>>> >> >>>> >> createSocket() calls to its SSLContect's socket factory.
>>> >> >>>> >>
>>> >> >>>> >> So why then are we initially able to connect but not after we
>>> have
>>> >> >>>> called
>>> >> >>>> >> execProt() and thus changed the connection factory?
>>> >> >>>> >>
>>> >> >>>> >> Moving back to the FTPSClient class now. When the initial
>>> >> connection
>>> >> >>>> is up,
>>> >> >>>> >> the _connectAction() method is called. It in turn calls
>>> >> >>>> sslNegotiation()
>>> >> >>>> >> which in turn sets up the secure socket. In the sslNegotiation()
>>> >> >>>> method, an
>>> >> >>>> >> SSLSocketFactory is not instantiated directly. It is given by
>>> the
>>> >> >>>> >> SSLContext's getSocketFactory() method. This connection factory
>>> >> >>>> obviously
>>> >> >>>> >> works otherwise the secure connection wouldn't work even
>>> initially.
>>> >> >>>> >>
>>> >> >>>> >> Compare this with what's being done in the execProt() method.
>>> Here
>>> >> the
>>> >> >>>> >> FTPSSocketFactory is instantiated directly with the SSLContext
>>> >> passed
>>> >> >>>> as a
>>> >> >>>> >> constructor argument. Obviously this factory does NOT work...
>>> >> >>>> >>
>>> >> >>>> >> Another question that comes to mind is why the socket factory
>>> >> created
>>> >> >>>> in
>>> >> >>>> >> the sslNegotiation() method is NOT set as the SocketClient's
>>> socket
>>> >> >>>> factory
>>> >> >>>> >> but the one created in the execProt() method is? It turns out
>>> that
>>> >> >>>> >> execProt() calls a generic sendCommand() method that sends the
>>> >> command
>>> >> >>>> and
>>> >> >>>> >> if the reply is OK, then sets the socket factory to null! One
>>> >> wonders
>>> >> >>>> why
>>> >> >>>> >> (and there is also a comment in the code: "Check this - is this
>>> >> >>>> necessary at
>>> >> >>>> >> all?"). Anyway, this is probably the reason why execProt() needs
>>> to
>>> >> >>>> "reset"
>>> >> >>>> >> the connection factory.
>>> >> >>>> >>
>>> >> >>>> >> Still sounds like a commons-net problem though. I'll try to post
>>> >> this
>>> >> >>>> at
>>> >> >>>> >> their user mailing list.
>>> >> >>>> >>
>>> >> >>>> >> Just one thought. Do we need to reuse the FTPSClient instance?
>>> >> Can't
>>> >> >>>> we
>>> >> >>>> >> just create a fresh new instance everytime we need to connect?
>>> >> >>>> >>
>>> >> >>>> >> /Bengt
>>> >> >>>> >>
>>> >> >>>> >>
>>> >> >>>> >>
>>> >> >>>> >> 2010/6/17 Claus Ibsen <cl...@gmail.com>
>>> >> >>>> >>
>>> >> >>>> >> Hi Bengt
>>> >> >>>> >>>
>>> >> >>>> >>> I think the issue should be created as a ticket for Apache
>>> Commons
>>> >> >>>> >>> Net. Then at least the people there can take a look, and maybe
>>> >> they
>>> >> >>>> >>> got some ideas. And can fix it in a future release.
>>> >> >>>> >>>
>>> >> >>>> >>> Bengt fell free to experiment yourself with the subclassing, as
>>> it
>>> >> >>>> >>> sounds like a good workaround.
>>> >> >>>> >>>
>>> >> >>>> >>> I assume the FTPSClient doesn't offer other methods to re
>>> connect
>>> >> >>>> >>> which can recover this error?
>>> >> >>>> >>>
>>> >> >>>> >>>
>>> >> >>>> >>> On Wed, Jun 16, 2010 at 2:44 PM, Bengt Rodehav <
>>> bengt@rodehav.com
>>> >> >
>>> >> >>>> wrote:
>>> >> >>>> >>> > I have mentioned this problem in a previous conversation but
>>> >> after
>>> >> >>>> >>> > investigating the subject further I decided to start a new
>>> >> thread
>>> >> >>>> for
>>> >> >>>> >>> this.
>>> >> >>>> >>> >
>>> >> >>>> >>> > If some kind of problem is encountered while using ftps (I
>>> >> tested
>>> >> >>>> this
>>> >> >>>> >>> using
>>> >> >>>> >>> > Filezilla server by copying a file that already existed which
>>> >> >>>> Filezilla
>>> >> >>>> >>> does
>>> >> >>>> >>> > not allow), then at the next attempt to connect to the ftps
>>> >> server,
>>> >> >>>> I
>>> >> >>>> >>> get
>>> >> >>>> >>> > the following exception in my logfile:
>>> >> >>>> >>> >
>>> >> >>>> >>> > 13:03:27,606 | ERROR | %7Bfile%3Aext%7D |
>>> >> GenericFileOnCompletion
>>> >> >>>> >>> |
>>> >> >>>> >>> > rg.apache.camel.processor.Logger 248 | Caused by:
>>> >> >>>> >>> >
>>> >> >>>>
>>> [org.apache.camel.component.file.GenericFileOperationFailedException -
>>> >> >>>> >>> File
>>> >> >>>> >>> > operation failed: null Unconnected sockets not implemented.
>>> >> Code:
>>> >> >>>> 221]
>>> >> >>>> >>> >
>>> >> >>>>
>>> org.apache.camel.component.file.GenericFileOperationFailedException:
>>> >> >>>> >>> File
>>> >> >>>> >>> > operation failed: null Unconnected sockets not implemented.
>>> >> Code:
>>> >> >>>> 221
>>> >> >>>> >>> > at
>>> >> >>>> >>> >
>>> >> >>>> >>>
>>> >> >>>>
>>> >>
>>> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:108)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>> >> >>>> >>> > at
>>> >> >>>> >>> >
>>> >> >>>> >>>
>>> >> >>>>
>>> >>
>>> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>> >> >>>> >>> > at
>>> >> >>>> >>> >
>>> >> >>>> >>>
>>> >> >>>>
>>> >>
>>> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>> >> >>>> >>> > at
>>> >> >>>> >>> >
>>> >> >>>> >>>
>>> >> >>>>
>>> >>
>>> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:123)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>> >> >>>> >>> > at
>>> >> >>>> >>> >
>>> >> >>>> >>>
>>> >> >>>>
>>> >>
>>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
>>> >> >>>> >>> > at
>>> >> >>>> >>> >
>>> >> >>>> >>>
>>> >> >>>>
>>> >>
>>> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>> >> >>>> >>> > at
>>> >> >>>> >>> >
>>> >> >>>> >>>
>>> >> >>>>
>>> >>
>>> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:91)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
>>> >> >>>> >>> >
>>> >> >>>> >>> > I added some logging to see the actual stack trace which is:
>>> >> >>>> >>> >
>>> >> >>>> >>> > java.net.SocketException: Unconnected sockets not implemented
>>> >> >>>> >>> > at
>>> >> >>>> javax.net.SocketFactory.createSocket(SocketFactory.java:104)
>>> >> >>>> >>> > at
>>> >> >>>> >>> >
>>> >> org.apache.commons.net.SocketClient.connect(SocketClient.java:175)
>>> >> >>>> >>> > at
>>> >> >>>> >>> >
>>> >> >>>> >>>
>>> >> >>>>
>>> >>
>>> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:91)
>>> >> >>>> >>> > at
>>> >> >>>> >>> >
>>> >> >>>> >>>
>>> >> >>>>
>>> >>
>>> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)
>>> >> >>>> >>> > at
>>> >> >>>> >>> >
>>> >> >>>> >>>
>>> >> >>>>
>>> >>
>>> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)
>>> >> >>>> >>> > at
>>> >> >>>> >>> >
>>> >> >>>> >>>
>>> >> >>>>
>>> >>
>>> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:99)
>>> >> >>>> >>> > at
>>> >> >>>> >>> >
>>> >> >>>> >>>
>>> >> >>>>
>>> >>
>>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)
>>> >> >>>> >>> > at
>>> >> >>>> >>> >
>>> >> >>>> >>>
>>> >> >>>>
>>> >>
>>> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)
>>> >> >>>> >>> >
>>> >> >>>> >>> > commons-net SocketClient class tries to call the socket
>>> >> factory's
>>> >> >>>> >>> > createSocket() method (with no parameters):
>>> >> >>>> >>> >
>>> >> >>>> >>> > _socket_= _socketFactory_.createSocket();
>>> >> >>>> >>> > _socket_.connect(new InetSocketAddress(hostname,
>>> port),
>>> >> >>>> >>> > connectTimeout);
>>> >> >>>> >>> >
>>> >> >>>> >>> > Then intention is to create an "unconnected socket" (hence
>>> the
>>> >> >>>> error
>>> >> >>>> >>> > message) and then subsequently connect it. In my case, the
>>> >> >>>> connection
>>> >> >>>> >>> > factory being used is an FTPSSocketFactory
>>> >> >>>> >>> > (package org.apache.commons.net.ftp) that inherits from
>>> >> >>>> >>> > SocketFactory(package javax.net). The SocketFactory class
>>> >> >>>> implements
>>> >> >>>> >>> the
>>> >> >>>> >>> > createSocket() method but throws the exception ("Unconnected
>>> >> >>>> sockets not
>>> >> >>>> >>> > implemented").
>>> >> >>>> >>> >
>>> >> >>>> >>> > The reason why this happens is that after having executed the
>>> >> >>>> >>> > FTPSClient.execPROT() method (package
>>> >> org.apache.commons.net.ftp),
>>> >> >>>> with
>>> >> >>>> >>> any
>>> >> >>>> >>> > other parameter than "C" (in my case I used "P"), the socket
>>> >> >>>> factory is
>>> >> >>>> >>> set
>>> >> >>>> >>> > to an FTPSSocketFactory as follows:
>>> >> >>>> >>> >
>>> >> >>>> >>> > setSocketFactory(new FTPSSocketFactory(context));
>>> >> >>>> >>> >
>>> >> >>>> >>> > After this, no connection attempt will succeed. This problem
>>> has
>>> >> >>>> been
>>> >> >>>> >>> > introduced recently when Claus (on my initiative) added
>>> support
>>> >> for
>>> >> >>>> >>> secure
>>> >> >>>> >>> > data channel in ftps. I'm thus to blame...
>>> >> >>>> >>> >
>>> >> >>>> >>> > I'm not sure how this should be fixed.
>>> >> >>>> >>> >
>>> >> >>>> >>> > It's a bit strange that all the connect methods
>>> >> >>>> >>> > in org.apache.commons.net.SocketClient (which is the ultimate
>>> >> base
>>> >> >>>> class
>>> >> >>>> >>> of
>>> >> >>>> >>> > FTPSClient) always try to create an unconnected socket first
>>> and
>>> >> >>>> then
>>> >> >>>> >>> > connects it. If commons-net uses this pattern, then it should
>>> >> make
>>> >> >>>> sure
>>> >> >>>> >>> that
>>> >> >>>> >>> > all its connection factor's support unconnected sockets but
>>> the
>>> >> >>>> >>> > FTPSSocketFactory dosn't. This sounds like a bug in
>>> commons-net.
>>> >> >>>> >>> >
>>> >> >>>> >>> > Knowing that commons-net doesn't release very frequently I
>>> think
>>> >> we
>>> >> >>>> need
>>> >> >>>> >>> a
>>> >> >>>> >>> > workaround for this. I guess it would be possible to subclass
>>> >> the
>>> >> >>>> >>> FTPClient
>>> >> >>>> >>> > class and override all the connect methods to make sure that
>>> no
>>> >> >>>> attempt
>>> >> >>>> >>> is
>>> >> >>>> >>> > made to create unconnected sockets. We would then change the
>>> >> >>>> >>> > createFtpClient() method in the FtpsEndpoint class to
>>> >> instantiate
>>> >> >>>> our
>>> >> >>>> >>> own
>>> >> >>>> >>> > subclass instead of an FTPSClient. Doesn't sound like a nice
>>> >> clean
>>> >> >>>> >>> solution
>>> >> >>>> >>> > though.
>>> >> >>>> >>> >
>>> >> >>>> >>> > Any ideas?
>>> >> >>>> >>> >
>>> >> >>>> >>> > /Bengt
>>> >> >>>> >>> >
>>> >> >>>> >>>
>>> >> >>>> >>>
>>> >> >>>> >>>
>>> >> >>>> >>> --
>>> >> >>>> >>> Claus Ibsen
>>> >> >>>> >>> Apache Camel Committer
>>> >> >>>> >>>
>>> >> >>>> >>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> >> >>>> >>> Open Source Integration: http://fusesource.com
>>> >> >>>> >>> Blog: http://davsclaus.blogspot.com/
>>> >> >>>> >>> Twitter: http://twitter.com/davsclaus
>>> >> >>>> >>>
>>> >> >>>> >>
>>> >> >>>> >>
>>> >> >>>> >
>>> >> >>>>
>>> >> >>>>
>>> >> >>>>
>>> >> >>>> --
>>> >> >>>> Claus Ibsen
>>> >> >>>> Apache Camel Committer
>>> >> >>>>
>>> >> >>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> >> >>>> Open Source Integration: http://fusesource.com
>>> >> >>>> Blog: http://davsclaus.blogspot.com/
>>> >> >>>> Twitter: http://twitter.com/davsclaus
>>> >> >>>>
>>> >> >>>
>>> >> >>>
>>> >> >>
>>> >> >
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Claus Ibsen
>>> >> Apache Camel Committer
>>> >>
>>> >> Author of Camel in Action: http://www.manning.com/ibsen/
>>> >> Open Source Integration: http://fusesource.com
>>> >> Blog: http://davsclaus.blogspot.com/
>>> >> Twitter: http://twitter.com/davsclaus
>>> >>
>>> >
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: "Unconnected sockets not implemented" exception in camel-ftp when
using ftps
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Jun 21, 2010 at 10:22 AM, Bengt Rodehav <be...@rodehav.com> wrote:
> Claus,
>
> Just tried and it works. Very simple way of achieving our goals I must say.
> Two questions:
>
It was just a quick test to see if that resolved it here and now as a
workaround.
> 1. How does the consumer side work? I haven't tried it yet but wouldn't it
> have the same problems?
>
Yes there is only one instance of the consumer and its not pooled. So
it may have the issue of not being able to re-connect due Apache FTP
issue.
So we may have to implement a better re-connect logic in camel-ftp.
Maybe re-creating the FTPClient which works
> 2. Regarding the option you would introduce to control this behaviour, will
> it default to "no-caching" for ftps? I guess it must, given the problems in
> FTPSClient.
>
Its a bit confusing if some is pooled and others are not. But I do
also think its best that it just works out of the box. We should add a
WARN / INFO etc to the wiki page so its easy to spot. But maybe we
should fix the re-connect logic and then the producer could still be
pooled.
> /Bengt
>
> 2010/6/21 Claus Ibsen <cl...@gmail.com>
>
>> On Mon, Jun 21, 2010 at 9:11 AM, Bengt Rodehav <be...@rodehav.com> wrote:
>> > OK
>> >
>>
>> Bengt could you try changing camel-ftp in the RemoteFileProducer
>> public class RemoteFileProducer<T> extends GenericFileProducer<T>
>> implements ServicePoolAware {
>> And remove the implements ServicePoolAware which should cause Camel to
>> not pool it and thus you get a new fresh instance each time.
>>
>> Then try testing with that to see if that fixes the re-connect issue.
>>
>> What we can then do is to add some nice option in the endpoint URI so
>> you can configure the pooling behavior.
>>
>>
>> > 2010/6/21 Claus Ibsen <cl...@gmail.com>
>> >
>> >> On Mon, Jun 21, 2010 at 8:52 AM, Bengt Rodehav <be...@rodehav.com>
>> wrote:
>> >> > I have now created a JIRA in commons-net for this issue:
>> >> >
>> >> > https://issues.apache.org/jira/browse/NET-327
>> >> >
>> >>
>> >> Good
>> >>
>> >> > <https://issues.apache.org/jira/browse/NET-327>However, I still think
>> we
>> >> > must fix this in camel-ftp by some kind of work-around (maybe similar
>> to
>> >> the
>> >> > one I proposed in
>> https://issues.apache.org/activemq/browse/CAMEL-2829).
>> >> > Have you had a chance to look at it Claus?
>> >> >
>> >>
>> >> No sorry I am to busy with other issues but we got the ticket so we
>> >> will look into it before 2.4 is cut.
>> >>
>> >>
>> >> > /Bengt
>> >> >
>> >> > 2010/6/18 Bengt Rodehav <be...@rodehav.com>
>> >> >
>> >> >> JIRA created:
>> >> >>
>> >> >> https://issues.apache.org/activemq/browse/CAMEL-2829
>> >> >>
>> >> >> <https://issues.apache.org/activemq/browse/CAMEL-2829>/Bengt
>> >> >>
>> >> >> 2010/6/18 Bengt Rodehav <be...@rodehav.com>
>> >> >>
>> >> >> Claus,
>> >> >>>
>> >> >>> I'm not sure what you mean by:
>> >> >>>
>> >> >>> "I think we should add an option on the endpoint to allow people to
>> >> >>> turn this on/off.
>> >> >>> Some would like to reuse existing connections to avoid the connect
>> ->
>> >> >>> upload -> disconnect cycle when they upload many files etc."
>> >> >>>
>> >> >>> IMO we would not disconnect more often than today. We would just
>> change
>> >> >>> the connect() method in FtpsOperations so that it creates a new
>> >> instance of
>> >> >>> FTPSClient prior to a connection attempt. I assume that since the
>> >> connect()
>> >> >>> method is called in the first place, we are not currently connected
>> and
>> >> >>> there is no way to reuse an existing connection. Am I right?
>> >> >>>
>> >> >>> I have made some modifications in camel-ftp along these lines. I
>> will
>> >> >>> create a JIRA ticket for this bug and attach my changes as diff
>> files
>> >> to the
>> >> >>> issue. I just tested my changes and they do solve the problem I have
>> >> >>> encountered but I haven't tested whether I have introduced any new
>> >> problems
>> >> >>> or not.
>> >> >>>
>> >> >>> Anyway, this is what I did:
>> >> >>>
>> >> >>> *FtpsEndpoint*
>> >> >>> I added the following members to keep track of the configuration of
>> the
>> >> >>> FTPSClient:
>> >> >>>
>> >> >>> private boolean isNeedClientAuth = false;
>> >> >>> private KeyManager keyManager;
>> >> >>> private TrustManager trustManager;
>> >> >>>
>> >> >>> I modified the createFtpClient() method so that it doesn't actually
>> >> create
>> >> >>> a FTPSClient but only calculates and stores the configuration in the
>> >> above
>> >> >>> members. I also renamed it to createFtpsConfiguration() to reflect
>> >> that.
>> >> >>>
>> >> >>> The createRemoteFileOperations() method was also changed
>> accordingly.
>> >> It
>> >> >>> no longer interacts directly with an FTPSClient (since it is not
>> >> created
>> >> >>> yet).
>> >> >>>
>> >> >>> *FtpsOperations*
>> >> >>> I added the private createFtpsClient() method that takes care of
>> >> creating
>> >> >>> an FTPSClient from configuration stored in the FtpsEndpoint. The
>> >> >>> createFtpsClient() is called in the connect() method just before the
>> >> call to
>> >> >>> the "real" connect() method in the superclass (FtpOperations).
>> >> >>>
>> >> >>> I changed the constructor so that it no longer takes a client as an
>> >> >>> argument. I send null as client to the super class constructor. This
>> is
>> >> of
>> >> >>> course a major change in behavior that you need to take a look at
>> it.
>> >> >>> Neither FtpsOperations nor its super class FtpOperations can no
>> longer
>> >> >>> assume that a client exists until the first connection attempt has
>> been
>> >> >>> made.
>> >> >>>
>> >> >>> *FtpOperations*
>> >> >>> I had to make the "client" member non-final since I now set it on
>> every
>> >> >>> call to the connect() method in FtpsOperations.
>> >> >>>
>> >> >>>
>> >> >>> BTW, I agree with you that the SFTPClient in commons-net is flawed.
>> It
>> >> >>> should provide a way to re-initialize itself so that a new connect()
>> >> could
>> >> >>> be made. The FTPClient seems to work that way and it also provides a
>> >> >>> disconnect() method. If they do not intend to support this, then
>> they
>> >> should
>> >> >>> at least state, as part of the documentation, that the SFTPClient
>> >> cannot be
>> >> >>> reused across multiple connections. I've posted about this on their
>> >> mailing
>> >> >>> list but hasn't got any reply yet. Meanwhile the above can be
>> regarded
>> >> as a
>> >> >>> (complicated...) work-around.
>> >> >>>
>> >> >>> /Bengt
>> >> >>>
>> >> >>>
>> >> >>> 2010/6/18 Claus Ibsen <cl...@gmail.com>
>> >> >>>
>> >> >>> On Thu, Jun 17, 2010 at 11:56 PM, Bengt Rodehav <be...@rodehav.com>
>> >> >>>> wrote:
>> >> >>>> > Claus,
>> >> >>>> >
>> >> >>>> > Having looked a bit more at the commons-net code it's becoming
>> clear
>> >> to
>> >> >>>> me
>> >> >>>> > that you probably can't just reconnect an FTPSClient once it has
>> >> >>>> entered
>> >> >>>> > secure communication mode. I can't find a way to re-initialise an
>> >> >>>> > FTPSClient. I tried the disconnect() method (in the super class
>> >> >>>> FTPClient)
>> >> >>>> > but it didn't get the FTPSClient out of secure communication
>> mode.
>> >> >>>> >
>> >> >>>> > To be safe I think we should always start with a newly
>> instantiated
>> >> >>>> > FTPSClient when we try to connect. Today the FTPSClient is
>> created
>> >> by
>> >> >>>> the
>> >> >>>> > FtpsEndpoint. I don't think it has to. It could just get all the
>> >> righ
>> >> >>>> > configuration parameters and pass it to FtpsOperations which
>> would
>> >> then
>> >> >>>> > instantiate a new FTPSClient at every connect attempt.
>> >> >>>> >
>> >> >>>> > If we do it this way for ftps I guess it would make sense with
>> the
>> >> same
>> >> >>>> > setup for plain ftp.
>> >> >>>> >
>> >> >>>> > What do you think?
>> >> >>>> >
>> >> >>>>
>> >> >>>> I think we should add an option on the endpoint to allow people to
>> >> >>>> turn this on/off.
>> >> >>>> Some would like to reuse existing connections to avoid the connect
>> ->
>> >> >>>> upload -> disconnect cycle when they upload many files etc.
>> >> >>>>
>> >> >>>>
>> >> >>>>
>> >> >>>>
>> >> >>>> And frankly I can't see the problem why a secure connection cannot
>> >> >>>> self heal and be able to re-establish itself. After all FTPSClient
>> got
>> >> >>>> all the connection information. And hence I would consider it a
>> >> >>>> missing feature / glitch in the FTP library.
>> >> >>>>
>> >> >>>> Imagine if using secure connections for JMS brokers which couldn't
>> >> >>>> self heal just because a network outage. That would cause a lot of
>> >> >>>> pain for clients having to re-start.
>> >> >>>>
>> >> >>>>
>> >> >>>> > /Bengt
>> >> >>>> >
>> >> >>>> > 2010/6/17 Bengt Rodehav <be...@rodehav.com>
>> >> >>>> >
>> >> >>>> >> Claus,
>> >> >>>> >>
>> >> >>>> >> Yeah, maybe this is a commons-net issue. As always I find it
>> >> >>>> surprising
>> >> >>>> >> that no one else has encountered this problem.
>> >> >>>> >>
>> >> >>>> >> As I started writing an email on the commons user list, I took
>> >> another
>> >> >>>> look
>> >> >>>> >> at the code in commons-net. It turns out that the
>> FTPSSocketFactory
>> >> >>>> class
>> >> >>>> >> actually does override the createSocket() methods in
>> SocketFactory.
>> >> A
>> >> >>>> >> similar work-around that I was talking about but on the factory
>> >> class
>> >> >>>> >> instead of in the SocketClient class. FTPSSocketFactory
>> delegates
>> >> the
>> >> >>>> >> createSocket() calls to its SSLContect's socket factory.
>> >> >>>> >>
>> >> >>>> >> So why then are we initially able to connect but not after we
>> have
>> >> >>>> called
>> >> >>>> >> execProt() and thus changed the connection factory?
>> >> >>>> >>
>> >> >>>> >> Moving back to the FTPSClient class now. When the initial
>> >> connection
>> >> >>>> is up,
>> >> >>>> >> the _connectAction() method is called. It in turn calls
>> >> >>>> sslNegotiation()
>> >> >>>> >> which in turn sets up the secure socket. In the sslNegotiation()
>> >> >>>> method, an
>> >> >>>> >> SSLSocketFactory is not instantiated directly. It is given by
>> the
>> >> >>>> >> SSLContext's getSocketFactory() method. This connection factory
>> >> >>>> obviously
>> >> >>>> >> works otherwise the secure connection wouldn't work even
>> initially.
>> >> >>>> >>
>> >> >>>> >> Compare this with what's being done in the execProt() method.
>> Here
>> >> the
>> >> >>>> >> FTPSSocketFactory is instantiated directly with the SSLContext
>> >> passed
>> >> >>>> as a
>> >> >>>> >> constructor argument. Obviously this factory does NOT work...
>> >> >>>> >>
>> >> >>>> >> Another question that comes to mind is why the socket factory
>> >> created
>> >> >>>> in
>> >> >>>> >> the sslNegotiation() method is NOT set as the SocketClient's
>> socket
>> >> >>>> factory
>> >> >>>> >> but the one created in the execProt() method is? It turns out
>> that
>> >> >>>> >> execProt() calls a generic sendCommand() method that sends the
>> >> command
>> >> >>>> and
>> >> >>>> >> if the reply is OK, then sets the socket factory to null! One
>> >> wonders
>> >> >>>> why
>> >> >>>> >> (and there is also a comment in the code: "Check this - is this
>> >> >>>> necessary at
>> >> >>>> >> all?"). Anyway, this is probably the reason why execProt() needs
>> to
>> >> >>>> "reset"
>> >> >>>> >> the connection factory.
>> >> >>>> >>
>> >> >>>> >> Still sounds like a commons-net problem though. I'll try to post
>> >> this
>> >> >>>> at
>> >> >>>> >> their user mailing list.
>> >> >>>> >>
>> >> >>>> >> Just one thought. Do we need to reuse the FTPSClient instance?
>> >> Can't
>> >> >>>> we
>> >> >>>> >> just create a fresh new instance everytime we need to connect?
>> >> >>>> >>
>> >> >>>> >> /Bengt
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>> >> 2010/6/17 Claus Ibsen <cl...@gmail.com>
>> >> >>>> >>
>> >> >>>> >> Hi Bengt
>> >> >>>> >>>
>> >> >>>> >>> I think the issue should be created as a ticket for Apache
>> Commons
>> >> >>>> >>> Net. Then at least the people there can take a look, and maybe
>> >> they
>> >> >>>> >>> got some ideas. And can fix it in a future release.
>> >> >>>> >>>
>> >> >>>> >>> Bengt fell free to experiment yourself with the subclassing, as
>> it
>> >> >>>> >>> sounds like a good workaround.
>> >> >>>> >>>
>> >> >>>> >>> I assume the FTPSClient doesn't offer other methods to re
>> connect
>> >> >>>> >>> which can recover this error?
>> >> >>>> >>>
>> >> >>>> >>>
>> >> >>>> >>> On Wed, Jun 16, 2010 at 2:44 PM, Bengt Rodehav <
>> bengt@rodehav.com
>> >> >
>> >> >>>> wrote:
>> >> >>>> >>> > I have mentioned this problem in a previous conversation but
>> >> after
>> >> >>>> >>> > investigating the subject further I decided to start a new
>> >> thread
>> >> >>>> for
>> >> >>>> >>> this.
>> >> >>>> >>> >
>> >> >>>> >>> > If some kind of problem is encountered while using ftps (I
>> >> tested
>> >> >>>> this
>> >> >>>> >>> using
>> >> >>>> >>> > Filezilla server by copying a file that already existed which
>> >> >>>> Filezilla
>> >> >>>> >>> does
>> >> >>>> >>> > not allow), then at the next attempt to connect to the ftps
>> >> server,
>> >> >>>> I
>> >> >>>> >>> get
>> >> >>>> >>> > the following exception in my logfile:
>> >> >>>> >>> >
>> >> >>>> >>> > 13:03:27,606 | ERROR | %7Bfile%3Aext%7D |
>> >> GenericFileOnCompletion
>> >> >>>> >>> |
>> >> >>>> >>> > rg.apache.camel.processor.Logger 248 | Caused by:
>> >> >>>> >>> >
>> >> >>>>
>> [org.apache.camel.component.file.GenericFileOperationFailedException -
>> >> >>>> >>> File
>> >> >>>> >>> > operation failed: null Unconnected sockets not implemented.
>> >> Code:
>> >> >>>> 221]
>> >> >>>> >>> >
>> >> >>>>
>> org.apache.camel.component.file.GenericFileOperationFailedException:
>> >> >>>> >>> File
>> >> >>>> >>> > operation failed: null Unconnected sockets not implemented.
>> >> Code:
>> >> >>>> 221
>> >> >>>> >>> > at
>> >> >>>> >>> >
>> >> >>>> >>>
>> >> >>>>
>> >>
>> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:108)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> >> >>>> >>> > at
>> >> >>>> >>> >
>> >> >>>> >>>
>> >> >>>>
>> >>
>> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> >> >>>> >>> > at
>> >> >>>> >>> >
>> >> >>>> >>>
>> >> >>>>
>> >>
>> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> >> >>>> >>> > at
>> >> >>>> >>> >
>> >> >>>> >>>
>> >> >>>>
>> >>
>> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:123)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> >> >>>> >>> > at
>> >> >>>> >>> >
>> >> >>>> >>>
>> >> >>>>
>> >>
>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
>> >> >>>> >>> > at
>> >> >>>> >>> >
>> >> >>>> >>>
>> >> >>>>
>> >>
>> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> >> >>>> >>> > at
>> >> >>>> >>> >
>> >> >>>> >>>
>> >> >>>>
>> >>
>> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:91)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
>> >> >>>> >>> >
>> >> >>>> >>> > I added some logging to see the actual stack trace which is:
>> >> >>>> >>> >
>> >> >>>> >>> > java.net.SocketException: Unconnected sockets not implemented
>> >> >>>> >>> > at
>> >> >>>> javax.net.SocketFactory.createSocket(SocketFactory.java:104)
>> >> >>>> >>> > at
>> >> >>>> >>> >
>> >> org.apache.commons.net.SocketClient.connect(SocketClient.java:175)
>> >> >>>> >>> > at
>> >> >>>> >>> >
>> >> >>>> >>>
>> >> >>>>
>> >>
>> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:91)
>> >> >>>> >>> > at
>> >> >>>> >>> >
>> >> >>>> >>>
>> >> >>>>
>> >>
>> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)
>> >> >>>> >>> > at
>> >> >>>> >>> >
>> >> >>>> >>>
>> >> >>>>
>> >>
>> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)
>> >> >>>> >>> > at
>> >> >>>> >>> >
>> >> >>>> >>>
>> >> >>>>
>> >>
>> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:99)
>> >> >>>> >>> > at
>> >> >>>> >>> >
>> >> >>>> >>>
>> >> >>>>
>> >>
>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)
>> >> >>>> >>> > at
>> >> >>>> >>> >
>> >> >>>> >>>
>> >> >>>>
>> >>
>> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)
>> >> >>>> >>> >
>> >> >>>> >>> > commons-net SocketClient class tries to call the socket
>> >> factory's
>> >> >>>> >>> > createSocket() method (with no parameters):
>> >> >>>> >>> >
>> >> >>>> >>> > _socket_= _socketFactory_.createSocket();
>> >> >>>> >>> > _socket_.connect(new InetSocketAddress(hostname,
>> port),
>> >> >>>> >>> > connectTimeout);
>> >> >>>> >>> >
>> >> >>>> >>> > Then intention is to create an "unconnected socket" (hence
>> the
>> >> >>>> error
>> >> >>>> >>> > message) and then subsequently connect it. In my case, the
>> >> >>>> connection
>> >> >>>> >>> > factory being used is an FTPSSocketFactory
>> >> >>>> >>> > (package org.apache.commons.net.ftp) that inherits from
>> >> >>>> >>> > SocketFactory(package javax.net). The SocketFactory class
>> >> >>>> implements
>> >> >>>> >>> the
>> >> >>>> >>> > createSocket() method but throws the exception ("Unconnected
>> >> >>>> sockets not
>> >> >>>> >>> > implemented").
>> >> >>>> >>> >
>> >> >>>> >>> > The reason why this happens is that after having executed the
>> >> >>>> >>> > FTPSClient.execPROT() method (package
>> >> org.apache.commons.net.ftp),
>> >> >>>> with
>> >> >>>> >>> any
>> >> >>>> >>> > other parameter than "C" (in my case I used "P"), the socket
>> >> >>>> factory is
>> >> >>>> >>> set
>> >> >>>> >>> > to an FTPSSocketFactory as follows:
>> >> >>>> >>> >
>> >> >>>> >>> > setSocketFactory(new FTPSSocketFactory(context));
>> >> >>>> >>> >
>> >> >>>> >>> > After this, no connection attempt will succeed. This problem
>> has
>> >> >>>> been
>> >> >>>> >>> > introduced recently when Claus (on my initiative) added
>> support
>> >> for
>> >> >>>> >>> secure
>> >> >>>> >>> > data channel in ftps. I'm thus to blame...
>> >> >>>> >>> >
>> >> >>>> >>> > I'm not sure how this should be fixed.
>> >> >>>> >>> >
>> >> >>>> >>> > It's a bit strange that all the connect methods
>> >> >>>> >>> > in org.apache.commons.net.SocketClient (which is the ultimate
>> >> base
>> >> >>>> class
>> >> >>>> >>> of
>> >> >>>> >>> > FTPSClient) always try to create an unconnected socket first
>> and
>> >> >>>> then
>> >> >>>> >>> > connects it. If commons-net uses this pattern, then it should
>> >> make
>> >> >>>> sure
>> >> >>>> >>> that
>> >> >>>> >>> > all its connection factor's support unconnected sockets but
>> the
>> >> >>>> >>> > FTPSSocketFactory dosn't. This sounds like a bug in
>> commons-net.
>> >> >>>> >>> >
>> >> >>>> >>> > Knowing that commons-net doesn't release very frequently I
>> think
>> >> we
>> >> >>>> need
>> >> >>>> >>> a
>> >> >>>> >>> > workaround for this. I guess it would be possible to subclass
>> >> the
>> >> >>>> >>> FTPClient
>> >> >>>> >>> > class and override all the connect methods to make sure that
>> no
>> >> >>>> attempt
>> >> >>>> >>> is
>> >> >>>> >>> > made to create unconnected sockets. We would then change the
>> >> >>>> >>> > createFtpClient() method in the FtpsEndpoint class to
>> >> instantiate
>> >> >>>> our
>> >> >>>> >>> own
>> >> >>>> >>> > subclass instead of an FTPSClient. Doesn't sound like a nice
>> >> clean
>> >> >>>> >>> solution
>> >> >>>> >>> > though.
>> >> >>>> >>> >
>> >> >>>> >>> > Any ideas?
>> >> >>>> >>> >
>> >> >>>> >>> > /Bengt
>> >> >>>> >>> >
>> >> >>>> >>>
>> >> >>>> >>>
>> >> >>>> >>>
>> >> >>>> >>> --
>> >> >>>> >>> Claus Ibsen
>> >> >>>> >>> Apache Camel Committer
>> >> >>>> >>>
>> >> >>>> >>> Author of Camel in Action: http://www.manning.com/ibsen/
>> >> >>>> >>> Open Source Integration: http://fusesource.com
>> >> >>>> >>> Blog: http://davsclaus.blogspot.com/
>> >> >>>> >>> Twitter: http://twitter.com/davsclaus
>> >> >>>> >>>
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>> >
>> >> >>>>
>> >> >>>>
>> >> >>>>
>> >> >>>> --
>> >> >>>> Claus Ibsen
>> >> >>>> Apache Camel Committer
>> >> >>>>
>> >> >>>> Author of Camel in Action: http://www.manning.com/ibsen/
>> >> >>>> Open Source Integration: http://fusesource.com
>> >> >>>> Blog: http://davsclaus.blogspot.com/
>> >> >>>> Twitter: http://twitter.com/davsclaus
>> >> >>>>
>> >> >>>
>> >> >>>
>> >> >>
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Claus Ibsen
>> >> Apache Camel Committer
>> >>
>> >> Author of Camel in Action: http://www.manning.com/ibsen/
>> >> Open Source Integration: http://fusesource.com
>> >> Blog: http://davsclaus.blogspot.com/
>> >> Twitter: http://twitter.com/davsclaus
>> >>
>> >
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: "Unconnected sockets not implemented" exception in camel-ftp when
using ftps
Posted by Bengt Rodehav <be...@rodehav.com>.
Claus,
Just tried and it works. Very simple way of achieving our goals I must say.
Two questions:
1. How does the consumer side work? I haven't tried it yet but wouldn't it
have the same problems?
2. Regarding the option you would introduce to control this behaviour, will
it default to "no-caching" for ftps? I guess it must, given the problems in
FTPSClient.
/Bengt
2010/6/21 Claus Ibsen <cl...@gmail.com>
> On Mon, Jun 21, 2010 at 9:11 AM, Bengt Rodehav <be...@rodehav.com> wrote:
> > OK
> >
>
> Bengt could you try changing camel-ftp in the RemoteFileProducer
> public class RemoteFileProducer<T> extends GenericFileProducer<T>
> implements ServicePoolAware {
> And remove the implements ServicePoolAware which should cause Camel to
> not pool it and thus you get a new fresh instance each time.
>
> Then try testing with that to see if that fixes the re-connect issue.
>
> What we can then do is to add some nice option in the endpoint URI so
> you can configure the pooling behavior.
>
>
> > 2010/6/21 Claus Ibsen <cl...@gmail.com>
> >
> >> On Mon, Jun 21, 2010 at 8:52 AM, Bengt Rodehav <be...@rodehav.com>
> wrote:
> >> > I have now created a JIRA in commons-net for this issue:
> >> >
> >> > https://issues.apache.org/jira/browse/NET-327
> >> >
> >>
> >> Good
> >>
> >> > <https://issues.apache.org/jira/browse/NET-327>However, I still think
> we
> >> > must fix this in camel-ftp by some kind of work-around (maybe similar
> to
> >> the
> >> > one I proposed in
> https://issues.apache.org/activemq/browse/CAMEL-2829).
> >> > Have you had a chance to look at it Claus?
> >> >
> >>
> >> No sorry I am to busy with other issues but we got the ticket so we
> >> will look into it before 2.4 is cut.
> >>
> >>
> >> > /Bengt
> >> >
> >> > 2010/6/18 Bengt Rodehav <be...@rodehav.com>
> >> >
> >> >> JIRA created:
> >> >>
> >> >> https://issues.apache.org/activemq/browse/CAMEL-2829
> >> >>
> >> >> <https://issues.apache.org/activemq/browse/CAMEL-2829>/Bengt
> >> >>
> >> >> 2010/6/18 Bengt Rodehav <be...@rodehav.com>
> >> >>
> >> >> Claus,
> >> >>>
> >> >>> I'm not sure what you mean by:
> >> >>>
> >> >>> "I think we should add an option on the endpoint to allow people to
> >> >>> turn this on/off.
> >> >>> Some would like to reuse existing connections to avoid the connect
> ->
> >> >>> upload -> disconnect cycle when they upload many files etc."
> >> >>>
> >> >>> IMO we would not disconnect more often than today. We would just
> change
> >> >>> the connect() method in FtpsOperations so that it creates a new
> >> instance of
> >> >>> FTPSClient prior to a connection attempt. I assume that since the
> >> connect()
> >> >>> method is called in the first place, we are not currently connected
> and
> >> >>> there is no way to reuse an existing connection. Am I right?
> >> >>>
> >> >>> I have made some modifications in camel-ftp along these lines. I
> will
> >> >>> create a JIRA ticket for this bug and attach my changes as diff
> files
> >> to the
> >> >>> issue. I just tested my changes and they do solve the problem I have
> >> >>> encountered but I haven't tested whether I have introduced any new
> >> problems
> >> >>> or not.
> >> >>>
> >> >>> Anyway, this is what I did:
> >> >>>
> >> >>> *FtpsEndpoint*
> >> >>> I added the following members to keep track of the configuration of
> the
> >> >>> FTPSClient:
> >> >>>
> >> >>> private boolean isNeedClientAuth = false;
> >> >>> private KeyManager keyManager;
> >> >>> private TrustManager trustManager;
> >> >>>
> >> >>> I modified the createFtpClient() method so that it doesn't actually
> >> create
> >> >>> a FTPSClient but only calculates and stores the configuration in the
> >> above
> >> >>> members. I also renamed it to createFtpsConfiguration() to reflect
> >> that.
> >> >>>
> >> >>> The createRemoteFileOperations() method was also changed
> accordingly.
> >> It
> >> >>> no longer interacts directly with an FTPSClient (since it is not
> >> created
> >> >>> yet).
> >> >>>
> >> >>> *FtpsOperations*
> >> >>> I added the private createFtpsClient() method that takes care of
> >> creating
> >> >>> an FTPSClient from configuration stored in the FtpsEndpoint. The
> >> >>> createFtpsClient() is called in the connect() method just before the
> >> call to
> >> >>> the "real" connect() method in the superclass (FtpOperations).
> >> >>>
> >> >>> I changed the constructor so that it no longer takes a client as an
> >> >>> argument. I send null as client to the super class constructor. This
> is
> >> of
> >> >>> course a major change in behavior that you need to take a look at
> it.
> >> >>> Neither FtpsOperations nor its super class FtpOperations can no
> longer
> >> >>> assume that a client exists until the first connection attempt has
> been
> >> >>> made.
> >> >>>
> >> >>> *FtpOperations*
> >> >>> I had to make the "client" member non-final since I now set it on
> every
> >> >>> call to the connect() method in FtpsOperations.
> >> >>>
> >> >>>
> >> >>> BTW, I agree with you that the SFTPClient in commons-net is flawed.
> It
> >> >>> should provide a way to re-initialize itself so that a new connect()
> >> could
> >> >>> be made. The FTPClient seems to work that way and it also provides a
> >> >>> disconnect() method. If they do not intend to support this, then
> they
> >> should
> >> >>> at least state, as part of the documentation, that the SFTPClient
> >> cannot be
> >> >>> reused across multiple connections. I've posted about this on their
> >> mailing
> >> >>> list but hasn't got any reply yet. Meanwhile the above can be
> regarded
> >> as a
> >> >>> (complicated...) work-around.
> >> >>>
> >> >>> /Bengt
> >> >>>
> >> >>>
> >> >>> 2010/6/18 Claus Ibsen <cl...@gmail.com>
> >> >>>
> >> >>> On Thu, Jun 17, 2010 at 11:56 PM, Bengt Rodehav <be...@rodehav.com>
> >> >>>> wrote:
> >> >>>> > Claus,
> >> >>>> >
> >> >>>> > Having looked a bit more at the commons-net code it's becoming
> clear
> >> to
> >> >>>> me
> >> >>>> > that you probably can't just reconnect an FTPSClient once it has
> >> >>>> entered
> >> >>>> > secure communication mode. I can't find a way to re-initialise an
> >> >>>> > FTPSClient. I tried the disconnect() method (in the super class
> >> >>>> FTPClient)
> >> >>>> > but it didn't get the FTPSClient out of secure communication
> mode.
> >> >>>> >
> >> >>>> > To be safe I think we should always start with a newly
> instantiated
> >> >>>> > FTPSClient when we try to connect. Today the FTPSClient is
> created
> >> by
> >> >>>> the
> >> >>>> > FtpsEndpoint. I don't think it has to. It could just get all the
> >> righ
> >> >>>> > configuration parameters and pass it to FtpsOperations which
> would
> >> then
> >> >>>> > instantiate a new FTPSClient at every connect attempt.
> >> >>>> >
> >> >>>> > If we do it this way for ftps I guess it would make sense with
> the
> >> same
> >> >>>> > setup for plain ftp.
> >> >>>> >
> >> >>>> > What do you think?
> >> >>>> >
> >> >>>>
> >> >>>> I think we should add an option on the endpoint to allow people to
> >> >>>> turn this on/off.
> >> >>>> Some would like to reuse existing connections to avoid the connect
> ->
> >> >>>> upload -> disconnect cycle when they upload many files etc.
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>> And frankly I can't see the problem why a secure connection cannot
> >> >>>> self heal and be able to re-establish itself. After all FTPSClient
> got
> >> >>>> all the connection information. And hence I would consider it a
> >> >>>> missing feature / glitch in the FTP library.
> >> >>>>
> >> >>>> Imagine if using secure connections for JMS brokers which couldn't
> >> >>>> self heal just because a network outage. That would cause a lot of
> >> >>>> pain for clients having to re-start.
> >> >>>>
> >> >>>>
> >> >>>> > /Bengt
> >> >>>> >
> >> >>>> > 2010/6/17 Bengt Rodehav <be...@rodehav.com>
> >> >>>> >
> >> >>>> >> Claus,
> >> >>>> >>
> >> >>>> >> Yeah, maybe this is a commons-net issue. As always I find it
> >> >>>> surprising
> >> >>>> >> that no one else has encountered this problem.
> >> >>>> >>
> >> >>>> >> As I started writing an email on the commons user list, I took
> >> another
> >> >>>> look
> >> >>>> >> at the code in commons-net. It turns out that the
> FTPSSocketFactory
> >> >>>> class
> >> >>>> >> actually does override the createSocket() methods in
> SocketFactory.
> >> A
> >> >>>> >> similar work-around that I was talking about but on the factory
> >> class
> >> >>>> >> instead of in the SocketClient class. FTPSSocketFactory
> delegates
> >> the
> >> >>>> >> createSocket() calls to its SSLContect's socket factory.
> >> >>>> >>
> >> >>>> >> So why then are we initially able to connect but not after we
> have
> >> >>>> called
> >> >>>> >> execProt() and thus changed the connection factory?
> >> >>>> >>
> >> >>>> >> Moving back to the FTPSClient class now. When the initial
> >> connection
> >> >>>> is up,
> >> >>>> >> the _connectAction() method is called. It in turn calls
> >> >>>> sslNegotiation()
> >> >>>> >> which in turn sets up the secure socket. In the sslNegotiation()
> >> >>>> method, an
> >> >>>> >> SSLSocketFactory is not instantiated directly. It is given by
> the
> >> >>>> >> SSLContext's getSocketFactory() method. This connection factory
> >> >>>> obviously
> >> >>>> >> works otherwise the secure connection wouldn't work even
> initially.
> >> >>>> >>
> >> >>>> >> Compare this with what's being done in the execProt() method.
> Here
> >> the
> >> >>>> >> FTPSSocketFactory is instantiated directly with the SSLContext
> >> passed
> >> >>>> as a
> >> >>>> >> constructor argument. Obviously this factory does NOT work...
> >> >>>> >>
> >> >>>> >> Another question that comes to mind is why the socket factory
> >> created
> >> >>>> in
> >> >>>> >> the sslNegotiation() method is NOT set as the SocketClient's
> socket
> >> >>>> factory
> >> >>>> >> but the one created in the execProt() method is? It turns out
> that
> >> >>>> >> execProt() calls a generic sendCommand() method that sends the
> >> command
> >> >>>> and
> >> >>>> >> if the reply is OK, then sets the socket factory to null! One
> >> wonders
> >> >>>> why
> >> >>>> >> (and there is also a comment in the code: "Check this - is this
> >> >>>> necessary at
> >> >>>> >> all?"). Anyway, this is probably the reason why execProt() needs
> to
> >> >>>> "reset"
> >> >>>> >> the connection factory.
> >> >>>> >>
> >> >>>> >> Still sounds like a commons-net problem though. I'll try to post
> >> this
> >> >>>> at
> >> >>>> >> their user mailing list.
> >> >>>> >>
> >> >>>> >> Just one thought. Do we need to reuse the FTPSClient instance?
> >> Can't
> >> >>>> we
> >> >>>> >> just create a fresh new instance everytime we need to connect?
> >> >>>> >>
> >> >>>> >> /Bengt
> >> >>>> >>
> >> >>>> >>
> >> >>>> >>
> >> >>>> >> 2010/6/17 Claus Ibsen <cl...@gmail.com>
> >> >>>> >>
> >> >>>> >> Hi Bengt
> >> >>>> >>>
> >> >>>> >>> I think the issue should be created as a ticket for Apache
> Commons
> >> >>>> >>> Net. Then at least the people there can take a look, and maybe
> >> they
> >> >>>> >>> got some ideas. And can fix it in a future release.
> >> >>>> >>>
> >> >>>> >>> Bengt fell free to experiment yourself with the subclassing, as
> it
> >> >>>> >>> sounds like a good workaround.
> >> >>>> >>>
> >> >>>> >>> I assume the FTPSClient doesn't offer other methods to re
> connect
> >> >>>> >>> which can recover this error?
> >> >>>> >>>
> >> >>>> >>>
> >> >>>> >>> On Wed, Jun 16, 2010 at 2:44 PM, Bengt Rodehav <
> bengt@rodehav.com
> >> >
> >> >>>> wrote:
> >> >>>> >>> > I have mentioned this problem in a previous conversation but
> >> after
> >> >>>> >>> > investigating the subject further I decided to start a new
> >> thread
> >> >>>> for
> >> >>>> >>> this.
> >> >>>> >>> >
> >> >>>> >>> > If some kind of problem is encountered while using ftps (I
> >> tested
> >> >>>> this
> >> >>>> >>> using
> >> >>>> >>> > Filezilla server by copying a file that already existed which
> >> >>>> Filezilla
> >> >>>> >>> does
> >> >>>> >>> > not allow), then at the next attempt to connect to the ftps
> >> server,
> >> >>>> I
> >> >>>> >>> get
> >> >>>> >>> > the following exception in my logfile:
> >> >>>> >>> >
> >> >>>> >>> > 13:03:27,606 | ERROR | %7Bfile%3Aext%7D |
> >> GenericFileOnCompletion
> >> >>>> >>> |
> >> >>>> >>> > rg.apache.camel.processor.Logger 248 | Caused by:
> >> >>>> >>> >
> >> >>>>
> [org.apache.camel.component.file.GenericFileOperationFailedException -
> >> >>>> >>> File
> >> >>>> >>> > operation failed: null Unconnected sockets not implemented.
> >> Code:
> >> >>>> 221]
> >> >>>> >>> >
> >> >>>>
> org.apache.camel.component.file.GenericFileOperationFailedException:
> >> >>>> >>> File
> >> >>>> >>> > operation failed: null Unconnected sockets not implemented.
> >> Code:
> >> >>>> 221
> >> >>>> >>> > at
> >> >>>> >>> >
> >> >>>> >>>
> >> >>>>
> >>
> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:108)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >> >>>> >>> > at
> >> >>>> >>> >
> >> >>>> >>>
> >> >>>>
> >>
> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >> >>>> >>> > at
> >> >>>> >>> >
> >> >>>> >>>
> >> >>>>
> >>
> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >> >>>> >>> > at
> >> >>>> >>> >
> >> >>>> >>>
> >> >>>>
> >>
> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:123)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >> >>>> >>> > at
> >> >>>> >>> >
> >> >>>> >>>
> >> >>>>
> >>
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
> >> >>>> >>> > at
> >> >>>> >>> >
> >> >>>> >>>
> >> >>>>
> >>
> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >> >>>> >>> > at
> >> >>>> >>> >
> >> >>>> >>>
> >> >>>>
> >>
> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:91)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
> >> >>>> >>> >
> >> >>>> >>> > I added some logging to see the actual stack trace which is:
> >> >>>> >>> >
> >> >>>> >>> > java.net.SocketException: Unconnected sockets not implemented
> >> >>>> >>> > at
> >> >>>> javax.net.SocketFactory.createSocket(SocketFactory.java:104)
> >> >>>> >>> > at
> >> >>>> >>> >
> >> org.apache.commons.net.SocketClient.connect(SocketClient.java:175)
> >> >>>> >>> > at
> >> >>>> >>> >
> >> >>>> >>>
> >> >>>>
> >>
> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:91)
> >> >>>> >>> > at
> >> >>>> >>> >
> >> >>>> >>>
> >> >>>>
> >>
> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)
> >> >>>> >>> > at
> >> >>>> >>> >
> >> >>>> >>>
> >> >>>>
> >>
> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)
> >> >>>> >>> > at
> >> >>>> >>> >
> >> >>>> >>>
> >> >>>>
> >>
> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:99)
> >> >>>> >>> > at
> >> >>>> >>> >
> >> >>>> >>>
> >> >>>>
> >>
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)
> >> >>>> >>> > at
> >> >>>> >>> >
> >> >>>> >>>
> >> >>>>
> >>
> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)
> >> >>>> >>> >
> >> >>>> >>> > commons-net SocketClient class tries to call the socket
> >> factory's
> >> >>>> >>> > createSocket() method (with no parameters):
> >> >>>> >>> >
> >> >>>> >>> > _socket_= _socketFactory_.createSocket();
> >> >>>> >>> > _socket_.connect(new InetSocketAddress(hostname,
> port),
> >> >>>> >>> > connectTimeout);
> >> >>>> >>> >
> >> >>>> >>> > Then intention is to create an "unconnected socket" (hence
> the
> >> >>>> error
> >> >>>> >>> > message) and then subsequently connect it. In my case, the
> >> >>>> connection
> >> >>>> >>> > factory being used is an FTPSSocketFactory
> >> >>>> >>> > (package org.apache.commons.net.ftp) that inherits from
> >> >>>> >>> > SocketFactory(package javax.net). The SocketFactory class
> >> >>>> implements
> >> >>>> >>> the
> >> >>>> >>> > createSocket() method but throws the exception ("Unconnected
> >> >>>> sockets not
> >> >>>> >>> > implemented").
> >> >>>> >>> >
> >> >>>> >>> > The reason why this happens is that after having executed the
> >> >>>> >>> > FTPSClient.execPROT() method (package
> >> org.apache.commons.net.ftp),
> >> >>>> with
> >> >>>> >>> any
> >> >>>> >>> > other parameter than "C" (in my case I used "P"), the socket
> >> >>>> factory is
> >> >>>> >>> set
> >> >>>> >>> > to an FTPSSocketFactory as follows:
> >> >>>> >>> >
> >> >>>> >>> > setSocketFactory(new FTPSSocketFactory(context));
> >> >>>> >>> >
> >> >>>> >>> > After this, no connection attempt will succeed. This problem
> has
> >> >>>> been
> >> >>>> >>> > introduced recently when Claus (on my initiative) added
> support
> >> for
> >> >>>> >>> secure
> >> >>>> >>> > data channel in ftps. I'm thus to blame...
> >> >>>> >>> >
> >> >>>> >>> > I'm not sure how this should be fixed.
> >> >>>> >>> >
> >> >>>> >>> > It's a bit strange that all the connect methods
> >> >>>> >>> > in org.apache.commons.net.SocketClient (which is the ultimate
> >> base
> >> >>>> class
> >> >>>> >>> of
> >> >>>> >>> > FTPSClient) always try to create an unconnected socket first
> and
> >> >>>> then
> >> >>>> >>> > connects it. If commons-net uses this pattern, then it should
> >> make
> >> >>>> sure
> >> >>>> >>> that
> >> >>>> >>> > all its connection factor's support unconnected sockets but
> the
> >> >>>> >>> > FTPSSocketFactory dosn't. This sounds like a bug in
> commons-net.
> >> >>>> >>> >
> >> >>>> >>> > Knowing that commons-net doesn't release very frequently I
> think
> >> we
> >> >>>> need
> >> >>>> >>> a
> >> >>>> >>> > workaround for this. I guess it would be possible to subclass
> >> the
> >> >>>> >>> FTPClient
> >> >>>> >>> > class and override all the connect methods to make sure that
> no
> >> >>>> attempt
> >> >>>> >>> is
> >> >>>> >>> > made to create unconnected sockets. We would then change the
> >> >>>> >>> > createFtpClient() method in the FtpsEndpoint class to
> >> instantiate
> >> >>>> our
> >> >>>> >>> own
> >> >>>> >>> > subclass instead of an FTPSClient. Doesn't sound like a nice
> >> clean
> >> >>>> >>> solution
> >> >>>> >>> > though.
> >> >>>> >>> >
> >> >>>> >>> > Any ideas?
> >> >>>> >>> >
> >> >>>> >>> > /Bengt
> >> >>>> >>> >
> >> >>>> >>>
> >> >>>> >>>
> >> >>>> >>>
> >> >>>> >>> --
> >> >>>> >>> Claus Ibsen
> >> >>>> >>> Apache Camel Committer
> >> >>>> >>>
> >> >>>> >>> Author of Camel in Action: http://www.manning.com/ibsen/
> >> >>>> >>> Open Source Integration: http://fusesource.com
> >> >>>> >>> Blog: http://davsclaus.blogspot.com/
> >> >>>> >>> Twitter: http://twitter.com/davsclaus
> >> >>>> >>>
> >> >>>> >>
> >> >>>> >>
> >> >>>> >
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>> --
> >> >>>> Claus Ibsen
> >> >>>> Apache Camel Committer
> >> >>>>
> >> >>>> Author of Camel in Action: http://www.manning.com/ibsen/
> >> >>>> Open Source Integration: http://fusesource.com
> >> >>>> Blog: http://davsclaus.blogspot.com/
> >> >>>> Twitter: http://twitter.com/davsclaus
> >> >>>>
> >> >>>
> >> >>>
> >> >>
> >> >
> >>
> >>
> >>
> >> --
> >> Claus Ibsen
> >> Apache Camel Committer
> >>
> >> Author of Camel in Action: http://www.manning.com/ibsen/
> >> Open Source Integration: http://fusesource.com
> >> Blog: http://davsclaus.blogspot.com/
> >> Twitter: http://twitter.com/davsclaus
> >>
> >
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
Re: "Unconnected sockets not implemented" exception in camel-ftp when
using ftps
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Jun 21, 2010 at 9:11 AM, Bengt Rodehav <be...@rodehav.com> wrote:
> OK
>
Bengt could you try changing camel-ftp in the RemoteFileProducer
public class RemoteFileProducer<T> extends GenericFileProducer<T>
implements ServicePoolAware {
And remove the implements ServicePoolAware which should cause Camel to
not pool it and thus you get a new fresh instance each time.
Then try testing with that to see if that fixes the re-connect issue.
What we can then do is to add some nice option in the endpoint URI so
you can configure the pooling behavior.
> 2010/6/21 Claus Ibsen <cl...@gmail.com>
>
>> On Mon, Jun 21, 2010 at 8:52 AM, Bengt Rodehav <be...@rodehav.com> wrote:
>> > I have now created a JIRA in commons-net for this issue:
>> >
>> > https://issues.apache.org/jira/browse/NET-327
>> >
>>
>> Good
>>
>> > <https://issues.apache.org/jira/browse/NET-327>However, I still think we
>> > must fix this in camel-ftp by some kind of work-around (maybe similar to
>> the
>> > one I proposed in https://issues.apache.org/activemq/browse/CAMEL-2829).
>> > Have you had a chance to look at it Claus?
>> >
>>
>> No sorry I am to busy with other issues but we got the ticket so we
>> will look into it before 2.4 is cut.
>>
>>
>> > /Bengt
>> >
>> > 2010/6/18 Bengt Rodehav <be...@rodehav.com>
>> >
>> >> JIRA created:
>> >>
>> >> https://issues.apache.org/activemq/browse/CAMEL-2829
>> >>
>> >> <https://issues.apache.org/activemq/browse/CAMEL-2829>/Bengt
>> >>
>> >> 2010/6/18 Bengt Rodehav <be...@rodehav.com>
>> >>
>> >> Claus,
>> >>>
>> >>> I'm not sure what you mean by:
>> >>>
>> >>> "I think we should add an option on the endpoint to allow people to
>> >>> turn this on/off.
>> >>> Some would like to reuse existing connections to avoid the connect ->
>> >>> upload -> disconnect cycle when they upload many files etc."
>> >>>
>> >>> IMO we would not disconnect more often than today. We would just change
>> >>> the connect() method in FtpsOperations so that it creates a new
>> instance of
>> >>> FTPSClient prior to a connection attempt. I assume that since the
>> connect()
>> >>> method is called in the first place, we are not currently connected and
>> >>> there is no way to reuse an existing connection. Am I right?
>> >>>
>> >>> I have made some modifications in camel-ftp along these lines. I will
>> >>> create a JIRA ticket for this bug and attach my changes as diff files
>> to the
>> >>> issue. I just tested my changes and they do solve the problem I have
>> >>> encountered but I haven't tested whether I have introduced any new
>> problems
>> >>> or not.
>> >>>
>> >>> Anyway, this is what I did:
>> >>>
>> >>> *FtpsEndpoint*
>> >>> I added the following members to keep track of the configuration of the
>> >>> FTPSClient:
>> >>>
>> >>> private boolean isNeedClientAuth = false;
>> >>> private KeyManager keyManager;
>> >>> private TrustManager trustManager;
>> >>>
>> >>> I modified the createFtpClient() method so that it doesn't actually
>> create
>> >>> a FTPSClient but only calculates and stores the configuration in the
>> above
>> >>> members. I also renamed it to createFtpsConfiguration() to reflect
>> that.
>> >>>
>> >>> The createRemoteFileOperations() method was also changed accordingly.
>> It
>> >>> no longer interacts directly with an FTPSClient (since it is not
>> created
>> >>> yet).
>> >>>
>> >>> *FtpsOperations*
>> >>> I added the private createFtpsClient() method that takes care of
>> creating
>> >>> an FTPSClient from configuration stored in the FtpsEndpoint. The
>> >>> createFtpsClient() is called in the connect() method just before the
>> call to
>> >>> the "real" connect() method in the superclass (FtpOperations).
>> >>>
>> >>> I changed the constructor so that it no longer takes a client as an
>> >>> argument. I send null as client to the super class constructor. This is
>> of
>> >>> course a major change in behavior that you need to take a look at it.
>> >>> Neither FtpsOperations nor its super class FtpOperations can no longer
>> >>> assume that a client exists until the first connection attempt has been
>> >>> made.
>> >>>
>> >>> *FtpOperations*
>> >>> I had to make the "client" member non-final since I now set it on every
>> >>> call to the connect() method in FtpsOperations.
>> >>>
>> >>>
>> >>> BTW, I agree with you that the SFTPClient in commons-net is flawed. It
>> >>> should provide a way to re-initialize itself so that a new connect()
>> could
>> >>> be made. The FTPClient seems to work that way and it also provides a
>> >>> disconnect() method. If they do not intend to support this, then they
>> should
>> >>> at least state, as part of the documentation, that the SFTPClient
>> cannot be
>> >>> reused across multiple connections. I've posted about this on their
>> mailing
>> >>> list but hasn't got any reply yet. Meanwhile the above can be regarded
>> as a
>> >>> (complicated...) work-around.
>> >>>
>> >>> /Bengt
>> >>>
>> >>>
>> >>> 2010/6/18 Claus Ibsen <cl...@gmail.com>
>> >>>
>> >>> On Thu, Jun 17, 2010 at 11:56 PM, Bengt Rodehav <be...@rodehav.com>
>> >>>> wrote:
>> >>>> > Claus,
>> >>>> >
>> >>>> > Having looked a bit more at the commons-net code it's becoming clear
>> to
>> >>>> me
>> >>>> > that you probably can't just reconnect an FTPSClient once it has
>> >>>> entered
>> >>>> > secure communication mode. I can't find a way to re-initialise an
>> >>>> > FTPSClient. I tried the disconnect() method (in the super class
>> >>>> FTPClient)
>> >>>> > but it didn't get the FTPSClient out of secure communication mode.
>> >>>> >
>> >>>> > To be safe I think we should always start with a newly instantiated
>> >>>> > FTPSClient when we try to connect. Today the FTPSClient is created
>> by
>> >>>> the
>> >>>> > FtpsEndpoint. I don't think it has to. It could just get all the
>> righ
>> >>>> > configuration parameters and pass it to FtpsOperations which would
>> then
>> >>>> > instantiate a new FTPSClient at every connect attempt.
>> >>>> >
>> >>>> > If we do it this way for ftps I guess it would make sense with the
>> same
>> >>>> > setup for plain ftp.
>> >>>> >
>> >>>> > What do you think?
>> >>>> >
>> >>>>
>> >>>> I think we should add an option on the endpoint to allow people to
>> >>>> turn this on/off.
>> >>>> Some would like to reuse existing connections to avoid the connect ->
>> >>>> upload -> disconnect cycle when they upload many files etc.
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> And frankly I can't see the problem why a secure connection cannot
>> >>>> self heal and be able to re-establish itself. After all FTPSClient got
>> >>>> all the connection information. And hence I would consider it a
>> >>>> missing feature / glitch in the FTP library.
>> >>>>
>> >>>> Imagine if using secure connections for JMS brokers which couldn't
>> >>>> self heal just because a network outage. That would cause a lot of
>> >>>> pain for clients having to re-start.
>> >>>>
>> >>>>
>> >>>> > /Bengt
>> >>>> >
>> >>>> > 2010/6/17 Bengt Rodehav <be...@rodehav.com>
>> >>>> >
>> >>>> >> Claus,
>> >>>> >>
>> >>>> >> Yeah, maybe this is a commons-net issue. As always I find it
>> >>>> surprising
>> >>>> >> that no one else has encountered this problem.
>> >>>> >>
>> >>>> >> As I started writing an email on the commons user list, I took
>> another
>> >>>> look
>> >>>> >> at the code in commons-net. It turns out that the FTPSSocketFactory
>> >>>> class
>> >>>> >> actually does override the createSocket() methods in SocketFactory.
>> A
>> >>>> >> similar work-around that I was talking about but on the factory
>> class
>> >>>> >> instead of in the SocketClient class. FTPSSocketFactory delegates
>> the
>> >>>> >> createSocket() calls to its SSLContect's socket factory.
>> >>>> >>
>> >>>> >> So why then are we initially able to connect but not after we have
>> >>>> called
>> >>>> >> execProt() and thus changed the connection factory?
>> >>>> >>
>> >>>> >> Moving back to the FTPSClient class now. When the initial
>> connection
>> >>>> is up,
>> >>>> >> the _connectAction() method is called. It in turn calls
>> >>>> sslNegotiation()
>> >>>> >> which in turn sets up the secure socket. In the sslNegotiation()
>> >>>> method, an
>> >>>> >> SSLSocketFactory is not instantiated directly. It is given by the
>> >>>> >> SSLContext's getSocketFactory() method. This connection factory
>> >>>> obviously
>> >>>> >> works otherwise the secure connection wouldn't work even initially.
>> >>>> >>
>> >>>> >> Compare this with what's being done in the execProt() method. Here
>> the
>> >>>> >> FTPSSocketFactory is instantiated directly with the SSLContext
>> passed
>> >>>> as a
>> >>>> >> constructor argument. Obviously this factory does NOT work...
>> >>>> >>
>> >>>> >> Another question that comes to mind is why the socket factory
>> created
>> >>>> in
>> >>>> >> the sslNegotiation() method is NOT set as the SocketClient's socket
>> >>>> factory
>> >>>> >> but the one created in the execProt() method is? It turns out that
>> >>>> >> execProt() calls a generic sendCommand() method that sends the
>> command
>> >>>> and
>> >>>> >> if the reply is OK, then sets the socket factory to null! One
>> wonders
>> >>>> why
>> >>>> >> (and there is also a comment in the code: "Check this - is this
>> >>>> necessary at
>> >>>> >> all?"). Anyway, this is probably the reason why execProt() needs to
>> >>>> "reset"
>> >>>> >> the connection factory.
>> >>>> >>
>> >>>> >> Still sounds like a commons-net problem though. I'll try to post
>> this
>> >>>> at
>> >>>> >> their user mailing list.
>> >>>> >>
>> >>>> >> Just one thought. Do we need to reuse the FTPSClient instance?
>> Can't
>> >>>> we
>> >>>> >> just create a fresh new instance everytime we need to connect?
>> >>>> >>
>> >>>> >> /Bengt
>> >>>> >>
>> >>>> >>
>> >>>> >>
>> >>>> >> 2010/6/17 Claus Ibsen <cl...@gmail.com>
>> >>>> >>
>> >>>> >> Hi Bengt
>> >>>> >>>
>> >>>> >>> I think the issue should be created as a ticket for Apache Commons
>> >>>> >>> Net. Then at least the people there can take a look, and maybe
>> they
>> >>>> >>> got some ideas. And can fix it in a future release.
>> >>>> >>>
>> >>>> >>> Bengt fell free to experiment yourself with the subclassing, as it
>> >>>> >>> sounds like a good workaround.
>> >>>> >>>
>> >>>> >>> I assume the FTPSClient doesn't offer other methods to re connect
>> >>>> >>> which can recover this error?
>> >>>> >>>
>> >>>> >>>
>> >>>> >>> On Wed, Jun 16, 2010 at 2:44 PM, Bengt Rodehav <bengt@rodehav.com
>> >
>> >>>> wrote:
>> >>>> >>> > I have mentioned this problem in a previous conversation but
>> after
>> >>>> >>> > investigating the subject further I decided to start a new
>> thread
>> >>>> for
>> >>>> >>> this.
>> >>>> >>> >
>> >>>> >>> > If some kind of problem is encountered while using ftps (I
>> tested
>> >>>> this
>> >>>> >>> using
>> >>>> >>> > Filezilla server by copying a file that already existed which
>> >>>> Filezilla
>> >>>> >>> does
>> >>>> >>> > not allow), then at the next attempt to connect to the ftps
>> server,
>> >>>> I
>> >>>> >>> get
>> >>>> >>> > the following exception in my logfile:
>> >>>> >>> >
>> >>>> >>> > 13:03:27,606 | ERROR | %7Bfile%3Aext%7D |
>> GenericFileOnCompletion
>> >>>> >>> |
>> >>>> >>> > rg.apache.camel.processor.Logger 248 | Caused by:
>> >>>> >>> >
>> >>>> [org.apache.camel.component.file.GenericFileOperationFailedException -
>> >>>> >>> File
>> >>>> >>> > operation failed: null Unconnected sockets not implemented.
>> Code:
>> >>>> 221]
>> >>>> >>> >
>> >>>> org.apache.camel.component.file.GenericFileOperationFailedException:
>> >>>> >>> File
>> >>>> >>> > operation failed: null Unconnected sockets not implemented.
>> Code:
>> >>>> 221
>> >>>> >>> > at
>> >>>> >>> >
>> >>>> >>>
>> >>>>
>> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:108)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> >>>> >>> > at
>> >>>> >>> >
>> >>>> >>>
>> >>>>
>> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> >>>> >>> > at
>> >>>> >>> >
>> >>>> >>>
>> >>>>
>> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> >>>> >>> > at
>> >>>> >>> >
>> >>>> >>>
>> >>>>
>> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:123)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> >>>> >>> > at
>> >>>> >>> >
>> >>>> >>>
>> >>>>
>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
>> >>>> >>> > at
>> >>>> >>> >
>> >>>> >>>
>> >>>>
>> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> >>>> >>> > at
>> >>>> >>> >
>> >>>> >>>
>> >>>>
>> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:91)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
>> >>>> >>> >
>> >>>> >>> > I added some logging to see the actual stack trace which is:
>> >>>> >>> >
>> >>>> >>> > java.net.SocketException: Unconnected sockets not implemented
>> >>>> >>> > at
>> >>>> javax.net.SocketFactory.createSocket(SocketFactory.java:104)
>> >>>> >>> > at
>> >>>> >>> >
>> org.apache.commons.net.SocketClient.connect(SocketClient.java:175)
>> >>>> >>> > at
>> >>>> >>> >
>> >>>> >>>
>> >>>>
>> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:91)
>> >>>> >>> > at
>> >>>> >>> >
>> >>>> >>>
>> >>>>
>> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)
>> >>>> >>> > at
>> >>>> >>> >
>> >>>> >>>
>> >>>>
>> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)
>> >>>> >>> > at
>> >>>> >>> >
>> >>>> >>>
>> >>>>
>> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:99)
>> >>>> >>> > at
>> >>>> >>> >
>> >>>> >>>
>> >>>>
>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)
>> >>>> >>> > at
>> >>>> >>> >
>> >>>> >>>
>> >>>>
>> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)
>> >>>> >>> >
>> >>>> >>> > commons-net SocketClient class tries to call the socket
>> factory's
>> >>>> >>> > createSocket() method (with no parameters):
>> >>>> >>> >
>> >>>> >>> > _socket_= _socketFactory_.createSocket();
>> >>>> >>> > _socket_.connect(new InetSocketAddress(hostname, port),
>> >>>> >>> > connectTimeout);
>> >>>> >>> >
>> >>>> >>> > Then intention is to create an "unconnected socket" (hence the
>> >>>> error
>> >>>> >>> > message) and then subsequently connect it. In my case, the
>> >>>> connection
>> >>>> >>> > factory being used is an FTPSSocketFactory
>> >>>> >>> > (package org.apache.commons.net.ftp) that inherits from
>> >>>> >>> > SocketFactory(package javax.net). The SocketFactory class
>> >>>> implements
>> >>>> >>> the
>> >>>> >>> > createSocket() method but throws the exception ("Unconnected
>> >>>> sockets not
>> >>>> >>> > implemented").
>> >>>> >>> >
>> >>>> >>> > The reason why this happens is that after having executed the
>> >>>> >>> > FTPSClient.execPROT() method (package
>> org.apache.commons.net.ftp),
>> >>>> with
>> >>>> >>> any
>> >>>> >>> > other parameter than "C" (in my case I used "P"), the socket
>> >>>> factory is
>> >>>> >>> set
>> >>>> >>> > to an FTPSSocketFactory as follows:
>> >>>> >>> >
>> >>>> >>> > setSocketFactory(new FTPSSocketFactory(context));
>> >>>> >>> >
>> >>>> >>> > After this, no connection attempt will succeed. This problem has
>> >>>> been
>> >>>> >>> > introduced recently when Claus (on my initiative) added support
>> for
>> >>>> >>> secure
>> >>>> >>> > data channel in ftps. I'm thus to blame...
>> >>>> >>> >
>> >>>> >>> > I'm not sure how this should be fixed.
>> >>>> >>> >
>> >>>> >>> > It's a bit strange that all the connect methods
>> >>>> >>> > in org.apache.commons.net.SocketClient (which is the ultimate
>> base
>> >>>> class
>> >>>> >>> of
>> >>>> >>> > FTPSClient) always try to create an unconnected socket first and
>> >>>> then
>> >>>> >>> > connects it. If commons-net uses this pattern, then it should
>> make
>> >>>> sure
>> >>>> >>> that
>> >>>> >>> > all its connection factor's support unconnected sockets but the
>> >>>> >>> > FTPSSocketFactory dosn't. This sounds like a bug in commons-net.
>> >>>> >>> >
>> >>>> >>> > Knowing that commons-net doesn't release very frequently I think
>> we
>> >>>> need
>> >>>> >>> a
>> >>>> >>> > workaround for this. I guess it would be possible to subclass
>> the
>> >>>> >>> FTPClient
>> >>>> >>> > class and override all the connect methods to make sure that no
>> >>>> attempt
>> >>>> >>> is
>> >>>> >>> > made to create unconnected sockets. We would then change the
>> >>>> >>> > createFtpClient() method in the FtpsEndpoint class to
>> instantiate
>> >>>> our
>> >>>> >>> own
>> >>>> >>> > subclass instead of an FTPSClient. Doesn't sound like a nice
>> clean
>> >>>> >>> solution
>> >>>> >>> > though.
>> >>>> >>> >
>> >>>> >>> > Any ideas?
>> >>>> >>> >
>> >>>> >>> > /Bengt
>> >>>> >>> >
>> >>>> >>>
>> >>>> >>>
>> >>>> >>>
>> >>>> >>> --
>> >>>> >>> Claus Ibsen
>> >>>> >>> Apache Camel Committer
>> >>>> >>>
>> >>>> >>> Author of Camel in Action: http://www.manning.com/ibsen/
>> >>>> >>> Open Source Integration: http://fusesource.com
>> >>>> >>> Blog: http://davsclaus.blogspot.com/
>> >>>> >>> Twitter: http://twitter.com/davsclaus
>> >>>> >>>
>> >>>> >>
>> >>>> >>
>> >>>> >
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Claus Ibsen
>> >>>> Apache Camel Committer
>> >>>>
>> >>>> Author of Camel in Action: http://www.manning.com/ibsen/
>> >>>> Open Source Integration: http://fusesource.com
>> >>>> Blog: http://davsclaus.blogspot.com/
>> >>>> Twitter: http://twitter.com/davsclaus
>> >>>>
>> >>>
>> >>>
>> >>
>> >
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: "Unconnected sockets not implemented" exception in camel-ftp when
using ftps
Posted by Bengt Rodehav <be...@rodehav.com>.
OK
2010/6/21 Claus Ibsen <cl...@gmail.com>
> On Mon, Jun 21, 2010 at 8:52 AM, Bengt Rodehav <be...@rodehav.com> wrote:
> > I have now created a JIRA in commons-net for this issue:
> >
> > https://issues.apache.org/jira/browse/NET-327
> >
>
> Good
>
> > <https://issues.apache.org/jira/browse/NET-327>However, I still think we
> > must fix this in camel-ftp by some kind of work-around (maybe similar to
> the
> > one I proposed in https://issues.apache.org/activemq/browse/CAMEL-2829).
> > Have you had a chance to look at it Claus?
> >
>
> No sorry I am to busy with other issues but we got the ticket so we
> will look into it before 2.4 is cut.
>
>
> > /Bengt
> >
> > 2010/6/18 Bengt Rodehav <be...@rodehav.com>
> >
> >> JIRA created:
> >>
> >> https://issues.apache.org/activemq/browse/CAMEL-2829
> >>
> >> <https://issues.apache.org/activemq/browse/CAMEL-2829>/Bengt
> >>
> >> 2010/6/18 Bengt Rodehav <be...@rodehav.com>
> >>
> >> Claus,
> >>>
> >>> I'm not sure what you mean by:
> >>>
> >>> "I think we should add an option on the endpoint to allow people to
> >>> turn this on/off.
> >>> Some would like to reuse existing connections to avoid the connect ->
> >>> upload -> disconnect cycle when they upload many files etc."
> >>>
> >>> IMO we would not disconnect more often than today. We would just change
> >>> the connect() method in FtpsOperations so that it creates a new
> instance of
> >>> FTPSClient prior to a connection attempt. I assume that since the
> connect()
> >>> method is called in the first place, we are not currently connected and
> >>> there is no way to reuse an existing connection. Am I right?
> >>>
> >>> I have made some modifications in camel-ftp along these lines. I will
> >>> create a JIRA ticket for this bug and attach my changes as diff files
> to the
> >>> issue. I just tested my changes and they do solve the problem I have
> >>> encountered but I haven't tested whether I have introduced any new
> problems
> >>> or not.
> >>>
> >>> Anyway, this is what I did:
> >>>
> >>> *FtpsEndpoint*
> >>> I added the following members to keep track of the configuration of the
> >>> FTPSClient:
> >>>
> >>> private boolean isNeedClientAuth = false;
> >>> private KeyManager keyManager;
> >>> private TrustManager trustManager;
> >>>
> >>> I modified the createFtpClient() method so that it doesn't actually
> create
> >>> a FTPSClient but only calculates and stores the configuration in the
> above
> >>> members. I also renamed it to createFtpsConfiguration() to reflect
> that.
> >>>
> >>> The createRemoteFileOperations() method was also changed accordingly.
> It
> >>> no longer interacts directly with an FTPSClient (since it is not
> created
> >>> yet).
> >>>
> >>> *FtpsOperations*
> >>> I added the private createFtpsClient() method that takes care of
> creating
> >>> an FTPSClient from configuration stored in the FtpsEndpoint. The
> >>> createFtpsClient() is called in the connect() method just before the
> call to
> >>> the "real" connect() method in the superclass (FtpOperations).
> >>>
> >>> I changed the constructor so that it no longer takes a client as an
> >>> argument. I send null as client to the super class constructor. This is
> of
> >>> course a major change in behavior that you need to take a look at it.
> >>> Neither FtpsOperations nor its super class FtpOperations can no longer
> >>> assume that a client exists until the first connection attempt has been
> >>> made.
> >>>
> >>> *FtpOperations*
> >>> I had to make the "client" member non-final since I now set it on every
> >>> call to the connect() method in FtpsOperations.
> >>>
> >>>
> >>> BTW, I agree with you that the SFTPClient in commons-net is flawed. It
> >>> should provide a way to re-initialize itself so that a new connect()
> could
> >>> be made. The FTPClient seems to work that way and it also provides a
> >>> disconnect() method. If they do not intend to support this, then they
> should
> >>> at least state, as part of the documentation, that the SFTPClient
> cannot be
> >>> reused across multiple connections. I've posted about this on their
> mailing
> >>> list but hasn't got any reply yet. Meanwhile the above can be regarded
> as a
> >>> (complicated...) work-around.
> >>>
> >>> /Bengt
> >>>
> >>>
> >>> 2010/6/18 Claus Ibsen <cl...@gmail.com>
> >>>
> >>> On Thu, Jun 17, 2010 at 11:56 PM, Bengt Rodehav <be...@rodehav.com>
> >>>> wrote:
> >>>> > Claus,
> >>>> >
> >>>> > Having looked a bit more at the commons-net code it's becoming clear
> to
> >>>> me
> >>>> > that you probably can't just reconnect an FTPSClient once it has
> >>>> entered
> >>>> > secure communication mode. I can't find a way to re-initialise an
> >>>> > FTPSClient. I tried the disconnect() method (in the super class
> >>>> FTPClient)
> >>>> > but it didn't get the FTPSClient out of secure communication mode.
> >>>> >
> >>>> > To be safe I think we should always start with a newly instantiated
> >>>> > FTPSClient when we try to connect. Today the FTPSClient is created
> by
> >>>> the
> >>>> > FtpsEndpoint. I don't think it has to. It could just get all the
> righ
> >>>> > configuration parameters and pass it to FtpsOperations which would
> then
> >>>> > instantiate a new FTPSClient at every connect attempt.
> >>>> >
> >>>> > If we do it this way for ftps I guess it would make sense with the
> same
> >>>> > setup for plain ftp.
> >>>> >
> >>>> > What do you think?
> >>>> >
> >>>>
> >>>> I think we should add an option on the endpoint to allow people to
> >>>> turn this on/off.
> >>>> Some would like to reuse existing connections to avoid the connect ->
> >>>> upload -> disconnect cycle when they upload many files etc.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> And frankly I can't see the problem why a secure connection cannot
> >>>> self heal and be able to re-establish itself. After all FTPSClient got
> >>>> all the connection information. And hence I would consider it a
> >>>> missing feature / glitch in the FTP library.
> >>>>
> >>>> Imagine if using secure connections for JMS brokers which couldn't
> >>>> self heal just because a network outage. That would cause a lot of
> >>>> pain for clients having to re-start.
> >>>>
> >>>>
> >>>> > /Bengt
> >>>> >
> >>>> > 2010/6/17 Bengt Rodehav <be...@rodehav.com>
> >>>> >
> >>>> >> Claus,
> >>>> >>
> >>>> >> Yeah, maybe this is a commons-net issue. As always I find it
> >>>> surprising
> >>>> >> that no one else has encountered this problem.
> >>>> >>
> >>>> >> As I started writing an email on the commons user list, I took
> another
> >>>> look
> >>>> >> at the code in commons-net. It turns out that the FTPSSocketFactory
> >>>> class
> >>>> >> actually does override the createSocket() methods in SocketFactory.
> A
> >>>> >> similar work-around that I was talking about but on the factory
> class
> >>>> >> instead of in the SocketClient class. FTPSSocketFactory delegates
> the
> >>>> >> createSocket() calls to its SSLContect's socket factory.
> >>>> >>
> >>>> >> So why then are we initially able to connect but not after we have
> >>>> called
> >>>> >> execProt() and thus changed the connection factory?
> >>>> >>
> >>>> >> Moving back to the FTPSClient class now. When the initial
> connection
> >>>> is up,
> >>>> >> the _connectAction() method is called. It in turn calls
> >>>> sslNegotiation()
> >>>> >> which in turn sets up the secure socket. In the sslNegotiation()
> >>>> method, an
> >>>> >> SSLSocketFactory is not instantiated directly. It is given by the
> >>>> >> SSLContext's getSocketFactory() method. This connection factory
> >>>> obviously
> >>>> >> works otherwise the secure connection wouldn't work even initially.
> >>>> >>
> >>>> >> Compare this with what's being done in the execProt() method. Here
> the
> >>>> >> FTPSSocketFactory is instantiated directly with the SSLContext
> passed
> >>>> as a
> >>>> >> constructor argument. Obviously this factory does NOT work...
> >>>> >>
> >>>> >> Another question that comes to mind is why the socket factory
> created
> >>>> in
> >>>> >> the sslNegotiation() method is NOT set as the SocketClient's socket
> >>>> factory
> >>>> >> but the one created in the execProt() method is? It turns out that
> >>>> >> execProt() calls a generic sendCommand() method that sends the
> command
> >>>> and
> >>>> >> if the reply is OK, then sets the socket factory to null! One
> wonders
> >>>> why
> >>>> >> (and there is also a comment in the code: "Check this - is this
> >>>> necessary at
> >>>> >> all?"). Anyway, this is probably the reason why execProt() needs to
> >>>> "reset"
> >>>> >> the connection factory.
> >>>> >>
> >>>> >> Still sounds like a commons-net problem though. I'll try to post
> this
> >>>> at
> >>>> >> their user mailing list.
> >>>> >>
> >>>> >> Just one thought. Do we need to reuse the FTPSClient instance?
> Can't
> >>>> we
> >>>> >> just create a fresh new instance everytime we need to connect?
> >>>> >>
> >>>> >> /Bengt
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >> 2010/6/17 Claus Ibsen <cl...@gmail.com>
> >>>> >>
> >>>> >> Hi Bengt
> >>>> >>>
> >>>> >>> I think the issue should be created as a ticket for Apache Commons
> >>>> >>> Net. Then at least the people there can take a look, and maybe
> they
> >>>> >>> got some ideas. And can fix it in a future release.
> >>>> >>>
> >>>> >>> Bengt fell free to experiment yourself with the subclassing, as it
> >>>> >>> sounds like a good workaround.
> >>>> >>>
> >>>> >>> I assume the FTPSClient doesn't offer other methods to re connect
> >>>> >>> which can recover this error?
> >>>> >>>
> >>>> >>>
> >>>> >>> On Wed, Jun 16, 2010 at 2:44 PM, Bengt Rodehav <bengt@rodehav.com
> >
> >>>> wrote:
> >>>> >>> > I have mentioned this problem in a previous conversation but
> after
> >>>> >>> > investigating the subject further I decided to start a new
> thread
> >>>> for
> >>>> >>> this.
> >>>> >>> >
> >>>> >>> > If some kind of problem is encountered while using ftps (I
> tested
> >>>> this
> >>>> >>> using
> >>>> >>> > Filezilla server by copying a file that already existed which
> >>>> Filezilla
> >>>> >>> does
> >>>> >>> > not allow), then at the next attempt to connect to the ftps
> server,
> >>>> I
> >>>> >>> get
> >>>> >>> > the following exception in my logfile:
> >>>> >>> >
> >>>> >>> > 13:03:27,606 | ERROR | %7Bfile%3Aext%7D |
> GenericFileOnCompletion
> >>>> >>> |
> >>>> >>> > rg.apache.camel.processor.Logger 248 | Caused by:
> >>>> >>> >
> >>>> [org.apache.camel.component.file.GenericFileOperationFailedException -
> >>>> >>> File
> >>>> >>> > operation failed: null Unconnected sockets not implemented.
> Code:
> >>>> 221]
> >>>> >>> >
> >>>> org.apache.camel.component.file.GenericFileOperationFailedException:
> >>>> >>> File
> >>>> >>> > operation failed: null Unconnected sockets not implemented.
> Code:
> >>>> 221
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:108)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:123)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:91)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
> >>>> >>> >
> >>>> >>> > I added some logging to see the actual stack trace which is:
> >>>> >>> >
> >>>> >>> > java.net.SocketException: Unconnected sockets not implemented
> >>>> >>> > at
> >>>> javax.net.SocketFactory.createSocket(SocketFactory.java:104)
> >>>> >>> > at
> >>>> >>> >
> org.apache.commons.net.SocketClient.connect(SocketClient.java:175)
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:91)
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:99)
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)
> >>>> >>> >
> >>>> >>> > commons-net SocketClient class tries to call the socket
> factory's
> >>>> >>> > createSocket() method (with no parameters):
> >>>> >>> >
> >>>> >>> > _socket_= _socketFactory_.createSocket();
> >>>> >>> > _socket_.connect(new InetSocketAddress(hostname, port),
> >>>> >>> > connectTimeout);
> >>>> >>> >
> >>>> >>> > Then intention is to create an "unconnected socket" (hence the
> >>>> error
> >>>> >>> > message) and then subsequently connect it. In my case, the
> >>>> connection
> >>>> >>> > factory being used is an FTPSSocketFactory
> >>>> >>> > (package org.apache.commons.net.ftp) that inherits from
> >>>> >>> > SocketFactory(package javax.net). The SocketFactory class
> >>>> implements
> >>>> >>> the
> >>>> >>> > createSocket() method but throws the exception ("Unconnected
> >>>> sockets not
> >>>> >>> > implemented").
> >>>> >>> >
> >>>> >>> > The reason why this happens is that after having executed the
> >>>> >>> > FTPSClient.execPROT() method (package
> org.apache.commons.net.ftp),
> >>>> with
> >>>> >>> any
> >>>> >>> > other parameter than "C" (in my case I used "P"), the socket
> >>>> factory is
> >>>> >>> set
> >>>> >>> > to an FTPSSocketFactory as follows:
> >>>> >>> >
> >>>> >>> > setSocketFactory(new FTPSSocketFactory(context));
> >>>> >>> >
> >>>> >>> > After this, no connection attempt will succeed. This problem has
> >>>> been
> >>>> >>> > introduced recently when Claus (on my initiative) added support
> for
> >>>> >>> secure
> >>>> >>> > data channel in ftps. I'm thus to blame...
> >>>> >>> >
> >>>> >>> > I'm not sure how this should be fixed.
> >>>> >>> >
> >>>> >>> > It's a bit strange that all the connect methods
> >>>> >>> > in org.apache.commons.net.SocketClient (which is the ultimate
> base
> >>>> class
> >>>> >>> of
> >>>> >>> > FTPSClient) always try to create an unconnected socket first and
> >>>> then
> >>>> >>> > connects it. If commons-net uses this pattern, then it should
> make
> >>>> sure
> >>>> >>> that
> >>>> >>> > all its connection factor's support unconnected sockets but the
> >>>> >>> > FTPSSocketFactory dosn't. This sounds like a bug in commons-net.
> >>>> >>> >
> >>>> >>> > Knowing that commons-net doesn't release very frequently I think
> we
> >>>> need
> >>>> >>> a
> >>>> >>> > workaround for this. I guess it would be possible to subclass
> the
> >>>> >>> FTPClient
> >>>> >>> > class and override all the connect methods to make sure that no
> >>>> attempt
> >>>> >>> is
> >>>> >>> > made to create unconnected sockets. We would then change the
> >>>> >>> > createFtpClient() method in the FtpsEndpoint class to
> instantiate
> >>>> our
> >>>> >>> own
> >>>> >>> > subclass instead of an FTPSClient. Doesn't sound like a nice
> clean
> >>>> >>> solution
> >>>> >>> > though.
> >>>> >>> >
> >>>> >>> > Any ideas?
> >>>> >>> >
> >>>> >>> > /Bengt
> >>>> >>> >
> >>>> >>>
> >>>> >>>
> >>>> >>>
> >>>> >>> --
> >>>> >>> Claus Ibsen
> >>>> >>> Apache Camel Committer
> >>>> >>>
> >>>> >>> Author of Camel in Action: http://www.manning.com/ibsen/
> >>>> >>> Open Source Integration: http://fusesource.com
> >>>> >>> Blog: http://davsclaus.blogspot.com/
> >>>> >>> Twitter: http://twitter.com/davsclaus
> >>>> >>>
> >>>> >>
> >>>> >>
> >>>> >
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Claus Ibsen
> >>>> Apache Camel Committer
> >>>>
> >>>> Author of Camel in Action: http://www.manning.com/ibsen/
> >>>> Open Source Integration: http://fusesource.com
> >>>> Blog: http://davsclaus.blogspot.com/
> >>>> Twitter: http://twitter.com/davsclaus
> >>>>
> >>>
> >>>
> >>
> >
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
Re: "Unconnected sockets not implemented" exception in camel-ftp when
using ftps
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Jun 21, 2010 at 8:52 AM, Bengt Rodehav <be...@rodehav.com> wrote:
> I have now created a JIRA in commons-net for this issue:
>
> https://issues.apache.org/jira/browse/NET-327
>
Good
> <https://issues.apache.org/jira/browse/NET-327>However, I still think we
> must fix this in camel-ftp by some kind of work-around (maybe similar to the
> one I proposed in https://issues.apache.org/activemq/browse/CAMEL-2829).
> Have you had a chance to look at it Claus?
>
No sorry I am to busy with other issues but we got the ticket so we
will look into it before 2.4 is cut.
> /Bengt
>
> 2010/6/18 Bengt Rodehav <be...@rodehav.com>
>
>> JIRA created:
>>
>> https://issues.apache.org/activemq/browse/CAMEL-2829
>>
>> <https://issues.apache.org/activemq/browse/CAMEL-2829>/Bengt
>>
>> 2010/6/18 Bengt Rodehav <be...@rodehav.com>
>>
>> Claus,
>>>
>>> I'm not sure what you mean by:
>>>
>>> "I think we should add an option on the endpoint to allow people to
>>> turn this on/off.
>>> Some would like to reuse existing connections to avoid the connect ->
>>> upload -> disconnect cycle when they upload many files etc."
>>>
>>> IMO we would not disconnect more often than today. We would just change
>>> the connect() method in FtpsOperations so that it creates a new instance of
>>> FTPSClient prior to a connection attempt. I assume that since the connect()
>>> method is called in the first place, we are not currently connected and
>>> there is no way to reuse an existing connection. Am I right?
>>>
>>> I have made some modifications in camel-ftp along these lines. I will
>>> create a JIRA ticket for this bug and attach my changes as diff files to the
>>> issue. I just tested my changes and they do solve the problem I have
>>> encountered but I haven't tested whether I have introduced any new problems
>>> or not.
>>>
>>> Anyway, this is what I did:
>>>
>>> *FtpsEndpoint*
>>> I added the following members to keep track of the configuration of the
>>> FTPSClient:
>>>
>>> private boolean isNeedClientAuth = false;
>>> private KeyManager keyManager;
>>> private TrustManager trustManager;
>>>
>>> I modified the createFtpClient() method so that it doesn't actually create
>>> a FTPSClient but only calculates and stores the configuration in the above
>>> members. I also renamed it to createFtpsConfiguration() to reflect that.
>>>
>>> The createRemoteFileOperations() method was also changed accordingly. It
>>> no longer interacts directly with an FTPSClient (since it is not created
>>> yet).
>>>
>>> *FtpsOperations*
>>> I added the private createFtpsClient() method that takes care of creating
>>> an FTPSClient from configuration stored in the FtpsEndpoint. The
>>> createFtpsClient() is called in the connect() method just before the call to
>>> the "real" connect() method in the superclass (FtpOperations).
>>>
>>> I changed the constructor so that it no longer takes a client as an
>>> argument. I send null as client to the super class constructor. This is of
>>> course a major change in behavior that you need to take a look at it.
>>> Neither FtpsOperations nor its super class FtpOperations can no longer
>>> assume that a client exists until the first connection attempt has been
>>> made.
>>>
>>> *FtpOperations*
>>> I had to make the "client" member non-final since I now set it on every
>>> call to the connect() method in FtpsOperations.
>>>
>>>
>>> BTW, I agree with you that the SFTPClient in commons-net is flawed. It
>>> should provide a way to re-initialize itself so that a new connect() could
>>> be made. The FTPClient seems to work that way and it also provides a
>>> disconnect() method. If they do not intend to support this, then they should
>>> at least state, as part of the documentation, that the SFTPClient cannot be
>>> reused across multiple connections. I've posted about this on their mailing
>>> list but hasn't got any reply yet. Meanwhile the above can be regarded as a
>>> (complicated...) work-around.
>>>
>>> /Bengt
>>>
>>>
>>> 2010/6/18 Claus Ibsen <cl...@gmail.com>
>>>
>>> On Thu, Jun 17, 2010 at 11:56 PM, Bengt Rodehav <be...@rodehav.com>
>>>> wrote:
>>>> > Claus,
>>>> >
>>>> > Having looked a bit more at the commons-net code it's becoming clear to
>>>> me
>>>> > that you probably can't just reconnect an FTPSClient once it has
>>>> entered
>>>> > secure communication mode. I can't find a way to re-initialise an
>>>> > FTPSClient. I tried the disconnect() method (in the super class
>>>> FTPClient)
>>>> > but it didn't get the FTPSClient out of secure communication mode.
>>>> >
>>>> > To be safe I think we should always start with a newly instantiated
>>>> > FTPSClient when we try to connect. Today the FTPSClient is created by
>>>> the
>>>> > FtpsEndpoint. I don't think it has to. It could just get all the righ
>>>> > configuration parameters and pass it to FtpsOperations which would then
>>>> > instantiate a new FTPSClient at every connect attempt.
>>>> >
>>>> > If we do it this way for ftps I guess it would make sense with the same
>>>> > setup for plain ftp.
>>>> >
>>>> > What do you think?
>>>> >
>>>>
>>>> I think we should add an option on the endpoint to allow people to
>>>> turn this on/off.
>>>> Some would like to reuse existing connections to avoid the connect ->
>>>> upload -> disconnect cycle when they upload many files etc.
>>>>
>>>>
>>>>
>>>>
>>>> And frankly I can't see the problem why a secure connection cannot
>>>> self heal and be able to re-establish itself. After all FTPSClient got
>>>> all the connection information. And hence I would consider it a
>>>> missing feature / glitch in the FTP library.
>>>>
>>>> Imagine if using secure connections for JMS brokers which couldn't
>>>> self heal just because a network outage. That would cause a lot of
>>>> pain for clients having to re-start.
>>>>
>>>>
>>>> > /Bengt
>>>> >
>>>> > 2010/6/17 Bengt Rodehav <be...@rodehav.com>
>>>> >
>>>> >> Claus,
>>>> >>
>>>> >> Yeah, maybe this is a commons-net issue. As always I find it
>>>> surprising
>>>> >> that no one else has encountered this problem.
>>>> >>
>>>> >> As I started writing an email on the commons user list, I took another
>>>> look
>>>> >> at the code in commons-net. It turns out that the FTPSSocketFactory
>>>> class
>>>> >> actually does override the createSocket() methods in SocketFactory. A
>>>> >> similar work-around that I was talking about but on the factory class
>>>> >> instead of in the SocketClient class. FTPSSocketFactory delegates the
>>>> >> createSocket() calls to its SSLContect's socket factory.
>>>> >>
>>>> >> So why then are we initially able to connect but not after we have
>>>> called
>>>> >> execProt() and thus changed the connection factory?
>>>> >>
>>>> >> Moving back to the FTPSClient class now. When the initial connection
>>>> is up,
>>>> >> the _connectAction() method is called. It in turn calls
>>>> sslNegotiation()
>>>> >> which in turn sets up the secure socket. In the sslNegotiation()
>>>> method, an
>>>> >> SSLSocketFactory is not instantiated directly. It is given by the
>>>> >> SSLContext's getSocketFactory() method. This connection factory
>>>> obviously
>>>> >> works otherwise the secure connection wouldn't work even initially.
>>>> >>
>>>> >> Compare this with what's being done in the execProt() method. Here the
>>>> >> FTPSSocketFactory is instantiated directly with the SSLContext passed
>>>> as a
>>>> >> constructor argument. Obviously this factory does NOT work...
>>>> >>
>>>> >> Another question that comes to mind is why the socket factory created
>>>> in
>>>> >> the sslNegotiation() method is NOT set as the SocketClient's socket
>>>> factory
>>>> >> but the one created in the execProt() method is? It turns out that
>>>> >> execProt() calls a generic sendCommand() method that sends the command
>>>> and
>>>> >> if the reply is OK, then sets the socket factory to null! One wonders
>>>> why
>>>> >> (and there is also a comment in the code: "Check this - is this
>>>> necessary at
>>>> >> all?"). Anyway, this is probably the reason why execProt() needs to
>>>> "reset"
>>>> >> the connection factory.
>>>> >>
>>>> >> Still sounds like a commons-net problem though. I'll try to post this
>>>> at
>>>> >> their user mailing list.
>>>> >>
>>>> >> Just one thought. Do we need to reuse the FTPSClient instance? Can't
>>>> we
>>>> >> just create a fresh new instance everytime we need to connect?
>>>> >>
>>>> >> /Bengt
>>>> >>
>>>> >>
>>>> >>
>>>> >> 2010/6/17 Claus Ibsen <cl...@gmail.com>
>>>> >>
>>>> >> Hi Bengt
>>>> >>>
>>>> >>> I think the issue should be created as a ticket for Apache Commons
>>>> >>> Net. Then at least the people there can take a look, and maybe they
>>>> >>> got some ideas. And can fix it in a future release.
>>>> >>>
>>>> >>> Bengt fell free to experiment yourself with the subclassing, as it
>>>> >>> sounds like a good workaround.
>>>> >>>
>>>> >>> I assume the FTPSClient doesn't offer other methods to re connect
>>>> >>> which can recover this error?
>>>> >>>
>>>> >>>
>>>> >>> On Wed, Jun 16, 2010 at 2:44 PM, Bengt Rodehav <be...@rodehav.com>
>>>> wrote:
>>>> >>> > I have mentioned this problem in a previous conversation but after
>>>> >>> > investigating the subject further I decided to start a new thread
>>>> for
>>>> >>> this.
>>>> >>> >
>>>> >>> > If some kind of problem is encountered while using ftps (I tested
>>>> this
>>>> >>> using
>>>> >>> > Filezilla server by copying a file that already existed which
>>>> Filezilla
>>>> >>> does
>>>> >>> > not allow), then at the next attempt to connect to the ftps server,
>>>> I
>>>> >>> get
>>>> >>> > the following exception in my logfile:
>>>> >>> >
>>>> >>> > 13:03:27,606 | ERROR | %7Bfile%3Aext%7D | GenericFileOnCompletion
>>>> >>> |
>>>> >>> > rg.apache.camel.processor.Logger 248 | Caused by:
>>>> >>> >
>>>> [org.apache.camel.component.file.GenericFileOperationFailedException -
>>>> >>> File
>>>> >>> > operation failed: null Unconnected sockets not implemented. Code:
>>>> 221]
>>>> >>> >
>>>> org.apache.camel.component.file.GenericFileOperationFailedException:
>>>> >>> File
>>>> >>> > operation failed: null Unconnected sockets not implemented. Code:
>>>> 221
>>>> >>> > at
>>>> >>> >
>>>> >>>
>>>> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:108)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>>> >>> > at
>>>> >>> >
>>>> >>>
>>>> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>>> >>> > at
>>>> >>> >
>>>> >>>
>>>> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>>> >>> > at
>>>> >>> >
>>>> >>>
>>>> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:123)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>>> >>> > at
>>>> >>> >
>>>> >>>
>>>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
>>>> >>> > at
>>>> >>> >
>>>> >>>
>>>> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>>> >>> > at
>>>> >>> >
>>>> >>>
>>>> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:91)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
>>>> >>> >
>>>> >>> > I added some logging to see the actual stack trace which is:
>>>> >>> >
>>>> >>> > java.net.SocketException: Unconnected sockets not implemented
>>>> >>> > at
>>>> javax.net.SocketFactory.createSocket(SocketFactory.java:104)
>>>> >>> > at
>>>> >>> > org.apache.commons.net.SocketClient.connect(SocketClient.java:175)
>>>> >>> > at
>>>> >>> >
>>>> >>>
>>>> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:91)
>>>> >>> > at
>>>> >>> >
>>>> >>>
>>>> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)
>>>> >>> > at
>>>> >>> >
>>>> >>>
>>>> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)
>>>> >>> > at
>>>> >>> >
>>>> >>>
>>>> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:99)
>>>> >>> > at
>>>> >>> >
>>>> >>>
>>>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)
>>>> >>> > at
>>>> >>> >
>>>> >>>
>>>> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)
>>>> >>> >
>>>> >>> > commons-net SocketClient class tries to call the socket factory's
>>>> >>> > createSocket() method (with no parameters):
>>>> >>> >
>>>> >>> > _socket_= _socketFactory_.createSocket();
>>>> >>> > _socket_.connect(new InetSocketAddress(hostname, port),
>>>> >>> > connectTimeout);
>>>> >>> >
>>>> >>> > Then intention is to create an "unconnected socket" (hence the
>>>> error
>>>> >>> > message) and then subsequently connect it. In my case, the
>>>> connection
>>>> >>> > factory being used is an FTPSSocketFactory
>>>> >>> > (package org.apache.commons.net.ftp) that inherits from
>>>> >>> > SocketFactory(package javax.net). The SocketFactory class
>>>> implements
>>>> >>> the
>>>> >>> > createSocket() method but throws the exception ("Unconnected
>>>> sockets not
>>>> >>> > implemented").
>>>> >>> >
>>>> >>> > The reason why this happens is that after having executed the
>>>> >>> > FTPSClient.execPROT() method (package org.apache.commons.net.ftp),
>>>> with
>>>> >>> any
>>>> >>> > other parameter than "C" (in my case I used "P"), the socket
>>>> factory is
>>>> >>> set
>>>> >>> > to an FTPSSocketFactory as follows:
>>>> >>> >
>>>> >>> > setSocketFactory(new FTPSSocketFactory(context));
>>>> >>> >
>>>> >>> > After this, no connection attempt will succeed. This problem has
>>>> been
>>>> >>> > introduced recently when Claus (on my initiative) added support for
>>>> >>> secure
>>>> >>> > data channel in ftps. I'm thus to blame...
>>>> >>> >
>>>> >>> > I'm not sure how this should be fixed.
>>>> >>> >
>>>> >>> > It's a bit strange that all the connect methods
>>>> >>> > in org.apache.commons.net.SocketClient (which is the ultimate base
>>>> class
>>>> >>> of
>>>> >>> > FTPSClient) always try to create an unconnected socket first and
>>>> then
>>>> >>> > connects it. If commons-net uses this pattern, then it should make
>>>> sure
>>>> >>> that
>>>> >>> > all its connection factor's support unconnected sockets but the
>>>> >>> > FTPSSocketFactory dosn't. This sounds like a bug in commons-net.
>>>> >>> >
>>>> >>> > Knowing that commons-net doesn't release very frequently I think we
>>>> need
>>>> >>> a
>>>> >>> > workaround for this. I guess it would be possible to subclass the
>>>> >>> FTPClient
>>>> >>> > class and override all the connect methods to make sure that no
>>>> attempt
>>>> >>> is
>>>> >>> > made to create unconnected sockets. We would then change the
>>>> >>> > createFtpClient() method in the FtpsEndpoint class to instantiate
>>>> our
>>>> >>> own
>>>> >>> > subclass instead of an FTPSClient. Doesn't sound like a nice clean
>>>> >>> solution
>>>> >>> > though.
>>>> >>> >
>>>> >>> > Any ideas?
>>>> >>> >
>>>> >>> > /Bengt
>>>> >>> >
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>> --
>>>> >>> Claus Ibsen
>>>> >>> Apache Camel Committer
>>>> >>>
>>>> >>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>> >>> Open Source Integration: http://fusesource.com
>>>> >>> Blog: http://davsclaus.blogspot.com/
>>>> >>> Twitter: http://twitter.com/davsclaus
>>>> >>>
>>>> >>
>>>> >>
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>
>>>
>>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: "Unconnected sockets not implemented" exception in camel-ftp when
using ftps
Posted by Bengt Rodehav <be...@rodehav.com>.
I have now created a JIRA in commons-net for this issue:
https://issues.apache.org/jira/browse/NET-327
<https://issues.apache.org/jira/browse/NET-327>However, I still think we
must fix this in camel-ftp by some kind of work-around (maybe similar to the
one I proposed in https://issues.apache.org/activemq/browse/CAMEL-2829).
Have you had a chance to look at it Claus?
/Bengt
2010/6/18 Bengt Rodehav <be...@rodehav.com>
> JIRA created:
>
> https://issues.apache.org/activemq/browse/CAMEL-2829
>
> <https://issues.apache.org/activemq/browse/CAMEL-2829>/Bengt
>
> 2010/6/18 Bengt Rodehav <be...@rodehav.com>
>
> Claus,
>>
>> I'm not sure what you mean by:
>>
>> "I think we should add an option on the endpoint to allow people to
>> turn this on/off.
>> Some would like to reuse existing connections to avoid the connect ->
>> upload -> disconnect cycle when they upload many files etc."
>>
>> IMO we would not disconnect more often than today. We would just change
>> the connect() method in FtpsOperations so that it creates a new instance of
>> FTPSClient prior to a connection attempt. I assume that since the connect()
>> method is called in the first place, we are not currently connected and
>> there is no way to reuse an existing connection. Am I right?
>>
>> I have made some modifications in camel-ftp along these lines. I will
>> create a JIRA ticket for this bug and attach my changes as diff files to the
>> issue. I just tested my changes and they do solve the problem I have
>> encountered but I haven't tested whether I have introduced any new problems
>> or not.
>>
>> Anyway, this is what I did:
>>
>> *FtpsEndpoint*
>> I added the following members to keep track of the configuration of the
>> FTPSClient:
>>
>> private boolean isNeedClientAuth = false;
>> private KeyManager keyManager;
>> private TrustManager trustManager;
>>
>> I modified the createFtpClient() method so that it doesn't actually create
>> a FTPSClient but only calculates and stores the configuration in the above
>> members. I also renamed it to createFtpsConfiguration() to reflect that.
>>
>> The createRemoteFileOperations() method was also changed accordingly. It
>> no longer interacts directly with an FTPSClient (since it is not created
>> yet).
>>
>> *FtpsOperations*
>> I added the private createFtpsClient() method that takes care of creating
>> an FTPSClient from configuration stored in the FtpsEndpoint. The
>> createFtpsClient() is called in the connect() method just before the call to
>> the "real" connect() method in the superclass (FtpOperations).
>>
>> I changed the constructor so that it no longer takes a client as an
>> argument. I send null as client to the super class constructor. This is of
>> course a major change in behavior that you need to take a look at it.
>> Neither FtpsOperations nor its super class FtpOperations can no longer
>> assume that a client exists until the first connection attempt has been
>> made.
>>
>> *FtpOperations*
>> I had to make the "client" member non-final since I now set it on every
>> call to the connect() method in FtpsOperations.
>>
>>
>> BTW, I agree with you that the SFTPClient in commons-net is flawed. It
>> should provide a way to re-initialize itself so that a new connect() could
>> be made. The FTPClient seems to work that way and it also provides a
>> disconnect() method. If they do not intend to support this, then they should
>> at least state, as part of the documentation, that the SFTPClient cannot be
>> reused across multiple connections. I've posted about this on their mailing
>> list but hasn't got any reply yet. Meanwhile the above can be regarded as a
>> (complicated...) work-around.
>>
>> /Bengt
>>
>>
>> 2010/6/18 Claus Ibsen <cl...@gmail.com>
>>
>> On Thu, Jun 17, 2010 at 11:56 PM, Bengt Rodehav <be...@rodehav.com>
>>> wrote:
>>> > Claus,
>>> >
>>> > Having looked a bit more at the commons-net code it's becoming clear to
>>> me
>>> > that you probably can't just reconnect an FTPSClient once it has
>>> entered
>>> > secure communication mode. I can't find a way to re-initialise an
>>> > FTPSClient. I tried the disconnect() method (in the super class
>>> FTPClient)
>>> > but it didn't get the FTPSClient out of secure communication mode.
>>> >
>>> > To be safe I think we should always start with a newly instantiated
>>> > FTPSClient when we try to connect. Today the FTPSClient is created by
>>> the
>>> > FtpsEndpoint. I don't think it has to. It could just get all the righ
>>> > configuration parameters and pass it to FtpsOperations which would then
>>> > instantiate a new FTPSClient at every connect attempt.
>>> >
>>> > If we do it this way for ftps I guess it would make sense with the same
>>> > setup for plain ftp.
>>> >
>>> > What do you think?
>>> >
>>>
>>> I think we should add an option on the endpoint to allow people to
>>> turn this on/off.
>>> Some would like to reuse existing connections to avoid the connect ->
>>> upload -> disconnect cycle when they upload many files etc.
>>>
>>>
>>>
>>>
>>> And frankly I can't see the problem why a secure connection cannot
>>> self heal and be able to re-establish itself. After all FTPSClient got
>>> all the connection information. And hence I would consider it a
>>> missing feature / glitch in the FTP library.
>>>
>>> Imagine if using secure connections for JMS brokers which couldn't
>>> self heal just because a network outage. That would cause a lot of
>>> pain for clients having to re-start.
>>>
>>>
>>> > /Bengt
>>> >
>>> > 2010/6/17 Bengt Rodehav <be...@rodehav.com>
>>> >
>>> >> Claus,
>>> >>
>>> >> Yeah, maybe this is a commons-net issue. As always I find it
>>> surprising
>>> >> that no one else has encountered this problem.
>>> >>
>>> >> As I started writing an email on the commons user list, I took another
>>> look
>>> >> at the code in commons-net. It turns out that the FTPSSocketFactory
>>> class
>>> >> actually does override the createSocket() methods in SocketFactory. A
>>> >> similar work-around that I was talking about but on the factory class
>>> >> instead of in the SocketClient class. FTPSSocketFactory delegates the
>>> >> createSocket() calls to its SSLContect's socket factory.
>>> >>
>>> >> So why then are we initially able to connect but not after we have
>>> called
>>> >> execProt() and thus changed the connection factory?
>>> >>
>>> >> Moving back to the FTPSClient class now. When the initial connection
>>> is up,
>>> >> the _connectAction() method is called. It in turn calls
>>> sslNegotiation()
>>> >> which in turn sets up the secure socket. In the sslNegotiation()
>>> method, an
>>> >> SSLSocketFactory is not instantiated directly. It is given by the
>>> >> SSLContext's getSocketFactory() method. This connection factory
>>> obviously
>>> >> works otherwise the secure connection wouldn't work even initially.
>>> >>
>>> >> Compare this with what's being done in the execProt() method. Here the
>>> >> FTPSSocketFactory is instantiated directly with the SSLContext passed
>>> as a
>>> >> constructor argument. Obviously this factory does NOT work...
>>> >>
>>> >> Another question that comes to mind is why the socket factory created
>>> in
>>> >> the sslNegotiation() method is NOT set as the SocketClient's socket
>>> factory
>>> >> but the one created in the execProt() method is? It turns out that
>>> >> execProt() calls a generic sendCommand() method that sends the command
>>> and
>>> >> if the reply is OK, then sets the socket factory to null! One wonders
>>> why
>>> >> (and there is also a comment in the code: "Check this - is this
>>> necessary at
>>> >> all?"). Anyway, this is probably the reason why execProt() needs to
>>> "reset"
>>> >> the connection factory.
>>> >>
>>> >> Still sounds like a commons-net problem though. I'll try to post this
>>> at
>>> >> their user mailing list.
>>> >>
>>> >> Just one thought. Do we need to reuse the FTPSClient instance? Can't
>>> we
>>> >> just create a fresh new instance everytime we need to connect?
>>> >>
>>> >> /Bengt
>>> >>
>>> >>
>>> >>
>>> >> 2010/6/17 Claus Ibsen <cl...@gmail.com>
>>> >>
>>> >> Hi Bengt
>>> >>>
>>> >>> I think the issue should be created as a ticket for Apache Commons
>>> >>> Net. Then at least the people there can take a look, and maybe they
>>> >>> got some ideas. And can fix it in a future release.
>>> >>>
>>> >>> Bengt fell free to experiment yourself with the subclassing, as it
>>> >>> sounds like a good workaround.
>>> >>>
>>> >>> I assume the FTPSClient doesn't offer other methods to re connect
>>> >>> which can recover this error?
>>> >>>
>>> >>>
>>> >>> On Wed, Jun 16, 2010 at 2:44 PM, Bengt Rodehav <be...@rodehav.com>
>>> wrote:
>>> >>> > I have mentioned this problem in a previous conversation but after
>>> >>> > investigating the subject further I decided to start a new thread
>>> for
>>> >>> this.
>>> >>> >
>>> >>> > If some kind of problem is encountered while using ftps (I tested
>>> this
>>> >>> using
>>> >>> > Filezilla server by copying a file that already existed which
>>> Filezilla
>>> >>> does
>>> >>> > not allow), then at the next attempt to connect to the ftps server,
>>> I
>>> >>> get
>>> >>> > the following exception in my logfile:
>>> >>> >
>>> >>> > 13:03:27,606 | ERROR | %7Bfile%3Aext%7D | GenericFileOnCompletion
>>> >>> |
>>> >>> > rg.apache.camel.processor.Logger 248 | Caused by:
>>> >>> >
>>> [org.apache.camel.component.file.GenericFileOperationFailedException -
>>> >>> File
>>> >>> > operation failed: null Unconnected sockets not implemented. Code:
>>> 221]
>>> >>> >
>>> org.apache.camel.component.file.GenericFileOperationFailedException:
>>> >>> File
>>> >>> > operation failed: null Unconnected sockets not implemented. Code:
>>> 221
>>> >>> > at
>>> >>> >
>>> >>>
>>> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:108)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>> >>> > at
>>> >>> >
>>> >>>
>>> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>> >>> > at
>>> >>> >
>>> >>>
>>> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>> >>> > at
>>> >>> >
>>> >>>
>>> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:123)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>> >>> > at
>>> >>> >
>>> >>>
>>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
>>> >>> > at
>>> >>> >
>>> >>>
>>> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>> >>> > at
>>> >>> >
>>> >>>
>>> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:91)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
>>> >>> >
>>> >>> > I added some logging to see the actual stack trace which is:
>>> >>> >
>>> >>> > java.net.SocketException: Unconnected sockets not implemented
>>> >>> > at
>>> javax.net.SocketFactory.createSocket(SocketFactory.java:104)
>>> >>> > at
>>> >>> > org.apache.commons.net.SocketClient.connect(SocketClient.java:175)
>>> >>> > at
>>> >>> >
>>> >>>
>>> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:91)
>>> >>> > at
>>> >>> >
>>> >>>
>>> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)
>>> >>> > at
>>> >>> >
>>> >>>
>>> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)
>>> >>> > at
>>> >>> >
>>> >>>
>>> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:99)
>>> >>> > at
>>> >>> >
>>> >>>
>>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)
>>> >>> > at
>>> >>> >
>>> >>>
>>> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)
>>> >>> >
>>> >>> > commons-net SocketClient class tries to call the socket factory's
>>> >>> > createSocket() method (with no parameters):
>>> >>> >
>>> >>> > _socket_= _socketFactory_.createSocket();
>>> >>> > _socket_.connect(new InetSocketAddress(hostname, port),
>>> >>> > connectTimeout);
>>> >>> >
>>> >>> > Then intention is to create an "unconnected socket" (hence the
>>> error
>>> >>> > message) and then subsequently connect it. In my case, the
>>> connection
>>> >>> > factory being used is an FTPSSocketFactory
>>> >>> > (package org.apache.commons.net.ftp) that inherits from
>>> >>> > SocketFactory(package javax.net). The SocketFactory class
>>> implements
>>> >>> the
>>> >>> > createSocket() method but throws the exception ("Unconnected
>>> sockets not
>>> >>> > implemented").
>>> >>> >
>>> >>> > The reason why this happens is that after having executed the
>>> >>> > FTPSClient.execPROT() method (package org.apache.commons.net.ftp),
>>> with
>>> >>> any
>>> >>> > other parameter than "C" (in my case I used "P"), the socket
>>> factory is
>>> >>> set
>>> >>> > to an FTPSSocketFactory as follows:
>>> >>> >
>>> >>> > setSocketFactory(new FTPSSocketFactory(context));
>>> >>> >
>>> >>> > After this, no connection attempt will succeed. This problem has
>>> been
>>> >>> > introduced recently when Claus (on my initiative) added support for
>>> >>> secure
>>> >>> > data channel in ftps. I'm thus to blame...
>>> >>> >
>>> >>> > I'm not sure how this should be fixed.
>>> >>> >
>>> >>> > It's a bit strange that all the connect methods
>>> >>> > in org.apache.commons.net.SocketClient (which is the ultimate base
>>> class
>>> >>> of
>>> >>> > FTPSClient) always try to create an unconnected socket first and
>>> then
>>> >>> > connects it. If commons-net uses this pattern, then it should make
>>> sure
>>> >>> that
>>> >>> > all its connection factor's support unconnected sockets but the
>>> >>> > FTPSSocketFactory dosn't. This sounds like a bug in commons-net.
>>> >>> >
>>> >>> > Knowing that commons-net doesn't release very frequently I think we
>>> need
>>> >>> a
>>> >>> > workaround for this. I guess it would be possible to subclass the
>>> >>> FTPClient
>>> >>> > class and override all the connect methods to make sure that no
>>> attempt
>>> >>> is
>>> >>> > made to create unconnected sockets. We would then change the
>>> >>> > createFtpClient() method in the FtpsEndpoint class to instantiate
>>> our
>>> >>> own
>>> >>> > subclass instead of an FTPSClient. Doesn't sound like a nice clean
>>> >>> solution
>>> >>> > though.
>>> >>> >
>>> >>> > Any ideas?
>>> >>> >
>>> >>> > /Bengt
>>> >>> >
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Claus Ibsen
>>> >>> Apache Camel Committer
>>> >>>
>>> >>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> >>> Open Source Integration: http://fusesource.com
>>> >>> Blog: http://davsclaus.blogspot.com/
>>> >>> Twitter: http://twitter.com/davsclaus
>>> >>>
>>> >>
>>> >>
>>> >
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>
>>
>
Re: "Unconnected sockets not implemented" exception in camel-ftp when
using ftps
Posted by Bengt Rodehav <be...@rodehav.com>.
JIRA created:
https://issues.apache.org/activemq/browse/CAMEL-2829
<https://issues.apache.org/activemq/browse/CAMEL-2829>/Bengt
2010/6/18 Bengt Rodehav <be...@rodehav.com>
> Claus,
>
> I'm not sure what you mean by:
>
> "I think we should add an option on the endpoint to allow people to
> turn this on/off.
> Some would like to reuse existing connections to avoid the connect ->
> upload -> disconnect cycle when they upload many files etc."
>
> IMO we would not disconnect more often than today. We would just change the
> connect() method in FtpsOperations so that it creates a new instance of
> FTPSClient prior to a connection attempt. I assume that since the connect()
> method is called in the first place, we are not currently connected and
> there is no way to reuse an existing connection. Am I right?
>
> I have made some modifications in camel-ftp along these lines. I will
> create a JIRA ticket for this bug and attach my changes as diff files to the
> issue. I just tested my changes and they do solve the problem I have
> encountered but I haven't tested whether I have introduced any new problems
> or not.
>
> Anyway, this is what I did:
>
> *FtpsEndpoint*
> I added the following members to keep track of the configuration of the
> FTPSClient:
>
> private boolean isNeedClientAuth = false;
> private KeyManager keyManager;
> private TrustManager trustManager;
>
> I modified the createFtpClient() method so that it doesn't actually create
> a FTPSClient but only calculates and stores the configuration in the above
> members. I also renamed it to createFtpsConfiguration() to reflect that.
>
> The createRemoteFileOperations() method was also changed accordingly. It no
> longer interacts directly with an FTPSClient (since it is not created yet).
>
> *FtpsOperations*
> I added the private createFtpsClient() method that takes care of creating
> an FTPSClient from configuration stored in the FtpsEndpoint. The
> createFtpsClient() is called in the connect() method just before the call to
> the "real" connect() method in the superclass (FtpOperations).
>
> I changed the constructor so that it no longer takes a client as an
> argument. I send null as client to the super class constructor. This is of
> course a major change in behavior that you need to take a look at it.
> Neither FtpsOperations nor its super class FtpOperations can no longer
> assume that a client exists until the first connection attempt has been
> made.
>
> *FtpOperations*
> I had to make the "client" member non-final since I now set it on every
> call to the connect() method in FtpsOperations.
>
>
> BTW, I agree with you that the SFTPClient in commons-net is flawed. It
> should provide a way to re-initialize itself so that a new connect() could
> be made. The FTPClient seems to work that way and it also provides a
> disconnect() method. If they do not intend to support this, then they should
> at least state, as part of the documentation, that the SFTPClient cannot be
> reused across multiple connections. I've posted about this on their mailing
> list but hasn't got any reply yet. Meanwhile the above can be regarded as a
> (complicated...) work-around.
>
> /Bengt
>
>
> 2010/6/18 Claus Ibsen <cl...@gmail.com>
>
> On Thu, Jun 17, 2010 at 11:56 PM, Bengt Rodehav <be...@rodehav.com> wrote:
>> > Claus,
>> >
>> > Having looked a bit more at the commons-net code it's becoming clear to
>> me
>> > that you probably can't just reconnect an FTPSClient once it has entered
>> > secure communication mode. I can't find a way to re-initialise an
>> > FTPSClient. I tried the disconnect() method (in the super class
>> FTPClient)
>> > but it didn't get the FTPSClient out of secure communication mode.
>> >
>> > To be safe I think we should always start with a newly instantiated
>> > FTPSClient when we try to connect. Today the FTPSClient is created by
>> the
>> > FtpsEndpoint. I don't think it has to. It could just get all the righ
>> > configuration parameters and pass it to FtpsOperations which would then
>> > instantiate a new FTPSClient at every connect attempt.
>> >
>> > If we do it this way for ftps I guess it would make sense with the same
>> > setup for plain ftp.
>> >
>> > What do you think?
>> >
>>
>> I think we should add an option on the endpoint to allow people to
>> turn this on/off.
>> Some would like to reuse existing connections to avoid the connect ->
>> upload -> disconnect cycle when they upload many files etc.
>>
>>
>>
>>
>> And frankly I can't see the problem why a secure connection cannot
>> self heal and be able to re-establish itself. After all FTPSClient got
>> all the connection information. And hence I would consider it a
>> missing feature / glitch in the FTP library.
>>
>> Imagine if using secure connections for JMS brokers which couldn't
>> self heal just because a network outage. That would cause a lot of
>> pain for clients having to re-start.
>>
>>
>> > /Bengt
>> >
>> > 2010/6/17 Bengt Rodehav <be...@rodehav.com>
>> >
>> >> Claus,
>> >>
>> >> Yeah, maybe this is a commons-net issue. As always I find it surprising
>> >> that no one else has encountered this problem.
>> >>
>> >> As I started writing an email on the commons user list, I took another
>> look
>> >> at the code in commons-net. It turns out that the FTPSSocketFactory
>> class
>> >> actually does override the createSocket() methods in SocketFactory. A
>> >> similar work-around that I was talking about but on the factory class
>> >> instead of in the SocketClient class. FTPSSocketFactory delegates the
>> >> createSocket() calls to its SSLContect's socket factory.
>> >>
>> >> So why then are we initially able to connect but not after we have
>> called
>> >> execProt() and thus changed the connection factory?
>> >>
>> >> Moving back to the FTPSClient class now. When the initial connection is
>> up,
>> >> the _connectAction() method is called. It in turn calls
>> sslNegotiation()
>> >> which in turn sets up the secure socket. In the sslNegotiation()
>> method, an
>> >> SSLSocketFactory is not instantiated directly. It is given by the
>> >> SSLContext's getSocketFactory() method. This connection factory
>> obviously
>> >> works otherwise the secure connection wouldn't work even initially.
>> >>
>> >> Compare this with what's being done in the execProt() method. Here the
>> >> FTPSSocketFactory is instantiated directly with the SSLContext passed
>> as a
>> >> constructor argument. Obviously this factory does NOT work...
>> >>
>> >> Another question that comes to mind is why the socket factory created
>> in
>> >> the sslNegotiation() method is NOT set as the SocketClient's socket
>> factory
>> >> but the one created in the execProt() method is? It turns out that
>> >> execProt() calls a generic sendCommand() method that sends the command
>> and
>> >> if the reply is OK, then sets the socket factory to null! One wonders
>> why
>> >> (and there is also a comment in the code: "Check this - is this
>> necessary at
>> >> all?"). Anyway, this is probably the reason why execProt() needs to
>> "reset"
>> >> the connection factory.
>> >>
>> >> Still sounds like a commons-net problem though. I'll try to post this
>> at
>> >> their user mailing list.
>> >>
>> >> Just one thought. Do we need to reuse the FTPSClient instance? Can't we
>> >> just create a fresh new instance everytime we need to connect?
>> >>
>> >> /Bengt
>> >>
>> >>
>> >>
>> >> 2010/6/17 Claus Ibsen <cl...@gmail.com>
>> >>
>> >> Hi Bengt
>> >>>
>> >>> I think the issue should be created as a ticket for Apache Commons
>> >>> Net. Then at least the people there can take a look, and maybe they
>> >>> got some ideas. And can fix it in a future release.
>> >>>
>> >>> Bengt fell free to experiment yourself with the subclassing, as it
>> >>> sounds like a good workaround.
>> >>>
>> >>> I assume the FTPSClient doesn't offer other methods to re connect
>> >>> which can recover this error?
>> >>>
>> >>>
>> >>> On Wed, Jun 16, 2010 at 2:44 PM, Bengt Rodehav <be...@rodehav.com>
>> wrote:
>> >>> > I have mentioned this problem in a previous conversation but after
>> >>> > investigating the subject further I decided to start a new thread
>> for
>> >>> this.
>> >>> >
>> >>> > If some kind of problem is encountered while using ftps (I tested
>> this
>> >>> using
>> >>> > Filezilla server by copying a file that already existed which
>> Filezilla
>> >>> does
>> >>> > not allow), then at the next attempt to connect to the ftps server,
>> I
>> >>> get
>> >>> > the following exception in my logfile:
>> >>> >
>> >>> > 13:03:27,606 | ERROR | %7Bfile%3Aext%7D | GenericFileOnCompletion
>> >>> |
>> >>> > rg.apache.camel.processor.Logger 248 | Caused by:
>> >>> > [org.apache.camel.component.file.GenericFileOperationFailedException
>> -
>> >>> File
>> >>> > operation failed: null Unconnected sockets not implemented. Code:
>> 221]
>> >>> > org.apache.camel.component.file.GenericFileOperationFailedException:
>> >>> File
>> >>> > operation failed: null Unconnected sockets not implemented. Code:
>> 221
>> >>> > at
>> >>> >
>> >>>
>> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:108)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> >>> > at
>> >>> >
>> >>>
>> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> >>> > at
>> >>> >
>> >>>
>> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> >>> > at
>> >>> >
>> >>>
>> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:123)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> >>> > at
>> >>> >
>> >>>
>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
>> >>> > at
>> >>> >
>> >>>
>> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> >>> > at
>> >>> >
>> >>>
>> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:91)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
>> >>> >
>> >>> > I added some logging to see the actual stack trace which is:
>> >>> >
>> >>> > java.net.SocketException: Unconnected sockets not implemented
>> >>> > at
>> javax.net.SocketFactory.createSocket(SocketFactory.java:104)
>> >>> > at
>> >>> > org.apache.commons.net.SocketClient.connect(SocketClient.java:175)
>> >>> > at
>> >>> >
>> >>>
>> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:91)
>> >>> > at
>> >>> >
>> >>>
>> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)
>> >>> > at
>> >>> >
>> >>>
>> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)
>> >>> > at
>> >>> >
>> >>>
>> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:99)
>> >>> > at
>> >>> >
>> >>>
>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)
>> >>> > at
>> >>> >
>> >>>
>> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)
>> >>> >
>> >>> > commons-net SocketClient class tries to call the socket factory's
>> >>> > createSocket() method (with no parameters):
>> >>> >
>> >>> > _socket_= _socketFactory_.createSocket();
>> >>> > _socket_.connect(new InetSocketAddress(hostname, port),
>> >>> > connectTimeout);
>> >>> >
>> >>> > Then intention is to create an "unconnected socket" (hence the error
>> >>> > message) and then subsequently connect it. In my case, the
>> connection
>> >>> > factory being used is an FTPSSocketFactory
>> >>> > (package org.apache.commons.net.ftp) that inherits from
>> >>> > SocketFactory(package javax.net). The SocketFactory class
>> implements
>> >>> the
>> >>> > createSocket() method but throws the exception ("Unconnected sockets
>> not
>> >>> > implemented").
>> >>> >
>> >>> > The reason why this happens is that after having executed the
>> >>> > FTPSClient.execPROT() method (package org.apache.commons.net.ftp),
>> with
>> >>> any
>> >>> > other parameter than "C" (in my case I used "P"), the socket factory
>> is
>> >>> set
>> >>> > to an FTPSSocketFactory as follows:
>> >>> >
>> >>> > setSocketFactory(new FTPSSocketFactory(context));
>> >>> >
>> >>> > After this, no connection attempt will succeed. This problem has
>> been
>> >>> > introduced recently when Claus (on my initiative) added support for
>> >>> secure
>> >>> > data channel in ftps. I'm thus to blame...
>> >>> >
>> >>> > I'm not sure how this should be fixed.
>> >>> >
>> >>> > It's a bit strange that all the connect methods
>> >>> > in org.apache.commons.net.SocketClient (which is the ultimate base
>> class
>> >>> of
>> >>> > FTPSClient) always try to create an unconnected socket first and
>> then
>> >>> > connects it. If commons-net uses this pattern, then it should make
>> sure
>> >>> that
>> >>> > all its connection factor's support unconnected sockets but the
>> >>> > FTPSSocketFactory dosn't. This sounds like a bug in commons-net.
>> >>> >
>> >>> > Knowing that commons-net doesn't release very frequently I think we
>> need
>> >>> a
>> >>> > workaround for this. I guess it would be possible to subclass the
>> >>> FTPClient
>> >>> > class and override all the connect methods to make sure that no
>> attempt
>> >>> is
>> >>> > made to create unconnected sockets. We would then change the
>> >>> > createFtpClient() method in the FtpsEndpoint class to instantiate
>> our
>> >>> own
>> >>> > subclass instead of an FTPSClient. Doesn't sound like a nice clean
>> >>> solution
>> >>> > though.
>> >>> >
>> >>> > Any ideas?
>> >>> >
>> >>> > /Bengt
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Claus Ibsen
>> >>> Apache Camel Committer
>> >>>
>> >>> Author of Camel in Action: http://www.manning.com/ibsen/
>> >>> Open Source Integration: http://fusesource.com
>> >>> Blog: http://davsclaus.blogspot.com/
>> >>> Twitter: http://twitter.com/davsclaus
>> >>>
>> >>
>> >>
>> >
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>
Re: "Unconnected sockets not implemented" exception in camel-ftp when
using ftps
Posted by Bengt Rodehav <be...@rodehav.com>.
Claus,
I'm not sure what you mean by:
"I think we should add an option on the endpoint to allow people to
turn this on/off.
Some would like to reuse existing connections to avoid the connect ->
upload -> disconnect cycle when they upload many files etc."
IMO we would not disconnect more often than today. We would just change the
connect() method in FtpsOperations so that it creates a new instance of
FTPSClient prior to a connection attempt. I assume that since the connect()
method is called in the first place, we are not currently connected and
there is no way to reuse an existing connection. Am I right?
I have made some modifications in camel-ftp along these lines. I will create
a JIRA ticket for this bug and attach my changes as diff files to the issue.
I just tested my changes and they do solve the problem I have encountered
but I haven't tested whether I have introduced any new problems or not.
Anyway, this is what I did:
*FtpsEndpoint*
I added the following members to keep track of the configuration of the
FTPSClient:
private boolean isNeedClientAuth = false;
private KeyManager keyManager;
private TrustManager trustManager;
I modified the createFtpClient() method so that it doesn't actually create a
FTPSClient but only calculates and stores the configuration in the above
members. I also renamed it to createFtpsConfiguration() to reflect that.
The createRemoteFileOperations() method was also changed accordingly. It no
longer interacts directly with an FTPSClient (since it is not created yet).
*FtpsOperations*
I added the private createFtpsClient() method that takes care of creating an
FTPSClient from configuration stored in the FtpsEndpoint. The
createFtpsClient() is called in the connect() method just before the call to
the "real" connect() method in the superclass (FtpOperations).
I changed the constructor so that it no longer takes a client as an
argument. I send null as client to the super class constructor. This is of
course a major change in behavior that you need to take a look at it.
Neither FtpsOperations nor its super class FtpOperations can no longer
assume that a client exists until the first connection attempt has been
made.
*FtpOperations*
I had to make the "client" member non-final since I now set it on every call
to the connect() method in FtpsOperations.
BTW, I agree with you that the SFTPClient in commons-net is flawed. It
should provide a way to re-initialize itself so that a new connect() could
be made. The FTPClient seems to work that way and it also provides a
disconnect() method. If they do not intend to support this, then they should
at least state, as part of the documentation, that the SFTPClient cannot be
reused across multiple connections. I've posted about this on their mailing
list but hasn't got any reply yet. Meanwhile the above can be regarded as a
(complicated...) work-around.
/Bengt
2010/6/18 Claus Ibsen <cl...@gmail.com>
> On Thu, Jun 17, 2010 at 11:56 PM, Bengt Rodehav <be...@rodehav.com> wrote:
> > Claus,
> >
> > Having looked a bit more at the commons-net code it's becoming clear to
> me
> > that you probably can't just reconnect an FTPSClient once it has entered
> > secure communication mode. I can't find a way to re-initialise an
> > FTPSClient. I tried the disconnect() method (in the super class
> FTPClient)
> > but it didn't get the FTPSClient out of secure communication mode.
> >
> > To be safe I think we should always start with a newly instantiated
> > FTPSClient when we try to connect. Today the FTPSClient is created by the
> > FtpsEndpoint. I don't think it has to. It could just get all the righ
> > configuration parameters and pass it to FtpsOperations which would then
> > instantiate a new FTPSClient at every connect attempt.
> >
> > If we do it this way for ftps I guess it would make sense with the same
> > setup for plain ftp.
> >
> > What do you think?
> >
>
> I think we should add an option on the endpoint to allow people to
> turn this on/off.
> Some would like to reuse existing connections to avoid the connect ->
> upload -> disconnect cycle when they upload many files etc.
>
>
>
>
> And frankly I can't see the problem why a secure connection cannot
> self heal and be able to re-establish itself. After all FTPSClient got
> all the connection information. And hence I would consider it a
> missing feature / glitch in the FTP library.
>
> Imagine if using secure connections for JMS brokers which couldn't
> self heal just because a network outage. That would cause a lot of
> pain for clients having to re-start.
>
>
> > /Bengt
> >
> > 2010/6/17 Bengt Rodehav <be...@rodehav.com>
> >
> >> Claus,
> >>
> >> Yeah, maybe this is a commons-net issue. As always I find it surprising
> >> that no one else has encountered this problem.
> >>
> >> As I started writing an email on the commons user list, I took another
> look
> >> at the code in commons-net. It turns out that the FTPSSocketFactory
> class
> >> actually does override the createSocket() methods in SocketFactory. A
> >> similar work-around that I was talking about but on the factory class
> >> instead of in the SocketClient class. FTPSSocketFactory delegates the
> >> createSocket() calls to its SSLContect's socket factory.
> >>
> >> So why then are we initially able to connect but not after we have
> called
> >> execProt() and thus changed the connection factory?
> >>
> >> Moving back to the FTPSClient class now. When the initial connection is
> up,
> >> the _connectAction() method is called. It in turn calls sslNegotiation()
> >> which in turn sets up the secure socket. In the sslNegotiation() method,
> an
> >> SSLSocketFactory is not instantiated directly. It is given by the
> >> SSLContext's getSocketFactory() method. This connection factory
> obviously
> >> works otherwise the secure connection wouldn't work even initially.
> >>
> >> Compare this with what's being done in the execProt() method. Here the
> >> FTPSSocketFactory is instantiated directly with the SSLContext passed as
> a
> >> constructor argument. Obviously this factory does NOT work...
> >>
> >> Another question that comes to mind is why the socket factory created in
> >> the sslNegotiation() method is NOT set as the SocketClient's socket
> factory
> >> but the one created in the execProt() method is? It turns out that
> >> execProt() calls a generic sendCommand() method that sends the command
> and
> >> if the reply is OK, then sets the socket factory to null! One wonders
> why
> >> (and there is also a comment in the code: "Check this - is this
> necessary at
> >> all?"). Anyway, this is probably the reason why execProt() needs to
> "reset"
> >> the connection factory.
> >>
> >> Still sounds like a commons-net problem though. I'll try to post this at
> >> their user mailing list.
> >>
> >> Just one thought. Do we need to reuse the FTPSClient instance? Can't we
> >> just create a fresh new instance everytime we need to connect?
> >>
> >> /Bengt
> >>
> >>
> >>
> >> 2010/6/17 Claus Ibsen <cl...@gmail.com>
> >>
> >> Hi Bengt
> >>>
> >>> I think the issue should be created as a ticket for Apache Commons
> >>> Net. Then at least the people there can take a look, and maybe they
> >>> got some ideas. And can fix it in a future release.
> >>>
> >>> Bengt fell free to experiment yourself with the subclassing, as it
> >>> sounds like a good workaround.
> >>>
> >>> I assume the FTPSClient doesn't offer other methods to re connect
> >>> which can recover this error?
> >>>
> >>>
> >>> On Wed, Jun 16, 2010 at 2:44 PM, Bengt Rodehav <be...@rodehav.com>
> wrote:
> >>> > I have mentioned this problem in a previous conversation but after
> >>> > investigating the subject further I decided to start a new thread for
> >>> this.
> >>> >
> >>> > If some kind of problem is encountered while using ftps (I tested
> this
> >>> using
> >>> > Filezilla server by copying a file that already existed which
> Filezilla
> >>> does
> >>> > not allow), then at the next attempt to connect to the ftps server, I
> >>> get
> >>> > the following exception in my logfile:
> >>> >
> >>> > 13:03:27,606 | ERROR | %7Bfile%3Aext%7D | GenericFileOnCompletion
> >>> |
> >>> > rg.apache.camel.processor.Logger 248 | Caused by:
> >>> > [org.apache.camel.component.file.GenericFileOperationFailedException
> -
> >>> File
> >>> > operation failed: null Unconnected sockets not implemented. Code:
> 221]
> >>> > org.apache.camel.component.file.GenericFileOperationFailedException:
> >>> File
> >>> > operation failed: null Unconnected sockets not implemented. Code: 221
> >>> > at
> >>> >
> >>>
> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:108)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >>> > at
> >>> >
> >>>
> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >>> > at
> >>> >
> >>>
> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >>> > at
> >>> >
> >>>
> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:123)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >>> > at
> >>> >
> >>>
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
> >>> > at
> >>> >
> >>>
> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >>> > at
> >>> >
> >>>
> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:91)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
> >>> >
> >>> > I added some logging to see the actual stack trace which is:
> >>> >
> >>> > java.net.SocketException: Unconnected sockets not implemented
> >>> > at
> javax.net.SocketFactory.createSocket(SocketFactory.java:104)
> >>> > at
> >>> > org.apache.commons.net.SocketClient.connect(SocketClient.java:175)
> >>> > at
> >>> >
> >>>
> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:91)
> >>> > at
> >>> >
> >>>
> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)
> >>> > at
> >>> >
> >>>
> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)
> >>> > at
> >>> >
> >>>
> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:99)
> >>> > at
> >>> >
> >>>
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)
> >>> > at
> >>> >
> >>>
> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)
> >>> >
> >>> > commons-net SocketClient class tries to call the socket factory's
> >>> > createSocket() method (with no parameters):
> >>> >
> >>> > _socket_= _socketFactory_.createSocket();
> >>> > _socket_.connect(new InetSocketAddress(hostname, port),
> >>> > connectTimeout);
> >>> >
> >>> > Then intention is to create an "unconnected socket" (hence the error
> >>> > message) and then subsequently connect it. In my case, the connection
> >>> > factory being used is an FTPSSocketFactory
> >>> > (package org.apache.commons.net.ftp) that inherits from
> >>> > SocketFactory(package javax.net). The SocketFactory class implements
> >>> the
> >>> > createSocket() method but throws the exception ("Unconnected sockets
> not
> >>> > implemented").
> >>> >
> >>> > The reason why this happens is that after having executed the
> >>> > FTPSClient.execPROT() method (package org.apache.commons.net.ftp),
> with
> >>> any
> >>> > other parameter than "C" (in my case I used "P"), the socket factory
> is
> >>> set
> >>> > to an FTPSSocketFactory as follows:
> >>> >
> >>> > setSocketFactory(new FTPSSocketFactory(context));
> >>> >
> >>> > After this, no connection attempt will succeed. This problem has been
> >>> > introduced recently when Claus (on my initiative) added support for
> >>> secure
> >>> > data channel in ftps. I'm thus to blame...
> >>> >
> >>> > I'm not sure how this should be fixed.
> >>> >
> >>> > It's a bit strange that all the connect methods
> >>> > in org.apache.commons.net.SocketClient (which is the ultimate base
> class
> >>> of
> >>> > FTPSClient) always try to create an unconnected socket first and then
> >>> > connects it. If commons-net uses this pattern, then it should make
> sure
> >>> that
> >>> > all its connection factor's support unconnected sockets but the
> >>> > FTPSSocketFactory dosn't. This sounds like a bug in commons-net.
> >>> >
> >>> > Knowing that commons-net doesn't release very frequently I think we
> need
> >>> a
> >>> > workaround for this. I guess it would be possible to subclass the
> >>> FTPClient
> >>> > class and override all the connect methods to make sure that no
> attempt
> >>> is
> >>> > made to create unconnected sockets. We would then change the
> >>> > createFtpClient() method in the FtpsEndpoint class to instantiate our
> >>> own
> >>> > subclass instead of an FTPSClient. Doesn't sound like a nice clean
> >>> solution
> >>> > though.
> >>> >
> >>> > Any ideas?
> >>> >
> >>> > /Bengt
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> Claus Ibsen
> >>> Apache Camel Committer
> >>>
> >>> Author of Camel in Action: http://www.manning.com/ibsen/
> >>> Open Source Integration: http://fusesource.com
> >>> Blog: http://davsclaus.blogspot.com/
> >>> Twitter: http://twitter.com/davsclaus
> >>>
> >>
> >>
> >
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
Re: "Unconnected sockets not implemented" exception in camel-ftp when
using ftps
Posted by Claus Ibsen <cl...@gmail.com>.
On Thu, Jun 17, 2010 at 11:56 PM, Bengt Rodehav <be...@rodehav.com> wrote:
> Claus,
>
> Having looked a bit more at the commons-net code it's becoming clear to me
> that you probably can't just reconnect an FTPSClient once it has entered
> secure communication mode. I can't find a way to re-initialise an
> FTPSClient. I tried the disconnect() method (in the super class FTPClient)
> but it didn't get the FTPSClient out of secure communication mode.
>
> To be safe I think we should always start with a newly instantiated
> FTPSClient when we try to connect. Today the FTPSClient is created by the
> FtpsEndpoint. I don't think it has to. It could just get all the righ
> configuration parameters and pass it to FtpsOperations which would then
> instantiate a new FTPSClient at every connect attempt.
>
> If we do it this way for ftps I guess it would make sense with the same
> setup for plain ftp.
>
> What do you think?
>
I think we should add an option on the endpoint to allow people to
turn this on/off.
Some would like to reuse existing connections to avoid the connect ->
upload -> disconnect cycle when they upload many files etc.
And frankly I can't see the problem why a secure connection cannot
self heal and be able to re-establish itself. After all FTPSClient got
all the connection information. And hence I would consider it a
missing feature / glitch in the FTP library.
Imagine if using secure connections for JMS brokers which couldn't
self heal just because a network outage. That would cause a lot of
pain for clients having to re-start.
> /Bengt
>
> 2010/6/17 Bengt Rodehav <be...@rodehav.com>
>
>> Claus,
>>
>> Yeah, maybe this is a commons-net issue. As always I find it surprising
>> that no one else has encountered this problem.
>>
>> As I started writing an email on the commons user list, I took another look
>> at the code in commons-net. It turns out that the FTPSSocketFactory class
>> actually does override the createSocket() methods in SocketFactory. A
>> similar work-around that I was talking about but on the factory class
>> instead of in the SocketClient class. FTPSSocketFactory delegates the
>> createSocket() calls to its SSLContect's socket factory.
>>
>> So why then are we initially able to connect but not after we have called
>> execProt() and thus changed the connection factory?
>>
>> Moving back to the FTPSClient class now. When the initial connection is up,
>> the _connectAction() method is called. It in turn calls sslNegotiation()
>> which in turn sets up the secure socket. In the sslNegotiation() method, an
>> SSLSocketFactory is not instantiated directly. It is given by the
>> SSLContext's getSocketFactory() method. This connection factory obviously
>> works otherwise the secure connection wouldn't work even initially.
>>
>> Compare this with what's being done in the execProt() method. Here the
>> FTPSSocketFactory is instantiated directly with the SSLContext passed as a
>> constructor argument. Obviously this factory does NOT work...
>>
>> Another question that comes to mind is why the socket factory created in
>> the sslNegotiation() method is NOT set as the SocketClient's socket factory
>> but the one created in the execProt() method is? It turns out that
>> execProt() calls a generic sendCommand() method that sends the command and
>> if the reply is OK, then sets the socket factory to null! One wonders why
>> (and there is also a comment in the code: "Check this - is this necessary at
>> all?"). Anyway, this is probably the reason why execProt() needs to "reset"
>> the connection factory.
>>
>> Still sounds like a commons-net problem though. I'll try to post this at
>> their user mailing list.
>>
>> Just one thought. Do we need to reuse the FTPSClient instance? Can't we
>> just create a fresh new instance everytime we need to connect?
>>
>> /Bengt
>>
>>
>>
>> 2010/6/17 Claus Ibsen <cl...@gmail.com>
>>
>> Hi Bengt
>>>
>>> I think the issue should be created as a ticket for Apache Commons
>>> Net. Then at least the people there can take a look, and maybe they
>>> got some ideas. And can fix it in a future release.
>>>
>>> Bengt fell free to experiment yourself with the subclassing, as it
>>> sounds like a good workaround.
>>>
>>> I assume the FTPSClient doesn't offer other methods to re connect
>>> which can recover this error?
>>>
>>>
>>> On Wed, Jun 16, 2010 at 2:44 PM, Bengt Rodehav <be...@rodehav.com> wrote:
>>> > I have mentioned this problem in a previous conversation but after
>>> > investigating the subject further I decided to start a new thread for
>>> this.
>>> >
>>> > If some kind of problem is encountered while using ftps (I tested this
>>> using
>>> > Filezilla server by copying a file that already existed which Filezilla
>>> does
>>> > not allow), then at the next attempt to connect to the ftps server, I
>>> get
>>> > the following exception in my logfile:
>>> >
>>> > 13:03:27,606 | ERROR | %7Bfile%3Aext%7D | GenericFileOnCompletion
>>> |
>>> > rg.apache.camel.processor.Logger 248 | Caused by:
>>> > [org.apache.camel.component.file.GenericFileOperationFailedException -
>>> File
>>> > operation failed: null Unconnected sockets not implemented. Code: 221]
>>> > org.apache.camel.component.file.GenericFileOperationFailedException:
>>> File
>>> > operation failed: null Unconnected sockets not implemented. Code: 221
>>> > at
>>> >
>>> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:108)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>> > at
>>> >
>>> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>> > at
>>> >
>>> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>> > at
>>> >
>>> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:123)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>> > at
>>> >
>>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
>>> > at
>>> >
>>> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>>> > at
>>> >
>>> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:91)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
>>> >
>>> > I added some logging to see the actual stack trace which is:
>>> >
>>> > java.net.SocketException: Unconnected sockets not implemented
>>> > at javax.net.SocketFactory.createSocket(SocketFactory.java:104)
>>> > at
>>> > org.apache.commons.net.SocketClient.connect(SocketClient.java:175)
>>> > at
>>> >
>>> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:91)
>>> > at
>>> >
>>> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)
>>> > at
>>> >
>>> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)
>>> > at
>>> >
>>> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:99)
>>> > at
>>> >
>>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)
>>> > at
>>> >
>>> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)
>>> >
>>> > commons-net SocketClient class tries to call the socket factory's
>>> > createSocket() method (with no parameters):
>>> >
>>> > _socket_= _socketFactory_.createSocket();
>>> > _socket_.connect(new InetSocketAddress(hostname, port),
>>> > connectTimeout);
>>> >
>>> > Then intention is to create an "unconnected socket" (hence the error
>>> > message) and then subsequently connect it. In my case, the connection
>>> > factory being used is an FTPSSocketFactory
>>> > (package org.apache.commons.net.ftp) that inherits from
>>> > SocketFactory(package javax.net). The SocketFactory class implements
>>> the
>>> > createSocket() method but throws the exception ("Unconnected sockets not
>>> > implemented").
>>> >
>>> > The reason why this happens is that after having executed the
>>> > FTPSClient.execPROT() method (package org.apache.commons.net.ftp), with
>>> any
>>> > other parameter than "C" (in my case I used "P"), the socket factory is
>>> set
>>> > to an FTPSSocketFactory as follows:
>>> >
>>> > setSocketFactory(new FTPSSocketFactory(context));
>>> >
>>> > After this, no connection attempt will succeed. This problem has been
>>> > introduced recently when Claus (on my initiative) added support for
>>> secure
>>> > data channel in ftps. I'm thus to blame...
>>> >
>>> > I'm not sure how this should be fixed.
>>> >
>>> > It's a bit strange that all the connect methods
>>> > in org.apache.commons.net.SocketClient (which is the ultimate base class
>>> of
>>> > FTPSClient) always try to create an unconnected socket first and then
>>> > connects it. If commons-net uses this pattern, then it should make sure
>>> that
>>> > all its connection factor's support unconnected sockets but the
>>> > FTPSSocketFactory dosn't. This sounds like a bug in commons-net.
>>> >
>>> > Knowing that commons-net doesn't release very frequently I think we need
>>> a
>>> > workaround for this. I guess it would be possible to subclass the
>>> FTPClient
>>> > class and override all the connect methods to make sure that no attempt
>>> is
>>> > made to create unconnected sockets. We would then change the
>>> > createFtpClient() method in the FtpsEndpoint class to instantiate our
>>> own
>>> > subclass instead of an FTPSClient. Doesn't sound like a nice clean
>>> solution
>>> > though.
>>> >
>>> > Any ideas?
>>> >
>>> > /Bengt
>>> >
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>
>>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: "Unconnected sockets not implemented" exception in camel-ftp when
using ftps
Posted by Bengt Rodehav <be...@rodehav.com>.
Claus,
Having looked a bit more at the commons-net code it's becoming clear to me
that you probably can't just reconnect an FTPSClient once it has entered
secure communication mode. I can't find a way to re-initialise an
FTPSClient. I tried the disconnect() method (in the super class FTPClient)
but it didn't get the FTPSClient out of secure communication mode.
To be safe I think we should always start with a newly instantiated
FTPSClient when we try to connect. Today the FTPSClient is created by the
FtpsEndpoint. I don't think it has to. It could just get all the righ
configuration parameters and pass it to FtpsOperations which would then
instantiate a new FTPSClient at every connect attempt.
If we do it this way for ftps I guess it would make sense with the same
setup for plain ftp.
What do you think?
/Bengt
2010/6/17 Bengt Rodehav <be...@rodehav.com>
> Claus,
>
> Yeah, maybe this is a commons-net issue. As always I find it surprising
> that no one else has encountered this problem.
>
> As I started writing an email on the commons user list, I took another look
> at the code in commons-net. It turns out that the FTPSSocketFactory class
> actually does override the createSocket() methods in SocketFactory. A
> similar work-around that I was talking about but on the factory class
> instead of in the SocketClient class. FTPSSocketFactory delegates the
> createSocket() calls to its SSLContect's socket factory.
>
> So why then are we initially able to connect but not after we have called
> execProt() and thus changed the connection factory?
>
> Moving back to the FTPSClient class now. When the initial connection is up,
> the _connectAction() method is called. It in turn calls sslNegotiation()
> which in turn sets up the secure socket. In the sslNegotiation() method, an
> SSLSocketFactory is not instantiated directly. It is given by the
> SSLContext's getSocketFactory() method. This connection factory obviously
> works otherwise the secure connection wouldn't work even initially.
>
> Compare this with what's being done in the execProt() method. Here the
> FTPSSocketFactory is instantiated directly with the SSLContext passed as a
> constructor argument. Obviously this factory does NOT work...
>
> Another question that comes to mind is why the socket factory created in
> the sslNegotiation() method is NOT set as the SocketClient's socket factory
> but the one created in the execProt() method is? It turns out that
> execProt() calls a generic sendCommand() method that sends the command and
> if the reply is OK, then sets the socket factory to null! One wonders why
> (and there is also a comment in the code: "Check this - is this necessary at
> all?"). Anyway, this is probably the reason why execProt() needs to "reset"
> the connection factory.
>
> Still sounds like a commons-net problem though. I'll try to post this at
> their user mailing list.
>
> Just one thought. Do we need to reuse the FTPSClient instance? Can't we
> just create a fresh new instance everytime we need to connect?
>
> /Bengt
>
>
>
> 2010/6/17 Claus Ibsen <cl...@gmail.com>
>
> Hi Bengt
>>
>> I think the issue should be created as a ticket for Apache Commons
>> Net. Then at least the people there can take a look, and maybe they
>> got some ideas. And can fix it in a future release.
>>
>> Bengt fell free to experiment yourself with the subclassing, as it
>> sounds like a good workaround.
>>
>> I assume the FTPSClient doesn't offer other methods to re connect
>> which can recover this error?
>>
>>
>> On Wed, Jun 16, 2010 at 2:44 PM, Bengt Rodehav <be...@rodehav.com> wrote:
>> > I have mentioned this problem in a previous conversation but after
>> > investigating the subject further I decided to start a new thread for
>> this.
>> >
>> > If some kind of problem is encountered while using ftps (I tested this
>> using
>> > Filezilla server by copying a file that already existed which Filezilla
>> does
>> > not allow), then at the next attempt to connect to the ftps server, I
>> get
>> > the following exception in my logfile:
>> >
>> > 13:03:27,606 | ERROR | %7Bfile%3Aext%7D | GenericFileOnCompletion
>> |
>> > rg.apache.camel.processor.Logger 248 | Caused by:
>> > [org.apache.camel.component.file.GenericFileOperationFailedException -
>> File
>> > operation failed: null Unconnected sockets not implemented. Code: 221]
>> > org.apache.camel.component.file.GenericFileOperationFailedException:
>> File
>> > operation failed: null Unconnected sockets not implemented. Code: 221
>> > at
>> >
>> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:108)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> > at
>> >
>> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> > at
>> >
>> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> > at
>> >
>> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:123)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> > at
>> >
>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
>> > at
>> >
>> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
>> > at
>> >
>> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:91)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
>> >
>> > I added some logging to see the actual stack trace which is:
>> >
>> > java.net.SocketException: Unconnected sockets not implemented
>> > at javax.net.SocketFactory.createSocket(SocketFactory.java:104)
>> > at
>> > org.apache.commons.net.SocketClient.connect(SocketClient.java:175)
>> > at
>> >
>> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:91)
>> > at
>> >
>> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)
>> > at
>> >
>> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)
>> > at
>> >
>> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:99)
>> > at
>> >
>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)
>> > at
>> >
>> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)
>> >
>> > commons-net SocketClient class tries to call the socket factory's
>> > createSocket() method (with no parameters):
>> >
>> > _socket_= _socketFactory_.createSocket();
>> > _socket_.connect(new InetSocketAddress(hostname, port),
>> > connectTimeout);
>> >
>> > Then intention is to create an "unconnected socket" (hence the error
>> > message) and then subsequently connect it. In my case, the connection
>> > factory being used is an FTPSSocketFactory
>> > (package org.apache.commons.net.ftp) that inherits from
>> > SocketFactory(package javax.net). The SocketFactory class implements
>> the
>> > createSocket() method but throws the exception ("Unconnected sockets not
>> > implemented").
>> >
>> > The reason why this happens is that after having executed the
>> > FTPSClient.execPROT() method (package org.apache.commons.net.ftp), with
>> any
>> > other parameter than "C" (in my case I used "P"), the socket factory is
>> set
>> > to an FTPSSocketFactory as follows:
>> >
>> > setSocketFactory(new FTPSSocketFactory(context));
>> >
>> > After this, no connection attempt will succeed. This problem has been
>> > introduced recently when Claus (on my initiative) added support for
>> secure
>> > data channel in ftps. I'm thus to blame...
>> >
>> > I'm not sure how this should be fixed.
>> >
>> > It's a bit strange that all the connect methods
>> > in org.apache.commons.net.SocketClient (which is the ultimate base class
>> of
>> > FTPSClient) always try to create an unconnected socket first and then
>> > connects it. If commons-net uses this pattern, then it should make sure
>> that
>> > all its connection factor's support unconnected sockets but the
>> > FTPSSocketFactory dosn't. This sounds like a bug in commons-net.
>> >
>> > Knowing that commons-net doesn't release very frequently I think we need
>> a
>> > workaround for this. I guess it would be possible to subclass the
>> FTPClient
>> > class and override all the connect methods to make sure that no attempt
>> is
>> > made to create unconnected sockets. We would then change the
>> > createFtpClient() method in the FtpsEndpoint class to instantiate our
>> own
>> > subclass instead of an FTPSClient. Doesn't sound like a nice clean
>> solution
>> > though.
>> >
>> > Any ideas?
>> >
>> > /Bengt
>> >
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>
Re: "Unconnected sockets not implemented" exception in camel-ftp when
using ftps
Posted by Bengt Rodehav <be...@rodehav.com>.
Claus,
Yeah, maybe this is a commons-net issue. As always I find it surprising that
no one else has encountered this problem.
As I started writing an email on the commons user list, I took another look
at the code in commons-net. It turns out that the FTPSSocketFactory class
actually does override the createSocket() methods in SocketFactory. A
similar work-around that I was talking about but on the factory class
instead of in the SocketClient class. FTPSSocketFactory delegates the
createSocket() calls to its SSLContect's socket factory.
So why then are we initially able to connect but not after we have called
execProt() and thus changed the connection factory?
Moving back to the FTPSClient class now. When the initial connection is up,
the _connectAction() method is called. It in turn calls sslNegotiation()
which in turn sets up the secure socket. In the sslNegotiation() method, an
SSLSocketFactory is not instantiated directly. It is given by the
SSLContext's getSocketFactory() method. This connection factory obviously
works otherwise the secure connection wouldn't work even initially.
Compare this with what's being done in the execProt() method. Here the
FTPSSocketFactory is instantiated directly with the SSLContext passed as a
constructor argument. Obviously this factory does NOT work...
Another question that comes to mind is why the socket factory created in the
sslNegotiation() method is NOT set as the SocketClient's socket factory but
the one created in the execProt() method is? It turns out that execProt()
calls a generic sendCommand() method that sends the command and if the reply
is OK, then sets the socket factory to null! One wonders why (and there is
also a comment in the code: "Check this - is this necessary at all?").
Anyway, this is probably the reason why execProt() needs to "reset" the
connection factory.
Still sounds like a commons-net problem though. I'll try to post this at
their user mailing list.
Just one thought. Do we need to reuse the FTPSClient instance? Can't we just
create a fresh new instance everytime we need to connect?
/Bengt
2010/6/17 Claus Ibsen <cl...@gmail.com>
> Hi Bengt
>
> I think the issue should be created as a ticket for Apache Commons
> Net. Then at least the people there can take a look, and maybe they
> got some ideas. And can fix it in a future release.
>
> Bengt fell free to experiment yourself with the subclassing, as it
> sounds like a good workaround.
>
> I assume the FTPSClient doesn't offer other methods to re connect
> which can recover this error?
>
>
> On Wed, Jun 16, 2010 at 2:44 PM, Bengt Rodehav <be...@rodehav.com> wrote:
> > I have mentioned this problem in a previous conversation but after
> > investigating the subject further I decided to start a new thread for
> this.
> >
> > If some kind of problem is encountered while using ftps (I tested this
> using
> > Filezilla server by copying a file that already existed which Filezilla
> does
> > not allow), then at the next attempt to connect to the ftps server, I get
> > the following exception in my logfile:
> >
> > 13:03:27,606 | ERROR | %7Bfile%3Aext%7D | GenericFileOnCompletion
> |
> > rg.apache.camel.processor.Logger 248 | Caused by:
> > [org.apache.camel.component.file.GenericFileOperationFailedException -
> File
> > operation failed: null Unconnected sockets not implemented. Code: 221]
> > org.apache.camel.component.file.GenericFileOperationFailedException: File
> > operation failed: null Unconnected sockets not implemented. Code: 221
> > at
> >
> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:108)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> > at
> >
> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> > at
> >
> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> > at
> >
> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:123)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> > at
> >
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
> > at
> >
> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> > at
> >
> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:91)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
> >
> > I added some logging to see the actual stack trace which is:
> >
> > java.net.SocketException: Unconnected sockets not implemented
> > at javax.net.SocketFactory.createSocket(SocketFactory.java:104)
> > at
> > org.apache.commons.net.SocketClient.connect(SocketClient.java:175)
> > at
> >
> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:91)
> > at
> >
> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)
> > at
> >
> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)
> > at
> >
> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:99)
> > at
> >
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)
> > at
> >
> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)
> >
> > commons-net SocketClient class tries to call the socket factory's
> > createSocket() method (with no parameters):
> >
> > _socket_= _socketFactory_.createSocket();
> > _socket_.connect(new InetSocketAddress(hostname, port),
> > connectTimeout);
> >
> > Then intention is to create an "unconnected socket" (hence the error
> > message) and then subsequently connect it. In my case, the connection
> > factory being used is an FTPSSocketFactory
> > (package org.apache.commons.net.ftp) that inherits from
> > SocketFactory(package javax.net). The SocketFactory class implements the
> > createSocket() method but throws the exception ("Unconnected sockets not
> > implemented").
> >
> > The reason why this happens is that after having executed the
> > FTPSClient.execPROT() method (package org.apache.commons.net.ftp), with
> any
> > other parameter than "C" (in my case I used "P"), the socket factory is
> set
> > to an FTPSSocketFactory as follows:
> >
> > setSocketFactory(new FTPSSocketFactory(context));
> >
> > After this, no connection attempt will succeed. This problem has been
> > introduced recently when Claus (on my initiative) added support for
> secure
> > data channel in ftps. I'm thus to blame...
> >
> > I'm not sure how this should be fixed.
> >
> > It's a bit strange that all the connect methods
> > in org.apache.commons.net.SocketClient (which is the ultimate base class
> of
> > FTPSClient) always try to create an unconnected socket first and then
> > connects it. If commons-net uses this pattern, then it should make sure
> that
> > all its connection factor's support unconnected sockets but the
> > FTPSSocketFactory dosn't. This sounds like a bug in commons-net.
> >
> > Knowing that commons-net doesn't release very frequently I think we need
> a
> > workaround for this. I guess it would be possible to subclass the
> FTPClient
> > class and override all the connect methods to make sure that no attempt
> is
> > made to create unconnected sockets. We would then change the
> > createFtpClient() method in the FtpsEndpoint class to instantiate our own
> > subclass instead of an FTPSClient. Doesn't sound like a nice clean
> solution
> > though.
> >
> > Any ideas?
> >
> > /Bengt
> >
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
Re: "Unconnected sockets not implemented" exception in camel-ftp when
using ftps
Posted by Claus Ibsen <cl...@gmail.com>.
Hi Bengt
I think the issue should be created as a ticket for Apache Commons
Net. Then at least the people there can take a look, and maybe they
got some ideas. And can fix it in a future release.
Bengt fell free to experiment yourself with the subclassing, as it
sounds like a good workaround.
I assume the FTPSClient doesn't offer other methods to re connect
which can recover this error?
On Wed, Jun 16, 2010 at 2:44 PM, Bengt Rodehav <be...@rodehav.com> wrote:
> I have mentioned this problem in a previous conversation but after
> investigating the subject further I decided to start a new thread for this.
>
> If some kind of problem is encountered while using ftps (I tested this using
> Filezilla server by copying a file that already existed which Filezilla does
> not allow), then at the next attempt to connect to the ftps server, I get
> the following exception in my logfile:
>
> 13:03:27,606 | ERROR | %7Bfile%3Aext%7D | GenericFileOnCompletion |
> rg.apache.camel.processor.Logger 248 | Caused by:
> [org.apache.camel.component.file.GenericFileOperationFailedException - File
> operation failed: null Unconnected sockets not implemented. Code: 221]
> org.apache.camel.component.file.GenericFileOperationFailedException: File
> operation failed: null Unconnected sockets not implemented. Code: 221
> at
> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:108)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> at
> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> at
> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> at
> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:123)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> at
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
> at
> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> at
> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:91)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
>
> I added some logging to see the actual stack trace which is:
>
> java.net.SocketException: Unconnected sockets not implemented
> at javax.net.SocketFactory.createSocket(SocketFactory.java:104)
> at
> org.apache.commons.net.SocketClient.connect(SocketClient.java:175)
> at
> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:91)
> at
> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)
> at
> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)
> at
> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:99)
> at
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)
> at
> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)
>
> commons-net SocketClient class tries to call the socket factory's
> createSocket() method (with no parameters):
>
> _socket_= _socketFactory_.createSocket();
> _socket_.connect(new InetSocketAddress(hostname, port),
> connectTimeout);
>
> Then intention is to create an "unconnected socket" (hence the error
> message) and then subsequently connect it. In my case, the connection
> factory being used is an FTPSSocketFactory
> (package org.apache.commons.net.ftp) that inherits from
> SocketFactory(package javax.net). The SocketFactory class implements the
> createSocket() method but throws the exception ("Unconnected sockets not
> implemented").
>
> The reason why this happens is that after having executed the
> FTPSClient.execPROT() method (package org.apache.commons.net.ftp), with any
> other parameter than "C" (in my case I used "P"), the socket factory is set
> to an FTPSSocketFactory as follows:
>
> setSocketFactory(new FTPSSocketFactory(context));
>
> After this, no connection attempt will succeed. This problem has been
> introduced recently when Claus (on my initiative) added support for secure
> data channel in ftps. I'm thus to blame...
>
> I'm not sure how this should be fixed.
>
> It's a bit strange that all the connect methods
> in org.apache.commons.net.SocketClient (which is the ultimate base class of
> FTPSClient) always try to create an unconnected socket first and then
> connects it. If commons-net uses this pattern, then it should make sure that
> all its connection factor's support unconnected sockets but the
> FTPSSocketFactory dosn't. This sounds like a bug in commons-net.
>
> Knowing that commons-net doesn't release very frequently I think we need a
> workaround for this. I guess it would be possible to subclass the FTPClient
> class and override all the connect methods to make sure that no attempt is
> made to create unconnected sockets. We would then change the
> createFtpClient() method in the FtpsEndpoint class to instantiate our own
> subclass instead of an FTPSClient. Doesn't sound like a nice clean solution
> though.
>
> Any ideas?
>
> /Bengt
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus