You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ftpserver-users@mina.apache.org by Frank van der Kleij <kl...@hotmail.com> on 2020/12/21 11:00:41 UTC
Re: Problems with sftp and vfs
Coming back to this, when we investigated this a few weeks ago we found that this was caused by the sftp server implementation getting stuck over a command that was trying to get some information about the files (id -G), clearly not supported by that implementation.
The workaround we found is to use a custom file object factory that can be used to tune the delegation of operations from the FTP file to the VFS file.
I provided one implementation (ShallowReadOnlyVfsFtpFileFactory) that forces read-only behavior and that does not delegate methods that are not relevant in that context - which worked for Stefan.
I already delivered the new binaries a while ago but I finally updated the documentation on the website (http://vfs-utils.sourceforge.net/ftpserver/configuration.html).
Cheers,
Frank
VFS FTPServer - VFS Utils FTPServer Configuration<http://vfs-utils.sourceforge.net/ftpserver/configuration.html>
VFS Utils FTPServer Configuration. To use this bridge you should just add the VFS libraries, the VFS Utils FTPServer library, the VFS Utils Utils library and, if needed, your custom VFS Provider and its dependencies to the common/lib directory of your Apache Mina FTP Server installation.
vfs-utils.sourceforge.net
________________________________
From: Frank van der Kleij <kl...@hotmail.com>
Sent: Wednesday, November 4, 2020 10:31 PM
To: ftpserver-users@mina.apache.org <ft...@mina.apache.org>
Subject: Re: Problems with sftp and vfs
Hi Stefan,
Maybe we can take the discussion privately until we find a solution for your problem?
For me it worked with small file that I downloaded from a local sftp server through an ftp server with vfs. Maybe for bigger files it behaves differently. It might not be very easy to reproduce for me since I don't have exactly the same environment as you, and it could be the problem is in the sftp provider of the VFS library, but I'll give it another shot.
SFTP does not make the difference between active and passive modes, that exists only in the ftp/ftps protocols.
Best regards,
Frank
________________________________
From: Stefan Ziegler <st...@gmail.com>
Sent: Wednesday, November 4, 2020 8:22 PM
To: ftpserver-users@mina.apache.org <ft...@mina.apache.org>
Subject: Re: Problems with sftp and vfs
When I use the ftp command in the terminal "ftp localhost 2221" I'm able to
login but when try to list with "dir" it hangs:
ftp> dir
200 Command PORT okay.
150 File status okay; about to open data connection.
Logfile:
2020-11-04 20:18:38.130 INFO 18691 --- [pool-3-thread-1]
o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: LIST
2020-11-04 20:18:38.154 DEBUG 18691 --- [pool-3-thread-1]
o.a.f.impl.IODataConnectionFactory : Opening active data connection
2020-11-04 20:18:38.155 DEBUG 18691 --- [pool-3-thread-1]
o.a.f.impl.IODataConnectionFactory : Binding active data connection
to /127.0.0.1:0
2020-11-04 20:18:39.803 DEBUG 18691 --- [pool-3-thread-1]
o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u000000:***@
u000000.your-storagebox.de/ch.so.agi.av.dm01avch24lv95d
2020-11-04 20:18:39.806 DEBUG 18691 --- [pool-3-thread-1]
o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u000000:***@
u000000.your-storagebox.de/ch.so.agi.geodata
2020-11-04 20:18:39.807 DEBUG 18691 --- [pool-3-thread-1]
o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u000000:***@
u246727.your-storagebox.de/ch.so.agi.av.dm01avch24lv95d
2020-11-04 20:18:39.807 DEBUG 18691 --- [pool-3-thread-1]
o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u000000:***@
u000000.your-storagebox.de/ch.so.agi.geodata
It this a acitve/passive issue? Can I set the VFS to passive?
Stefan
On Tue, Nov 3, 2020 at 9:33 PM Frank van der Kleij <kl...@hotmail.com>
wrote:
> Hi Stefan,
>
> The authenticate method is always called and the user is passed to the
> VFS. Then depending on how the VFS is initialised it will use some
> information from the user; in your case it should just take the home
> directory from the user.
>
> For me it is working - though I didn't use your code example, but I had to
> add the jsch and jcifs libraries to the classpath of the ftp server because
> they are needed for sftp (that is using ssh).
>
>
> This is my configuration, but you can do it in code too:
>
> <server xmlns="http://mina.apache.org/ftpserver/spring/v1"
> xmlns:beans="http://www.springframework.org/schema/beans"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="
> http://mina.apache.org/ftpserver/spring/v1
> http://mina.apache.org/ftpserver/ftpserver-1.0.xsd
> "
> id="myServer">
> <listeners>
> <nio-listener name="default" port="2323">
> <ssl>
> <keystore file="./res/ftpserver.jks" password="password" />
> </ssl>
> </nio-listener>
> </listeners>
> <!-- user manager configuration -->
> <file-user-manager file="res/conf/users.properties"
> encrypt-passwords="true"/>
>
> <!-- VFS based file system manager configuration -->
> <filesystem>
> <beans:bean
> class="org.vfsutils.ftpserver.filesystem.VfsFileSystemFactory">
> <beans:property name="authenticator.vfsRoot"
> value="sftp://myusr:mypwd@mysftpsrvr:22/"/>
> <beans:property name="authenticator.vfsType"
> value="virtual"/>
> </beans:bean>
> </filesystem>
>
> </server>
>
> Hope this helps,
>
> Frank
>
> ________________________________
> From: Stefan Ziegler <st...@gmail.com>
> Sent: Tuesday, November 3, 2020 5:21 PM
> To: ftpserver-users@mina.apache.org <ft...@mina.apache.org>
> Subject: Re: Problems with sftp and vfs
>
> I tried to narrow it down but it's still strange: The exact same code works
> when using "ftp://" but does not work with "sftp://" in the setVfsRoot()
> methode. With "sftp://" I get an error like:
>
> org.apache.ftpserver.ftplet.FtpException: Error resolving home directory /
> of user demo.
> at
>
> org.vfsutils.ftpserver.filesystem.VfsFileSystemFactory.createFileSystemView(VfsFileSystemFactory.java:64)
> ~[vfsutils-ftpserver-0.2.0-SNAPSHOT.jar:na]
>
> The demo user is the login for the local ftp server. Why does it use the
> local user for creating the virtual file system? For me these are
> completely different things. And why does it work with plain "ftp://" vfs?
> As far as I understand it should also use
> VfsFileSystemFactory.createFileSystemView.
>
> Stefan
>
> On Tue, Nov 3, 2020 at 12:49 PM Frank van der Kleij <kl...@hotmail.com>
> wrote:
>
> > Hi Stefan,
> >
> > I haven't worked with this for a while now, but I can set up an
> > environment to test this one of these days.
> >
> > In the code you are using "ftp://" as virtual file system but in the
> logs
> > I see "sftp://".
> >
> > So it was working when your client was using "ftp://" and your VFS was
> > using "ftp://" too? And now you want your VFS to be "sftp://"?
> > Have you tried with "ftps://" for the VFS? As sftp goes over SSH a
> > different VFS provider is used.
> >
> > As for the approach, I am not 100% sure but I don't think you have to
> call
> > the createFileSystemView method, that should be done automatically; this
> > part might be superfluous:
> >
> > BaseUser vfsUser = new BaseUser();
> > vfsUser.setName(ftpUserHetzner);
> > vfsUser.setPassword(ftpPwdHetzner);
> > vfsUser.setHomeDirectory("/");
> >
> > vfsFileSystemFactory.createFileSystemView(vfsUser);
> >
> > Cheers,
> >
> > Frank
> >
> >
> >
> >
> >
> > ________________________________
> > From: Stefan Ziegler <st...@gmail.com>
> > Sent: Sunday, November 1, 2020 8:20 PM
> > To: ftpserver-users@mina.apache.org <ft...@mina.apache.org>
> > Subject: Problems with sftp and vfs
> >
> > Hi
> >
> > I'm setting up a plain ftp server in front of another ftp server as
> virtual
> > file system. This works great since the ftp server used in the virtual
> file
> > system does not allow anonymous logins. Switching from plain ftp to ftps
> in
> > the virtual file systems ftp server I'm not able to download files
> anymore.
> > It still shows the directories but when trying to download a file it
> seems
> > like it stalls somehow.
> >
> > According to the logs it seems that it tries to download as "anonymous"
> > user and not the user from the virtual file system ftp server (but I can
> > misinterpret this).
> >
> > Has anyone an idea or some hints? Do I mess up something in my code (with
> > the users)? Thanks for any help. See code and logs below.
> >
> > best regards
> > Stefan
> >
> > PropertiesUserManagerFactory userManagerFactory = new
> > PropertiesUserManagerFactory();
> > UserManager userManager = userManagerFactory.createUserManager();
> > BaseUser user = new BaseUser();
> > user.setName("anonymous");
> > userManager.save(user);
> >
> > ListenerFactory listenerFactory = new ListenerFactory();
> > listenerFactory.setPort(2221);
> >
> > ConnectionConfigFactory connectionConfigFactory = new
> > ConnectionConfigFactory();
> > connectionConfigFactory.setAnonymousLoginEnabled(true);
> > connectionConfigFactory.setMaxLogins(10);
> > connectionConfigFactory.setMaxThreads(10);
> >
> > FtpServerFactory factory = new FtpServerFactory();
> >
> >
> factory.setConnectionConfig(connectionConfigFactory.createConnectionConfig());
> > factory.setUserManager(userManager);
> > factory.addListener("default", listenerFactory.createListener());
> >
> > VfsFileSystemFactory vfsFileSystemFactory = new VfsFileSystemFactory();
> > VfsAuthenticator vfsAuthentificator = new VfsAuthenticator();
> > vfsAuthentificator.setVfsRoot("ftp://
> > "+ftpUserHetzner+":"+ftpPwdHetzner+"@"+ftpServerHetzner);
> > vfsAuthentificator.setVfsType("virtual");
> >
> > vfsFileSystemFactory.setAuthenticator(vfsAuthentificator);
> >
> > BaseUser vfsUser = new BaseUser();
> > vfsUser.setName(ftpUserHetzner);
> > vfsUser.setPassword(ftpPwdHetzner);
> > vfsUser.setHomeDirectory("/");
> >
> > vfsFileSystemFactory.createFileSystemView(vfsUser);
> > factory.setFileSystem(vfsFileSystemFactory);
> >
> > FtpServer server = factory.createServer();
> > server.start();
> >
> > 2020-11-01 19:50:31.043 INFO 117121 --- [ NioProcessor-4]
> > o.a.f.listener.nio.FtpLoggingFilter : CREATED
> > 2020-11-01 19:50:31.045 INFO 117121 --- [pool-3-thread-1]
> > o.a.f.listener.nio.FtpLoggingFilter : OPENED
> > 2020-11-01 19:50:31.047 INFO 117121 --- [pool-3-thread-2]
> > o.a.f.listener.nio.FtpLoggingFilter : SENT: 220 Service ready for
> new
> > user.
> >
> > 2020-11-01 19:50:31.060 INFO 117121 --- [pool-3-thread-1]
> > o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: USER anonymous
> > 2020-11-01 19:50:31.061 DEBUG 117121 --- [pool-3-thread-1]
> > org.apache.ftpserver.command.impl.USER : Currently 1 out of 10
> anonymous
> > users logged in
> > 2020-11-01 19:50:31.061 DEBUG 117121 --- [pool-3-thread-1]
> > org.apache.ftpserver.command.impl.USER : Currently 1 out of 5 users
> > logged in
> > 2020-11-01 19:50:31.063 INFO 117121 --- [pool-3-thread-2]
> > o.a.f.listener.nio.FtpLoggingFilter : SENT: 331 Guest login okay,
> send
> > your complete e-mail address as password.
> >
> > 2020-11-01 19:50:31.064 INFO 117121 --- [pool-3-thread-2]
> > o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: PASS *****
> > 2020-11-01 19:50:31.064 DEBUG 117121 --- [pool-3-thread-2]
> > org.apache.ftpserver.command.impl.PASS : Currently 1 out of 10
> anonymous
> > users logged in
> > 2020-11-01 19:50:31.064 DEBUG 117121 --- [pool-3-thread-2]
> > org.apache.ftpserver.command.impl.PASS : Currently 1 out of 5 users
> > logged in
> > 2020-11-01 19:50:31.066 DEBUG 117121 --- [pool-3-thread-2]
> > o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u000000:***@
> > u000000.your-storagebox.de/
> > 2020-11-01 19:50:31.089 INFO 117121 --- [pool-3-thread-2]
> > o.v.f.filesystem.VfsAuthenticator : Authenticated user anonymous
> > based on filesystem
> org.apache.commons.vfs2.impl.VirtualFileSystem@53de3b94
> > 2020-11-01 19:50:31.090 INFO 117121 --- [pool-3-thread-2]
> > org.apache.ftpserver.command.impl.PASS : Anonymous login success -
> > anonymous@example.com
> > 2020-11-01 19:50:31.091 INFO 117121 --- [pool-3-thread-1]
> > o.a.f.listener.nio.FtpLoggingFilter : SENT: 230 User logged in,
> > proceed.
> >
> > 2020-11-01 19:50:31.092 INFO 117121 --- [pool-3-thread-1]
> > o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: OPTS UTF8 ON
> > 2020-11-01 19:50:31.093 INFO 117121 --- [pool-3-thread-2]
> > o.a.f.listener.nio.FtpLoggingFilter : SENT: 200 Command OPTS okay.
> >
> > 2020-11-01 19:50:31.094 INFO 117121 --- [pool-3-thread-2]
> > o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: OPTS MLST
> > size;modify;type;
> > 2020-11-01 19:50:31.096 INFO 117121 --- [pool-3-thread-1]
> > o.a.f.listener.nio.FtpLoggingFilter : SENT: 200 Command OPTS okay.
> >
> > 2020-11-01 19:50:31.098 INFO 117121 --- [pool-3-thread-2]
> > o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: CWD
> > /ch.so.agi.av.dm01avch24lv95d
> > 2020-11-01 19:50:31.099 DEBUG 117121 --- [pool-3-thread-2]
> > o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u000000:***@
> > u000000.your-storagebox.de/ch.so.agi.av.dm01avch24lv95d
> > 2020-11-01 19:50:31.100 DEBUG 117121 --- [pool-3-thread-2]
> > o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u000000:***@
> > u000000.your-storagebox.de/ch.so.agi.av.dm01avch24lv95d
> > 2020-11-01 19:50:31.442 INFO 117121 --- [pool-3-thread-1]
> > o.a.f.listener.nio.FtpLoggingFilter : SENT: 250 Directory changed to
> > /ch.so.agi.av.dm01avch24lv95d
> >
> > 2020-11-01 19:50:31.443 INFO 117121 --- [pool-3-thread-1]
> > o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: TYPE I
> > 2020-11-01 19:50:31.446 INFO 117121 --- [pool-3-thread-2]
> > o.a.f.listener.nio.FtpLoggingFilter : SENT: 200 Command TYPE okay.
> >
> > 2020-11-01 19:50:31.447 INFO 117121 --- [pool-3-thread-2]
> > o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: PASV
> > 2020-11-01 19:50:31.450 DEBUG 117121 --- [pool-3-thread-2]
> > o.a.f.impl.IODataConnectionFactory : Initiating passive data
> > connection
> > 2020-11-01 19:50:31.450 DEBUG 117121 --- [pool-3-thread-2]
> > o.a.f.impl.IODataConnectionFactory : Opening passive data
> connection
> > on address "/127.0.0.1" and port 0
> > 2020-11-01 19:50:31.451 DEBUG 117121 --- [pool-3-thread-2]
> > o.a.f.impl.IODataConnectionFactory : Passive data connection
> created
> > on address "/127.0.0.1" and port 0
> > 2020-11-01 19:50:31.454 INFO 117121 --- [pool-3-thread-1]
> > o.a.f.listener.nio.FtpLoggingFilter : SENT: 227 Entering Passive
> Mode
> > (127,0,0,1,163,47)
> >
> > 2020-11-01 19:50:31.456 INFO 117121 --- [pool-3-thread-1]
> > o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: RETR 257900.itf.zip
> > 2020-11-01 19:50:31.457 DEBUG 117121 --- [pool-3-thread-1]
> > o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u000000:***@
> > u000000.your-storagebox.de/ch.so.agi.av.dm01avch24lv95d/257900.itf.zip
> > 2020-11-01 19:50:31.457 DEBUG 117121 --- [pool-3-thread-1]
> > o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u000000:***@
> > u000000.your-storagebox.de/ch.so.agi.av.dm01avch24lv95d/257900.itf.zip
> >
> >
> >
> >
> >
>