Apache SSHD file stores

["Wells, Chris" <ch...@hp.com>]

First, thanks to mina team for an amazing product.  We were recently presented with the challenge of providing an sftp front end for an AWS S3 store.  I went out searching the web and found apache mina and a few examples using it.  After a few hours of tinkering, I had a working prototype that did 90% of what we need (and most of that time was actually spent tweaking our S3 code to perform file chunking intelligently, because we want to support resuming uploads).  The hooks and interfaces put in place were easy to understand and override: we only had to override org.apache.sshd's SshFile, FileSystemFactory, and FileSystemView, as well as java's InputStream and OutputStream.

That was using org.apache.sshd.sshd-core 0.11.0 (and a bit later, 0.14.0).

I discovered that org.apache.sshd.sshd-core 1.0.0 was available and tried updating to that version.  It looks like a lot has changed, at least from a file store perspective.  To update to using 1.0, it looks like I'll need to override/implement a lot of low level nio interfaces and classes, including Path, FileSystemProvider,  FileChannel, and FileSystem.  There's a lot more effort involved to make this happen.  From the perspective of a dev who wants to override sshd-core's file store backend, it looks like 1.0 is a step backwards from simplicity.  Hopefully future versions will address this, but I know that no product can accommodate every user's needs all of the time.

Again, thanks for a terrific product.

[http://h30067.www3.hp.com/allentries/JPG_T/hp_sig_logo_020807.jpg]

Chris Wells
Software Designer
859.422.4776 | 810 Bull Lea Run | Lexington, KY 40511
chris.wells@hp.com<ma...@hp.com>

RE: Apache SSHD file stores

[Lyor Goldstein <lg...@vmware.com>]

You make some excellent points about the difficulty that the transition from the old files view to Java 7 NIO model poses to older code. However, the idea behind this transition was to align the code with existing Java mechanisms thus making it standard (see SftpFileSystem implementation). I agree that implementing the NIO model is more complex, but eventually it may provide a very useful "side-effect". The effort will no longer be usable only for SSHD, but rather for ANY Java code that uses NIO API. In other words, if you develop a Java NIO FileSystem for SSHD, you will be able to offer it as a Java "plugin" to any code that needs S3 access (in this case) using the same APIs it would use for accessing local files (and not even be aware of this...). In other words, one can develop a JAR (Maven artifact) that simply provides an S3-based FileSystem implementation, which does not depend on SSHD - although can be used for SSHD, but not only for it - thus contributing to the open-source community a useful tool.

-----Original Message-----
From: Wells, Chris [mailto:chris.wells@hp.com] 
Sent: Friday, January 29, 2016 16:47
To: dev@mina.apache.org
Subject: Apache SSHD file stores

First, thanks to mina team for an amazing product.  We were recently presented with the challenge of providing an sftp front end for an AWS S3 store.  I went out searching the web and found apache mina and a few examples using it.  After a few hours of tinkering, I had a working prototype that did 90% of what we need (and most of that time was actually spent tweaking our S3 code to perform file chunking intelligently, because we want to support resuming uploads).  The hooks and interfaces put in place were easy to understand and override: we only had to override org.apache.sshd's SshFile, FileSystemFactory, and FileSystemView, as well as java's InputStream and OutputStream.

That was using org.apache.sshd.sshd-core 0.11.0 (and a bit later, 0.14.0).

I discovered that org.apache.sshd.sshd-core 1.0.0 was available and tried updating to that version.  It looks like a lot has changed, at least from a file store perspective.  To update to using 1.0, it looks like I'll need to override/implement a lot of low level nio interfaces and classes, including Path, FileSystemProvider,  FileChannel, and FileSystem.  There's a lot more effort involved to make this happen.  From the perspective of a dev who wants to override sshd-core's file store backend, it looks like 1.0 is a step backwards from simplicity.  Hopefully future versions will address this, but I know that no product can accommodate every user's needs all of the time.

Again, thanks for a terrific product.

Chris Wells
Software Designer
859.422.4776 | 810 Bull Lea Run | Lexington, KY 40511 chris.wells@hp.com<ma...@hp.com>