[VFS]: Integration - too early?

[Paul Smith <Pa...@lawlex.com.au>]

[Apologise in advance for the lengthy email]

Hi All,

I've been thinking about trying to integrate VFS into log4j's Chainsaw v2
(http://logging.apache.org/log4j/docs/chainsaw.html) that we are developing.
A great feature would be able to read in remote log files for viewing.  We
currently support this via a direct URL mechanism, but one has to know the
full url, and it doesn't have the full power that VFS seems to have like
browsing a remote directory.

Aside from some technical questions, which I will get to in a sec, I just
wanted to clear whether the following would be ok with the commons
community.

For a Java Web Start launched version to include VFS, I would need to sign
several commons jars using a personal email certificate so that Java Web
Start will validate and approve them as part of the Chainsaw launch.   If
any of the jars in the Web Start app are not signed, it fails miserably for
security reasons.

The certificate that I would signing with would display my psmith@apache.org
email inside it.  Now obviously I wouldn't have written a single line of the
commons code, and just wanted to make sure no-one can foresee a problem with
that.  We'd obviously give appropriate credit within Chainsaw to the
commons-dev team.

Now onto the technical side of things....

* Is VFS still in a state of flux?  

* Does VFS have a feature to dynamically detect which file systems are
'available'? ie. whether HttpClient is available via a class loader etc.
Since Chainsaw might be delivered by Web Start, or just Ant'd up, it'd be
nice to degrade gracefully if a particular file system, say, Http, cannot be
used, because dependant libraries cannot be found.  I'm unsure whether any
of the FileSystemManager implementations can help me here.  Any pointers?

* This might be File system specific, but does anyone know whether, for a
given FileObject, one might be able to read a portion of the, say, text
file, and only grab the first X lines of it for a preview?  I'd like
Chainsaw to be able to show a short preview before possibly incurring the
full expense of streaming the whole log file from the remote file system.

* I don't know much about SFTP, but does this support the ability to
'browse' a remote directory?  It sounds like it does! (Cool!).  I assume if
one has setup certificate-based authentication for a local ssh with a remote
location, the password wouldn't even be needed here.  Would that work? (ie
the usual ~/.ssh/authorized_keys file etc)

That's all I can think of for now.  If anyone is interested in trying out
the current stable version of Chainsaw, they can try out this Java Web Start
link:

http://logging.apache.org/log4j/docs/webstart/chainsaw/chainsawWebStart.jnlp


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org

Re: [VFS]: Integration - too early?

[Mario Ivankovits <im...@apache.org>]

Adam R. B. Jack wrote:

>Would you want a Bugzilla report on this?
>
Yes - please post everything in bugzilla, that way it cant be lost - and 
much more important - i could inform you about the progress.
Please prefix your summary with [vfs] - so i easily could find it - as a 
sandbox project it has no own component in bugzilla.

Every request is welcome.

Thanks!!
-- Mario


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org

Re: [VFS]: Integration - too early?

["Adam R. B. Jack" <aj...@trysybase.com>]

> >* Is VFS still in a state of flux?
> >
> >
> Well - quite some time not much happened in VFS - i picked it up now and
> try to implement what ever comes in my head.

Awesome! As a long time user of VFS (first Ruper, now Depot), this is great
news. VFS is nicely done, and it'd be a real a shame to be without it.

BTW: Depot is the only dependency on Gump that claims usage of VFS (and it'd
be nice to see more):


http://lsd.student.utwente.nl/gump/commons-vfs/commons-vfs/details.html#Project+Dependees

http://lsd.student.utwente.nl/gump/depot/depot-update-ant-sample-vfs/index.html

> >* Does VFS have a feature to dynamically detect which file systems are
> >'available'?
> Yes

Could you make it not log at a WARNING level if it doesn't find something?
Depot has Ant tasks that create the file systems (each run), and the ant
build log is horribly poluted with insignificant things (about semi-obscure
protocols). [I got so frustrated with this not being changed (I reported a
year or more ago) I tried writting this myself, avoiding the
StabdardFileSystem. Technically the API wouldn't allow me to safely do it &
I had to limp back, tail between legs. Would you want me to enter a change
request for this?

Also, last time Depot ran (using VFS) on Gump it crashed when VFS tried to
set a null into HttpClient as a username (not behaviour the user can
control, it seems). I posted here about it, and either need to know how to
set a user (couldn't see it) or some fix/workaround. [I found it odd that
VFS unit tests pass]. Would you want a Bugzilla report on this? [BTW: When
Gump next runs Depot over VFS the output ought be here (below) and ought
show the stack trace.]


http://lsd.student.utwente.nl/gump/depot/depot-update-ant-sample-vfs/index.html

Finally, thanks for stepping up to VFS, I look forward to your progress with
baited breath. :-)

regards,

Adam



---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org

Re: [VFS]: Integration - too early?

[Mario Ivankovits <im...@apache.org>]

Paul Smith wrote:

>The certificate that I would signing with would display my psmith@apache.org
>email inside it.  Now obviously I wouldn't have written a single line of the
>commons code, and just wanted to make sure no-one can foresee a problem with
>that.  We'd obviously give appropriate credit within Chainsaw to the
>commons-dev team.
>  
>
As far as i understand the community idea - there should be no problem 
with this. At least not for me.

>* Is VFS still in a state of flux?
>  
>
Well - quite some time not much happened in VFS - i picked it up now and 
try to implement what ever comes in my head.
However - i try not to change the current api too much. Maybe to one or 
other method will be added.
I will be happy to work together with the log4j team to make the 
integration into chainsaw reality.

>* Does VFS have a feature to dynamically detect which file systems are
>'available'? 
>  
>
Yes - one of the methods to get a filesystemmanager is by using the 
StandardFilesytemManager - it uses a providers.xml where the possible 
filesystem-types are defined and also the needet classes.
If the dependency check failes the provider is not available.

e.g.
    <provider 
class-name="org.apache.commons.vfs.provider.http.HttpFileProvider">
        <scheme name="http"/>
        <if-available 
class-name="org.apache.commons.httpclient.HttpClient"/>
    </provider>

>* This might be File system specific, but does anyone know whether, for a
>given FileObject, one might be able to read a portion of the, say, text
>file, and only grab the first X lines of it for a preview?
>
Currently a simple InputStream is provided. As always you could wrap it 
e.g. into an BufferedReader and use readLine().
So the answer is YES.
However - we have to check how e.g ftp react if you close the stream in 
the mid of the transfer.

One of the things i would implement in the future is to provide access 
to the file by some sort of RandomAccessFile - for sure - for the 
filesystems where it is possible.
But it should be possible to use the restart-capability of some 
ftp-server to simulate this behaviour.

>* I don't know much about SFTP, but does this support the ability to
>'browse' a remote directory?
>
YES.
This brings me back to the previous question.
Currently the SFTP provider reads the complete file into memory - i have 
to check if this is still needet.
But even if i change this - it is transparent to the above outlined 
solution.

>I assume if one has setup certificate-based authentication for a local ssh with a remote
>location, the password wouldn't even be needed here.  Would that work? (ie
>the usual ~/.ssh/authorized_keys file etc)
>  
>
This should work too - but i will test this again.
Another problem here is the fact, that it is not easily possible to find 
the location of the key-files on the client side - the current code 
tries to figure out some places:
1) check "vfs.ftp.sshdir" system-property
2) check "user.home"/.ssh
3) check c:\\cygwin\\home\\"user.name"\\.ssh

I DEFINITELY WILL CHANGE THIS.
I know from some old communication between the original authors and me, 
that they never ever wanted to use system-properties - all has to be 
configured by the application.
In the past it was a problem to send some configuration to the 
filessystem - now it is not.

>That's all I can think of for now.  If anyone is interested in trying out
>the current stable version of Chainsaw
>
I currently use the cvs-head version of chainsaw2 (and a patched 
commons-logging). The chainsaw stuff was motivation enough to change our 
complete custom loggin solution to commons-logging with log4j as backend.
Keep on going !!!

-- Mario


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org