You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Jörg Schaible <jo...@gmx.de> on 2009/08/06 11:07:28 UTC
[vfs] VFS.getManager and FileCache
Hi gang,
the docs of VFS praise VFS.getManager() as simplest possibility to work with
VFS. However, this setup uses a SortRefFileCache as default implementation.
This caused our server app to fail, since it means that FileObject
instances are only garbage collected if the Java process runs out of
memory. However, if the instance refer some other limited resource (in our
case for SFTP socket connections to the SFTP server) they may never be
released. Our SFTP server simply runs out of available connections over
time.
Therefore we should possibly reconsider the default choice and use the
WeakRefFileCache instead. Since the FileObject instances are often linked
anyway with their parent/children references, the weak references are
normally not freed while the client has one reference left.
IMHO this behaviour matches the "simplest" (and less error-prone) approach
using VFS.getManager() most. WDYT?
- Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [vfs] VFS.getManager and FileCache
Posted by Jörg Schaible <jo...@gmx.de>.
Hi Mario,
coming back to this now again ... ;-)
Mario Ivankovits wrote at Donnerstag, 6. August 2009 17:36:
> Hi!
>
> Using weak ref as default seems not too bad. However, since the file
> objects have just their parent cached it might lead to much more file
> system access then now as a parent might need to refresh its children.
>
> I tried to kick a discussion about caching at all already [1].
Ah, yes, interesting.
> I still think going the mount/unmount way and leaving all the
> concurrency/threading issues up to the application makes more sense.
+1, we might provide a synchronizing wrapper like it is done with
Collections.synchronizedXXX() methods. Actually I'd deprecate the
FM.resolveFile here in favor to mount/unmount simply to make it explicit
that you're doing something fundamental different than with FO.resolveFile.
> Means, VFS is no longer thread safe by default and you have to deal with
> thread safety only if you use the object returned from mount() in
> differente threads. Also, by unmounting (equals filesystem.close()) we can
> easily get rid of any used underlaying resources.
Currently I am diving deeper into the code, since even with the
WeakRefFilesCache our connections to the SFTP server are not closed even
when we keep no reference to any FileObject anymore and they are GCed ...
go hunting ;-)
- Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [vfs] VFS.getManager and FileCache
Posted by Mario Ivankovits <ma...@ops.co.at>.
Hi!
Using weak ref as default seems not too bad. However, since the file objects have just their parent cached it might lead to much more file system access then now as a parent might need to refresh its children.
I tried to kick a discussion about caching at all already [1].
I still think going the mount/unmount way and leaving all the concurrency/threading issues up to the application makes more sense.
Means, VFS is no longer thread safe by default and you have to deal with thread safety only if you use the object returned from mount() in differente threads.
Also, by unmounting (equals filesystem.close()) we can easily get rid of any used underlaying resources.
Ciao,
Mario
[1] http://www.mail-archive.com/dev@commons.apache.org/msg10387.html
-----Ursprüngliche Nachricht-----
Von: news [mailto:news@ger.gmane.org] Im Auftrag von Jörg Schaible
Gesendet: Donnerstag, 06. August 2009 11:07
An: dev@commons.apache.org
Betreff: [vfs] VFS.getManager and FileCache
Hi gang,
the docs of VFS praise VFS.getManager() as simplest possibility to work with
VFS. However, this setup uses a SortRefFileCache as default implementation.
This caused our server app to fail, since it means that FileObject
instances are only garbage collected if the Java process runs out of
memory. However, if the instance refer some other limited resource (in our
case for SFTP socket connections to the SFTP server) they may never be
released. Our SFTP server simply runs out of available connections over
time.
Therefore we should possibly reconsider the default choice and use the
WeakRefFileCache instead. Since the FileObject instances are often linked
anyway with their parent/children references, the weak references are
normally not freed while the client has one reference left.
IMHO this behaviour matches the "simplest" (and less error-prone) approach
using VFS.getManager() most. WDYT?
- Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org