[vfs] Disabling default file caching

[Robert Enyedi <re...@gmail.com>]

Hi,

During the migration of JCommander's backend to VFS I found out about the
default file caching that VFS does.
For instance FileObject.getChildren() by default returns the same children
across multiple calls regardless of the changes that happened on the disk
between two identical calls.

I'm not entirely sure if having the cache activated by default is
necessarily a good idea, but this is a separate discussion.

Now I am trying to disable the caching by using a StandardFileManager
instance configured this way:

StandardFileSystemManager standardManager = new StandardFileSystemManager();
standardManager.setFilesCache(new NullFilesCache());
standardManager.setConfiguration(configFile); // configFile is a valid URL
standardManager.init();

I intend to use this manager instead of the default one obtained using the
handy VFS.getManager() call.

Unfortunately this isn't really working because it seems that the file
system providers aren't really loaded. And I also get a NullPointerException
while trying to get the parent file for a root FileObject.

Could someone help out with what I'm trying to achieve? Has anyone dealt
with this issue before?

Thanks,
Robert

Re: [vfs] Disabling default file caching

[Robert Enyedi <re...@gmail.com>]

Thanks :-) Yeah, one of those days... Good luck with your work!

On 11/23/05, Mario Ivankovits <ma...@ops.co.at> wrote:
>
> Hi Robert,
>
> > I did not want to offend anyone.
> Sorry, maybe I used the wrong words, a little bit too much work today.
>
> I know you are a good fellow and you didnt mean it harm :-)
>
> ---
> Mario
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>
>

Re: [vfs] Disabling default file caching

[Mario Ivankovits <ma...@ops.co.at>]

Hi Robert,

> I did not want to offend anyone.
Sorry, maybe I used the wrong words, a little bit too much work today.

I know you are a good fellow and you didnt mean it harm :-)

---
Mario


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org

Re: [vfs] Disabling default file caching

[Robert Enyedi <re...@gmail.com>]

Hi Mario,

I did not want to offend anyone. I just wanted somebody to clarify to me the
cache related behavior. And I think that now I have a clearer picture. If
caching redesign is high on the priority list for the next version that is
enough for my needs :-)

And for now I have a functional workaround. Using the .close() method is OK.
Let's focus on the next release ;-)

Regards,
Robert


On 11/23/05, Mario Ivankovits <ma...@ops.co.at> wrote:
>
> Hi Robert,
>
> having a system bulid around a cache is an architectural design flaw,
> there is nothing else to say - we all know that - POINT.
>
> To excuse myself: when I jumped into VFS development this cache was
> already a long living, persistent, *never freed* map.
> The first thing I did was to fix this "memory leak". Unhappily, this
> time I havent had enough knowledge to remove the cache at all.
>
> Wrong decision, well ... maybe.
> As I said, it is on the todo list - and at a rather high position.
>
> For now please call .close() and lets find a CLEAN solution later.
>
> Thanks!
>
> ---
> Mario
>
>
>

Re: [vfs] Disabling default file caching

[Mario Ivankovits <ma...@ops.co.at>]

Hi Robert,

having a system bulid around a cache is an architectural design flaw, 
there is nothing else to say - we all know that - POINT.

To excuse myself: when I jumped into VFS development this cache was 
already a long living, persistent, *never freed* map.
The first thing I did was to fix this "memory leak". Unhappily, this 
time I havent had enough knowledge to remove the cache at all.

Wrong decision, well ... maybe.
As I said, it is on the todo list - and at a rather high position.

For now please call .close() and lets find a CLEAN solution later.

Thanks!

---
Mario

Re: [vfs] Disabling default file caching

[Robert Enyedi <re...@gmail.com>]

Hi,

Well, I've thought the same thing for a while.
> But not using caching at all will have quite an
> impact on speed. So turning it off completely by
> default ...don't know if that makes sense on the
> protocol level. You might want to have something
> in between.


If you want something in between then you should explicitly state it in the
code. But by default to not support concurrent access to the filesystem is
at least strange...

We ended up just properly closing everything which
> made all problems disappear - for us.


So you mean that having a directory.close() call before
directory.getChildren() is a nice approach?

Don't get me wrong, I also believe that caching is a very useful feature to
have. But I think that future development should look into the default
policy and ways to dynamically update that policy.

Regards,
Robert

Re: [vfs] Disabling default file caching

[Torsten Curdt <tc...@apache.org>]

> Calling directory.close() before directory.getChildren() helps  
> indeed. But
> it isn't nice at all especially because this is the default  
> behavior. It
> will definitely require addressing.
>
> I think a distinction should be made between short runs (e.g. Ant  
> task) and
> long runs (like a file manager) for VFS. By default a long running  
> task
> should be considered as happening (so no caching). If a client  
> knows that it
> needs a short running task then it should configure VFS  
> accordingly. However
> the default setting should provide a real time view of the file  
> systems.
>
> I don't know, but shouldn't this behavior considered as an  
> (architectural)
> bug?

Well, I've thought the same thing for a while.
But not using caching at all will have quite an
impact on speed. So turning it off completely by
default ...don't know if that makes sense on the
protocol level. You might want to have something
in between.

We ended up just properly closing everything which
made all problems disappear - for us.

My 2 cents

cheers
--
Torsten

Re: [vfs] Disabling default file caching

[Robert Enyedi <re...@gmail.com>]

Hi Mario,

Calling directory.close() before directory.getChildren() helps indeed. But
it isn't nice at all especially because this is the default behavior. It
will definitely require addressing.

I think a distinction should be made between short runs (e.g. Ant task) and
long runs (like a file manager) for VFS. By default a long running task
should be considered as happening (so no caching). If a client knows that it
needs a short running task then it should configure VFS accordingly. However
the default setting should provide a real time view of the file systems.

I don't know, but shouldn't this behavior considered as an (architectural)
bug?

Regards,
Robert

On 11/23/05, Mario Ivankovits <ma...@ops.co.at> wrote:
>
> Hi Robert!
> > For instance FileObject.getChildren() by default returns the same
> children
> > across multiple calls regardless of the changes that happened on the
> disk
> > between two identical calls.
> >
> This is already a todo on "VfsNext".
>
> For now, you can call "foDirectory.close()" to refresh its children data.
>
> > standardManager.setFilesCache(new NullFilesCache());
> >
> Please dont use any other FilesCache than the default, they will lead to
> memory leaks and/or some other problems as you found out.
> And this sort of cache is other than the local cache the *FileObject
> uses, so the FileObject wont refresh its cache even if you use this
> NullFilesCache.
>
>
> Hope this helps for now.
> ---
> Mario
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>
>

Re: [vfs] Disabling default file caching

[Mario Ivankovits <ma...@ops.co.at>]

Hi Robert!
> For instance FileObject.getChildren() by default returns the same children
> across multiple calls regardless of the changes that happened on the disk
> between two identical calls.
>   
This is already a todo on "VfsNext".

For now, you can call "foDirectory.close()" to refresh its children data.

> standardManager.setFilesCache(new NullFilesCache());
>   
Please dont use any other FilesCache than the default, they will lead to 
memory leaks and/or some other problems as you found out.
And this sort of cache is other than the local cache the *FileObject 
uses, so the FileObject wont refresh its cache even if you use this 
NullFilesCache.


Hope this helps for now.
---
Mario


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org