You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Filip Defoort <fi...@cirquedigital.com> on 2006/07/12 03:08:12 UTC
[VFS] accessing FileObjects from different threads
Hi,
I'm accessing VFS FileObjects from different threads and running into some
synchronization problems (using the ON_RESOLVE cache strategy).
Sometimes, the FileObject gets detached while it's still being used by
another
thread causing exceptions and sometimes the FileObject go into a deadlock.
What are the rules for accessing FileObjects from different threads ?
Many thanks in advance,
- Filip
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [VFS] accessing FileObjects from different threads
Posted by Filip Defoort <fi...@cirquedigital.com>.
Hi Mario,
Follow-up on today's testing: indeed, no more deadlock problems. Yeah!
I did notice one thing, that is that the behaviour of
FileObject.listChildren() seems
to have changed. It now throw an exception when you do a list children on a
file (as opposed to a folder). I believe it simply returned null or an
empty array
before.
Not sure if this change is intended or if it's a side-effect of some of
the changes...
Thanks!
- Filip
Filip Defoort wrote:
> Hi Mario,
>
> Indeed -- it's been a while again... too little time, too many things
> to do;
> but I keep coming back to VFS :-D. Still would love to find the time
> to improve the normalization...
>
> It's primarily reads of metadata about the FileObjects that I think
> are causing
> the problems (listChildren(), getType(),...) -- I don't touch the
> actual content
> often enough to show any signs there. It showed up after switching
> to RC8-snapshot to work around the cache freshness problem and some
> changes in my code to page in information on demand as the main
> tables scroll independently.
>
> The symptom is either deadlock on FileObject synchronization or NPE,
> depending on the test.
>
> I'll have testing done tomorrow with HEAD and see if it props up with
> the very latest
> VFS (without the SynchronizedFileObject decoration) and will let you
> know... If the problem is still there, I can get you the thread dumps...
>
> Thanks and Cheers!
> - Filip
>
>
>
>
> Mario Ivankovits wrote:
>> Hi Filip, nice to read you again.
>>
>>
>>> Sometimes, the FileObject gets detached while it's still being used by
>>> another
>>> thread causing exceptions and sometimes the FileObject go into a
>>> deadlock.
>>>
>> As you already noticed ( you are bleedingly fast) :-) I committed a try
>> to fix this issue.
>> Maybe if you simply try the svn head version it might behave better
>> already without any additional change. But for sure, if one thread tries
>> to copy the file and another one deletes it at the same time you'll get
>> exceptions again.
>> Could you please provide more informations about the operations you try
>> in parallel on the same fileObject?
>>
>> The next NEW thing you can try is, to decorate the file object so that
>> it will synchronize each call.
>>
>> StandardFileSystemManager manager = new
>> StandardFileSystemManager();
>> manager.setCacheStrategy(CacheStrategy.ON_RESOLVE);
>> manager.setFileObjectDecorator(SynchronizedFileObject.class);
>> manager.init();
>>
>> if you use instanceof with your FileObjects you have to change this to
>> FileObjectUtils.isInstanceOf
>>
>> However, the deadlock problem might be more problematic. We need a "full
>> thread dump" to figure out whats the problem.
>> Also the SynchronizedFileObject might introduce new deadlocks as it
>> synchronize against the fileObject instance, internally the FileObject
>> itself has to synchronize against the fileSystem .... but hey, its a
>> try.
>>
>> Could you please check this out.
>>
>> Thanks!
>> Ciao,
>> Mario
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [VFS] accessing FileObjects from different threads
Posted by Filip Defoort <fi...@cirquedigital.com>.
Hi Mario,
Indeed -- it's been a while again... too little time, too many things to do;
but I keep coming back to VFS :-D. Still would love to find the time
to improve the normalization...
It's primarily reads of metadata about the FileObjects that I think are
causing
the problems (listChildren(), getType(),...) -- I don't touch the actual
content
often enough to show any signs there. It showed up after switching
to RC8-snapshot to work around the cache freshness problem and some
changes in my code to page in information on demand as the main
tables scroll independently.
The symptom is either deadlock on FileObject synchronization or NPE,
depending on the test.
I'll have testing done tomorrow with HEAD and see if it props up with
the very latest
VFS (without the SynchronizedFileObject decoration) and will let you
know... If the problem is still there, I can get you the thread dumps...
Thanks and Cheers!
- Filip
Mario Ivankovits wrote:
> Hi Filip, nice to read you again.
>
>
>> Sometimes, the FileObject gets detached while it's still being used by
>> another
>> thread causing exceptions and sometimes the FileObject go into a
>> deadlock.
>>
> As you already noticed ( you are bleedingly fast) :-) I committed a try
> to fix this issue.
> Maybe if you simply try the svn head version it might behave better
> already without any additional change. But for sure, if one thread tries
> to copy the file and another one deletes it at the same time you'll get
> exceptions again.
> Could you please provide more informations about the operations you try
> in parallel on the same fileObject?
>
> The next NEW thing you can try is, to decorate the file object so that
> it will synchronize each call.
>
> StandardFileSystemManager manager = new StandardFileSystemManager();
> manager.setCacheStrategy(CacheStrategy.ON_RESOLVE);
> manager.setFileObjectDecorator(SynchronizedFileObject.class);
> manager.init();
>
> if you use instanceof with your FileObjects you have to change this to
> FileObjectUtils.isInstanceOf
>
> However, the deadlock problem might be more problematic. We need a "full
> thread dump" to figure out whats the problem.
> Also the SynchronizedFileObject might introduce new deadlocks as it
> synchronize against the fileObject instance, internally the FileObject
> itself has to synchronize against the fileSystem .... but hey, its a try.
>
> Could you please check this out.
>
> Thanks!
> Ciao,
> Mario
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [VFS] accessing FileObjects from different threads
Posted by Mario Ivankovits <ma...@ops.co.at>.
Hi Filip, nice to read you again.
> Sometimes, the FileObject gets detached while it's still being used by
> another
> thread causing exceptions and sometimes the FileObject go into a
> deadlock.
As you already noticed ( you are bleedingly fast) :-) I committed a try
to fix this issue.
Maybe if you simply try the svn head version it might behave better
already without any additional change. But for sure, if one thread tries
to copy the file and another one deletes it at the same time you'll get
exceptions again.
Could you please provide more informations about the operations you try
in parallel on the same fileObject?
The next NEW thing you can try is, to decorate the file object so that
it will synchronize each call.
StandardFileSystemManager manager = new StandardFileSystemManager();
manager.setCacheStrategy(CacheStrategy.ON_RESOLVE);
manager.setFileObjectDecorator(SynchronizedFileObject.class);
manager.init();
if you use instanceof with your FileObjects you have to change this to
FileObjectUtils.isInstanceOf
However, the deadlock problem might be more problematic. We need a "full
thread dump" to figure out whats the problem.
Also the SynchronizedFileObject might introduce new deadlocks as it
synchronize against the fileObject instance, internally the FileObject
itself has to synchronize against the fileSystem .... but hey, its a try.
Could you please check this out.
Thanks!
Ciao,
Mario
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [VFS] accessing FileObjects from different threads
Posted by Filip Defoort <fi...@cirquedigital.com>.
Saw some fixes in the latest svn trunk -- I believe those address the
problem
I was having....
Keeping fingers crossed...
- Filip
Filip Defoort wrote:
> Hi,
>
> I'm accessing VFS FileObjects from different threads and running into
> some
> synchronization problems (using the ON_RESOLVE cache strategy).
>
> Sometimes, the FileObject gets detached while it's still being used by
> another
> thread causing exceptions and sometimes the FileObject go into a
> deadlock.
>
> What are the rules for accessing FileObjects from different threads ?
>
> Many thanks in advance,
> - Filip
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org