You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by bu...@apache.org on 2006/02/16 21:17:02 UTC
DO NOT REPLY [Bug 38686] New: - Default VFS cache behavior
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG�
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=38686>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND�
INSERTED IN THE BUG DATABASE.
http://issues.apache.org/bugzilla/show_bug.cgi?id=38686
Summary: Default VFS cache behavior
Product: Commons
Version: Nightly Builds
Platform: All
OS/Version: All
Status: NEW
Severity: enhancement
Priority: P3
Component: VFS
AssignedTo: commons-dev@jakarta.apache.org
ReportedBy: jb@nanthrax.net
It's really a bug but a explanation.
The default VFS cache is SoftRefFilesCache.
In my process, I have a FileManipulator which is simply a singleton. This
singleton create one (and only one max) instance of FileSystemManager using
VFS.getManager().
This process is called every 10mn to perform a copy from a FileObject to
another. In earch loop, I perform something which looks like :
FIleManipulator fileManipulator = FileManipulator.getInstance();
fileManipulator.copy("tgz:http://remote/archive.tar.gz!/dir/file", "/tmp/file");
Either the tgz:http://remote/archive.tar.gz!/dir/file has not change, in the
vfs_cache directory, I have :
tmp_19071_file
tmp_19076_file
So a copy if performed every loop and neither release which take a lot of space
on the filesystem.
How can I avoid this behavior ? Should I create a FileManipulator (and so a
FileSystemManager) and use the same in the loop ? Do I need to define my own
filesystemmanager with another FileCache system (such as NullFileCache) ? Is it
possible to define another VFS cache system for the default filesystem manager
got with VFS.getManager() ?
Many thanks for your help.
--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
DO NOT REPLY [Bug 38686] - [VFS] Default VFS cache behavior
Posted by bu...@apache.org.
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG�
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=38686>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND�
INSERTED IN THE BUG DATABASE.
http://issues.apache.org/bugzilla/show_bug.cgi?id=38686
jb@nanthrax.net changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution| |FIXED
------- Additional Comments From jb@nanthrax.net 2006-02-27 14:35 -------
It seems to work.
Thanks a lot for this workaround !!!
--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
DO NOT REPLY [Bug 38686] - [VFS] Default VFS cache behavior
Posted by bu...@apache.org.
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG�
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=38686>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND�
INSERTED IN THE BUG DATABASE.
http://issues.apache.org/bugzilla/show_bug.cgi?id=38686
mario@ops.co.at changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|RESOLVED |REOPENED
Resolution|FIXED |
------- Additional Comments From mario@ops.co.at 2006-02-27 17:33 -------
Ok, so lets see if we can add this replicator to VFS ;-)
--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
DO NOT REPLY [Bug 38686] - [VFS] Default VFS cache behavior
Posted by bu...@apache.org.
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG�
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=38686>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND�
INSERTED IN THE BUG DATABASE.
http://issues.apache.org/bugzilla/show_bug.cgi?id=38686
------- Additional Comments From jb@nanthrax.net 2006-02-21 11:43 -------
Thanks a lot, I'm going to try it !!!!
--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
DO NOT REPLY [Bug 38686] - [vfs] Default VFS cache behavior
Posted by bu...@apache.org.
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG�
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=38686>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND�
INSERTED IN THE BUG DATABASE.
http://issues.apache.org/bugzilla/show_bug.cgi?id=38686
Sandy@McArthur.org changed:
What |Removed |Added
----------------------------------------------------------------------------
Summary|Default VFS cache behavior |[vfs] Default VFS cache
| |behavior
--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
DO NOT REPLY [Bug 38686] - [VFS] Default VFS cache behavior
Posted by bu...@apache.org.
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG�
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=38686>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND�
INSERTED IN THE BUG DATABASE.
http://issues.apache.org/bugzilla/show_bug.cgi?id=38686
------- Additional Comments From troeger@alaun.de 2006-02-21 10:29 -------
Hi,
the tmp files are created by the FileReplicator and are deleted during shutdown
of the VFS-Manager.
You may set your own implementation of a FileReplicator (that does a better Job)
via:
((DefaultFileSystemManager) VFS.getManager()).setReplicator(...);
You may gracefully shutdown the default replicator
(VFS.getManager()).getReplicator()) before.
Be sure to do this at application start up!!!
After accessing your tar.gz you should shutdown the implicitly created file
system like so:
((DefaultFileSystemManager)
VFS.getManger()).closeFileSystem(someFileObjectInYourTarGz.getFilesSystem());
This will release the TarFileSystem instance that holds the reference to the tmp
File object.
As far as I know there�s no event (or something like that) to keep track on file
system shutdowns. So you may use a weak reference to the file object (you
replicated) to cleanup the unused temp file:
1. create a class that extends java.lang.ref.WeakReference:
public class WeakFileRef extends WeakReference {
private String absPath;
public WeakFileRef(File file, ReferenceQueue queue) {
super(file, queue);
this.absPath = file.getAbsolutePath();
}
public String getPath() {
return this.absPath;
}
}
2. Do your own FileReplicator implementation. You may copy and slightly modify
org.apache.commons.vfs.impl.DefaultFileReplicator. Attention: don�t use any hard
reference on file objects here - use the WeakFileRef class.
3. In your FileReplicator implementation create a field of type ReferenceQueue
private ReferenceQueue queue = new ReferenceQueue();
4. Change allocateFile(String):
public File allocateFile(final String baseName) throws FileSystemException
{
WeakFileRef fileRef = (WeakFileRef) queue.poll(); // poll the refence queue
and delete finalized files
while (fileRef != null) {
File toDelete = new File(fileRef.getAbsPath());
try {
if (toDelete.exists()) {
toDelete.delete();
}
} catch (IOException) {
// do some exception handling or throw a FileSystemExeption
}
copies.remove(fileRef);
fileRef = (WeakFileRef) queue.poll();
}
// Create a unique-ish file name
final String basename = createFilename(baseName);
filecount++;
final File file = createFile(tempDir, basename);
// Keep track to delete later
copies.add(new WeakFileRef(file, queue)); // <- changed
return file;
}
5. Change the close() method -> ArrayList 'copies' now holds WeakFileRef objects!
Notice that the files are not cleaned up immediately - it depends on garbage
collection.
Hope this helps/(works) ;-)
--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
DO NOT REPLY [Bug 38686] - [VFS] Default VFS cache behavior
Posted by bu...@apache.org.
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG�
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=38686>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND�
INSERTED IN THE BUG DATABASE.
http://issues.apache.org/bugzilla/show_bug.cgi?id=38686
jb@nanthrax.net changed:
What |Removed |Added
----------------------------------------------------------------------------
Summary|[vfs] Default VFS cache |[VFS] Default VFS cache
|behavior |behavior
------- Additional Comments From jb@nanthrax.net 2006-02-17 06:30 -------
Sorry, I was tired yesterday evening (in France) when I have wrote the message.
I correct some typing mistakes :
Sentence 1 : "It's NOT really a bug but an explanation"
Sentence 7 : "In EACH loop, I perform something like that : "
Sentence 16 : "So a copy IS performed every loop and neither deleted which take
a lot of space on the filesystem".
Sorry, thanks a lot
--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
DO NOT REPLY [Bug 38686] - [VFS] Default VFS cache behavior
Posted by bu...@apache.org.
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG�
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=38686>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND�
INSERTED IN THE BUG DATABASE.
http://issues.apache.org/bugzilla/show_bug.cgi?id=38686
------- Additional Comments From jb@nanthrax.net 2006-02-21 07:19 -------
Any news about this ?
Many thanks
--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org