You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by bu...@apache.org on 2005/05/30 20:53:13 UTC
DO NOT REPLY [Bug 35126] New: -
[vfs][PATCH] Default External File Monitor
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG�
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=35126>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND�
INSERTED IN THE BUG DATABASE.
http://issues.apache.org/bugzilla/show_bug.cgi?id=35126
Summary: [vfs][PATCH] Default External File Monitor
Product: Commons
Version: unspecified
Platform: Other
OS/Version: other
Status: NEW
Severity: normal
Priority: P2
Component: VFS
AssignedTo: commons-dev@jakarta.apache.org
ReportedBy: xknight@users.sourceforge.net
Did some code cleanup and fixed a bug
1) Merged the addStack and deleteStack into a ModifiedQueue object. In case the
order of adding and removing file objects from the monitored map matters in the
future.
2) Added a call to close a file object before it fires a create event. Fixes
following bug: File created in file system, file create event fired, file
deleted from file system, file delete event fired, same file (name, date/time
modified, file path) created again (copied from another location for example) no
event fired.
3) Reset the parent's children list on queue of a file delete instead of on
removal from map. There should be a slimmer chance of a missed file creation (or
re-creation) because of this.
/*
* Copyright 2002-2005 The Apache Software Foundation.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.commons.vfs.impl;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.commons.vfs.FileListener;
import org.apache.commons.vfs.FileMonitor;
import org.apache.commons.vfs.FileName;
import org.apache.commons.vfs.FileObject;
import org.apache.commons.vfs.FileSystemException;
import org.apache.commons.vfs.FileType;
import org.apache.commons.vfs.provider.AbstractFileSystem;
import java.util.HashMap;
import java.util.Map;
import java.util.Stack;
import java.util.ArrayList;
/**
* A polling {@link FileMonitor} implementation.<br />
* <br />
* The DefaultFileMonitor is a Thread based polling file system monitor with a 1
* second delay.<br />
* <br />
* <b>Design:</b>
* <p/>
* There is a Map of monitors known as FileMonitorAgents. With the thread running,
* each FileMonitorAgent object is asked to "check" on the file it is
* responsible for.
* To do this check, the cache is cleared.
* </p>
* <ul>
* <li>If the file existed before the refresh and it no longer exists, a delete
* event is fired.</li>
* <li>If the file existed before the refresh and it still exists, check the
* last modified timestamp to see if that has changed.</li>
* <li>If it has, fire a change event.</li>
* </ul>
* <p/>
* With each file delete, the FileMonitorAgent of the parent is asked to
* re-build its
* list of children, so that they can be accurately checked when there are new
* children.<br/>
* New files are detected during each "check" as each file does a check for new
* children.
* If new children are found, create events are fired recursively if recursive
* descent is
* enabled.
* </p>
* <p/>
* For performance reasons, added a delay that increases as the number of files
* monitored
* increases. The default is a delay of 1 second for every 1000 files processed.
* </p>
* <p/>
* <br /><b>Example usage:</b><pre>
* FileSystemManager fsManager = VFS.getManager();
* FileObject listendir = fsManager.resolveFile("/home/username/monitored/");
* <p/>
* DefaultFileMonitor fm = new DefaultFileMonitor(new CustomFileListener());
* fm.setRecursive(true);
* fm.addFile(listendir);
* fm.start();
* </pre>
* <i>(where CustomFileListener is a class that implements the FileListener
* interface.)</i>
*
* @author <a href="mailto:xknight@users.sourceforge.net">Christopher Ottley</a>
* @version $Revision: 170205 $ $Date: 2005-05-15 08:21:59Z $
*/
public class DefaultFileMonitor implements Runnable, FileMonitor
{
private final static Log log = LogFactory.getLog(DefaultFileMonitor.class);
/**
* Map from FileName to FileObject being monitored.
*/
private final Map monitorMap = new HashMap();
/**
* The low priority thread used for checking the files being monitored.
*/
private Thread monitorThread;
/**
* File objects to be modify the monitor map (added to / deleted from the map)
*/
private ModificationQueue mQueue = new ModificationQueue();
/**
* A flag used to determine if the monitor thread should be running.
*/
private boolean shouldRun = true;
/**
* A flag used to determine if adding files to be monitored should be recursive.
*/
private boolean recursive = false;
/**
* Set the delay between checks
*/
private long delay = 1000;
/**
* Set the number of files to check until a delay will be inserted
*/
private int checksPerRun = 1000;
/**
* A listener object that if set, is notified on file creation and deletion.
*/
private final FileListener listener;
public DefaultFileMonitor(final FileListener listener)
{
this.listener = listener;
}
/**
* Access method to get the recursive setting when adding files for monitoring.
*/
public boolean isRecursive()
{
return this.recursive;
}
/**
* Access method to set the recursive setting when adding files for monitoring.
*/
public void setRecursive(final boolean newRecursive)
{
this.recursive = newRecursive;
}
/**
* Access method to get the current FileListener object notified when there
* are changes with the files added.
*/
FileListener getFileListener()
{
return this.listener;
}
/**
* Adds a file to be monitored.
*/
public void addFile(final FileObject file)
{
synchronized (this.monitorMap)
{
if (this.monitorMap.get(file.getName()) == null)
{
this.monitorMap.put(file.getName(), new FileMonitorAgent(this,
file));
try
{
if (this.listener != null)
{
file.getFileSystem().addListener(file, this.listener);
}
if (file.getType().hasChildren() && this.recursive)
{
// Traverse the children
final FileObject[] children = file.getChildren();
for (int i = 0; i < children.length; i++)
{
this.addFile(children[i]); // Add depth first
}
}
}
catch (FileSystemException fse)
{
log.error(fse.getLocalizedMessage(), fse);
}
}
}
}
/**
* Queues a file for addition to be monitored.
*/
protected void queueAddFile(final FileObject file)
{
this.mQueue.join(new ChangeRec(ChangeRec.ADD, file));
}
/**
* Removes a file from being monitored.
*/
public void removeFile(final FileObject file)
{
synchronized (this.monitorMap)
{
FileName fn = file.getName();
if (this.monitorMap.get(fn) != null)
{
this.monitorMap.remove(fn);
}
}
}
/**
* Queues a file for removal from being monitored.
* Resets the children list on queue instead on processing of removal
*/
protected void queueRemoveFile(final FileObject file)
{
synchronized (this.monitorMap)
{
FileObject parent;
try
{
parent = file.getParent();
}
catch (FileSystemException fse)
{
parent = null;
}
if (parent != null)
{ // Not the root
FileMonitorAgent parentAgent =
(FileMonitorAgent) this.monitorMap.get(parent.getName());
if (parentAgent != null)
{
parentAgent.resetChildrenList();
}
}
}
this.mQueue.join(new ChangeRec(ChangeRec.DELETE, file));
}
/**
* Get the delay between runs
*/
public long getDelay()
{
return delay;
}
/**
* Set the delay between runs
*/
public void setDelay(long delay)
{
if (delay > 0)
{
this.delay = delay;
}
else
{
this.delay = 1000;
}
}
/**
* get the number of files to check per run
*/
public int getChecksPerRun()
{
return checksPerRun;
}
/**
* set the number of files to check per run.
* a additional delay will be added if there are more files to check
*
* @param checksPerRun a value less than 1 will disable this feature
*/
public void setChecksPerRun(int checksPerRun)
{
this.checksPerRun = checksPerRun;
}
/**
* Starts monitoring the files that have been added.
*/
public void start()
{
if (this.monitorThread == null)
{
this.monitorThread = new Thread(this);
this.monitorThread.setDaemon(true);
this.monitorThread.setPriority(Thread.MIN_PRIORITY);
}
this.monitorThread.start();
}
/**
* Stops monitoring the files that have been added.
*/
public void stop()
{
this.shouldRun = false;
}
/**
* Asks the agent for each file being monitored to check its file for changes.
*/
public void run()
{
mainloop:
while (!Thread.currentThread().isInterrupted() && this.shouldRun)
{
while (!this.mQueue.isEmpty())
{
ChangeRec currRec = (ChangeRec)this.mQueue.leave();
if (currRec.type() == ChangeRec.DELETE)
{
this.removeFile((FileObject) currRec.file());
}
else if (currRec.type() == ChangeRec.ADD)
{
this.addFile((FileObject) currRec.file());
}
}
// For each entry in the map
Object fileNames[];
synchronized (this.monitorMap)
{
fileNames = this.monitorMap.keySet().toArray();
}
for (int iterFileNames = 0; iterFileNames < fileNames.length;
iterFileNames++)
{
FileName fileName = (FileName) fileNames[iterFileNames];
FileMonitorAgent agent;
synchronized (this.monitorMap)
{
agent = (FileMonitorAgent) this.monitorMap.get(fileName);
}
if (agent != null)
{
agent.check();
}
if (getChecksPerRun() > 0)
{
if ((iterFileNames % getChecksPerRun()) == 0)
{
try
{
Thread.sleep(getDelay());
}
catch (InterruptedException e)
{
}
}
}
if (Thread.currentThread().isInterrupted() || !this.shouldRun)
{
continue mainloop;
}
}
try
{
Thread.sleep(getDelay());
}
catch (InterruptedException e)
{
continue;
}
}
this.shouldRun = true;
}
/**
* A queue holding modifications that are to be done to the map.
*/
private static class ModificationQueue {
ArrayList store = new ArrayList();
public synchronized boolean join(Object o)
{
return store.add(o);
}
public synchronized Object leave()
{
Object result = null;
try
{
result = store.remove(0);
}
catch (Exception e)
{
result = null;
}
return result;
}
public synchronized int size()
{
return store.size();
}
public synchronized boolean isEmpty()
{
return store.isEmpty();
}
public synchronized Object peek()
{
Object result = null;
try
{
result = store.get(0);
}
catch (Exception e)
{
result = null;
}
return result;
}
public synchronized boolean containsKey(String key, KeyRetriever kr)
{
boolean result = false;
for (int i = 0; i < store.size(); i++)
{
if (kr.getKey(store.get(i)).equals(key))
{
result = true;
break;
}
}
return result;
}
public synchronized void clear()
{
store.clear();
}
}
private static class KeyRetriever {
public String getKey(Object o)
{
return ((ChangeRec)o).location();
}
}
/**
* Record holding an actionable record.
*/
private static class ChangeRec {
/** A delete action is to be performed. */
public static final int DELETE = 0;
/** An add action is to be performed. */
public static final int ADD = 1;
/** The type of action to perform for the document. */
private int type = -1;
/** A unique string identifier of object to be deleted. */
private String location;
/** The file that's changing. */
private FileObject file;
public ChangeRec(int ptype, FileObject pfile)
{
this.type = ptype;
this.file = pfile;
this.location = this.file.getName().toString();
}
public final String location()
{
return this.location;
}
public final int type()
{
return this.type;
}
public final FileObject file()
{
return this.file;
}
}
/**
* File monitor agent.
*/
private static class FileMonitorAgent
{
private final FileObject file;
private final DefaultFileMonitor fm;
private boolean exists;
private long timestamp;
private Map children = null;
private FileMonitorAgent(DefaultFileMonitor fm, FileObject file)
{
this.fm = fm;
this.file = file;
this.refresh();
this.resetChildrenList();
try
{
this.exists = this.file.exists();
}
catch (FileSystemException fse)
{
this.exists = false;
}
try
{
this.timestamp = this.file.getContent().getLastModifiedTime();
}
catch (FileSystemException fse)
{
this.timestamp = -1;
}
}
private void resetChildrenList()
{
try
{
this.refresh();
if (this.file.getType() == FileType.FOLDER)
{
this.children = new HashMap();
FileObject[] childrenList = this.file.getChildren();
for (int i = 0; i < childrenList.length; i++)
{
this.children.put(childrenList[i].getName(), new
Object()); // null?
}
}
}
catch (FileSystemException fse)
{
this.children = null;
}
}
/**
* Clear the cache and re-request the file object
*/
private void refresh()
{
try
{
// this.file = ((AbstractFileSystem)
this.file.getFileSystem()).resolveFile(this.file.getName(), false);
// close the file - this will detach and reattach its resources
(for this thread) on the
// next access
this.file.close();
}
catch (FileSystemException fse)
{
log.error(fse.getLocalizedMessage(), fse);
}
}
/**
* Recursively fires create events for all children if recursive descent is
* enabled. Otherwise the create event is only fired for the initial
* FileObject.
*/
private void fireAllCreate(FileObject child)
{
// Add listener so that it can be triggered
if (this.fm.getFileListener() != null)
{
child.getFileSystem().addListener(child, this.fm.getFileListener());
}
try
{
// If the child is not closed a file that is present, then deleted
from
// the file system, then added back would not fire a created event.
child.close();
}
catch (FileSystemException fse)
{
log.error(fse.getLocalizedMessage(), fse);
}
((AbstractFileSystem) child.getFileSystem()).fireFileCreated(child);
// Remove it because a listener is added in the queueAddFile
if (this.fm.getFileListener() != null)
{
child.getFileSystem().removeListener(child,
this.fm.getFileListener());
}
this.fm.queueAddFile(child); // Add
try
{
if (this.fm.isRecursive())
{
if (child.getType() == FileType.FOLDER)
{
FileObject[] newChildren = child.getChildren();
for (int i = 0; i < newChildren.length; i++)
{
fireAllCreate(newChildren[i]);
}
}
}
}
catch (FileSystemException fse)
{
log.error(fse.getLocalizedMessage(), fse);
}
}
/**
* Only checks for new children. If children are removed, they'll
* eventually be checked.
*/
private void checkForNewChildren()
{
try
{
if (this.file.getType() == FileType.FOLDER)
{
FileObject[] newChildren = this.file.getChildren();
if (this.children != null)
{
// See which new children are not listed in the current
children map.
Map newChildrenMap = new HashMap();
Stack missingChildren = new Stack();
for (int i = 0; i < newChildren.length; i++)
{
newChildrenMap.put(newChildren[i].getName(), new
Object()); // null ?
// If the child's not there
if
(!this.children.containsKey(newChildren[i].getName()))
{
missingChildren.push(newChildren[i]);
}
}
this.children = newChildrenMap;
// If there were missing children
if (!missingChildren.empty())
{
while (!missingChildren.empty())
{
FileObject child = (FileObject)
missingChildren.pop();
this.fireAllCreate(child);
}
}
}
else
{ // TODO: Check - I don't think this block is ever called.
// First set of children - Break out the cigars
if (newChildren.length > 0)
{
this.children = new HashMap();
}
for (int i = 0; i < newChildren.length; i++)
{
this.children.put(newChildren[i].getName(), new
Object()); // null?
this.fireAllCreate(newChildren[i]);
}
}
}
}
catch (FileSystemException fse)
{
log.error(fse.getLocalizedMessage(), fse);
}
}
private void check()
{
this.refresh();
try
{
// If the file existed and now doesn't
if (this.exists && !this.file.exists())
{
this.exists = this.file.exists();
this.timestamp = -1;
// Fire delete event
((AbstractFileSystem)
this.file.getFileSystem()).fireFileDeleted(this.file);
// Remove listener in case file is re-created. Don't want to
fire twice.
if (this.fm.getFileListener() != null)
{
this.file.getFileSystem().removeListener(this.file,
this.fm.getFileListener());
}
// Remove from map
this.fm.queueRemoveFile(this.file);
}
else if (this.exists && this.file.exists())
{
// Check the timestamp to see if it has been modified
if (this.timestamp !=
this.file.getContent().getLastModifiedTime())
{
this.timestamp =
this.file.getContent().getLastModifiedTime();
// Fire change event
// Don't fire if it's a folder because new file children
// and deleted files in a folder have their own event
triggered.
if (this.file.getType() != FileType.FOLDER)
{
((AbstractFileSystem)
this.file.getFileSystem()).fireFileChanged(this.file);
}
}
}
this.checkForNewChildren();
}
catch (FileSystemException fse)
{
log.error(fse.getLocalizedMessage(), fse);
}
}
}
}
--
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org