You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ant.apache.org by Magesh Umasankar <um...@apache.org> on 2001/12/22 01:43:48 UTC
Virtual FileSystem Layer
I have been spending some time now on the VFS
layer... Nothing major to report yet, but I just wanted
to sound off so that if I am going down the wrong
route, I correct it right away.
I evaluated at WebNFS, NetBeansFS (NBFS) and
JNDI.
1. WebNFS seems to be going nowhere. It has
been dormant for quite sometime now. Licensing
is rigid. Technically, it doesn't look so bad as it
closely replicates java.io.File's API. But then,
that really gives us very little.
2. NBFS looks OK. It has got a few filesystems
already built. There may be some licensing issues,
I don't know, but that shouldn't concern us too
much as, according to Peter, it is Mozilla (I haven't
really check the license out, sorry). But, as far as I
can see, it seems to lack in sophisticated API features
like searching based on attributes, etc., which
we will definitely be needing for the Selector APIs.
3. JNDI, by far, beats the above to, in my
evaluation. It is generic enough. We don't have
any licensing issues. It has also become part of
the core JRE (1.4 onwards). Technically, it fits to a T
what we are looking for - virtual file system that
provides search controls, access attributes,
url mounting, etc. Furthermore, there's been
some ground work already done for us at Jakarta/Apache
(Catalina). I have written a SPI for a FTPFileSystem
- though it is in a real crude stage right now. I believe
this is the way to go because Ant's code would be
operating at the (Dir)Context level and we can keep
adding SPIs as we need them. Furthermore,
JNDI has been stable for quite sometime now and
we can depend on a widely used API.
I don't think JNDI is a heavyweight API for our needs.
It seems to be the only one, so far, which encompasses
at the APIP level, all the new functionalities that we
desire to introduce.
Let me know if my approach, so far, to go the JNDI
route seems reasonable.
Cheers,
Magesh
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Virtual FileSystem Layer
Posted by Bill Burton <bi...@progress.com>.
Hi Magesh,
Magesh Umasankar wrote:
>
> From: "Bill Burton" <bi...@progress.com>
>
> > Has anyone looked at jEdit's VFS (http://www.jedit.org)? However, even if
>
> I did look at it, but felt that NetBeans FS provided better features,
> comparitively. But again, I may have missed something. WIll
> you please be able to provide some details on where jEdit's VFS
> wins over NetBeans's?
I couldn't speak to the differences as I'm not familiar with the NetBeans
FS. I brought up the jEdit VFS assuming it had been overlooked.
> Cheers,
> Magesh
-Bill
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Virtual FileSystem Layer
Posted by Magesh Umasankar <um...@apache.org>.
From: "Bill Burton" <bi...@progress.com>
> Has anyone looked at jEdit's VFS (http://www.jedit.org)? However, even if
I did look at it, but felt that NetBeans FS provided better features,
comparitively. But again, I may have missed something. WIll
you please be able to provide some details on where jEdit's VFS
wins over NetBeans's?
Cheers,
Magesh
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Virtual FileSystem Layer
Posted by Bill Burton <bi...@progress.com>.
Hello,
Has anyone looked at jEdit's VFS (http://www.jedit.org)? However, even if
it is technically good, it's under the GPL license--but maybe the author
could be persuaded to contribute/license the VFS functionality under the
ASF license?
Here are a few links to some of the VFS related functionality:
* Core I/O - support for file, filesystem roots, URL, favorite locations
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/jedit/jEdit/org/gjt/sp/jedit/io/
* FTP plugin -
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/jedit/plugins/FTP/ftp/
* SQL plugin -
http://plugins.jedit.org/plugins/SQL
BTW, jEdit supports Ant builds via the AntFarm plug-in.
-Bill
Magesh Umasankar wrote:
>
> I have been spending some time now on the VFS
> layer... Nothing major to report yet, but I just wanted
> to sound off so that if I am going down the wrong
> route, I correct it right away.
>
> I evaluated at WebNFS, NetBeansFS (NBFS) and
> JNDI.
>
> 1. WebNFS seems to be going nowhere. It has
> been dormant for quite sometime now. Licensing
> is rigid. Technically, it doesn't look so bad as it
> closely replicates java.io.File's API. But then,
> that really gives us very little.
>
> 2. NBFS looks OK. It has got a few filesystems
> already built. There may be some licensing issues,
> I don't know, but that shouldn't concern us too
> much as, according to Peter, it is Mozilla (I haven't
> really check the license out, sorry). But, as far as I
> can see, it seems to lack in sophisticated API features
> like searching based on attributes, etc., which
> we will definitely be needing for the Selector APIs.
>
> 3. JNDI, by far, beats the above to, in my
> evaluation. It is generic enough. We don't have
> any licensing issues. It has also become part of
> the core JRE (1.4 onwards). Technically, it fits to a T
> what we are looking for - virtual file system that
> provides search controls, access attributes,
> url mounting, etc. Furthermore, there's been
> some ground work already done for us at Jakarta/Apache
> (Catalina). I have written a SPI for a FTPFileSystem
> - though it is in a real crude stage right now. I believe
> this is the way to go because Ant's code would be
> operating at the (Dir)Context level and we can keep
> adding SPIs as we need them. Furthermore,
> JNDI has been stable for quite sometime now and
> we can depend on a widely used API.
>
> I don't think JNDI is a heavyweight API for our needs.
> It seems to be the only one, so far, which encompasses
> at the APIP level, all the new functionalities that we
> desire to introduce.
>
> Let me know if my approach, so far, to go the JNDI
> route seems reasonable.
>
> Cheers,
> Magesh
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Virtual FileSystem Layer
Posted by Peter Donald <pe...@apache.org>.
On Sat, 29 Dec 2001 06:50, Stefan Bodewig wrote:
> On Sat, 22 Dec 2001, Peter Donald <pe...@apache.org> wrote:
> > JNDI is nice. It will be slower - a lot slower - and a bit more
> > memory hungry ... whether this is an issue or not is another thing
> > altogether.
>
> This could become a big issue, at least the "a lot slower" part. All
> the crude "are we doing a slow scan" parts in DirectoryScanner are
> there because Ant 1.1 has simply been too slow when scanning large
> directory trees. Even now, most time of the javac task will be spent
> scanning the source tree for files that match **/*.java - the
> compilation itself will probably be a lot faster (at least when using
> jikes or modern). <javac>'s perfomance is one of Ant's big pros, so
> we shouldn't destroy that 8-)
>
> Any estimate how bad the performance of JNDI compared to direct file
> access is? Can we implement a hybrid solution that uses direct file
> system access when we know that we are talking about plain local
> files?
A lot of it will come down to how we actually implement things. (ie do we
keep the JNDI tree in memeory or create objects on demand or whatever). If we
cached the tree in memory then the speed of JNDI would actually be faster
that native file access (on most filesystems).
The cost of file access is significantly higher than constructing the JNDI
tree. It could also be slower (maybe 10%->1% depending on the JVM) due to the
allocation of lots of objects everytime you look up things and add things to
tree. FWIW I used to refer to JNDI as a object fountain because of this sort
of behaviour.
So I guess it really depends on how it is implemented. We could implement it
so that it is speedy as but it will always be more memory hungry. So when I
say slower it is not really in same league as slowscan/fastscan much less.
All comes down to code-fu.
However the main penalty of JNDI is that it is a generic interface. I don't
mind it because I have written oodles of providers, and dozens of apps that
use naming/directory services but it would not be as simple as a custom API.
So JNDI is
* more memory hungry
* maybe faster or slower depending on implementation
* much more complex an API (but standard so others may already know it).
--
Cheers,
Pete
----------------------------------
Ancient Go proverb: "Only amateurs
try to come up with 'fancy' moves"
----------------------------------
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Virtual FileSystem Layer
Posted by Jose Alberto Fernandez <j_...@yahoo.com>.
--- Stefan Bodewig <bo...@apache.org> wrote: > On
Sat, 22 Dec 2001, Peter Donald <pe...@apache.org>
> wrote:
>
> > JNDI is nice. It will be slower - a lot slower -
> and a bit more
> > memory hungry ... whether this is an issue or not
> is another thing
> > altogether.
>
> This could become a big issue, at least the "a lot
> slower" part. All
> the crude "are we doing a slow scan" parts in
> DirectoryScanner are
> there because Ant 1.1 has simply been too slow when
> scanning large
> directory trees. Even now, most time of the javac
> task will be spent
> scanning the source tree for files that match
> **/*.java - the
> compilation itself will probably be a lot faster (at
> least when using
> jikes or modern). <javac>'s perfomance is one of
> Ant's big pros, so
> we shouldn't destroy that 8-)
>
> Any estimate how bad the performance of JNDI
> compared to direct file
> access is? Can we implement a hybrid solution that
> uses direct file
> system access when we know that we are talking about
> plain local
> files?
>
JNDI is an empty shell. It all depends on the
implementation of the providers. They can be very
smart or they can be very stupid. For example the
directory search capabilities of JNDI can be
implemented in a provider so that it understands ´**´
and so on directly and evaluates them on the native
provider code (smart) or it can be scanning and
creating JNDI objects which are filtered after being
created (stupid).
The problem with JNDI is that it is easy to write
stupid providers, and it could be hard work to
implement smart ones.
Jose Alberto
__________________________________________________
Do You Yahoo!?
Everything you'll ever need on one web page
from News and Sport to Email and Music Charts
http://uk.my.yahoo.com
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Virtual FileSystem Layer
Posted by Adam Murdoch <ad...@yahoo.com>.
> From: Stefan Bodewig [mailto:bodewig@apache.org]
>
> Any estimate how bad the performance of JNDI compared to direct file
> access is? Can we implement a hybrid solution that uses direct file
> system access when we know that we are talking about plain local
> files?
>
This is a good idea.
Probably the best way to deal with this kind of issue is to put together our
own VFS API, and layer it on top of the file system providers (e.g. JNDI in
this case). This gives us the flexibility to use JNDI whilst prototyping
the VFS, and add in a direct file provider once it stabilises (though, I
would probably do it the other way round - start with direct file access and
add JNDI later).
Granted, we don't know whether JNDI performance is an issue or not.
However, technologies like JNDI (or java.io.File for that matter) almost
always require an app to compensate for bugs, performance problems, or
missing bits of functionality. This is especially likely in ant's case,
given how heavily dependent the tasks are on the file system (almost every
task touches the file system - and there are an awful lot of tasks), and the
range of platforms (OS/JVM) that Ant is used on. Having a single spot to
deal with these issues is far better than having them dealt with in every
task (inconsistently dealt with, of course).
An added bonus of using our own API is, we end up with an API that does what
the task writers actually need. Having a good semantic match means simpler
task implementations, more consistent behaviour, and less "how do i .."
questions. The file system is by far the most heavily used service in the
current set of tasks. There are a huge number of tasks, both built-in and
custom, and an equally large number of task developers. Making the file
system as easy to use as possible should be the primary motivation of the
VFS. Even if the implementation was limited to local files only, it would
still be a huge win.
Of course, there's only one way to know whether an API is useable or not,
and that is to write code against it. So, I'd like to do up a prototype of
an API that a task writer might use to get at the file system. This will
let us compare writing tasks against a custom API, and a JDNI based API (of
course, the custom API may well sit on top of JNDI).
Probably another useful thing for us to do is decide on a model for the file
system - that is, what is a "file"? what do they contain? how are they
structured? how are they named? what operations can be performed on them?
Here's a starting point:
- A file may have content, which is simply a chunk of binary data, plus some
attributes that describe the content:
- MIME type
- size
- encoding
- last modified timestamp
- etc.
- Files are arranged hierarchically - that is, a file has a single parent.
- A file can either have content or child files, but not both.
- A file is identified by a URI (including relative name + base URI).
- A file has some attributes:
- access control info
- etc
- Operations on a file:
- create.
- delete (including all child files).
- move to a new parent, rename, or both.
- get or set attributes.
- Operations on a file's content:
- read or write as a stream only (no random access).
- get or set attributes.
I don't think we need anything more detailed than that.
Adam
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Virtual FileSystem Layer
Posted by Stefan Bodewig <bo...@apache.org>.
On Sat, 22 Dec 2001, Peter Donald <pe...@apache.org> wrote:
> JNDI is nice. It will be slower - a lot slower - and a bit more
> memory hungry ... whether this is an issue or not is another thing
> altogether.
This could become a big issue, at least the "a lot slower" part. All
the crude "are we doing a slow scan" parts in DirectoryScanner are
there because Ant 1.1 has simply been too slow when scanning large
directory trees. Even now, most time of the javac task will be spent
scanning the source tree for files that match **/*.java - the
compilation itself will probably be a lot faster (at least when using
jikes or modern). <javac>'s perfomance is one of Ant's big pros, so
we shouldn't destroy that 8-)
Any estimate how bad the performance of JNDI compared to direct file
access is? Can we implement a hybrid solution that uses direct file
system access when we know that we are talking about plain local
files?
Stefan
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Virtual FileSystem Layer
Posted by Peter Donald <pe...@apache.org>.
On Sat, 22 Dec 2001 11:43, Magesh Umasankar wrote:
> I have been spending some time now on the VFS
> layer... Nothing major to report yet, but I just wanted
> to sound off so that if I am going down the wrong
> route, I correct it right away.
kool. I wouldn't mind seeing it checked into CVS just so we can browse your
progress aswell ;)
> 3. JNDI, by far, beats the above to, in my
> evaluation. It is generic enough. We don't have
> any licensing issues. It has also become part of
> the core JRE (1.4 onwards). Technically, it fits to a T
> what we are looking for - virtual file system that
> provides search controls, access attributes,
> url mounting, etc. Furthermore, there's been
> some ground work already done for us at Jakarta/Apache
> (Catalina). I have written a SPI for a FTPFileSystem
> - though it is in a real crude stage right now. I believe
> this is the way to go because Ant's code would be
> operating at the (Dir)Context level and we can keep
> adding SPIs as we need them. Furthermore,
> JNDI has been stable for quite sometime now and
> we can depend on a widely used API.
JNDI is nice. It will be slower - a lot slower - and a bit more memory hungry
... whether this is an issue or not is another thing altogether.
You may want to check out org.apache.avalon.excalibur.naming in the
jakarta-avalon-excalibur CVS. It is a framework for writing JNDI providers.
In actually used it to provide a File based Context a while back.
> Let me know if my approach, so far, to go the JNDI
> route seems reasonable.
hard to say at this stage. Just keep experimenting and checking stuff into
CVS and we can see where it goes.
--
Cheers,
Pete
*----------------------------------------------*
| The best defense against logic is ignorance. |
*----------------------------------------------*
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Virtual FileSystem Layer
Posted by Magesh Umasankar <um...@apache.org>.
From: "Adam Murdoch" <ad...@yahoo.com>
First of all, Adam, this material is quite something that
you have put together. While going through it, I couldn't
help but notice the similarities of what you have listed to
that available in the JNDI API, which in fact further
boosts my sales pitch for using JNDI ;-)
> The API also has to be general enough to deal with stuff we haven't
thought
> of yet. On that note, I personally think that JNDI might be a touch too
> general for what we need.
JNDI is designed to be a bit general, but it seems like we do
need *a lot* of what it provides. I will try to map your inputs to
APIs/nomenclature in the JNDI world.
> So, the features. Note that many of these will be optional - not every
> feature will be available for every node in the file system. I've used
the
javax.naming.OperationNotSupportedException will be thrown
by those service providers which do not implement a certain
method.
> term "node" to mean both directories and files. I'm not suggesting we
Node is roughly equivalent to DirContext in the JNDI world. DirContext
is a collection of sub DirContexts and 'FileResources' (aka files).
> actually call them "nodes" in the API. I've used the term "root node" to
> mean the root of a file system.
Docbase for a DirContext is its root.
> * Naming
>
> - Locate a node by absolute name.
Context.PROVIDER_URL can be used to locate by abolute name
or to identify the docbase, apart from resolving protocol, userid,
password, etc.
> - Get the absolute name for a node.
> - Resolve a name to a node, relative to some base node - like
> FileUtils.resolveFile().
> - Get the relative name of a node, relative to some base node.
> - Determine the base name (with and without the extension), and extension
of
> the node.
> - Deal with file systems that are case sensitive, and case insentitive.
Custom APIs implemented when extending the DirContext. Also
implementing lookup, lookuplink, etc. would solve portions of this.
> * Properties
Properties map to DirContext "Attributes" in JNDI and you
can perform all the operations you have mentioned below by
implementing the Attribute interface.
> - Determine what properties are available on the node.
> - Determine if the node exists.
> - Determine the type of node (file vs. directory, could be "has-content"
vs
> "has-children").
> - Determine if the node is readable.
> - Determine if the node is writeable.
> - Get/set the permissions on the node. This covers things like chmod &
> chown, making read-only, making executable, etc.
>
> * Content
>
> - Determine if the node can/does have content.
> - Get the size of the node.
> - Get/set the last-modified time of the node.
> - Get/set the mime-type of the node.
> - Get/set the encoding of the node.
> - Get a checksum of the node.
> - Get content as InputStream.
> - Get content as Reader.
> - Set content as an OutputStream.
> - Set content as a Writer.
> - Implicit creation of node and its ancestors when content is written.
> - Compare nodes for equality (last modified timestamp, checksum, bytewise
> compare).
You would be able to handle streams using the FileResource.
>
> * Hierarchy
>
> - Get the parent node of a node.
> - Get the child nodes of a node.
lookup, lookuplink...
> - Iterate over (or visit) the descendants of a node.
> - With or without a selector.
> - In various orders - depthwise, etc.
> - Be able to modify the nodes during traversal.
Implementation of DirContext's search methoids by defining
SearchControls.
>
> * Modification
>
> - Create a new node of a particular type. Create all missing ancestors.
createSubContext
> - Move, copy, delete a node.
DirContext.destroySubContext, rename, unbind, rebind, etc.
> - All descendants.
> - Optional selector. E.g. ignore empty dirs, ignore default excludes,
etc.
> - Optional filter.
DirContext.modifyAttributes
> * Conversion
>
> - Convert the node to a java.net.URL.
> - Make the node content available as a local file (to hand off to external
> commands).
> - Get the OS specific *filename* for a node.
> - Resolve an OS specific *filename* to a node.
Custom APIs.
>
> * File System Types
>
> - Local file.
> - HTTP.
> - FTP.
> - Classloader, uses Classloader.getResource().
> - Temporary files.
> - etc ...
These will be individual JNDI Service Providers.
> - Compound file system. Made up of a bunch of mount points. The VFS
> itself.
>
> - Layered file systems (that sit on top of another file system or node):
> - zip, bzip, jar, tar
This is equivalent CompositeName in the JNDI world.
> - filtering - token replacement, etc
>
> - Factories for creating and configuring file system root nodes.
InitialContextFactory in JNDI...
> - Ability to easily add new file system implementations.
It would take defining a JNDI ServiceProvider...
> * Other things
>
> - Maybe some way to explicitly close a node and release all resources used
> by it.
>
> - Maybe detection of concurrent updates, in the case of parallel tasks.
Implementing SPI in a threaded model.
>
> - Netbeans has an event model in its VFS. Something like this might be
> useful in dependency management.
JNDI too has an event model. javax.naming.event, which offers
sophisticated functionality.
Summarizing, I just can't help but notice the similarities that
JNDI provides to what you have put together. I don't want us to
reinvent the wheel if we can avoid it.
Cheers,
Magesh
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Virtual FileSystem Layer
Posted by Peter Donald <pe...@apache.org>.
Hi,
Brilliant! I just copied your email and added it into CVS under
docs/ant2/VFS.txt - hope thats ok ;)
On Sat, 22 Dec 2001 13:06, Adam Murdoch wrote:
...snip...
> What's missing? What shouldn't be on the list?
I guess there was only two other things you missed that I would add - both
are related. These are
* efficient way to search a particular node type. Different node types will
require different strategies to search and index and thus we will need to be
able to create particular search controls/scanners for different node types.
The nodesets return by these search controls may not even be fully resolved
until you actually try to access elements from the nodelist. Does that make
sense?
* caching of indexing information. Ideally you don't want to load the index
for a zip over and over again. Nor do you want to keep index information for
all VFS nodes in memory. We need an efficient caching mechanism for this sort
of stuff.
--
Cheers,
Pete
---------------------------------------------------
Murphy's law - "Anything that can go wrong, will."
(Actually, this is Finagle's law, which in itself
shows that Finagle was right.)
---------------------------------------------------
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Virtual FileSystem Layer
Posted by Adam Murdoch <ad...@yahoo.com>.
Hi,
I've also been doing a bit of work on the VFS. No code yet - instead, I've
done a survey of the Ant 1 code, to help get a better idea of what we need
the VFS to actually do.
I've put together a rough list of the sort of features the current tasks
require from the file system. This list is entirely from the task writer's
POV. I've ignored the build file writer completely - though, the action
list is a good summary of the build file writer's concerns. I've tried to
steer clear of assumptions about what is actually going to provide each
feature to the tasks, or what the API will look like to the tasks.
The goal for doing up this list, was to help identify the features we want
to support, and the API that the tasks will use to get at them. This should
be largely independent of how we decide to represent the file system in the
build files. In addition, it doesn't matter too much whether the list below
is complete (I'm sure it isn't), or that the VFS provide every single one of
the features. Whatever it doesn't provide, can stay up in the tasks, and be
refactored down later.
The assumption here is that we do actually want to put together a file
system API. I think it's a good idea to at least put together some
interfaces, even if the implementation is stolen from somewhere else.
Without a doubt, the file system is the most widely used "service" in the
current crop of tasks. The API that we choose has to have a good semantic
match with what the tasks need to do, so that writing the tasks is easy.
The API also has to be general enough to deal with stuff we haven't thought
of yet. On that note, I personally think that JNDI might be a touch too
general for what we need.
So, the features. Note that many of these will be optional - not every
feature will be available for every node in the file system. I've used the
term "node" to mean both directories and files. I'm not suggesting we
actually call them "nodes" in the API. I've used the term "root node" to
mean the root of a file system.
* Naming
- Locate a node by absolute name.
- Get the absolute name for a node.
- Resolve a name to a node, relative to some base node - like
FileUtils.resolveFile().
- Get the relative name of a node, relative to some base node.
- Determine the base name (with and without the extension), and extension of
the node.
- Deal with file systems that are case sensitive, and case insentitive.
* Properties
- Determine what properties are available on the node.
- Determine if the node exists.
- Determine the type of node (file vs. directory, could be "has-content" vs
"has-children").
- Determine if the node is readable.
- Determine if the node is writeable.
- Get/set the permissions on the node. This covers things like chmod &
chown, making read-only, making executable, etc.
* Content
- Determine if the node can/does have content.
- Get the size of the node.
- Get/set the last-modified time of the node.
- Get/set the mime-type of the node.
- Get/set the encoding of the node.
- Get a checksum of the node.
- Get content as InputStream.
- Get content as Reader.
- Set content as an OutputStream.
- Set content as a Writer.
- Implicit creation of node and its ancestors when content is written.
- Compare nodes for equality (last modified timestamp, checksum, bytewise
compare).
* Hierarchy
- Get the parent node of a node.
- Get the child nodes of a node.
- Iterate over (or visit) the descendants of a node.
- With or without a selector.
- In various orders - depthwise, etc.
- Be able to modify the nodes during traversal.
* Modification
- Create a new node of a particular type. Create all missing ancestors.
- Move, copy, delete a node.
- All descendants.
- Optional selector. E.g. ignore empty dirs, ignore default excludes, etc.
- Optional filter.
* Conversion
- Convert the node to a java.net.URL.
- Make the node content available as a local file (to hand off to external
commands).
- Get the OS specific *filename* for a node.
- Resolve an OS specific *filename* to a node.
* File System Types
- Local file.
- HTTP.
- FTP.
- Classloader, uses Classloader.getResource().
- Temporary files.
- etc ...
- Compound file system. Made up of a bunch of mount points. The VFS
itself.
- Layered file systems (that sit on top of another file system or node):
- zip, bzip, jar, tar
- filtering - token replacement, etc
- Factories for creating and configuring file system root nodes.
- Ability to easily add new file system implementations.
* Task Container
- A mechanism for a task to get hold of the project's root node.
- A mechanism that allows a task to create its own private root nodes,
without letting it mess with the project's file system, or the file systems
of other tasks.
- A mechanism for cleaning up all the node InputStream, OutputStream, Reader
and Writers opened by a task during its execute() method. Cleaning up files
is one thing the current tasks don't do very well at all. Something like
this would take care of anything the task did not explictly close. Would
include root nodes it created.
* Other things
- Maybe some way to explicitly close a node and release all resources used
by it.
- Maybe detection of concurrent updates, in the case of parallel tasks.
- Netbeans has an event model in its VFS. Something like this might be
useful in dependency management.
- nodesets. The replacement for, or generalisation of, FileSet, Path,
FileList, etc
- A nodeset that contains the descendents of a node that match a selector
(like the current FileSet implementation).
- A nodeset that contains arbitrary nodes.
- An aggregating nodeset.
- Custom nodeset implementations.
- Reimplement the Ant 1 Fileset, Path and Filelist as adaptors sitting on
top of the VFS.
- A classloader that can load classes from a node.
- etc ..
What's missing? What shouldn't be on the list?
Adam
> -----Original Message-----
> From: Magesh Umasankar [mailto:umagesh@apache.org]
> Sent: Saturday, 22 December 2001 10:44 AM
> To: ant-dev@jakarta.apache.org
> Subject: Virtual FileSystem Layer
>
>
> I have been spending some time now on the VFS
> layer... Nothing major to report yet, but I just wanted
> to sound off so that if I am going down the wrong
> route, I correct it right away.
>
> I evaluated at WebNFS, NetBeansFS (NBFS) and
> JNDI.
>
> 1. WebNFS seems to be going nowhere. It has
> been dormant for quite sometime now. Licensing
> is rigid. Technically, it doesn't look so bad as it
> closely replicates java.io.File's API. But then,
> that really gives us very little.
>
> 2. NBFS looks OK. It has got a few filesystems
> already built. There may be some licensing issues,
> I don't know, but that shouldn't concern us too
> much as, according to Peter, it is Mozilla (I haven't
> really check the license out, sorry). But, as far as I
> can see, it seems to lack in sophisticated API features
> like searching based on attributes, etc., which
> we will definitely be needing for the Selector APIs.
>
> 3. JNDI, by far, beats the above to, in my
> evaluation. It is generic enough. We don't have
> any licensing issues. It has also become part of
> the core JRE (1.4 onwards). Technically, it fits to a T
> what we are looking for - virtual file system that
> provides search controls, access attributes,
> url mounting, etc. Furthermore, there's been
> some ground work already done for us at Jakarta/Apache
> (Catalina). I have written a SPI for a FTPFileSystem
> - though it is in a real crude stage right now. I believe
> this is the way to go because Ant's code would be
> operating at the (Dir)Context level and we can keep
> adding SPIs as we need them. Furthermore,
> JNDI has been stable for quite sometime now and
> we can depend on a widely used API.
>
> I don't think JNDI is a heavyweight API for our needs.
> It seems to be the only one, so far, which encompasses
> at the APIP level, all the new functionalities that we
> desire to introduce.
>
> Let me know if my approach, so far, to go the JNDI
> route seems reasonable.
>
> Cheers,
> Magesh
>
>
>
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>