You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by "Michael D. Hirsch" <mh...@nubridges.com> on 2004/09/17 20:01:07 UTC
[VFS and Net] Dealing with non-standard FTP servers
Hello,
I'm new to this list, but I've been a happy user of commons-vfs for several
months, now. It's saved me tons of work, but now I'm running up against what
I think are its limits. I'm looking for advice on how to get around a
problem.
I am is using commons-vfs to retrieve files from various sites. In
particular, I'm now trying to get files from a server which is not supported
in the standard collection of FTPFileEntryParsers.
There are several obstacles for me to do this--some are of my own making and
others are in the commons code. I'll try to separate them out.
The main problem is that even though the server presents non-standard file
lists, it claims to be of system type "unix". This means that commons-net
will attempt to parse it like a unix server, and that doesn't work on this
output.
I could define my own FtpFileEntryParser and register it with
DefaultFTPFileEntryParserFactory, but there doesn't seem to be a way to
register new parsers--it appears to be hard coded into createFileEntryParser
in DefaultFTPFileEntryParserFactory. Even worse, that wouldn't work cause I
can't distinguish this server from a unix server.
Another option is to explicitly pass the FQCN for my new parser into
DefaultFTPFileEntryParserFactory, but there I'm stymied because, AFAICT,
there is no way to pass the FQCN through the vfs layer to the net layer.
I keep hoping that I'm just missing something in the API that lets me tell VFS
what I really want to do at the net layer. I need to be able to set the FQCN
for the FTPFileEntryParser to use on a given connection.
Does anyone have any suggestions for how to proceed? I'm not opposed to
changing the vfs or net code if you think the changes would be accepted back
into CVS.
Thanks,
Michael Hirsch
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [VFS and Net] Dealing with non-standard FTP servers
Posted by "Jeffrey D. Brekke" <jb...@wi.rr.com>.
>>>>> On Fri, 17 Sep 2004 14:01:07 -0400, "Michael D. Hirsch" <mh...@nubridges.com> said:
> Does anyone have any suggestions for how to proceed? I'm not
> opposed to changing the vfs or net code if you think the changes
> would be accepted back into CVS.
If you have ideas on how the api could be cleaner, you're welcome to
present your ideas. I know the commons-net developers are a pretty
busy bunch, but we do get around to looking at the posts eventually.
--
=====================================================================
Jeffrey D. Brekke jbrekke@wi.rr.com
Wisconsin, USA brekke@apache.org
ekkerbj@yahoo.com
ekkerbj@gmail.com
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [VFS and Net] Dealing with non-standard FTP servers
Posted by Mario Ivankovits <ma...@ops.co.at>.
Steve Cohen wrote:
>Thinking over our small disagreement, it occurs to me that's what's behind it
>is my feeling that a need has not been shown thus far for parser factories
>other than the default.
>
>> ---tail cuted---
>
Now i got you, and must admit you are right.
I thought the method name should transport the real possible function
behind, but now i understand that this is mostly not what the users want
- in fact they want to set a custom parser (or help the host detection
mechanism) only for a single connection - not more ore less.
Even if it is just a method name - it is interresting how things could
change if people talk (write) together.
Thanks!
Ciao,
Mario
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [VFS and Net] Dealing with non-standard FTP servers
Posted by Steve Cohen <sc...@javactivity.org>.
On Monday 20 September 2004 6:30 am, Mario Ivankovits wrote:
> Steve Cohen wrote:
> >Well, I don't really agree with you about your reasons not to like
> >setParserKey(). It's a key that ultimately gets translated into a parser.
>
> Thats right as long as the user uses the default factory implementation
> else it is the key which is passed to the factory - whatever it does
> with this data.
>
> >Still, I'm all for "shorter is better" and therefore I am in favor of the
> > two renamings you propose as improvements over the current names.
>
> Well then, lets do it.
>
That's good.
Thinking over our small disagreement, it occurs to me that's what's behind it
is my feeling that a need has not been shown thus far for parser factories
other than the default. Everything we've thus far been asked to do was
doable using just the default parser factory. Thus far, the
DefaultFTPFileEntryParserFactory has proven itself sufficiently flexible to
do the job.
I understand that ultimately such a need may arise, but until it does, my
preference would be to avoid unnecessary complexity and not feature the
factory concept so prominently; rather we could remain secure in the
knowledge that it's there if we ever need it.
Writing a new parser is beyond the scope of most developers. We can count on
one hand the number that have been written and contributed. Still further
beyond that scope is the writing of parser factories.
That is where I was coming from.
But as I said, I am okay with your changes.
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [VFS and Net] Dealing with non-standard FTP servers
Posted by Mario Ivankovits <ma...@ops.co.at>.
Steve Cohen wrote:
>Well, I don't really agree with you about your reasons not to like
>setParserKey(). It's a key that ultimately gets translated into a parser.
>
Thats right as long as the user uses the default factory implementation
else it is the key which is passed to the factory - whatever it does
with this data.
>Still, I'm all for "shorter is better" and therefore I am in favor of the two
>renamings you propose as improvements over the current names.
>
>
Well then, lets do it.
---
Mario
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [VFS and Net] Dealing with non-standard FTP servers
Posted by Steve Cohen <sc...@javactivity.org>.
Well, I don't really agree with you about your reasons not to like
setParserKey(). It's a key that ultimately gets translated into a parser.
The name doesn't have to imply ownership, and I'm still not sure what the
problem would be even if a VFS user thought it did. If the user cares about
the underlying object model, he's got the source. But it's up to you.
Still, I'm all for "shorter is better" and therefore I am in favor of the two
renamings you propose as improvements over the current names.
Steve
On Monday 20 September 2004 1:11 am, Mario Ivankovits wrote:
> Steve Cohen wrote:
> >I must say, though that the name of your function is as ugly a method name
> > as any I've ever seen.
>
> I know it, i hate it too, but ...
>
> For sure, i can drop the word FTP as it is redundant (counted to how
> often the word FTP appears in the FQCN of the builder-class)
>
> I dont like "setParserKey()" as this might look like it is an argument
> to the parser class and not to its factory. Even if this should not
> concern VFS - this is how it is.
>
> Next thing is "setParserFactoryKey()", but then - which parser factory
> is meant here ?
>
> What do you think about renaming
> *) setFTPFileEntryParserFactoryKey() to setEntryParserFactoryKey()
> *) setFTPFileEntryParserFactory() to setEntryParserFactory()
> ?
>
> --
> Mario
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [VFS and Net] Dealing with non-standard FTP servers
Posted by Mario Ivankovits <ma...@ops.co.at>.
Steve Cohen wrote:
>I must say, though that the name of your function is as ugly a method name as
>any I've ever seen.
>
I know it, i hate it too, but ...
For sure, i can drop the word FTP as it is redundant (counted to how
often the word FTP appears in the FQCN of the builder-class)
I dont like "setParserKey()" as this might look like it is an argument
to the parser class and not to its factory. Even if this should not
concern VFS - this is how it is.
Next thing is "setParserFactoryKey()", but then - which parser factory
is meant here ?
What do you think about renaming
*) setFTPFileEntryParserFactoryKey() to setEntryParserFactoryKey()
*) setFTPFileEntryParserFactory() to setEntryParserFactory()
?
--
Mario
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [VFS and Net] Dealing with non-standard FTP servers
Posted by Steve Cohen <sc...@javactivity.org>.
You da man! as we used to say here in the States. Nice work. And so fast. I
appreciate your taking the suggestion. I wrote the hook and nobody was using
it.
I must say, though that the name of your function is as ugly a method name as
any I've ever seen. I try to make it a rule to limit method names to four
nouns. You have six, or eight if you expand "FTP". I know, I know, most of
those nouns came from me. But still! God forbid we should combine this
factory with something else! Once the name goes over 80 chars we won't be
able to write the method name in code without getting a nag from the style
gurus. :-)
All kidding aside, I think setParserKey() would be a better name for your
method. What is being set is the key that will be used to instantiate a
parser. The rest is internal details. The fact that this is happening
through a factory should not be of concern to users of VFS. The FTP part of
the method is implied by the name of your class.
On Sunday 19 September 2004 1:30 pm, Mario Ivankovits wrote:
> Steve Cohen wrote:
> >Sorry, please disregard my earlier reply. I didn't read your original
> >carefully enough. Mario has provided one workaround. I would suggest
> > that VFS should be enhanced to take advantage of this little-known
> > capability of Commons-Net.
>
> Done!
>
> FtpFileSystemConfigBuilder.setFTPFileEntryParserFactoryKey()
> setFTPFileEntryParserFactoryKey: allows to set the "key" used in
> FTPFileEntryParserFactory. In case of the default commons-net
> implementation this can be a FQCN (which could be your own
> implementation) or one of the already known ftp-entry parsers - see
> org.apache.commons.net.ftp.parser.DefaultFTPFileEntryParserFactory for
> details.
>
>
> Ciao,
> Mario
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [VFS and Net] Dealing with non-standard FTP servers
Posted by Mario Ivankovits <ma...@ops.co.at>.
Steve Cohen wrote:
>Sorry, please disregard my earlier reply. I didn't read your original
>carefully enough. Mario has provided one workaround. I would suggest that
>VFS should be enhanced to take advantage of this little-known capability of
>Commons-Net.
>
>
Done!
FtpFileSystemConfigBuilder.setFTPFileEntryParserFactoryKey()
setFTPFileEntryParserFactoryKey: allows to set the "key" used in
FTPFileEntryParserFactory. In case of the default commons-net
implementation this can be a FQCN (which could be your own
implementation) or one of the already known ftp-entry parsers - see
org.apache.commons.net.ftp.parser.DefaultFTPFileEntryParserFactory for
details.
Ciao,
Mario
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [VFS and Net] Dealing with non-standard FTP servers
Posted by Steve Cohen <sc...@javactivity.org>.
Sorry, please disregard my earlier reply. I didn't read your original
carefully enough. Mario has provided one workaround. I would suggest that
VFS should be enhanced to take advantage of this little-known capability of
Commons-Net. I have had it in the back of my mind to do the same for Ant but
have not found the time yet.
On Sunday 19 September 2004 9:53 am, Steve Cohen wrote:
> On Friday 17 September 2004 1:01 pm, Michael D. Hirsch wrote:
> > Hello,
> >
> > I'm new to this list, but I've been a happy user of commons-vfs for
> > several months, now. It's saved me tons of work, but now I'm running up
> > against what I think are its limits. I'm looking for advice on how to
> > get around a problem.
> >
> > I am is using commons-vfs to retrieve files from various sites. In
> > particular, I'm now trying to get files from a server which is not
> > supported in the standard collection of FTPFileEntryParsers.
> >
> > There are several obstacles for me to do this--some are of my own making
> > and others are in the commons code. I'll try to separate them out.
> >
> > The main problem is that even though the server presents non-standard
> > file lists, it claims to be of system type "unix". This means that
> > commons-net will attempt to parse it like a unix server, and that
> > doesn't work on this output.
> >
> > I could define my own FtpFileEntryParser and register it with
> > DefaultFTPFileEntryParserFactory, but there doesn't seem to be a way to
> > register new parsers--it appears to be hard coded into
> > createFileEntryParser in DefaultFTPFileEntryParserFactory. Even worse,
> > that wouldn't work cause I can't distinguish this server from a unix
> > server.
> >
> > Another option is to explicitly pass the FQCN for my new parser into
> > DefaultFTPFileEntryParserFactory, but there I'm stymied because, AFAICT,
> > there is no way to pass the FQCN through the vfs layer to the net layer.
> >
> > I keep hoping that I'm just missing something in the API that lets me
> > tell VFS what I really want to do at the net layer. I need to be able to
> > set the FQCN for the FTPFileEntryParser to use on a given connection.
> >
> > Does anyone have any suggestions for how to proceed? I'm not opposed to
> > changing the vfs or net code if you think the changes would be accepted
> > back into CVS.
> >
> > Thanks,
> >
> > Michael Hirsch
>
> You are missing something in the API.
> Here is the source for
> DefaultFTPFileEntryParserFactory.createFileEntryParser():
>
> Note that the first thing it tries to do is resolve the key parameter as a
> fully qualified class name of a class derived from FTPFileEntryParser.
> Only when that fails, does the behavior of parsing key as a SYST return
> value kick in.
>
> public FTPFileEntryParser createFileEntryParser(String key)
> {
> Class parserClass = null;
> try
> {
> parserClass = Class.forName(key);
> return (FTPFileEntryParser) parserClass.newInstance();
> }
> catch (ClassNotFoundException e)
> {
> String ukey = null;
> if (null != key)
> {
> ukey = key.toUpperCase();
> }
> if (ukey.indexOf("UNIX") >= 0)
> {
> return createUnixFTPEntryParser();
> }
> else if (ukey.indexOf("VMS") >= 0)
> {
> return createVMSVersioningFTPEntryParser();
> }
> ...
>
> So how would you use this to create and use your own parser?
> The easiest way is to call
> FTPClient.listFiles(String parserKey, String pathname) passing in the fully
> qualified name of your class as the first parameter.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [VFS and Net] Dealing with non-standard FTP servers
Posted by Steve Cohen <sc...@javactivity.org>.
On Friday 17 September 2004 1:01 pm, Michael D. Hirsch wrote:
> Hello,
>
> I'm new to this list, but I've been a happy user of commons-vfs for several
> months, now. It's saved me tons of work, but now I'm running up against
> what I think are its limits. I'm looking for advice on how to get around a
> problem.
>
> I am is using commons-vfs to retrieve files from various sites. In
> particular, I'm now trying to get files from a server which is not
> supported in the standard collection of FTPFileEntryParsers.
>
> There are several obstacles for me to do this--some are of my own making
> and others are in the commons code. I'll try to separate them out.
>
> The main problem is that even though the server presents non-standard file
> lists, it claims to be of system type "unix". This means that commons-net
> will attempt to parse it like a unix server, and that doesn't work on this
> output.
>
> I could define my own FtpFileEntryParser and register it with
> DefaultFTPFileEntryParserFactory, but there doesn't seem to be a way to
> register new parsers--it appears to be hard coded into
> createFileEntryParser in DefaultFTPFileEntryParserFactory. Even worse,
> that wouldn't work cause I can't distinguish this server from a unix
> server.
>
> Another option is to explicitly pass the FQCN for my new parser into
> DefaultFTPFileEntryParserFactory, but there I'm stymied because, AFAICT,
> there is no way to pass the FQCN through the vfs layer to the net layer.
>
> I keep hoping that I'm just missing something in the API that lets me tell
> VFS what I really want to do at the net layer. I need to be able to set
> the FQCN for the FTPFileEntryParser to use on a given connection.
>
> Does anyone have any suggestions for how to proceed? I'm not opposed to
> changing the vfs or net code if you think the changes would be accepted
> back into CVS.
>
> Thanks,
>
> Michael Hirsch
You are missing something in the API.
Here is the source for
DefaultFTPFileEntryParserFactory.createFileEntryParser():
Note that the first thing it tries to do is resolve the key parameter as a
fully qualified class name of a class derived from FTPFileEntryParser. Only
when that fails, does the behavior of parsing key as a SYST return value kick
in.
public FTPFileEntryParser createFileEntryParser(String key)
{
Class parserClass = null;
try
{
parserClass = Class.forName(key);
return (FTPFileEntryParser) parserClass.newInstance();
}
catch (ClassNotFoundException e)
{
String ukey = null;
if (null != key)
{
ukey = key.toUpperCase();
}
if (ukey.indexOf("UNIX") >= 0)
{
return createUnixFTPEntryParser();
}
else if (ukey.indexOf("VMS") >= 0)
{
return createVMSVersioningFTPEntryParser();
}
...
So how would you use this to create and use your own parser?
The easiest way is to call
FTPClient.listFiles(String parserKey, String pathname) passing in the fully
qualified name of your class as the first parameter.
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [VFS and Net] Dealing with non-standard FTP servers
Posted by "Michael D. Hirsch" <mh...@nubridges.com>.
On Friday 17 September 2004 02:30 pm, Mario Ivankovits wrote:
> Michael D. Hirsch wrote:
> >Another option is to explicitly pass the FQCN for my new parser into
> >DefaultFTPFileEntryParserFactory, but there I'm stymied because, AFAICT,
> >there is no way to pass the FQCN through the vfs layer to the net layer.
>
> You cant pass directly a FTPFileEntryParser, but you can pass your own
> FtpFileEntryParserFactory which in turn can create your FTPFileEntryParser.
>
> FileSystemOptions fso = new FileSystemOptions();
>
> FtpFileSystemConfigBuilder.getInstance().setFTPFileEntryParserFactory(fso,
> new YourFtpFileEntryParserFactory());
> FileObject fo =
> VFS.getManager().resolveFile("ftp://to/your/file", fso);
> FileObject fo2 = fo.resolveFile("a_child");
> FileObject fo3 =
> VFS.getManager().resolveFile("ftp://to/your/file");
>
> In this example the filesystem configuration will be used for the
> objects "fo" and "fo2", but not "fo3" as this will create a new
> filesystem without special options.
>
> Hope this helps.
Indeed. This might do the trick for me. I'll give it a whirl and see how it
works.
Thanks,
Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [VFS and Net] Dealing with non-standard FTP servers
Posted by Mario Ivankovits <ma...@ops.co.at>.
Michael D. Hirsch wrote:
>Another option is to explicitly pass the FQCN for my new parser into
>DefaultFTPFileEntryParserFactory, but there I'm stymied because, AFAICT,
>there is no way to pass the FQCN through the vfs layer to the net layer.
>
>
You cant pass directly a FTPFileEntryParser, but you can pass your own
FtpFileEntryParserFactory which in turn can create your FTPFileEntryParser.
FileSystemOptions fso = new FileSystemOptions();
FtpFileSystemConfigBuilder.getInstance().setFTPFileEntryParserFactory(fso,
new YourFtpFileEntryParserFactory());
FileObject fo =
VFS.getManager().resolveFile("ftp://to/your/file", fso);
FileObject fo2 = fo.resolveFile("a_child");
FileObject fo3 = VFS.getManager().resolveFile("ftp://to/your/file");
In this example the filesystem configuration will be used for the
objects "fo" and "fo2", but not "fo3" as this will create a new
filesystem without special options.
Hope this helps.
Mario
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org