You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by dl...@comcast.net on 2012/09/20 04:04:28 UTC
[VFS] HDFS provider
I did not see a provider for HDFS on the list of supported file systems. I created a read-only HDFS provider while working on an Accumulo ticket. I was wondering if one of the devs had time to review to see if I made any glaring mistakes. I am in the process of becoming an Accumulo committer, so I'm assuming that this file system provider could be included in the VFS project if it makes sense. Thanks,
Dave Marion
Re: [VFS] HDFS provider
Posted by Gary Gregory <ga...@gmail.com>.
I would welcome a new file systems too. The tricky part is unit testing.
Most file systems are now tested in the build with embedded servers, for
examples see HTTP, FTP, SFTP and WebDAV.
Did you create your FS with unit tests? If yes, in the same style as VFS?
Can you use an embedded server?
JIRA is the place to start for a submission.
Thank you,
Gary
On Wed, Sep 19, 2012 at 10:04 PM, <dl...@comcast.net> wrote:
>
> I did not see a provider for HDFS on the list of supported file systems. I
> created a read-only HDFS provider while working on an Accumulo ticket. I
> was wondering if one of the devs had time to review to see if I made any
> glaring mistakes. I am in the process of becoming an Accumulo committer, so
> I'm assuming that this file system provider could be included in the VFS
> project if it makes sense. Thanks,
>
> Dave Marion
>
--
E-Mail: garydgregory@gmail.com | ggregory@apache.org
JUnit in Action, 2nd Ed: <http://goog_1249600977>http://bit.ly/ECvg0
Spring Batch in Action: <http://s.apache.org/HOq>http://bit.ly/bqpbCK
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory
Re: [VFS] HDFS provider
Posted by Simone Tripodi <si...@apache.org>.
Hi Dave!
Welcome to commons! :)
What you are proposing makes a lot of sense IMHO! Unfortunately I am
not so familiar with VFS to review & apply your modifications, guess
it would be easier to do if you could provide a patch via JIRA.
Good luck and all the best!
-Simo
http://people.apache.org/~simonetripodi/
http://simonetripodi.livejournal.com/
http://twitter.com/simonetripodi
http://www.99soft.org/
On Thu, Sep 20, 2012 at 4:04 AM, <dl...@comcast.net> wrote:
>
> I did not see a provider for HDFS on the list of supported file systems. I created a read-only HDFS provider while working on an Accumulo ticket. I was wondering if one of the devs had time to review to see if I made any glaring mistakes. I am in the process of becoming an Accumulo committer, so I'm assuming that this file system provider could be included in the VFS project if it makes sense. Thanks,
>
> Dave Marion
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
RE: [VFS] HDFS provider
Posted by Dave Marion <dl...@comcast.net>.
For those that were interested, I submitted an initial patch in VFS-442. I'm working on the suggested changes.
Dave
-----Original Message-----
From: dlmarion@comcast.net [mailto:dlmarion@comcast.net]
Sent: Thursday, September 20, 2012 1:17 PM
To: dev@commons.apache.org
Subject: Re: [VFS] HDFS provider
I forgot to subscribe to the dev mailing list before sending the email. I just read the responses on the archive and subscribed.
I do have unit tests, but they do not use an embedded server. They are more like functional tests and require a running instance of HDFS on the system. I have not looked [yet] to see if HDFS has a mock server or one that can be embedded. My code is up at https://github.com/dlmarion/ACCUMULO-708 in case anyone is interested. I'm still working on the ticket, so I'll look to see if I can modify the unit tests. Thanks,
Dave
----- Original Message -----
From: dlmarion@comcast.net
To: dev@commons.apache.org
Cc: dlmarion@comcast.net
Sent: Wednesday, September 19, 2012 10:04:28 PM
Subject: [VFS] HDFS provider
I did not see a provider for HDFS on the list of supported file systems. I created a read-only HDFS provider while working on an Accumulo ticket. I was wondering if one of the devs had time to review to see if I made any glaring mistakes. I am in the process of becoming an Accumulo committer, so I'm assuming that this file system provider could be included in the VFS project if it makes sense. Thanks,
Dave Marion
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [VFS] HDFS provider
Posted by dl...@comcast.net.
I forgot to subscribe to the dev mailing list before sending the email. I just read the responses on the archive and subscribed.
I do have unit tests, but they do not use an embedded server. They are more like functional tests and require a running instance of HDFS on the system. I have not looked [yet] to see if HDFS has a mock server or one that can be embedded. My code is up at https://github.com/dlmarion/ACCUMULO-708 in case anyone is interested. I'm still working on the ticket, so I'll look to see if I can modify the unit tests. Thanks,
Dave
----- Original Message -----
From: dlmarion@comcast.net
To: dev@commons.apache.org
Cc: dlmarion@comcast.net
Sent: Wednesday, September 19, 2012 10:04:28 PM
Subject: [VFS] HDFS provider
I did not see a provider for HDFS on the list of supported file systems. I created a read-only HDFS provider while working on an Accumulo ticket. I was wondering if one of the devs had time to review to see if I made any glaring mistakes. I am in the process of becoming an Accumulo committer, so I'm assuming that this file system provider could be included in the VFS project if it makes sense. Thanks,
Dave Marion