HADOOP-8545 swift reviewers needed

[Steve Loughran <st...@hortonworks.com>]

Hi,

I've been through suresh's review of the HADOOP-8545 patch and pulled in
the changes -that patch is now ready for review by anyone, testing even
better.


I'd like to get this in not just because we've been working on it for a
long time, but because I'm trying to specify more rigorously what
filesystems should do ( https://issues.apache.org/jira/browse/HADOOP-9361 )
-a lot of those swift tests can be pulled up to be test cases, and I'd like
to have the swift: FS tested the way I'm setting up s3 and ftp.

-steve

Re: HADOOP-8545 swift reviewers needed

[Steve Loughran <st...@hortonworks.com>]

Great

Its designed to skip all tests if you don't create a file
test/resources/auth-keys.xml. It should be documented in the index.apt.vm
file -if not, that's something to ask for.

Note that the test are slow, as in "go and do something else" slow- lots of
operations being tried, including some big files and many small ones.
Public clouds sometimes throttle PUT/DELETE operations, which the code is
designed to handle by being forgiving of delays, but it can really kill
test teardown times

-steve

On 9 July 2013 19:04, Stephen Chu <sc...@cloudera.com> wrote:

> Hi Steve,
>
> I'll help out testing the Hadoop on Swift work. I have a single-node Swift
> setup and will first try running the unit tests against the setup.
>
> I'm guessing the most up-to-date documentation is the
> hadoop-openstack/src/site/apt/index.apt.vm
> added in HADOOP-8545? Thanks,
> Stephen
>
>
> On Tue, Jul 9, 2013 at 8:15 AM, Steve Loughran <stevel@hortonworks.com
> >wrote:
>
> > Hi,
> >
> > I've been through suresh's review of the HADOOP-8545 patch and pulled in
> > the changes -that patch is now ready for review by anyone, testing even
> > better.
> >
> >
> > I'd like to get this in not just because we've been working on it for a
> > long time, but because I'm trying to specify more rigorously what
> > filesystems should do (
> https://issues.apache.org/jira/browse/HADOOP-9361)
> > -a lot of those swift tests can be pulled up to be test cases, and I'd
> like
> > to have the swift: FS tested the way I'm setting up s3 and ftp.
> >
> > -steve
> >
>

Re: HADOOP-8545 swift reviewers needed

[Stephen Chu <sc...@cloudera.com>]

Hi Steve,

I'll help out testing the Hadoop on Swift work. I have a single-node Swift
setup and will first try running the unit tests against the setup.

I'm guessing the most up-to-date documentation is the
hadoop-openstack/src/site/apt/index.apt.vm
added in HADOOP-8545? Thanks,
Stephen


On Tue, Jul 9, 2013 at 8:15 AM, Steve Loughran <st...@hortonworks.com>wrote:

> Hi,
>
> I've been through suresh's review of the HADOOP-8545 patch and pulled in
> the changes -that patch is now ready for review by anyone, testing even
> better.
>
>
> I'd like to get this in not just because we've been working on it for a
> long time, but because I'm trying to specify more rigorously what
> filesystems should do ( https://issues.apache.org/jira/browse/HADOOP-9361)
> -a lot of those swift tests can be pulled up to be test cases, and I'd like
> to have the swift: FS tested the way I'm setting up s3 and ftp.
>
> -steve
>