[jira] [Updated] (HADOOP-8136) Enhance hadoop to use a newer version (0.8.1) of the jets3t library

["Jagane Sundar (Updated) (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-8136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jagane Sundar updated HADOOP-8136:
----------------------------------

    Target Version/s: 1.0.2
              Status: Patch Available  (was: Open)

Here's my first cut at a patch for this jira. Some notes:
1. Most of the changes are on account of the jets3t-0.8.1 library throwing a ServiceException, instead of a S3ServiceException.
2. There is also a piece of brittle code that was comparing an exception string with 'ResponseCode=404' to decide whether the S3 server threw a 404 error. I replaced that with a more rational test of response code.
3. I turned down the logging by setting
org.apache.hadoop.fs.s3native=ERROR
org.jets3t.service.impl.rest.httpclient.RestStorageService=ERROR
in log4j.properties
                
> Enhance hadoop to use a newer version (0.8.1) of the jets3t library
> -------------------------------------------------------------------
>
>                 Key: HADOOP-8136
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8136
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs/s3
>    Affects Versions: 1.0.0, 0.22.0, 0.23.3
>         Environment: Ubuntu 11.04, 64 bit, JDK 1.6.0_30
>            Reporter: Jagane Sundar
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Hadoop is built against, and includes, an older version of the Jets3t library - version 0.6.1.
> The current version of the Jets3t library(as of March 2012) is 0.8.1. This new version includes many improvements such as support for "Requester-Pays" buckets.
> Since hadoop includes a copy of the version 0.6.1 jets3t library, and since this version ends up early in the CLASSPATH, any Map Reduce application that wants to use the jets3t library ends up getting version 0.6.1 of the jets3t library. The MR application fails, usually with an error stating that the method signature of some method in the Jets3t library does not match.
> It would be useful to enhance Jets3tNativeFileSystemStore.java to use the API published by the 0.8.1 version of the jets3t library.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira