You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2014/11/04 10:41:35 UTC
[jira] [Updated] (HADOOP-11263) NativeS3FileSystem doesn't work
with hadoop-client unless jets3t added to classpath
[ https://issues.apache.org/jira/browse/HADOOP-11263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated HADOOP-11263:
------------------------------------
Summary: NativeS3FileSystem doesn't work with hadoop-client unless jets3t added to classpath (was: NativeS3FileSystem doesn't work with hadoop-client)
> NativeS3FileSystem doesn't work with hadoop-client unless jets3t added to classpath
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-11263
> URL: https://issues.apache.org/jira/browse/HADOOP-11263
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.4.0
> Reporter: Sangjin Lee
>
> When you start using the NativeS3FileSystem (which is in hadoop-common) based on the hadoop-client set of jars, it fails with a NoClassDefFoundError:
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/jets3t/service/ServiceException
> at org.apache.hadoop.fs.s3native.NativeS3FileSystem.createDefaultStore(NativeS3FileSystem.java:280)
> at org.apache.hadoop.fs.s3native.NativeS3FileSystem.initialize(NativeS3FileSystem.java:270)
> at com.twitter.twadoop.util.hadoop.NativeS3FileSystemWrapper.initialize(NativeS3FileSystemWrapper.java:34)
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2438)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:90)
> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2472)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2454)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:384)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
> {noformat}
> NativeS3FileSystem depends on a library called jets3t, which is not found in the hadoop-client build. It turns out that this library was specifically excluded in the hadoop-client pom.xml:
> {noformat}
> <exclusion>
> <groupId>net.java.dev.jets3t</groupId>
> <artifactId>jets3t</artifactId>
> </exclusion>
> {noformat}
> This strikes me as an issue, as a component that's part of hadoop-common cannot run with a hadoop-client build.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)