[jira] [Commented] (HDFS-9394) branch-2 hadoop-hdfs-client fails during FileSystem ServiceLoader initialization, because HftpFileSystem is missing.

["Chris Nauroth (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14993229#comment-14993229 ] 

Chris Nauroth commented on HDFS-9394:
-------------------------------------

The easiest way to see this problem is to run the hadoop-hdfs-client tests on branch-2.  Several of the tests will fail like this:

{code}
Running org.apache.hadoop.fs.TestUrlStreamHandlerFactory
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.2 sec <<< FAILURE! - in org.apache.hadoop.fs.TestUrlStreamHandlerFactory
testConcurrency(org.apache.hadoop.fs.TestUrlStreamHandlerFactory)  Time elapsed: 0.148 sec  <<< ERROR!
java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider org.apache.hadoop.hdfs.web.HftpFileSystem not found
	at java.util.ServiceLoader.fail(ServiceLoader.java:231)
	at java.util.ServiceLoader.access$300(ServiceLoader.java:181)
	at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:365)
	at java.util.ServiceLoader$1.next(ServiceLoader.java:445)
	at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:2662)
	at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2673)
	at org.apache.hadoop.fs.FsUrlStreamHandlerFactory.<init>(FsUrlStreamHandlerFactory.java:62)
	at org.apache.hadoop.fs.FsUrlStreamHandlerFactory.<init>(FsUrlStreamHandlerFactory.java:55)
	at org.apache.hadoop.fs.TestUrlStreamHandlerFactory.singleRun(TestUrlStreamHandlerFactory.java:48)
	at org.apache.hadoop.fs.TestUrlStreamHandlerFactory.testConcurrency(TestUrlStreamHandlerFactory.java:43)
{code}

This also implies that end users wouldn't be successful in depending on hadoop-hdfs-client only.  They'd still have to get hadoop-hdfs.jar onto the classpath as a workaround.

This is not a problem on trunk, where {{HftpFileSystem}} and {{HsftpFileSystem}} have been removed.

FYI [~liuml07] and [~wheat9].  Thanks!

> branch-2 hadoop-hdfs-client fails during FileSystem ServiceLoader initialization, because HftpFileSystem is missing.
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-9394
>                 URL: https://issues.apache.org/jira/browse/HDFS-9394
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client
>            Reporter: Chris Nauroth
>            Priority: Critical
>
> On branch-2, hadoop-hdfs-client contains a {{FileSystem}} service descriptor that lists {{HftpFileSystem}} and {{HsftpFileSystem}}.  These classes do not reside in hadoop-hdfs-client.  Instead, they reside in hadoop-hdfs.  If the application has hadoop-hdfs-client.jar on the classpath, but not hadoop-hdfs.jar, then this can cause a {{ServiceConfigurationError}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)