You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Eli Collins (Created) (JIRA)" <ji...@apache.org> on 2012/02/23 05:27:48 UTC
[jira] [Created] (HADOOP-8105) HADOOP-8013 breaks the Hive 0.8
tests
HADOOP-8013 breaks the Hive 0.8 tests
-------------------------------------
Key: HADOOP-8105
URL: https://issues.apache.org/jira/browse/HADOOP-8105
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 0.23.1
Reporter: Eli Collins
Priority: Blocker
The FilterFileSystem javadoc says "The class FilterFileSystem itself simply overrides all methods of FileSystem with versions that pass all requests to the contained file system." initialize(URI, Configuration) is part of the FileSystem API, and before HADOOP-8013 it simply called fs.initialize(). After HADOOP-8013 it calls super.initialize() rather than delegating. Hive's ProxyLocalFileSystem would be able to work around this problem by calling setConf, but unfortunately FilterFileSystem does not override that method. We need to make FilterFileSystem call fs.initialize (instead of using super) and also override setConf.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-8105) FilterFileSystem should delegate
initialize rather than use super
Posted by "Amareshwari Sriramadasu (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu resolved HADOOP-8105.
---------------------------------------------
Resolution: Duplicate
Eli, This is fixed through HADOOP-8054
> FilterFileSystem should delegate initialize rather than use super
> -----------------------------------------------------------------
>
> Key: HADOOP-8105
> URL: https://issues.apache.org/jira/browse/HADOOP-8105
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Eli Collins
> Priority: Blocker
>
> The FilterFileSystem javadoc says "The class FilterFileSystem itself simply overrides all methods of FileSystem with versions that pass all requests to the contained file system." initialize(URI, Configuration) is part of the FileSystem API, and before HADOOP-8013 it simply called fs.initialize(). After HADOOP-8013 it calls super.initialize() rather than delegating. This breaks a number of the Hive tests. Hive's ProxyLocalFileSystem would be able to work around this problem by calling setConf, but unfortunately FilterFileSystem does not override that method. We need to make FilterFileSystem call fs.initialize (instead of using super) and also override setConf.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8105) FilterFileSystem should delegate
initialize rather than use super
Posted by "Eli Collins (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214297#comment-13214297 ]
Eli Collins commented on HADOOP-8105:
-------------------------------------
Here's the particular problem you'll see, we NPE in FileSystem because conf is now null (and there's no way to set conf because the setConf method is not overridden).
{noformat}
java.lang.NullPointerException
at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:330)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1249)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1289)
at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:512)
at org.apache.hadoop.fs.FilterFileSystem.listStatus(FilterFileSystem.java:207)
at org.apache.hadoop.fs.ProxyFileSystem.listStatus(ProxyFileSystem.java:174)
at org.apache.hadoop.fs.FilterFileSystem.listStatus(FilterFileSystem.java:207)
at org.apache.hadoop.hive.ql.QTestUtil.cleanUp(QTestUtil.java:385)
at org.apache.hadoop.hive.ql.QTestUtil.shutdown(QTestUtil.java:255)
{noformat}
> FilterFileSystem should delegate initialize rather than use super
> -----------------------------------------------------------------
>
> Key: HADOOP-8105
> URL: https://issues.apache.org/jira/browse/HADOOP-8105
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Eli Collins
> Priority: Blocker
>
> The FilterFileSystem javadoc says "The class FilterFileSystem itself simply overrides all methods of FileSystem with versions that pass all requests to the contained file system." initialize(URI, Configuration) is part of the FileSystem API, and before HADOOP-8013 it simply called fs.initialize(). After HADOOP-8013 it calls super.initialize() rather than delegating. This breaks a number of the Hive tests. Hive's ProxyLocalFileSystem would be able to work around this problem by calling setConf, but unfortunately FilterFileSystem does not override that method. We need to make FilterFileSystem call fs.initialize (instead of using super) and also override setConf.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8105) FilterFileSystem should delegate
initialize rather than use super
Posted by "Eli Collins (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-8105:
--------------------------------
Description: The FilterFileSystem javadoc says "The class FilterFileSystem itself simply overrides all methods of FileSystem with versions that pass all requests to the contained file system." initialize(URI, Configuration) is part of the FileSystem API, and before HADOOP-8013 it simply called fs.initialize(). After HADOOP-8013 it calls super.initialize() rather than delegating. This breaks a number of the Hive tests. Hive's ProxyLocalFileSystem would be able to work around this problem by calling setConf, but unfortunately FilterFileSystem does not override that method. We need to make FilterFileSystem call fs.initialize (instead of using super) and also override setConf. (was: The FilterFileSystem javadoc says "The class FilterFileSystem itself simply overrides all methods of FileSystem with versions that pass all requests to the contained file system." initialize(URI, Configuration) is part of the FileSystem API, and before HADOOP-8013 it simply called fs.initialize(). After HADOOP-8013 it calls super.initialize() rather than delegating. Hive's ProxyLocalFileSystem would be able to work around this problem by calling setConf, but unfortunately FilterFileSystem does not override that method. We need to make FilterFileSystem call fs.initialize (instead of using super) and also override setConf.)
> FilterFileSystem should delegate initialize rather than use super
> -----------------------------------------------------------------
>
> Key: HADOOP-8105
> URL: https://issues.apache.org/jira/browse/HADOOP-8105
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Eli Collins
> Priority: Blocker
>
> The FilterFileSystem javadoc says "The class FilterFileSystem itself simply overrides all methods of FileSystem with versions that pass all requests to the contained file system." initialize(URI, Configuration) is part of the FileSystem API, and before HADOOP-8013 it simply called fs.initialize(). After HADOOP-8013 it calls super.initialize() rather than delegating. This breaks a number of the Hive tests. Hive's ProxyLocalFileSystem would be able to work around this problem by calling setConf, but unfortunately FilterFileSystem does not override that method. We need to make FilterFileSystem call fs.initialize (instead of using super) and also override setConf.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8105) FilterFileSystem should delegate
initialize rather than use super
Posted by "Eli Collins (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-8105:
--------------------------------
Summary: FilterFileSystem should delegate initialize rather than use super (was: HADOOP-8013 breaks the Hive 0.8 tests)
> FilterFileSystem should delegate initialize rather than use super
> -----------------------------------------------------------------
>
> Key: HADOOP-8105
> URL: https://issues.apache.org/jira/browse/HADOOP-8105
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Eli Collins
> Priority: Blocker
>
> The FilterFileSystem javadoc says "The class FilterFileSystem itself simply overrides all methods of FileSystem with versions that pass all requests to the contained file system." initialize(URI, Configuration) is part of the FileSystem API, and before HADOOP-8013 it simply called fs.initialize(). After HADOOP-8013 it calls super.initialize() rather than delegating. Hive's ProxyLocalFileSystem would be able to work around this problem by calling setConf, but unfortunately FilterFileSystem does not override that method. We need to make FilterFileSystem call fs.initialize (instead of using super) and also override setConf.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira