You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Amareshwari Sriramadasu (Created) (JIRA)" <ji...@apache.org> on 2012/02/10 09:19:59 UTC
[jira] [Created] (HADOOP-8054) NPE with RawLocalFileSystem
NPE with RawLocalFileSystem
---------------------------
Key: HADOOP-8054
URL: https://issues.apache.org/jira/browse/HADOOP-8054
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 0.23.1
Reporter: Amareshwari Sriramadasu
Priority: Critical
While running Hive tests, I'm seeing the following exception with 0.23.1,
{noformat}
ava.lang.NullPointerException
at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
{noformat}
Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with RawLocalFileSystem
Posted by "Robert Joseph Evans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207792#comment-13207792 ]
Robert Joseph Evans commented on HADOOP-8054:
---------------------------------------------
I am very sorry that HADOOP-8013 has broken the hive unit tests. The change as Daryn stated was intentional, and was made to allow viewfs to work properly in connection with FilterFileSystem. I am happy to put in a fix but I would like to better understand what has broken and how it was broken so we can fix the correct thing. Otherwise we will be back here again as Daryn tries yet again to fix FilterFileSystem.
@Daryn, why is it that after this patch is applied that no unit tests fail on our side in common? If it really is critical behavior and is something that we want to be part of the contract between filter file system and the file systems it wraps, then we need a unit test to enforce it.
> NPE with RawLocalFileSystem
> ---------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Priority: Critical
> Attachments: HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210371#comment-13210371 ]
Hudson commented on HADOOP-8054:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1755 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1755/])
HADOOP-8054. NPE with FilterFileSystem (Daryn Sharp via bobby) (Revision 1245637)
Result = FAILURE
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245637
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalFileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFilterFileSystem.java
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Fix For: 0.23.2
>
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Amareshwari Sriramadasu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-8054:
--------------------------------------------
Status: Patch Available (was: Open)
Resubmitting for hudson
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with RawLocalFileSystem
Posted by "Daryn Sharp (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206992#comment-13206992 ]
Daryn Sharp commented on HADOOP-8054:
-------------------------------------
The "problem" with reverting the change is that the contained filesystem, which has already been initialized, will be re-initialized with the uri of the filter. Since {{LocalFileSystem}} instantiates a {{RawLocalFileSystem}} with no conf, it's probably safer for {{LocalFileSystem#initialize}} to just call {{setConf}} on the raw filesystem.
> NPE with RawLocalFileSystem
> ---------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Priority: Critical
> Attachments: HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with RawLocalFileSystem
Posted by "Daryn Sharp (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207804#comment-13207804 ]
Daryn Sharp commented on HADOOP-8054:
-------------------------------------
I added tests that used LocalFileSystem so I'm not sure why it's failing. Offline I've requested the test code that is breaking because perhaps it's the test doing something unusual.
> NPE with RawLocalFileSystem
> ---------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Priority: Critical
> Attachments: HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with RawLocalFileSystem
Posted by "Amareshwari Sriramadasu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208198#comment-13208198 ]
Amareshwari Sriramadasu commented on HADOOP-8054:
-------------------------------------------------
The attached patch has the regression test (It fails without the patch).
> NPE with RawLocalFileSystem
> ---------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Priority: Critical
> Attachments: HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with RawLocalFileSystem
Posted by "Amareshwari Sriramadasu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206796#comment-13206796 ]
Amareshwari Sriramadasu commented on HADOOP-8054:
-------------------------------------------------
The issue was because of HADOOP-8013 and the following change:
{noformat}
@@ -77,7 +79,11 @@
* @param conf the configuration
*/
public void initialize(URI name, Configuration conf) throws IOException {
- fs.initialize(name, conf);
+ super.initialize(name, conf);
+
{noformat}
Robert/Daryn, was this change required or is it by mistake?
FilterFileSystem uses the other contained file system as the basic file system and overrides all the methods of FileSystem with versions that pass all requests to the contained file system. It does not use super class methods.
I will upload a patch undoing this change soon.
> NPE with RawLocalFileSystem
> ---------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Priority: Critical
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209229#comment-13209229 ]
Hadoop QA commented on HADOOP-8054:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12514771/HADOOP-8054.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/604//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/604//console
This message is automatically generated.
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210366#comment-13210366 ]
Hudson commented on HADOOP-8054:
--------------------------------
Integrated in Hadoop-Common-trunk-Commit #1743 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1743/])
HADOOP-8054. NPE with FilterFileSystem (Daryn Sharp via bobby) (Revision 1245637)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245637
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalFileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFilterFileSystem.java
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Fix For: 0.23.2
>
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210956#comment-13210956 ]
Hudson commented on HADOOP-8054:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk #994 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/994/])
HADOOP-8054. Committing to 0.23.1. (Revision 1245837)
HADOOP-8054. NPE with FilterFileSystem (Daryn Sharp via bobby) (Revision 1245637)
Result = SUCCESS
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245837
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245637
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalFileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFilterFileSystem.java
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210370#comment-13210370 ]
Hudson commented on HADOOP-8054:
--------------------------------
Integrated in Hadoop-Common-0.23-Commit #565 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/565/])
svn merge -c 1245637 from trunk to 0.23 FIXES HADOOP-8054 NPE with FilterFileSystem (Daryn Sharp via bobby) (Revision 1245643)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245643
Files :
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalFileSystem.java
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFilterFileSystem.java
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Fix For: 0.23.2
>
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with RawLocalFileSystem
Posted by "Amareshwari Sriramadasu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208237#comment-13208237 ]
Amareshwari Sriramadasu commented on HADOOP-8054:
-------------------------------------------------
bq. The change as Daryn stated was intentional, and was made to allow viewfs to work properly in connection with FilterFileSystem.
I would like to better understand what has broken and how it was broken so we can fix the correct thing.
What was broken - "The contained FileSystem is no more initialized after the patch". The change makes all the sub classes of FilterFileSystem call initialize for contained FileSystem - making it incompatible.
bq. why is it that after this patch is applied that no unit tests fail on our side in common?
Because LocalFileSystem was changed to initialize the contained FileSystem in HADOOP-8013. So, no tests were failing. Added the failing test in the attached patch.
> NPE with RawLocalFileSystem
> ---------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Priority: Critical
> Attachments: HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210365#comment-13210365 ]
Hudson commented on HADOOP-8054:
--------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1817 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1817/])
HADOOP-8054. NPE with FilterFileSystem (Daryn Sharp via bobby) (Revision 1245637)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245637
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalFileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFilterFileSystem.java
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Fix For: 0.23.2
>
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8054) NPE with RawLocalFileSystem
Posted by "Amareshwari Sriramadasu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-8054:
--------------------------------------------
Assignee: Amareshwari Sriramadasu
Status: Patch Available (was: Open)
> NPE with RawLocalFileSystem
> ---------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Priority: Critical
> Attachments: HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210764#comment-13210764 ]
Hudson commented on HADOOP-8054:
--------------------------------
Integrated in Hadoop-Common-0.23-Commit #570 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/570/])
HADOOP-8054. Committing to 0.23.1. (Revision 1245839)
Result = SUCCESS
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245839
Files :
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Fix For: 0.23.2
>
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210919#comment-13210919 ]
Hudson commented on HADOOP-8054:
--------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #172 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/172/])
HADOOP-8054. Committing to 0.23.1. (Revision 1245839)
svn merge -c 1245637 from trunk to 0.23 FIXES HADOOP-8054 NPE with FilterFileSystem (Daryn Sharp via bobby) (Revision 1245643)
Result = SUCCESS
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245839
Files :
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245643
Files :
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalFileSystem.java
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFilterFileSystem.java
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Daryn Sharp (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209479#comment-13209479 ]
Daryn Sharp commented on HADOOP-8054:
-------------------------------------
I'd like to add a test that does a little more comprehensive testing of the conf setting. I'll post a patch after I burn through my email.
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with RawLocalFileSystem
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206856#comment-13206856 ]
Hadoop QA commented on HADOOP-8054:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12514345/HADOOP-8054.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/589//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/589//console
This message is automatically generated.
> NPE with RawLocalFileSystem
> ---------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Priority: Critical
> Attachments: HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210763#comment-13210763 ]
Hudson commented on HADOOP-8054:
--------------------------------
Integrated in Hadoop-Common-trunk-Commit #1748 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1748/])
HADOOP-8054. Committing to 0.23.1. (Revision 1245837)
Result = SUCCESS
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245837
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Fix For: 0.23.2
>
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210781#comment-13210781 ]
Hudson commented on HADOOP-8054:
--------------------------------
Integrated in Hadoop-Mapreduce-0.23-Commit #573 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/573/])
HADOOP-8054. Committing to 0.23.1. (Revision 1245839)
Result = ABORTED
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245839
Files :
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Fix For: 0.23.2
>
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210948#comment-13210948 ]
Hudson commented on HADOOP-8054:
--------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #200 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/200/])
HADOOP-8054. Committing to 0.23.1. (Revision 1245839)
svn merge -c 1245637 from trunk to 0.23 FIXES HADOOP-8054 NPE with FilterFileSystem (Daryn Sharp via bobby) (Revision 1245643)
Result = FAILURE
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245839
Files :
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245643
Files :
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalFileSystem.java
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFilterFileSystem.java
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210762#comment-13210762 ]
Hudson commented on HADOOP-8054:
--------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1822 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1822/])
HADOOP-8054. Committing to 0.23.1. (Revision 1245837)
Result = SUCCESS
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245837
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Fix For: 0.23.2
>
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Robert Joseph Evans (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated HADOOP-8054:
----------------------------------------
Resolution: Fixed
Fix Version/s: 0.23.2
Status: Resolved (was: Patch Available)
Thanks Daryn, I just committed this to trunk and branch 0.23
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Fix For: 0.23.2
>
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Daryn Sharp (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208545#comment-13208545 ]
Daryn Sharp commented on HADOOP-8054:
-------------------------------------
Basically, if an instance of {{FilterFileSystem}} uses a different uri (scheme and/or authority) than the embedded fs, the double init made the embedded fs incorrectly report the filtered's uri. This breaks all kinds of things like cached fs instances, dupping of the fs via its uri, fs statistics, setting of the token service which is based on the uri's authority, etc. I think this patch didn't break existing tests because RFS is hardcoded to return file:/// instead of the uri passed to init. This masks the problem that was fixed.
Other {{FilterFileSystem}} instances use the ctor that takes the embedded {{FileSystem}}, which for the aforementioned reason must not be re-inited, in particular a chrooted fs. {{LocalFileSystem}} is unique in that its ctor instantiates a RLFS with no conf and relies on the double init to set the conf in the RLFS. I ran into this jira's issue, so I modified {{LocalFileSystem}} instead of {{FilterFileSystem}}, to continue to re-init the RLFS because it would be no worse than before.
W/o seeing the source code, {{ProxyFileSystem}} appears to be instantiating a RLFS with the default ctor and then "neglects" to call {{setConf}} or {{initialize}} because it could "get away with it" when it was wrapped in a {{FilteredFileSystem}}. As an aside, I think it was a mistake that {{RawLocalFileSystem}} exposed the default ctor, instead of forcing the use of the ctor that takes a conf...
Long story (sorry): for backwards-compatibility, I'll see if propagating {{FilterFileSystem#setConf(conf)}} to the embedded fs will fix the problem.
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Attachments: HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with RawLocalFileSystem
Posted by "Daryn Sharp (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206995#comment-13206995 ]
Daryn Sharp commented on HADOOP-8054:
-------------------------------------
I'll clarify that the contained filesystem is already initialized in the case of {{new FilterFileSystem(otherFs)}}.
> NPE with RawLocalFileSystem
> ---------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Priority: Critical
> Attachments: HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Daryn Sharp (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HADOOP-8054:
--------------------------------
Attachment: HADOOP-8054.patch
Ran into issues simply propagating the {{setConf}}. I considered having RLFS's {{setConf}} or ctor call its own {{initialize}} since it has a hardcoded uri, but that causes problems with unchecked exceptions.
So, what I've done is have {{FilterFileSystem#initialize}} call {{initialize}} on its embedded fs iff it has a null conf (ie. hasn't been initialized). Not perfect, but should restore former behavior for existing filesystems.
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208712#comment-13208712 ]
Hadoop QA commented on HADOOP-8054:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12514687/HADOOP-8054.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/600//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/600//console
This message is automatically generated.
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209567#comment-13209567 ]
Hadoop QA commented on HADOOP-8054:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12514828/HADOOP-8054.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/606//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/606//console
This message is automatically generated.
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Daryn Sharp (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HADOOP-8054:
--------------------------------
Status: Patch Available (was: Open)
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Amareshwari Sriramadasu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-8054:
--------------------------------------------
Attachment: HADOOP-8054.patch
Updated the patch with test included in my earlier patch
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210765#comment-13210765 ]
Hudson commented on HADOOP-8054:
--------------------------------
Integrated in Hadoop-Hdfs-0.23-Commit #557 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/557/])
HADOOP-8054. Committing to 0.23.1. (Revision 1245839)
Result = SUCCESS
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245839
Files :
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Fix For: 0.23.2
>
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Amareshwari Sriramadasu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-8054:
--------------------------------------------
Component/s: fs
Assignee: Daryn Sharp (was: Amareshwari Sriramadasu)
Summary: NPE with FilterFileSystem (was: NPE with RawLocalFileSystem)
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Attachments: HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated HADOOP-8054:
----------------------------------
Fix Version/s: (was: 0.23.2)
0.23.1
Merged to 23.1 too.
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Amareshwari Sriramadasu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209204#comment-13209204 ]
Amareshwari Sriramadasu commented on HADOOP-8054:
-------------------------------------------------
Patch works for me.
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210389#comment-13210389 ]
Hudson commented on HADOOP-8054:
--------------------------------
Integrated in Hadoop-Mapreduce-0.23-Commit #568 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/568/])
svn merge -c 1245637 from trunk to 0.23 FIXES HADOOP-8054 NPE with FilterFileSystem (Daryn Sharp via bobby) (Revision 1245643)
Result = ABORTED
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245643
Files :
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalFileSystem.java
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFilterFileSystem.java
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Fix For: 0.23.2
>
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210909#comment-13210909 ]
Hudson commented on HADOOP-8054:
--------------------------------
Integrated in Hadoop-Hdfs-trunk #959 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/959/])
HADOOP-8054. Committing to 0.23.1. (Revision 1245837)
HADOOP-8054. NPE with FilterFileSystem (Daryn Sharp via bobby) (Revision 1245637)
Result = SUCCESS
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245837
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245637
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalFileSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFilterFileSystem.java
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8054) NPE with RawLocalFileSystem
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated HADOOP-8054:
----------------------------------
Status: Open (was: Patch Available)
Cancelling patch while Daryn investigates this further. Thanks Amarsri!
> NPE with RawLocalFileSystem
> ---------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Priority: Critical
> Attachments: HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210369#comment-13210369 ]
Hudson commented on HADOOP-8054:
--------------------------------
Integrated in Hadoop-Hdfs-0.23-Commit #552 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/552/])
svn merge -c 1245637 from trunk to 0.23 FIXES HADOOP-8054 NPE with FilterFileSystem (Daryn Sharp via bobby) (Revision 1245643)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245643
Files :
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalFileSystem.java
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFilterFileSystem.java
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Fix For: 0.23.2
>
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210780#comment-13210780 ]
Hudson commented on HADOOP-8054:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1760 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1760/])
HADOOP-8054. Committing to 0.23.1. (Revision 1245837)
Result = ABORTED
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1245837
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Fix For: 0.23.2
>
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Robert Joseph Evans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210355#comment-13210355 ]
Robert Joseph Evans commented on HADOOP-8054:
---------------------------------------------
Amareshwari said the patch works. The tests look good, and the patch looks good. +1
Because this breaks HIVE and is a regression I would like to see this go into 0.23.1. I will put it into trunk and 0.23.2, but please consider this for 0.23.1.
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8054) NPE with RawLocalFileSystem
Posted by "Amareshwari Sriramadasu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-8054:
--------------------------------------------
Attachment: HADOOP-8054.patch
Patch with the fix and regression test.
> NPE with RawLocalFileSystem
> ---------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Priority: Critical
> Attachments: HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Amareshwari Sriramadasu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-8054:
--------------------------------------------
Status: Open (was: Patch Available)
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Daryn Sharp (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210374#comment-13210374 ]
Daryn Sharp commented on HADOOP-8054:
-------------------------------------
I'd like to echo that the issue is not specific to hive so I recommend it for 23.1. Hive happened to expose the issue with one of their filesystems, but the issue may break other non-core filesystems. This is a trivial patch to restore former questionable behavior to avoid more unforeseen incompatibility issues.
Arguably it's a mistake for some filesystems to have exposed their default ctors because it creates an un-configed & thus unusable fs instance. I feel a fs should always be instantiated with a conf. FFS hid the problem by re-initing the embedded fs, albeit with its own uri which is wrong. Ie. if the uris don't match and/or the init performs what should be one-time operations.
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Fix For: 0.23.2
>
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8054) NPE with FilterFileSystem
Posted by "Daryn Sharp (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HADOOP-8054:
--------------------------------
Attachment: HADOOP-8054.patch
Amareshwari, I removed the one test you added to my patch, and replaced with a bunch of superset sets. They verify that the init of stacked filtered filesystems correctly sets the conf in all the chained filesystems.
> NPE with FilterFileSystem
> -------------------------
>
> Key: HADOOP-8054
> URL: https://issues.apache.org/jira/browse/HADOOP-8054
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.1
> Reporter: Amareshwari Sriramadasu
> Assignee: Daryn Sharp
> Priority: Critical
> Attachments: HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch, HADOOP-8054.patch
>
>
> While running Hive tests, I'm seeing the following exception with 0.23.1,
> {noformat}
> ava.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:1901)
> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:447)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.ProxyFileSystem.getFileStatus(ProxyFileSystem.java:247)
> at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:351)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1165)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:390)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:242)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
> {noformat}
> Did not see this with 0.23.0, though.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira