You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Ravi Gummadi (JIRA)" <ji...@apache.org> on 2009/01/09 10:14:59 UTC
[jira] Created: (HADOOP-5002) 2 core tests TestFileOutputFormat and
TestHarFileSystem are failing in branch 19
2 core tests TestFileOutputFormat and TestHarFileSystem are failing in branch 19
--------------------------------------------------------------------------------
Key: HADOOP-5002
URL: https://issues.apache.org/jira/browse/HADOOP-5002
Project: Hadoop Core
Issue Type: Bug
Affects Versions: 0.19.0
Environment: tests fail on my RHEL desktop in branch 19
Reporter: Ravi Gummadi
Attachments: patch-5002.txt
I see 2 of the core tests(TestHarFileSystem and TestFileOutputFormat) failing in branch 19. These are passing in branch 20 and in trunk.
I see nullpointer exceptions for the attempts of reducde tasks in the logs(for example, see the following):
2009-01-09 13:53:34,112 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(484)) - Error from attempt_200901091353_0001_r_000000_0: java.lang.NullPointerException
at org.apache.hadoop.fs.Path.<init>(Path.java:61)
at org.apache.hadoop.fs.Path.<init>(Path.java:50)
at org.apache.hadoop.tools.HadoopArchives$HArchivesReducer.configure(HadoopArchives.java:552)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:337)
at org.apache.hadoop.mapred.Child.main(Child.java:155)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5002) 2 core tests TestFileOutputFormat and
TestHarFileSystem are failing in branch 19
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated HADOOP-5002:
---------------------------------
Release Note: This patch solves the null pointer exception issue in the 2 core tests TestFileOutputFormat and TestHarFileSystem in branch 19.
Status: Patch Available (was: Open)
> 2 core tests TestFileOutputFormat and TestHarFileSystem are failing in branch 19
> --------------------------------------------------------------------------------
>
> Key: HADOOP-5002
> URL: https://issues.apache.org/jira/browse/HADOOP-5002
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.19.0
> Environment: tests fail on my RHEL desktop in branch 19
> Reporter: Ravi Gummadi
> Assignee: Amareshwari Sriramadasu
> Attachments: patch-5002.txt
>
>
> I see 2 of the core tests(TestHarFileSystem and TestFileOutputFormat) failing in branch 19. These are passing in branch 20 and in trunk.
> I see nullpointer exceptions for the attempts of reducde tasks in the logs(for example, see the following):
> 2009-01-09 13:53:34,112 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(484)) - Error from attempt_200901091353_0001_r_000000_0: java.lang.NullPointerException
> at org.apache.hadoop.fs.Path.<init>(Path.java:61)
> at org.apache.hadoop.fs.Path.<init>(Path.java:50)
> at org.apache.hadoop.tools.HadoopArchives$HArchivesReducer.configure(HadoopArchives.java:552)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:337)
> at org.apache.hadoop.mapred.Child.main(Child.java:155)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5002) 2 core tests TestFileOutputFormat and
TestHarFileSystem are failing in branch 19
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5002:
--------------------------------------------
Attachment: patch-5002.txt
Attaching the patch with fix
> 2 core tests TestFileOutputFormat and TestHarFileSystem are failing in branch 19
> --------------------------------------------------------------------------------
>
> Key: HADOOP-5002
> URL: https://issues.apache.org/jira/browse/HADOOP-5002
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.19.0
> Environment: tests fail on my RHEL desktop in branch 19
> Reporter: Ravi Gummadi
> Attachments: patch-5002.txt
>
>
> I see 2 of the core tests(TestHarFileSystem and TestFileOutputFormat) failing in branch 19. These are passing in branch 20 and in trunk.
> I see nullpointer exceptions for the attempts of reducde tasks in the logs(for example, see the following):
> 2009-01-09 13:53:34,112 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(484)) - Error from attempt_200901091353_0001_r_000000_0: java.lang.NullPointerException
> at org.apache.hadoop.fs.Path.<init>(Path.java:61)
> at org.apache.hadoop.fs.Path.<init>(Path.java:50)
> at org.apache.hadoop.tools.HadoopArchives$HArchivesReducer.configure(HadoopArchives.java:552)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:337)
> at org.apache.hadoop.mapred.Child.main(Child.java:155)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-5002) 2 core tests TestFileOutputFormat
and TestHarFileSystem are failing in branch 19
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi reassigned HADOOP-5002:
------------------------------------
Assignee: Amareshwari Sriramadasu
> 2 core tests TestFileOutputFormat and TestHarFileSystem are failing in branch 19
> --------------------------------------------------------------------------------
>
> Key: HADOOP-5002
> URL: https://issues.apache.org/jira/browse/HADOOP-5002
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.19.0
> Environment: tests fail on my RHEL desktop in branch 19
> Reporter: Ravi Gummadi
> Assignee: Amareshwari Sriramadasu
> Attachments: patch-5002.txt
>
>
> I see 2 of the core tests(TestHarFileSystem and TestFileOutputFormat) failing in branch 19. These are passing in branch 20 and in trunk.
> I see nullpointer exceptions for the attempts of reducde tasks in the logs(for example, see the following):
> 2009-01-09 13:53:34,112 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(484)) - Error from attempt_200901091353_0001_r_000000_0: java.lang.NullPointerException
> at org.apache.hadoop.fs.Path.<init>(Path.java:61)
> at org.apache.hadoop.fs.Path.<init>(Path.java:50)
> at org.apache.hadoop.tools.HadoopArchives$HArchivesReducer.configure(HadoopArchives.java:552)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:337)
> at org.apache.hadoop.mapred.Child.main(Child.java:155)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5002) 2 core tests TestFileOutputFormat and
TestHarFileSystem are failing in branch 19
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5002:
--------------------------------------------
Component/s: mapred
Priority: Blocker (was: Major)
Affects Version/s: (was: 0.19.0)
0.19.1
Fix Version/s: 0.19.1
This is due to HADOOP-4847
> 2 core tests TestFileOutputFormat and TestHarFileSystem are failing in branch 19
> --------------------------------------------------------------------------------
>
> Key: HADOOP-5002
> URL: https://issues.apache.org/jira/browse/HADOOP-5002
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.1
> Environment: tests fail on my RHEL desktop in branch 19
> Reporter: Ravi Gummadi
> Assignee: Amareshwari Sriramadasu
> Priority: Blocker
> Fix For: 0.19.1
>
> Attachments: patch-5002.txt
>
>
> I see 2 of the core tests(TestHarFileSystem and TestFileOutputFormat) failing in branch 19. These are passing in branch 20 and in trunk.
> I see nullpointer exceptions for the attempts of reducde tasks in the logs(for example, see the following):
> 2009-01-09 13:53:34,112 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(484)) - Error from attempt_200901091353_0001_r_000000_0: java.lang.NullPointerException
> at org.apache.hadoop.fs.Path.<init>(Path.java:61)
> at org.apache.hadoop.fs.Path.<init>(Path.java:50)
> at org.apache.hadoop.tools.HadoopArchives$HArchivesReducer.configure(HadoopArchives.java:552)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:337)
> at org.apache.hadoop.mapred.Child.main(Child.java:155)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5002) 2 core tests TestFileOutputFormat
and TestHarFileSystem are failing in branch 19
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12665329#action_12665329 ]
Amareshwari Sriramadasu commented on HADOOP-5002:
-------------------------------------------------
With HADOOP-4847, setting of work output path is moved to method, initialize(), which is the first method called from run() of Map/Reduce Task, after starting communication thread.
In branch 0.19, ReduceTask.run() does instantiation of class Reducer in the beginning of the method. i.e. before initialize() call, which is not necessary.
The patch moves the instantiation of reducer down, where the reducer is actually called i.e. after shuffle and sort.
> 2 core tests TestFileOutputFormat and TestHarFileSystem are failing in branch 19
> --------------------------------------------------------------------------------
>
> Key: HADOOP-5002
> URL: https://issues.apache.org/jira/browse/HADOOP-5002
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.1
> Environment: tests fail on my RHEL desktop in branch 19
> Reporter: Ravi Gummadi
> Assignee: Amareshwari Sriramadasu
> Priority: Blocker
> Fix For: 0.19.1
>
> Attachments: patch-5002.txt
>
>
> I see 2 of the core tests(TestHarFileSystem and TestFileOutputFormat) failing in branch 19. These are passing in branch 20 and in trunk.
> I see nullpointer exceptions for the attempts of reducde tasks in the logs(for example, see the following):
> 2009-01-09 13:53:34,112 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(484)) - Error from attempt_200901091353_0001_r_000000_0: java.lang.NullPointerException
> at org.apache.hadoop.fs.Path.<init>(Path.java:61)
> at org.apache.hadoop.fs.Path.<init>(Path.java:50)
> at org.apache.hadoop.tools.HadoopArchives$HArchivesReducer.configure(HadoopArchives.java:552)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:337)
> at org.apache.hadoop.mapred.Child.main(Child.java:155)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5002) 2 core tests TestFileOutputFormat and
TestHarFileSystem are failing in branch 19
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-5002:
--------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Amareshwari!
> 2 core tests TestFileOutputFormat and TestHarFileSystem are failing in branch 19
> --------------------------------------------------------------------------------
>
> Key: HADOOP-5002
> URL: https://issues.apache.org/jira/browse/HADOOP-5002
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.1
> Environment: tests fail on my RHEL desktop in branch 19
> Reporter: Ravi Gummadi
> Assignee: Amareshwari Sriramadasu
> Priority: Blocker
> Fix For: 0.19.1
>
> Attachments: patch-5002.txt
>
>
> I see 2 of the core tests(TestHarFileSystem and TestFileOutputFormat) failing in branch 19. These are passing in branch 20 and in trunk.
> I see nullpointer exceptions for the attempts of reducde tasks in the logs(for example, see the following):
> 2009-01-09 13:53:34,112 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(484)) - Error from attempt_200901091353_0001_r_000000_0: java.lang.NullPointerException
> at org.apache.hadoop.fs.Path.<init>(Path.java:61)
> at org.apache.hadoop.fs.Path.<init>(Path.java:50)
> at org.apache.hadoop.tools.HadoopArchives$HArchivesReducer.configure(HadoopArchives.java:552)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:337)
> at org.apache.hadoop.mapred.Child.main(Child.java:155)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5002) 2 core tests TestFileOutputFormat
and TestHarFileSystem are failing in branch 19
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12662376#action_12662376 ]
Ravi Gummadi commented on HADOOP-5002:
--------------------------------------
Patch looks fine.
Patch testing gave
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
As this patch makes the 2 tests pass, I guess this doesn't need extra testcases to be written.
Unit tests also passed on my local machine.
> 2 core tests TestFileOutputFormat and TestHarFileSystem are failing in branch 19
> --------------------------------------------------------------------------------
>
> Key: HADOOP-5002
> URL: https://issues.apache.org/jira/browse/HADOOP-5002
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.19.0
> Environment: tests fail on my RHEL desktop in branch 19
> Reporter: Ravi Gummadi
> Assignee: Amareshwari Sriramadasu
> Attachments: patch-5002.txt
>
>
> I see 2 of the core tests(TestHarFileSystem and TestFileOutputFormat) failing in branch 19. These are passing in branch 20 and in trunk.
> I see nullpointer exceptions for the attempts of reducde tasks in the logs(for example, see the following):
> 2009-01-09 13:53:34,112 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(484)) - Error from attempt_200901091353_0001_r_000000_0: java.lang.NullPointerException
> at org.apache.hadoop.fs.Path.<init>(Path.java:61)
> at org.apache.hadoop.fs.Path.<init>(Path.java:50)
> at org.apache.hadoop.tools.HadoopArchives$HArchivesReducer.configure(HadoopArchives.java:552)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:337)
> at org.apache.hadoop.mapred.Child.main(Child.java:155)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.