You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Vinod K V (JIRA)" <ji...@apache.org> on 2009/11/26 10:28:39 UTC
[jira] Created: (MAPREDUCE-1245) TestFairScheduler fails with "too
many open files" error
TestFairScheduler fails with "too many open files" error
--------------------------------------------------------
Key: MAPREDUCE-1245
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1245
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 0.21.0
Reporter: Vinod K V
Fix For: 0.21.0
This was caused by MAPREDUCE-1103 and was observed after MAPREDUCE-1239.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1245) TestFairScheduler fails with
"too many open files" error
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782909#action_12782909 ]
Hadoop QA commented on MAPREDUCE-1245:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12426216/1245_v1.patch
against trunk revision 884531.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/271/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/271/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/271/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/271/console
This message is automatically generated.
> TestFairScheduler fails with "too many open files" error
> --------------------------------------------------------
>
> Key: MAPREDUCE-1245
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1245
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.21.0
> Reporter: Vinod K V
> Fix For: 0.21.0
>
> Attachments: 1245_v1.patch
>
>
> This was caused by MAPREDUCE-1103 and was observed after MAPREDUCE-1239.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1245) TestFairScheduler fails with
"too many open files" error
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783082#action_12783082 ]
Hadoop QA commented on MAPREDUCE-1245:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12426216/1245_v1.patch
against trunk revision 884628.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/273/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/273/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/273/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/273/console
This message is automatically generated.
> TestFairScheduler fails with "too many open files" error
> --------------------------------------------------------
>
> Key: MAPREDUCE-1245
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1245
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.21.0
> Reporter: Vinod K V
> Fix For: 0.21.0
>
> Attachments: 1245_v1.patch
>
>
> This was caused by MAPREDUCE-1103 and was observed after MAPREDUCE-1239.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1245) TestFairScheduler fails with "too
many open files" error
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal updated MAPREDUCE-1245:
--------------------------------------
Status: Patch Available (was: Open)
> TestFairScheduler fails with "too many open files" error
> --------------------------------------------------------
>
> Key: MAPREDUCE-1245
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1245
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.21.0
> Reporter: Vinod K V
> Fix For: 0.21.0
>
> Attachments: 1245_v1.patch
>
>
> This was caused by MAPREDUCE-1103 and was observed after MAPREDUCE-1239.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1245) TestFairScheduler fails with
"too many open files" error
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12784918#action_12784918 ]
Hudson commented on MAPREDUCE-1245:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #162 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/162/])
. Fix TestFairScheduler failures by instantiating lightweight Jobtracker. Contributed by Sharad Agarwal.
> TestFairScheduler fails with "too many open files" error
> --------------------------------------------------------
>
> Key: MAPREDUCE-1245
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1245
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.22.0
> Reporter: Vinod K V
> Assignee: Sharad Agarwal
> Fix For: 0.22.0
>
> Attachments: 1245_v1.patch
>
>
> This was caused by MAPREDUCE-1103 and was observed after MAPREDUCE-1239.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1245) TestFairScheduler fails with
"too many open files" error
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12784714#action_12784714 ]
Hudson commented on MAPREDUCE-1245:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #140 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/140/])
. Fix TestFairScheduler failures by instantiating lightweight Jobtracker. Contributed by Sharad Agarwal.
> TestFairScheduler fails with "too many open files" error
> --------------------------------------------------------
>
> Key: MAPREDUCE-1245
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1245
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.22.0
> Reporter: Vinod K V
> Assignee: Sharad Agarwal
> Fix For: 0.22.0
>
> Attachments: 1245_v1.patch
>
>
> This was caused by MAPREDUCE-1103 and was observed after MAPREDUCE-1239.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1245) TestFairScheduler fails with
"too many open files" error
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12784649#action_12784649 ]
Vinod K V commented on MAPREDUCE-1245:
--------------------------------------
Patch looks good. +1.
> TestFairScheduler fails with "too many open files" error
> --------------------------------------------------------
>
> Key: MAPREDUCE-1245
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1245
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.21.0
> Reporter: Vinod K V
> Fix For: 0.21.0
>
> Attachments: 1245_v1.patch
>
>
> This was caused by MAPREDUCE-1103 and was observed after MAPREDUCE-1239.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1245) TestFairScheduler fails with
"too many open files" error
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782813#action_12782813 ]
Vinod K V commented on MAPREDUCE-1245:
--------------------------------------
Here's the problem as mentioned [here|https://issues.apache.org/jira/browse/MAPREDUCE-1239?focusedCommentId=12782742&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12782742]:
{code}
[junit] 09/11/25 16:34:08 INFO mapred.JobTracker: JobTracker up at: 32801
[junit] 09/11/25 16:34:08 INFO mapred.JobTracker: JobTracker webserver: 44348
[junit] 09/11/25 16:34:08 INFO mapred.JobTracker: Cleaning up the system directory
[junit] 09/11/25 16:34:08 INFO mapred.JobTracker: problem cleaning system directory: file:/tmp/hadoop-sreerama/mapred/system
[junit] java.io.IOException: Cannot run program "chmod": java.io.IOException: error=24, Too many open files
[junit] at java.lang.ProcessBuilder.start(ProcessBuilder.java:474)
[junit] at org.apache.hadoop.util.Shell.runCommand(Shell.java:188)
[junit] at org.apache.hadoop.util.Shell.run(Shell.java:170)
[junit] at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:363)
[junit] at org.apache.hadoop.util.Shell.execCommand(Shell.java:449)
[junit] at org.apache.hadoop.util.Shell.execCommand(Shell.java:432)
[junit] at org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:540)
[junit] at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:532)
[junit] at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:281)
[junit] at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:295)
[junit] at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:1477)
[junit] at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:1306)
[junit] at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:1299)
[junit] at org.apache.hadoop.mapred.UtilsForTests.getJobTracker(UtilsForTests.java:712)
[junit] at org.apache.hadoop.mapred.TestFairScheduler.testPoolAssignment(TestFairScheduler.java:2546)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit] at java.lang.reflect.Method.invoke(Method.java:616)
[junit] at junit.framework.TestCase.runTest(TestCase.java:168)
[junit] at junit.framework.TestCase.runBare(TestCase.java:134)
[junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
[junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
[junit] at junit.framework.TestResult.run(TestResult.java:113)
[junit] at junit.framework.TestCase.run(TestCase.java:124)
[junit] at junit.framework.TestSuite.runTest(TestSuite.java:232)
[junit] at junit.framework.TestSuite.run(TestSuite.java:227)
[junit] at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:79)
[junit] at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
[junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
[junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
[junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
[junit] Caused by: java.io.IOException: java.io.IOException: error=24, Too many open files
[junit] at java.lang.UNIXProcess.<init>(UNIXProcess.java:164)
[junit] at java.lang.ProcessImpl.start(ProcessImpl.java:81)
[junit] at java.lang.ProcessBuilder.start(ProcessBuilder.java:467)
[junit] ... 31 more
{code}
This happens because every {{submitJob()}} creates a new {{JobTracker}} object. The same problem plagues other tests too and drastically slows them down. Creating a single object and use it repetitively should suffice.
> TestFairScheduler fails with "too many open files" error
> --------------------------------------------------------
>
> Key: MAPREDUCE-1245
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1245
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.21.0
> Reporter: Vinod K V
> Fix For: 0.21.0
>
>
> This was caused by MAPREDUCE-1103 and was observed after MAPREDUCE-1239.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1245) TestFairScheduler fails with "too
many open files" error
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal updated MAPREDUCE-1245:
--------------------------------------
Status: Patch Available (was: Open)
> TestFairScheduler fails with "too many open files" error
> --------------------------------------------------------
>
> Key: MAPREDUCE-1245
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1245
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.21.0
> Reporter: Vinod K V
> Fix For: 0.21.0
>
> Attachments: 1245_v1.patch
>
>
> This was caused by MAPREDUCE-1103 and was observed after MAPREDUCE-1239.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1245) TestFairScheduler fails with "too
many open files" error
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal updated MAPREDUCE-1245:
--------------------------------------
Affects Version/s: (was: 0.21.0)
0.22.0
> TestFairScheduler fails with "too many open files" error
> --------------------------------------------------------
>
> Key: MAPREDUCE-1245
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1245
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.22.0
> Reporter: Vinod K V
> Assignee: Sharad Agarwal
> Fix For: 0.22.0
>
> Attachments: 1245_v1.patch
>
>
> This was caused by MAPREDUCE-1103 and was observed after MAPREDUCE-1239.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1245) TestFairScheduler fails with "too
many open files" error
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal updated MAPREDUCE-1245:
--------------------------------------
Resolution: Fixed
Fix Version/s: (was: 0.21.0)
0.22.0
Assignee: Sharad Agarwal
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this. Thanks Vinod for catching this.
> TestFairScheduler fails with "too many open files" error
> --------------------------------------------------------
>
> Key: MAPREDUCE-1245
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1245
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.21.0
> Reporter: Vinod K V
> Assignee: Sharad Agarwal
> Fix For: 0.22.0
>
> Attachments: 1245_v1.patch
>
>
> This was caused by MAPREDUCE-1103 and was observed after MAPREDUCE-1239.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1245) TestFairScheduler fails with "too
many open files" error
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal updated MAPREDUCE-1245:
--------------------------------------
Attachment: 1245_v1.patch
Use a lightweight JT for tests.
> TestFairScheduler fails with "too many open files" error
> --------------------------------------------------------
>
> Key: MAPREDUCE-1245
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1245
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.21.0
> Reporter: Vinod K V
> Fix For: 0.21.0
>
> Attachments: 1245_v1.patch
>
>
> This was caused by MAPREDUCE-1103 and was observed after MAPREDUCE-1239.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1245) TestFairScheduler fails with "too
many open files" error
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sharad Agarwal updated MAPREDUCE-1245:
--------------------------------------
Status: Open (was: Patch Available)
Retrying hudson
> TestFairScheduler fails with "too many open files" error
> --------------------------------------------------------
>
> Key: MAPREDUCE-1245
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1245
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.21.0
> Reporter: Vinod K V
> Fix For: 0.21.0
>
> Attachments: 1245_v1.patch
>
>
> This was caused by MAPREDUCE-1103 and was observed after MAPREDUCE-1239.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.